The CHIME Fast Radio Burst Project: System Overview

The custom electronics that perform the digitization, frequency channelization, and "corner-turn" (see below) are housed in two 20 ft steel shipping containers outfitted with radio-frequency-shielded enclosures that provide >100 dB of shielding of the focal line from the high-speed electronics within. These enclosures (labeled the East and West receiver huts) are located midway between the first and second cylinders and the third and fourth cylinders, halfway along their lengths, to minimize and equalize the coaxial cable lengths that connect to the feed lines.

The F-Engine system consists of 128 "ICE" motherboards (Bandura et al. 2016a) housed in eight rack-mounted crates and interconnected with custom high-speed, full-mesh backplanes. The system is described in detail by Bandura et al. (2016b). Briefly, 16 amplified and filtered analog sky signals are digitized in the second Nyquist zone on daughter cards attached to each motherboard at a data rate of 800 MHz with 8-bit accuracy. This information is transmitted to a Field Programmable Gate Array (FPGA) located on each motherboard. The total data rate digitized by the F-Engine is 13.1 Tb s⁻¹ for the 2048 time streams. The 400 MHz bandwidth of each time stream is channelized into 1024 frequency bins, each 390 kHz wide, using a polyphase filter bank. A programmable gain and phase offset are applied to each frequency channel, and the data are rounded to 4 + 4 bit complex numbers, providing a total data rate of 6.5 Tb s⁻¹.

The "corner-turn" modules in the F-Engine reorganize the channelized data from all the motherboards in a crate in order to concentrate the data for a subset of frequencies into a single FPGA. A second corner-turn module reorganizes the data between a pair of crates located in the same enclosure before offloading the data to the X-Engine GPU nodes (see Section 3.2) through 1024 optical high-speed transceivers using standard 10 Gb s⁻¹ ethernet protocol. Data from all eight ICE motherboard crates are assembled in each GPU node, forming the last stage of the corner-turn. At this point, data for four frequency bins originating from all 2048 digitizers are assembled in a single GPU node, ready to be spatially correlated.

The X-Engine consists of 256 processing nodes, each receiving 25.6 Gb s⁻¹ of frequency-channelized data on 4 × 10 GbE ports. An Intel Xeon E5-2620v3 CPU handles data transfer, using the Intel Data Plane Development Kit (DPDK¹⁹ ) to reliably achieve high throughput. Each node has 64 GB of DDR4 RAM, sufficient to buffer resampled data for up to 20 s; an upgrade to at least 96 GB (for a 31 s buffer) is planned for the near future (see Section 5.3). The nodes occupy custom 4U rack-mount chassis and have no persistent local storage, but instead, a local set of file servers boot the nodes over the network.

Each node processes four input frequency channels of ∼390 kHz bandwidth each. These are processed by two dual-chip AMD FirePro S9300x2 GPUs, with each chip operating independently to perform all spatial and in-band processing on a single frequency channel. The processed data, including the resampled stream for FRB searching, are exported to the back ends over a pair of GbE links.

The X-Engine is entirely liquid cooled, using direct-to-chip cooling on the CPUs and GPUs and coupling the coolant to the ambient outside air with a 3 × 120 mm radiator in the front of each node, without any active chilling. Each rack operates a sealed and independent coolant loop, which is coupled to externally circulating coolant through a CoolIT CHx40 liquid-handling unit. The external coolant exhausts heat to the outside air through a high-capacity dry cooler. The X-Engine is housed in two 40 ft shipping containers directly adjacent to the cylinders (see Figure 1). To limit self-generated interference, a custom-built Faraday cage in each container provides >100 dB of shielding from 1 MHz to 10 GHz.

In order to take full advantage of the large instantaneous FOV of CHIME while maintaining the full sensitivity, we have developed a hybrid beam-forming pipeline (hereafter referred to as the "Level-0" or L0 process; see Figure 4) to be employed in the X-Engine correlator. Details of this pipeline are described in Ng et al. (2017). In summary, we synthesize 256 formed beams via a fast Fourier transform (FFT) algorithm along the north–south direction. The formalism of FFT beam-forming in the context of a radio interferometer can be found in, e.g., Tegmark & Zaldarriaga (2009) or Masui et al. (2017). We zero-fill the FFT by a factor of two and resample the result to improve spatial alignment of the 256 formed beams across the 400 MHz observing bandwidth.The final N–S beams are tiled across a runtime-defined range of angles (nominally ∼110°), evenly spaced in $\sin \theta$ space, where θ is the zenith angle. In the east–west direction, we form four beams via exact phasing. These give a total of 1024 discrete formed static beams that are closely spaced—with the exact spacing a tunable parameter—and tile the entire primary beam continuously. For beams spaced evenly in $\sin \theta$ from θ = −60° to θ = +60°, beam centers are separated by 04, to be compared with the beam FWHM of 05 at 400 MHz and 025 at 800 MHz. Thus, there is significant beam overlap at the lower frequencies. (For reference, for θ = −90° to θ = +90°, which is not obviously desirable in spite of the larger FOV because of primary beam fall-off and occultation by nearby mountains, the beam separation would be 046.) Simulations show that this planned beam spacing results in detections in 6–10 beams for bright (several janskys or more) events that are near the center of a beam, and typically just one for faint events. For bright sources in between beams, as many as 12 beams may detect it.

Zoom In Zoom Out Reset image size

To increase spectral resolution, 128 successive 2.56 μs voltage samples are collected and Fourier-transformed. The square of the magnitude of this spectrum is downsampled in frequency by a factor of 8. Three successive downsampled transforms are averaged together, and the two orthogonal polarizations are summed, producing a Stokes I data stream with a cadence of 0.983 ms and a spectral resolution of 24.4 kHz. The output data thus consist of 1024 total intensity beams, with 16k frequency channels at ∼1 ms cadence. These data are scaled, offset, and packed into 8-bit integers for transmission. In principle, it is possible to perform coherent dedispersion to a selected DM on phase data in the X-Engine; however, this feature has yet to be implemented.

Similar to the X-Engine, the CHIME/FRB instrument consists of a total of 132 compute nodes housed in a customized RF-shielded 40 ft shipping container sitting adjacent to the telescope cylinders (Figure 1). Each node contains two 10-core Intel E5-2630v4 CPUs and 128 GB of RAM in a 4U height (17.8 cm) case. L1 processing, namely, dedispersion (Section 4.3), is the most computationaly intensive and runs on 128 of the nodes (see Figure 4). Each L1 node processes eight beams using 16 cores. The remaining four cores in each node are utilized for assembling the incoming packets, processing the outgoing headers, and intensity data retrieval (see Section 4.7). L2 and L3 share a node, and L4 runs on a node of its own. All nodes are network-booted for flexibility to allow dynamic mapping of nodes to specific functions. To alleviate single-point failures, the L2/L3 and L4 nodes each have a backup node that will automatically take over processing in the event of a node failure. For on-site data storage, a Storinator S45 Turbo unit (manufactured by 45 Drives Inc.) is fitted with 45 10 TB Western Digital Gold drives. The X-Engine and FRB systems are monitored and controlled in part using the Intelligent Platform Management Interface (IPMI),²⁰ using the Prometheus²¹ and Grafana²² software packages.

The FRB instrument is liquid cooled, using the same direct-to-chip cooling as for the X-Engine (Section 3.2). However, as this system does not have GPUs, there is far less heat loading per node (∼200 W) compared to the X-Engine (∼1000 W). A smaller 240 × 120 mm radiator couples the cooling fluid to forced air circulating in each node. One CoolIT CHx40 liquid-handling unit is used to service two full racks that together contain a total of 20 nodes.

The X-Engine simultaneously transmits data to CHIME/Cosmology data-handling nodes, the FRB search back end, and CHIME/Pulsar. For the FRB search back end, each X-Engine node processes 64 24.4 kHz frequency channels for all 1024 beams, while each L1 node processes all 16k frequencies for eight beams. Thus, each L0 node must send data to each L1 node, which is achieved by employing three layers of network switches, as shown in Figure 5. Groups of 20 L0 nodes are connected, via two 1 Gb s⁻¹ links, to a CISCO Catalyst switch (model no. 3650-48TQ-L), which routes the data to a central data switch (CISCO Nexus 3132Q-X) via 4 × 10 GbE outputs. The data are then routed through a second set of Catalyst switches to groups of 10 L1 nodes on four 1 Gb s⁻¹ links. In total, this network consists of 21 Catalyst (14 for L0 and 7 for L1) and two Nexus switches. One of the 1 Gb s⁻¹ links on L0 is dedicated to transmitting data to the FRB and Pulsar back ends, at a rate of 0.55 and 0.25 Gb s⁻¹, respectively, with the transmission of each data packet timed to avoid data loss of the UDP packets. Each L1 node receives data at a rate of 1.1 Gb s⁻¹ for a total data rate for the FRB system of 142 Gb s⁻¹. The CHIME/Cosmology data and FRB baseband data (when requested—see Section 5.3), are transmitted on the second L0 1 Gb s⁻¹ link to archive machines. The network traffic is continuously monitored for missing or corrupted data packets so that these effects can be included in the sensitivity calculations for the FRB search (Section 5.2).

Zoom In Zoom Out Reset image size

The most computationally expensive part of the CHIME/FRB search is the "dedispersion transform," which converts intensity data from time and frequency into time and DM, allowing efficient detection of dispersed impulse signals (see Figure 4). In our implementation of the dedispersion transform, there are five key FRB parameters: DM and arrival time (each of which ranges over a large number of trial values), spectral index, scattering time, and intrinsic width (which range over a few trial values). Thus, the output of the dedispersion transform is a 5D array of signal-to-noise ratios (S/Ns), to which we apply a tunable threshold, currently set at 10σ given our current experience with fainter pulsar signals (see Section 6), to identify candidate events for processing by subsequent stages of the pipeline.

The computational challenge of the CHIME/FRB search is immense: the input data rate is 1.5 PB day^–1, and the dedispersion transform computes 10¹¹ S/N values per second (total for all beams). This is orders of magnitude larger than any FRB search to date, and comparable to the future search planned for the Square Kilometer Array.²³ For example, the data rate of the 36-beam ASKAP "fly's-eye" search (Bannister et al. 2017) is 1700 times smaller than for CHIME/FRB.

To meet these challenges, we have developed a new FRB search code "bonsai," which we will release publicly in the near future. This code will be described in detail in a forthcoming paper (K. M. Smith et al. 2018, in preparation). Here we briefly summarize its main attributes.

There are two general approaches to dedispersion: direct algorithms (see, e.g., Manchester & Taylor 1977) and tree algorithms (Taylor 1974; Zackay & Ofek 2017). The computational cost of tree dedispersion is ${ \mathcal O }({TF}\mathrm{log}F)$, versus ${ \mathcal O }({{TF}}^{2})$ for direct dedispersion, where T and F are the number of time and frequency samples, respectively. Thus, tree dedispersion is parametrically faster but suffers from two problems in practice. First, tree dedispersion tends to perform badly owing to memory bandwidth bottlenecks. In a straightforward implementation, a large memory region (larger than the Level 3 cache available in the CPU of our nodes) must be processed in each of ${\mathrm{log}}_{2}(F)$ iterations of the algorithm. The cost of these memory transactions ends up being much larger than the computational cost. Second, in some implementations, tree dedispersion may make approximations to the ν⁻² dispersion delay, which results in loss of S/N.

Our implementation of tree dedispersion, bonsai, includes two new features that solve these problems. To address memory bandwidth bottlenecks, we have implemented a version of tree dedispersion that is "blocked," in the sense that the data are processed in a small number of passes, in blocks having size tuned to that of the CPU cache. This minimizes the number of times the data go through the CPU Level 3 cache, resulting in a factor of ∼30 speed-up on multicore machines. An interesting property of the blocked tree algorithm is that it is also incremental: given sufficiently large S/N, the search triggers within a few seconds, even for an FRB whose dispersion delay in the CHIME band is much larger. This low latency is critical for the triggered baseband recording system (Section 5.3), since the X-engine can currently only buffer data for 20 s, and this window must include both the dedispersion delay of the FRB and the latency of the search.

Second, we have made tree dedispersion close to statistically optimal through appropriate choices of internal weightings. For CHIME/FRB, we choose parameters that are a compromise between computational cost and statistical optimality. We search to a maximum DM of 13,000 pc cm⁻³ and a maximum pulse width of 100 ms, with two trial spectral indices, −3 and +3, to provide added sensitivity to bursts with nonuniform spectra. Though DM 13,000 pc cm⁻³ is well beyond the maximum DM yet observed, it requires little extra computational time to reach this value over, for example, 3,000 pc cm⁻³, near the maximum yet seen (Bhandari et al. 2018). After initial RFI removal (see Section 4.8), the search consists of multiple dedispersion trees with different levels of time downsampling, to cover different parts of the (DM, pulse width) parameter space. The search is >80% optimal over most of this parameter space, occupying 0.75 cores beam⁻¹, and a memory footprint of 7 GB beam⁻¹. This more than suffices to meet the initial science goals of the CHIME/FRB search.

The bonsai code produces S/N estimates for a grid of DMs and times for a discrete set of spectral indices, scattering times, and intrinsic width parameters. As a detail, the output array of S/N estimates is "coarse-grained": we divide the grid of trial arrival times and DMs into coarse cells and take the maximum of all S/N values over each cell. We set a (tunable) threshold at 10σ and isolate local maxima within a neighborhood of size ΔDM ≈ 10 pc cm⁻³ and Δt ≈ 0.25 s. Once we have identified the peak S/N parameters, we generate a lightweight description of the coarse-grained candidate event, the "L1 Header." This includes the DM, time, and coordinates of the trigger; an RFI rating (see Section 4.8); and S/N values for slices along each parameter dimension (holding the remaining parameters fixed at their peak values).

To validate the accuracy and sensitivity of our candidate identification, we have performed injection analyses and processed CHIME Pathfinder (Bandura et al. 2014) acquisitions containing pulsations from PSR B0329+54 and PSR B0531+21. Compared to a pipeline based on RRATTrap (Karako-Argaman et al. 2015) and PRESTO²⁴ single-pulse search (e.g., Chawla et al. 2017), we find in initial analyses that the CHIME/FRB L1 pipeline has a lower rate of RFI false positives and is more effective at finding faint pulses (see Section 6).

Per-beam event detection reports from the L1 bonsai dedispersion analysis (see Section 4.3) are streamed to a single endpoint where they are buffered. These reports consist of L1 headers or null results. Once all beams have been reported for a given data block, the existence of multibeam events is investigated by grouping candidates in time, DM, and sky position. The cluster detection is performed with a simplified implementation of the DBSCAN²⁵ algorithm (Ester et al. 1996), which has ${ \mathcal O }(n\mathrm{log}n)$ complexity, where n is the number of events above our threshold and is capable of handling large event rates without issue. For any event in a group, there exists another whose differences in time, DM, and sky position are all below dimension-specific thresholds. For position, the threshold ensures that grouped events are spatially connected through beam adjacency. The DM and time thresholds reflect the uncertainties built into the coarse-grained event properties (see Section 4.3). After cluster detection, the event is classified as either RFI (see Section 4.8) or astrophysical, and the L2 header is formed, containing one or more L1 headers from the grouping. RFI events are sent straight to the L4 database (Section 4.6), while astrophysical events are passed to L3, after position refinement and event flux estimation, described next.

Position refinement is performed by comparing the detected S/Ns from each beam to what is expected from a frequency-dependent beam model. For multibeam events, the detected S/Ns are compared to those in a precomputed look-up table containing relative S/Ns for a grid of sky positions and spectral indexes to form a refined estimate of these parameters. χ² minimization is then used to refine the initial guess and estimate the uncertainty region. For single-beam events, a precomputed mapping between event S/N and the location and uncertainty region is used. The mapping is constructed from the distribution of simulated events drawn from the beam model that produce nondetections in adjacent beams. For very bright (S/N ∼ 1000) events having spectra well described by a power law and that hit randomly with respect to beam centers, simulations of our real-time pipeline show that positions can in principle be determined to ∼03, and spectral indexes to within ±0.5.

With a refined position and estimate of the on-axis S/N, radio fluxes of events are then estimated in real time. The estimate is based on the radiometer equation for single pulses (see, e.g., Cordes & McLaughlin 2003). Besides instrument and pipeline-specific parameters, we use the S/N and pulse width as estimated by bonsai as input. The S/N is corrected for the beam sensitivity at the location of the event, and the receiver bandwidth is corrected for the masked fraction from RFI excision. The sky temperature at the location of the event is interpolated from the reprocessed Haslam 408 MHz sky temperature map (Remazeilles et al. 2015) and scaled to our survey central frequency (600 MHz) using a power-law index of −2.6. The intrinsic pulse width is estimated by correcting for the sampling time, intrachannel smearing, and scattering, in quadrature. Intergalactic scattering times are estimated as in Section 3.11 of Yao et al. (2017), where we estimate the Galactic scattering contribution using the estimated Galactic DM contribution and the empirical DM–scattering relation from Krishnakumar et al. (2015), ignoring potential scattering in the host galaxy of the source.

Astrophysical events can be of Galactic or extragalactic origin and can come from either a known or an unknown source. The processing-related actions triggered for an event depend on its identification, and, as detailed below, the pipeline makes this decision in L3 based on the determined DM and localized position. By automatically recognizing events that come from known sources, we can provide alerts (see Section 5.1) only for FRBs, and not for Galactic objects. However, all astrophysical L1 events are stored, and additional data products for non-FRB events can also be stored for selected interesting sources.

We maintain a database of known sources that initially consists of all known pulsars, RRATs, and FRBs and their relevant parameters recorded in the ATNF pulsar catalog²⁶ (Manchester et al. 2005), the "RRATalog,"²⁷ and the FRB catalog²⁸ (Petroff et al. 2016). As time progresses, discoveries by CHIME/FRB and other FRB searches will be added to the database. For each astrophysical event, we compare the sky position and DM, weighted by the measurement error, to the same parameters of the sources in the database. The likelihood ratio (i.e., the Bayes factor) of association with neighboring known sources is calculated. A threshold for association is determined based on the Receiver Operating Characteristic (ROC) curve (the true-positive vs. the false-positive rate) for a set of simulated events. Currently, only the sky location and DM of an event are being compared with those of the known sources, but the framework allows for future additions (such as a comparison of the periodicity or the flux).

For events that cannot be associated with a known source, the predicted maximum Galactic DM along the line of sight is calculated using models described by Cordes & Lazio (2001) and Yao et al. (2017) of the Galactic free electron density. The difference in predicted maximum Galactic DM between the two models is taken as a systematic uncertainty; this is then added in quadrature with the L1-estimated DM uncertainty to yield a full uncertainty, σ. We impose the condition that a σ less than a minimum value (currently set to 20%) is set to this minimum, to avoid underestimating the true uncertainty when the two models agree coincidentally. In the real-time pipeline, a source is deemed extragalactic if its measured DM exceeds the maximum Galactic DM predicted by both models by at least 5σ. Sources with a measured DM only 2σ–5σ in excess of the predicted maximum Galactic DM fall in an ambiguous class of sources. Though otherwise not treated differently, this class is invoked to highlight sources that may be Galactic, warranting a multiwavelength study to identify possible Galactic foregrounds such as H ii regions. Alternatively, they may be extragalactic, but relatively nearby, and hence potentially good targets for multiwavelength counterpart identification.

Finally, a set of predefined rules determine which actions to trigger for each detection. Possible actions are to ignore an event, store the event header in the events database (the default action), call back for buffered intensity data from L1 (see Section 4.7), dump buffered baseband data from L0 (see Section 5.3), ask the CHIME/Pulsar observing system to track the source position in search mode, or send out an alert (Section 5.1). The rules can be set for individual sources or for groups of sources, with all rules being stored in a configuration database that is accessible through a web interface. Some examples of action rules are the following: "if the flux of a binary millisecond pulsar pulse is 100 times higher than its nominal flux, store the event header and trigger an intensity callback," and "if the source is unknown and the event extragalactic, store the event header, trigger an intensity callback and a baseband dump, and alert the community."

L4 first implements the actions selected in L3 for each event, such as requests to L1 for intensity data callbacks, requests for monitoring with CHIME/Pulsar, or sending out real-time alerts to the community for FRB detections. L4 is also host to the CHIME/FRB archive, which saves the header information and its associated analysis products in a relational database for every event sent past the L1 stage regardless of whether it is classified as astrophysical or RFI in the L2 stage. Each event header that arrives at the L4 stage of the pipeline is assigned a unique event number to identify the event and track all data products associated with it.

The size and quantity of data products stored for each event depend on the actions and priorities determined at the previous stages of the pipeline. Due to the current limited bandwidth out of the observatory, a local archiver with 450 TB of disk space has been installed. The data will be regularly transferred to a long-term offsite staging facility. In order to manage file transfers between different locations, we employ a custom file management software alpenhorn²⁹ (Hincks et al. 2015). Alpenhorn ensures reliable data transfer between multiple originating, intermediate, and archival storage sites. Among its features are keeping at least two copies of the data at different sites at all times, verifying data integrity through checksums, and handling data transfer among sites. The FRB pipeline database is periodically updated with the current status of the data files tracked by alpenhorn, so that a user accessing the live website has access to their location.

The database is integrated into a web display that allows for searching and filtering of events, displaying the best data measurements, interacting with data plots, tracing the location, and requesting downloads of raw data. The web display also incorporates user accounts, allowing users to rank, classify, and comment on candidate events. Users can also update action criteria and known source properties.

Since the data rate of CHIME/FRB is very high, we cannot save to disk all the intensity data taken by the telescope. Instead, each L1 node stores a buffer of recent data that can be retrieved if an interesting event is detected by L3 (see Section 4.5). We save data from the beams in which the event is detected independently, as well as the immediately adjacent beams.

CHIME/FRB searches up to a DM of 13,000 pc cm⁻³. At this maximal DM, a pulse can take up to ∼250 s to sweep through the CHIME frequency band. The L1 nodes do not have enough memory to store this much intensity data at full resolution, so we implemented a telescoping ring buffer: older data are progressively downsampled and stored with coarser time resolution to optimize memory usage. Initially, we keep 60 s of the most recent data at full resolution, 120 s binned by a factor of two, and 240 s binned by a factor of four.

As the L1 nodes do not have local storage, called-back intensity data are written to a network-shared archiver (see Section 4.6), where they are available for additional offline analysis and visualization. Apart from generating the dynamic spectra ("waterfall plots") for visualization of single-pulse events, we refine the event parameters (DM, arrival time, pulse width) determined by the L1 pipeline. We perform an improved localization using the intensity data from neighboring beams (considering detections as well as nondetections) and obtain more accurate values for spectral index and flux.

The CHIME/FRB software pipeline mitigates RFI using different criteria at different stages in order to minimize the misclassification of terrestrial signals as astrophysical transients.

In L1, we remove RFI signals from the intensity data by iteratively detrending and subtracting a time series obtained by assuming zero DM (Eatough et al. 2009) from the intensity data, while clipping amplitude and standard-deviation outliers. We have developed fast C++/assembly kernels that form a sequence of "transforms" at a high level. Each transform operates on arrays of intensity data with a parallel array of weights (where masked intensities are represented as zeros). The output of each transform is the input to the next. We have utilized low-level ring buffering and transparent resizing and resampling of the data, since the transforms need not use the same block size or downsampling.

Our current default RFI removal scheme consists of a sequence of nine clipping transforms (five with 3σ threshold and four with 5σ threshold), which are iterated six times. The sequence is repeated twice; after each iteration, the intensity is detrended by two polynomial (degree 4) and spline (equivalent degree 12) transforms, one along the time axis and one along frequency, respectively. We have found empirically that this transform chain removes RFI in data we have captured so far (e.g., from the CHIME Pathfinder, the Galt 26 m telescope, or with a CHIME incoherent beam—see Section 6) to a sufficient level that simulated FRBs near detection threshold dominate the output of the dedispersion transform. Our real-time detrending and RFI excision pipeline is currently in a proof-of-concept stage, and we are still experimenting with the implementation.

After candidate events have been identified (see Section 4.3), and before they are forwarded to L2 (see Section 4.4), a second round of RFI excision is performed by subjecting the S/N behavior surrounding significant detections to a machine-learning classifier—in this case, a support vector machine (Cortes & Vapnik 1995). Currently, we include two features with the goal of capturing both local and global S/N behaviors. For the former, we use the S/N fall-off at nearby but suboptimal DMs. For the latter, we consider the distribution of other above-threshold locations in the DM–time plane, noting that RFI tends to manifest as vertical streaks of events covering a large range of DMs. As our chosen machine-learning method requires supervision, we have developed tools to allow for efficient event labeling; training the classifier and refining feature definitions will be an ongoing process throughout commissioning.

The candidate events detected in single beams are transmitted from all L1 nodes to the L2 node. At this stage we group events detected in spatially neighboring beams and within a DM and time threshold. We further classify these events as astrophysical or RFI based on the distribution of S/Ns in neighboring beams and the shape of the grouped beams. Astrophysical events are in the far field of the telescope (≳25–50 km for CHIME's operating frequency band) and are expected to have a sharply focused intensity distribution pattern, whereas RFI events are typically in the near field and likely detected in the horizon-directed sidelobes, causing a significantly broader intensity distribution pattern.

In L2, we use a machine-learning classifier, implemented using scikit-learn³⁰ (Pedregosa et al. 2011), to calculate the probability of an event being astrophysical or RFI. Ultimately, multiple different classifiers can be loaded, and their calculated probabilities are assumed to be independent by the framework and multiplied together. Currently, the classifier is based on a stochastic gradient descent classifier with a linear combination of the following features: (1) the pattern of beams in a 3 × 3 grid in which the event was detected above threshold with respect to the beam with the highest S/N, (2) the ratio of the second-highest S/N to the highest S/N in the group, (3) the ratio of the average S/N of the detected beams to the highest S/N in the group, (4) the total number of beams where the event was detected, and (5) the highest L1 grade associated with the group from the L1 classifier. For events detected in single beams, no extra information is available, so the grade from the L1 classifier is the only value used for classification. In the future, we may implement features based on (1) the total L1 event activity (number of candidates detected in the same time window) at the time of event detection, (2) ratio of the FFT-formed beam intensity to that of an incoherent beam formed from all CHIME feeds, and (3) location of the beams on the sky (as RFI seems more likely to be detected at lower elevation beams).

As with L1, we are developing event labeling and generation of training sets for the classifier. At the L2 stage, all L1 candidates that are sent have their bonsai-determined information (S/N, DM, width, beam coordinates, etc.) stored in the database irrespective of their RFI/astrophysical classification. This allows us to reclassify and retrain the classifiers as required in the future.

Rapid reporting of FRB discoveries is important for coordination of detections with FRB searches at other radio bands, for finding repeat bursts from FRBs, and for discovering or ruling out possible electromagnetic or gravitational wave counterparts. Specifically for CHIME's FOV, facilities such as the Owens Valley Long Wavelength Array (23–88 MHz; G. Hallinan et al. 2018, in preparation) or the DSA-10 (1.4 GHz; V. Ravi et al. 2018, in preparation) could provide constraints on the spectral indices, spectral ranges, absorption, and scattering properties of FRBs. The non-Poissonian clustering of FRB 121102 (Scholz et al. 2016) makes the detection of repeat bursts more likely after the initial detection. If this characteristic holds for other FRBs as well, the monitoring of CHIME/FRB-discovered sources with tracking telescopes would be an important test of repetition, as well as an opportunity for interferometric localization (e.g., with the Jansky Very Large Array³¹ in New Mexico as in Chatterjee et al. 2017).

The CHIME/FRB real-time alert system will follow the VOEvent implementation suggested by Petroff et al. (2017) to be compatible with the other FRB search efforts around the world. After a period of human confirmation at the beginning of the CHIME/FRB program, we plan to announce every detected FRB along with the significance of the detection and other relevant characteristics in real time. The latency of the CHIME/FRB real-time pipeline from the arrival of the lowest frequency signal is ∼2–3 s. FRBs for which the intensity data are stored and reanalyzed (see Section 4.7), or for which there has been a baseband dump and subsequent improved parameter determination (see Section 5.3), will have updates to their reported characteristics announced as soon as possible.

CHIME/FRB is a complex instrument with multiple components that must be regularly calibrated and monitored to reach its science goals. Here we describe the three planned stages of the process: phase calibration of CHIME, monitoring of the pipeline performance to track CHIME/FRB's exposure to the sky, and measurement of the sensitivity and completeness of the CHIME/FRB pipeline.

In order to form phased beams, each RF input signal must be phase-calibrated regularly. While ultimately milliradian phase calibration will be available for CHIME in order to meet the stringent requirements of the cosmology experiment, this calibration accuracy will not be available at the commencement of FRB searching. For the CHIME/FRB system, we will determine the complex gains in a near real-time system using transits of bright effective point sources like Cassiopeia A and Cygnus A (Trotter et al. 2017), approximately once per day. We will observe such a calibration source with a full set of N(N − 1)/2 visibilities, where N is the number of feeds, using the CHIME cosmology X-Engine at a full complement of radio frequencies. We will then compensate for the delay expected from the physical locations of the antennas and the calibrator sky location. Any observed residual delay in the response to the calibrator will be instrumental, which we record and compensate for in the FRB beam-former in L0. The observations of the calibrator sources will also be used to calibrate the CHIME bandpass.

Most scientific goals of CHIME/FRB—measurements of the FRB rate and of property distributions—require a thorough understanding of CHIME/FRB's detection sensitivity as a function of time and sky location (i.e., the exposure of the survey) and the completeness of its survey as a function of burst fluence, burst width, DM, etc. We take two approaches that allow us to cross-check our results. To determine the exposure and the sensitivity as a function of time, we have to monitor metrics from the data flow and compute a sensitivity as a function of time and beam location using the radiometer equation (McLaughlin & Cordes 2003). However, the multiple complex stages of RFI filtering and excision make it challenging to understand and study the completeness, especially in the presence of rapidly changing RFI that is correlated across phased beams. Hence, we also plan to create a real-time multibeam FRB injection and detection system to estimate the survey completeness in real time.

To account for the total sensitivity and exposure of CHIME/FRB, we have to keep track of the sensitivity of each coherent beam and the number of beams that are being searched for FRBs at a given time. At the L0 stage, CHIME/FRB will monitor which feeds from the telescope are usable (informing beam shape and sensitivity) and which L0 nodes are active (informing the bandwidth and usable frequency channels). At the L1 stage, we monitor the UDP packet loss from L0 nodes to the L1 nodes (again informing usable bandwidth and frequencies), the fraction of data masked by the first-stage RFI removal algorithm, and the number of L1 nodes that are actively searching for FRBs, informing the sensitivity of each beam on the sky. At the L2/L3 stage, the number and identity of the reporting L1 nodes and the delay in TCP packets are tracked.

Most of these metrics will be reported on a few-minute timescale and stored in a time-tagged database. A database scraping script will then collect the metrics and calculate the sensitivity and exposure of CHIME/FRB to the sky.

In order to test the response of our RFI excision and filtering, we plan to inject FRB signals of known properties, accounting for the spatial and spectral responses of the coherent beams, into a small number of the beams and verify that we recover them as expected. Figure 6 shows a schematic of the injection pipeline that will run in parallel to the main CHIME/FRB search pipeline. A grid of 4 × 4 coherent beams will be chosen randomly among the 1024 beams to sample different north–south locations in the primary beam. An injection server will generate a set of FRB parameters (location, flux, DM, width, scattering, and spectral index) from a prior distribution. The effective signal strength in each beam and frequency band will then be computed based on the coherent beam model. A simulator code will inject the signal into the intensity data from the corresponding beams. The rest of the data analysis will proceed identically to the CHIME/FRB search pipeline. The injected parameters and the recovered parameters will be stored in a temporary database where the detection completeness of the pipeline as a function of FRB parameters can be analyzed. A detailed paper reporting the measured sensitivity of the CHIME/FRB search pipeline will be forthcoming once commissioning (Section 6) is complete.

Zoom In Zoom Out Reset image size

We plan to have triggered recordings of baseband data containing FRBs. This system will provide CHIME/FRB with sensitivity to polarization, greatly improved ability to localize sources, and spectral and time resolution constrained only by the uncertainty principle ΔfΔt ≥ 1, and it will open possibilities to perform very longbaseline interferometry (VLBI) of FRBs with other telescopes. This baseband system is currently under development and will be commissioned once CHIME/FRB science operations are underway. Here we provide our current plans for this system.

Baseband data from all 2048 correlator inputs that have been spectrally channelized by the F-Engine will be buffered in the X-Engine prior to beam-forming. These data will be written to disk on an archiver system upon receipt of a trigger from the FRB search pipeline (see Section 4.5). Ultimately, this system will allot approximately 90 GB of random access memory in each X-Engine node, providing a 31 s ring buffer—long enough for the few seconds of latency in the FRB search pipeline, as well as the dispersion delay across the band for DMs up to ∼1500 pc cm⁻³. To economize network resources and disk space, only the 100 ms of data surrounding the burst will normally be read out and stored to disk, for each frequency channel. These data will be analyzed offline in the initial implementations of the system. The buffer system will be designed so that upon receipt of a trigger only a small block of memory containing the desired 100 ms of data will be frozen, with the rest of the memory continuing to participate in the ring buffer. As such, the system will have no down time during readout of baseband data, only a slight shortening of the buffer.

Among the most compelling applications for the baseband system is improved localization precision of FRB sources. Rather than sources being localized to a formed beam width of ∼λ/D (where λ is the observing wavelength and D is the telescope extent, notably different in the north/south versus east/west direction), the baseband system permits interferometric localizations with precision (Masui et al. 2017)

$$\begin{eqnarray}&&{\rm{\Delta }}\theta =\displaystyle \frac{\sqrt{6}}{2\pi }\displaystyle \frac{\lambda }{D}\displaystyle \frac{1}{{\rm{S}}/{\rm{N}}}\,\,\mathrm{rad}.\end{eqnarray} \tag{ 1 }$$

Here S/N is the S/N of the burst as detected in the baseband data, and the geometric factor of $\sqrt{6}$ accounts for CHIME's baseline distribution. This will amount to arcminute uncertainties for most bursts.

The baseband system, through the use of coherent dedispersion (Hankins & Rickett 1975), will also permit the identification of structure in the FRB signals at time resolution well under the 1 ms provided to the CHIME/FRB search engine. Any such structure will constrain FRB emission physics. Similarly, the baseband data will allow the detailed investigation of FRB scattering and scintillation timescales. These properties can be used to constrain the astrophysical environments in which FRBs occur (e.g., Masui et al. 2015).

Finally, baseband data will contain full Stokes information, allowing us to determine the degree to which each FRB is polarized (Masui et al. 2015; Petroff et al. 2015a; Ravi et al. 2016; Michilli et al. 2018). In cases where an FRB has a linearly polarized component, the dependence of the polarization angle on observing frequency can then be used to calculate the Faraday rotation measure (RM). The RM corresponds to the integral of the electron-density-weighted line-of-sight magnetic field between the observer and the FRB and thus is a powerful probe of an FRB's immediate environment, host galaxy, and the intervening intergalactic medium (Petroff et al. 2015c; Akahori et al. 2016; Ravi et al. 2016; Michilli et al. 2018).