Open AccessArticle

VIIRS Edition 1 Cloud Properties for CERES, Part 2: Evaluation with CALIPSO

Christopher R. Yost

^1,*

Patrick Minnis

¹,

Sunny Sun-Mack

¹,

William L. Smith, Jr.

² and

Qing Z. Trepte

Science Systems and Applications, Inc., Hampton, VA 23666, USA

Science Directorate, NASA Langley Research Center, Hampton, VA 23681, USA

Author to whom correspondence should be addressed.

Remote Sens. 2023, 15(5), 1349; https://doi.org/10.3390/rs15051349

Submission received: 15 December 2022 / Revised: 15 February 2023 / Accepted: 22 February 2023 / Published: 28 February 2023

(This article belongs to the Special Issue VIIRS 2011–2021: Ten Years of Success in Earth Observations)

Download

Browse Figures

Figure 1
Gridded (5° × 5°) mean daytime (a–c) and nighttime (d–f) cloud fractions from CV1S (a,d) and CALIPSO (HA ≤ 5 km; (b,e)) and their differences (c,f) using all matched data from JAJO 2015–2016. "> Figure 2
Mean zonal (a) day and (b) night cloud fractions from CV1S and CALIOP for four horizontal averaging scales, JAJO 2015–2016, using all matched data. "> Figure 3
Dependence of CV1S cloud detection (a) fraction correct, (b) bias, (c) false alarm rate, and (d) hit rate on CALIOP horizontal averaging, JAJO, 2015–2016, for 0/100 matched pixels. "> Figure 4
Average (a,b) daytime and (c,d) nighttime (a,c) liquid and (b,d) ice cloud amounts for CV1S and CALIOP pixels collocated within ±15 min, JAJO 2015–2016. "> Figure 5
CV1S 0/100 (a) day and (b) night scene type fraction for various classifications as determined from CALIOP HA ≤ 5 km data for collocations with ±2.5 min during JAJO 2015–2016. * denotes single-layer, single phase clouds. Brackets indicate all CALIOP classifications treated as liquid-phase (blue) and ice-phase (red) clouds in height and COD discussion. "> Figure 6
CV1S versus CALIOP cloud-top heights for 0/100 liquid clouds, JAJO 2015–2016, (a,b): All liquid-dominant clouds, (c,d): Single-layer liquid clouds only. (a,c): day, (b,d): night. "> Figure 7
Regional mean cloud-top height differences between CV1S (CTHV) and CALIOP (CTHC) for 0/100 liquid water clouds, JAJO 2015–2016. (a,c): Single-layer (SL), (b,d): all with liquid tops, (a,b): day, (c,d): night. "> Figure 8
CV1S versus CALIOP cloud-top heights for all 0/100 single-layer ice clouds, JAJO 2015–2016, (a,b): Daytime, (c,d): Nighttime. (a,c): opaque, (b,d): non-opaque. "> Figure 9
Probability distributions of cloud-top height differences, CTHV − CTHC, regardless of phase, JAJO 2015–2016. (a,b): day, (c,d): night, (a,c): all matches, (b,d): overcast matches. "> Figure 10
Cloud-clear detection FC as function of collocation time window and VIIRS VZA for 0/100 cases, JAJO 2015–2016. Mean values are shown for low VZA (0–30°) in black and high VZA (30–80°) in magenta for day (a,d,g,j) and night (b,e,h,k). Sampling density (c,f,i,l) is presented as function of scan time difference and VZA in the (c,f,i,l). "> Figure 11
Dependence of CV1S true positive rate on CODC for 0/100 ice clouds, HA ≤ 1 km, JAJO 2015–2016. Solid (hashed) bars indicate fraction of cloud profiles for (a) day and (b) night observations. "> Figure 12
Variation of CV1S (a) day and (b) night scene identification and phase for multilayer non-opaque cirrus over low clouds as a function of CALIOP cirrus optical depth (CODu) using HA ≤ 80 km data collocated within 15 min. Number of samples used as ratio to total CALIOP cases is indicated in the blue box. The asterisks denote the CODu of the median and 90th percentile. "> Figure 13
CV1S classification of (a) day and (b) night clouds having effective temperatures between 0.0 °C and −40 °C for three different categories liquid, ice, and mixed phase clouds as determined from CALIOP profiles using the method of [<a href="#B30-remotesensing-15-01349" class="html-bibr">30</a>]. Percentages of each CALIOP category classified by CV1S as liquid or ice cloud are indicated in blue and red, respectively. "> Figure 14
Frequency of occurrence of CV1S COD for liquid water clouds, 2012, 30°S–60°S, (a) day and (b) night. "> Figure 15
Histograms of SIF ocean CTH differences as a function of the brightness temperature differences (BTD) between the VIIRS 11 and 12 µm channels (a,c) and the CV1S cloud optical depth (b,d) from daytime (a,b) and nighttime (c,d) data for 0/100 SL overcast liquid water clouds, JAJO 2015–2016. Fractional frequencies of occurrence are provided in the bottom portion of each plot. Mean differences are indicated by horizontal lines in each bar. Shaded area indicates the range of data for each bar. "> Figure 16
Scatterplots of 0/100 CALIOP and CV1S nighttime, single-layer, ice cloud top heights for the (a,c) tropics (latitude ≤ 30°) and (b,d) mid-latitudes (30° < |latitude| ≤ 60°) for opaque (a,b) and non-opaque (c,d) clouds, JAJO 2015–2016, time window of ±15 min. "> Figure 17
Temperature profiles near Fernando de Noronha, Brazil, 12 UTC, 15 July 2022 at 3.85°S, 32.41°W. (a) full profile to 28 km, (b) closeup of profile between 10 and 20 km. Thick dashed blue line is from G541 used by CERES and the solid red line is from a radiosonde. Zp and the thin dashed line mark the tropopause height from G541. Approximate boundaries of the tropical tropopause layer shown as green lines. "> Figure 18
(a) CV1S daytime cloud phase selection for different CALIOP cloud types and (b–i) scatterplots of 0/100 CTHC and CTHV pairs for each cloud type. Data are from JAJO 2015–2016 and were constrained to CALIOP/CV1S observation time differences of |∆t| ≤ 15 min. CALIOP HA ≤ 5 km. "> Figure 19
Same as <a href="#remotesensing-15-01349-f018" class="html-fig">Figure 18</a>, except for nighttime data. "> Figure 20
Cloud-top height biases, ∆Z, as a function of CALIOP/CV1S cloud-top phase agreement for CALIOP VFM cloud types, JAJO 2015–2016, |∆t| ≤ 15 min. W and I represent water and ice, respectively. (a,b): Biases, (c,d): Percent of each phase pair comprising the cloud type samples. (a,c): Day. (b,d): Night. ">

Review Reports Versions Notes

Abstract

The decades-long Clouds and Earth’s Radiant Energy System (CERES) Project includes both cloud and radiation measurements from instruments on the Aqua, Terra, and Suomi National Polar-orbiting Partnership (SNPP) satellites. To build a reliable long-term climate data record, it is important to determine the accuracies of the parameters retrieved from the sensors on each satellite. Cloud amount, phase, and top height derived from radiances taken by the Visible Infrared Imaging Radiometer Suite (VIIRS) on the SNPP are evaluated relative to the same quantities determined from measurements by the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) on the Cloud Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) spacecraft. The accuracies of the VIIRS cloud fractions are found to be as good as or better than those for the CERES amounts determined from Aqua MODerate-resolution Imaging Spectroradiometer (MODIS) data and for cloud fractions estimated by two other operational algorithms. Sensitivities of cloud fraction bias to CALIOP resolution, matching time window, and viewing zenith angle are examined. VIIRS cloud phase biases are slightly greater than their CERES MODIS counterparts. A majority of cloud phase errors are due to multilayer clouds during the daytime and supercooled liquid water clouds at night. CERES VIIRS cloud-top height biases are similar to those from CERES MODIS, except for ice clouds, which are smaller than those from CERES MODIS. CERES VIIRS cloud phase and top height uncertainties overall are very similar to or better than several operational algorithms, but fail to match the accuracies of experimental machine learning techniques. The greatest errors occur for multilayered clouds and clouds with phase misclassification. Cloud top heights can be improved by relaxing tropopause constraints, improving lapse-rate to model temperature profiles, and accounting for multilayer clouds. Other suggestions for improving the retrievals are also discussed.

Keywords:

cloud; Clouds and the Earth’s Radiant Energy System (CERES); CALIPSO; cloud height; cloud phase; cloud optical depth; cloud remote sensing; Suomi National Polar-orbiting Partnership (SNPP); Visible Infrared Imaging Radiometer Suite (VIIRS); validation

1. Introduction

Since 1998, the Clouds and Earth’s Radiant Energy System (CERES) Project has been generating climate data records (CDRs) consisting of Earth radiation budget parameters and cloud properties [1,2]. These quantities are based on the interpretation of broadband and narrowband radiance measurements taken by radiometers and imagers on several satellites. Currently, the CERES broadband radiometers operate on four satellites, Terra, Aqua, the Suomi National Polar-orbiting Partnership (SNPP) platform, and NOAA-20. The last three are in Sun-synchronous orbits with equatorial crossing times near 1330 LT, while Terra crosses the Equator around 1030 LT. SNPP and NOAA-20 carry the imager, Visible Infrared Imaging Radiometer Suite (VIIRS) [3], while the other two have the MODerate-resolution Imaging Spectroradiometer (MODIS). The imagers are used to estimate cloud properties coincident with the CERES broadband measurements. It is important for the CERES CDRs that both the radiation and cloud parameters be not only consistent, on average, across platform, but they should also be of similar accuracy.

The CERES SNPP VIIRS Edition 1 (CV1S) cloud properties were described and discussed in Part 1 of this paper [4]. With some notable exceptions, the average results for most parameters were found to be reasonably consistent with the corresponding means from the CERES Aqua MODIS Edition 4 (CM4A) cloud products. Quantitative details of the differences are provided by [4]. The direct comparisons of the average CV1S and CM4A properties comprise a significant step toward evaluating their consistency. Determining the instantaneous accuracy of the CV1S product and the sources of errors should provide additional information needed for improving the retrieved parameters in future editions and will aid enhancement of the consistency with the CERES MODIS cloud properties in future editions.

Over the past 16 years, retrievals of cloud parameters from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) on the Cloud Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) spacecraft [5] have served as a cloud-truth reference for assessing several key cloud properties determined from passive satellite measurements (e.g., [6,7,8,9,10,11,12,13,14,15,16,17,18]). Recently, refs. [19,20] employed the CALIOP products along with CloudSat Cloud Profiling Radar (CPR) data [21] to determine the accuracy of several CM4A cloud properties. This paper applies the same methods used for CM4A to evaluate the uncertainties in many of the same parameters such as cloud fraction, phase, and top height, but retrieved with the CV1S algorithms. The comparisons will then be examined to determine the instantaneous consistency with the CM4A uncertainties, the accuracy of CV1S relative to other algorithms, and the sources of the differences between CV1S and CALIOP.

2. Materials and Methods

The instantaneous CV1S cloud properties are matched with CALIOP cloud profiles to evaluate the CV1S cloud and phase amounts, cloud-top height, and cirrus cloud optical depth, and ice water path for the same time period used by [19] for the CM4-CALIPSO comparisons: January, April, July, and October (JAJO), 2015–2016.

2.1. Data

2.1.1. CERES VIIRS Edition 1 Cloud Products

The CERES cloud products are based on analyses of multiple VIIRS channel radiances that are sampled from the original VIIRS images. The VIIRS channel resolutions are either 750 m or 375 m and CERES uses radiances from both types of channels. To reduce processing and storage, CERES samples the 750 m (375 m) VIIRS data at every fourth (eighth) pixel and every other (fourth) scan line to yield an effective nominal resolution of 6 km × 1.5 km. Thus, along the track, each pixel represents a distance of 1.5 km.

Each pixel is classified as clear or cloudy. If the latter, cloud phase is determined as liquid, ice, or no retrieval. If liquid or ice, the cloud effective temperature CET, the cloud optical depth COD, and the particle effective radius CER are retrieved. These three quantities are employed to estimate the cloud effective and top heights, CEH and CTH, respectively, along with the cloud physical thickness and base height. All clouds in a given pixel are assumed to be single-layered in the retrieval. An experimental technique was also used to detect multilayered (ice over water) clouds and their properties. While the multilayer results are included in the standard CERES Single Satellite Footprint product, they are experimental and are not examined here. The algorithms and processes used to retrieve cloud properties from the VIIRS data are given in [4].

2.1.2. CALIPSO Data

The CALIOP Release 4.20 Vertical Feature Mask (VFM) and Cloud Layers products [22,23] are used to assess cloud fraction, CTH, and cloud top thermodynamic phase [24], and to identify cloud layering. The Cloud Layers product contains cloud information determined for five horizontal averaging (HA) distances of 1/3, 1, 5, 20, and 80 km. The lower resolution products comprise averages of the 1/3 km lidar backscatter intensity profiles over lengthening horizontal distances to detect very faint, very low optical depth clouds. The CALIOP cloud optical depth [25], COD_C, minimum decreases with increasing HA distance. Because the 5 km Cloud Layers product lacks information for clouds detected at HA < 5 km, the 1/3- and 1 km Cloud Layers products are also incorporated in the collocating process so that clouds from all HA are represented in the resulting matched dataset to facilitate analyses based on HA. As the matched dataset includes the 1/3- and 1 km data, the cloud fraction, CF_C, in a given 5 km CALIOP footprint can range from 0.0 to 1.0. The subscript, C, indicates a CALIOP quantity or retrieval. The cloud-layer top heights and bases detectable by CALIOP along with their retrieved optical depths are also included in the matched dataset. COD_C represents the cumulative optical depth of all layers having a retrieval. It is typically limited to values less than 5.0 because the lidar signal is usually attenuated at greater optical depths.

All sampled VIIRS pixels within 2.5 km of the center of each 5 km segment of the CALIPSO ground track were matched to that segment whenever it was within ±2.5 min of the VIIRS observations. Matches were also made for larger time intervals to examine matching and view angle dependencies. Because the CERES VIIRS data are sampled, this results in only 1–5 VIIRS pixels for a given CALIOP 5 km pixel. In this study, any cloud layers detected only with the 20 km or 80 km averaging are, as a default, treated as being cloud free, unless otherwise noted. In general, a cloud that is only detectable at the lowest CALIOP resolutions is less likely to be noticed in passive imager data. However, this reduced sensitivity to the thinnest clouds will be examined by comparing the VIIRS retrievals to lower resolution CALIOP properties.

The SNPP is in a Sun-synchronous orbit at an altitude of 834 km with a 1330 LT crossing time. Its orbit has a slightly longer period and greater altitude than both Aqua and CALIPSO, but the same nominal equatorial crossing time. Thus, it has a significant number of contemporaneous views with both Aqua MODIS and CALIPSO, but often with different viewing zenith angles (VZA). The CALIPSO view is 3° from nadir, while the MODIS views matched with CALIPSO are between about 0° and 18° off nadir, because Aqua and CALIPSO are in nearly the same orbits. The VIIRS VZA is partially a function of the scan time difference between the VIIRS and CALIOP views of the same location. For the 5 min time window considered here, VZA varies from 0° to 45°. Matches between Aqua MODIS and CALIOP data are typically within 3 min or less.

Because of the VZA differences between VIIRS and CALIOP, the VIIRS data are adjusted for parallax by adjusting the location of the pixels according to CV1S CTH retrievals. While this adjustment will be satisfactory for overcast, single-layered (SL) cloud systems, it cannot always account for the various parallax corrections required for different layers of clouds in a given CALIOP pixel. Additionally, cloud motion is not considered, so matched views can have increasingly different components as the matching window enlarges.

2.2. Comparison Process and Statistics

Cloud detection is evaluated following the methods of [20]. Two datasets are employed to examine cloud fraction. In one, the 50/50 case, a cloudy outcome is determined if either VIIRS cloud fraction, CF_V, or the CALIOP cloud fraction, CF_C, is greater than or equal to 0.50. The second case, 0/100, requires that both VIIRS and CALIOP be completely clear or cloudy. The 0/100 data comprise a subset of the 50/50 cases. A third dataset, which includes CALIOP data at all resolutions, is used to explore the sensitivity to optically thin clouds. While it is assumed that VIIRS is unable to detect very weakly scattering/absorbing clouds that are detected at the very low resolutions by CALIOP, it is important to know which clouds are not detected and how often very low optical-depth clouds are detected.

Uncertainties in cloud phase are examined by comparing the phase selections from CV1S to those from CALIOP. Only those matches for which CALIOP identified the presence of a single cloud phase in the column are used for the phase comparisons. Multiple cloud layers may be present, but it was required that all layers have the same phase or at least that one phase clearly predominates. The CALIOP algorithms sometimes result in indeterminate “unknown” phase, but that does not necessarily disqualify the data from the comparisons here if the presence of adjacent liquid or ice layers made it possible to infer a liquid or ice cloud phase.

Both cloud fraction and phase evaluations are based on parameters derived from the confusion matrix illustrated in Table 1. The true and false negatives outcomes are Tn and Fn, respectively, while their sum, n, is the total number of CV1S negatives. For the phase evaluation, negative indicates a liquid cloud, while negative refers to clear for the cloud fraction assessment. Fp and Tp indicate false and true positive outcomes, and p is the total number of CV1S positive results.

Two approaches are employed here to interpret the confusion matrix. To facilitate comparison with results from various studies, the method described by [19,20] is presented in Section 3. It is designated the fraction correct method, FCM. For comparison with more recent analyses using VIIRS data, the technique of [26], denoted as the balanced accuracy method BAM, is presented in Section 4. The approaches have some overlap, but employ different terminology.

The relevant FCM parameters are defined as follows.

Fraction Correct: FC = (Tp + Tn)/(n + p).

Hit Rate: HR = Tp/(Fn + Tp).

False Alarm Rate: FAR = Fp/p.

False Alarm Rate, water: FARw = Fn/n.

False Alarm Rate, ice: FARi = Fp/p.

Bias: B = (Fp − Fn)/(n + p).

Hansen-Kuiper Skill Score: HKSS = ((Tp × Tn) − (Fp × Fn))/((Tp + Fn) × (Fp + Tn)).

The same matrix variables are used to compute the following BAM parameters.

True positive rate: TPR = Tp/(Tp + Fn).

True negative rate: TNR = Tn/(Tn + Fp).

Accuracy: ACC = (Tp + Tn)/(n + p).

Balanced accuracy: BACC = (TPR +TNR)/2.

Note that ACC is the same as FC, also referred to as probability of detection (e.g., [20]) and hit rate (e.g., [19]). Here, hit rate refers to the fraction of clouds detected by CALIOP that are identified by the passive sensor. Other statistics computed include the number of pixels N, the fraction of cloudy pixels used, and the Hanssen-Kuipers’ skill score (HKSS) defined in [27]. Values of HKSS range from −1 to 1, where 1 signifies that all pixels were correctly identified and scores less than or equal to zero indicate no skill. In the context of cloud detection validation, a positive (negative) bias indicates that CV1S overestimates (underestimates) cloudiness. Cloud phase fractions and biases are also computed. In the context of cloud phase validation, a positive (negative) bias indicates that CV1S identified too many cloudy pixels as ice (liquid) phase. For the other parameters, such as cloud-top height, differences for each pixel are computed and averaged for a given category. The standard deviations of the differences (SDD) are also determined.

3. Results

3.1. Cloud Amount

Figure 1 plots the gridded (5° × 5°) global distributions of the CV1S and CALIOP cloud fractions from all of the matched data along with their differences. During the day, CF_V (Figure 1a) and CF_C (Figure 1b) appear very similar, except for areas over land where the CF_C map is redder than that from VIIRS. This difference is more easily seen in Figure 1c where the differences over some regions, such as northern South America and eastern Africa reach nearly −0.1. The absolute differences are smallest over many tropical marine areas and maritime midlatitude storm tracks. At night, the distributions of CF_V (Figure 1d) and CF_C (Figure 1e) have the same patterns, but it is evident that CF_V < CF_C in many areas. This is highlighted by the differences in Figure 1f, where large areas of deeper blue occur over the tropical and subtropical oceans as well as the polar regions. Over desert areas, CF_V tends to overestimate CF_C. The smallest differences in magnitude are found over parts of Siberia and the Southern Ocean.

These differences are further summarized in the zonal averages (Figure 2). Figure 2a reveals that CF_V is only slightly less than CF_C for HA ≤ 5 km everywhere during the day, except for the northern polar regions where the difference approaches −0.05. On average, the daytime difference is −0.030, but varies from −0.01 to −0.08 over the HA range. At night (Figure 2b), the discrepancy between CF_V and CF_C is readily seen for all HA’s with mean differences between −0.02 and −0.116 for HA = 1 km and HA ≤ 80 km, respectively. The VIIRS and CALIOP averages diverge most over the polar regions and the southern tropics.

Table 2 summarizes the cloud mask comparisons to CALIOP (HA ≤ 5 km) for the 0/100 and 50/50 cases (in parentheses). In computing the global values, all pixels are given equal weight. However, because of the more frequent sampling of the polar regions relative to their area, the influence of the polar areas on the global means is greater than warranted. Thus, more representative averages are computed by using areal weighting of the polar and nonpolar means. For example, in Table 2, the polar regions account for 28–37% of the pixel-weighted global means, while representing only 13% of the global surface area. The areal weighting produces improved global averages as might be expected since the polar statistics are uniformly worse than their nonpolar counterparts.

Over all areas, the fraction correct and hit rate exceed 0.9 for the daytime 0/100 cases (Table 2). Using cloud detection biases B as rough estimates for biases of cloud fraction, the CV1S cloud fraction, CF_V, is less than its CALIOP counterpart, CF_C, over polar regions by 0.024, while it is only half of that over nonpolar surfaces. Globally, daytime CF_V is 0.013 less than CF_C. These mean differences are not the same as those in Figure 2 because the 50/50 cases apparently have more cloud cover for some of the “clear” scenes than clear portions have for the “cloudy” cases. For example, the average cloud cover in the partly cloudy CALIOP cases deemed clear for the 50/50 cases (CF_C < 0.5) could be 30%, but for the mostly cloudy cases (CF_C > 0.5), it could be 95%, an imbalance of 25% that would yield a mean cloud amount for the binary 50/50 cases that is less than the true cloud amount. Clear pixels are infrequently mistaken as cloudy during the day, as FAR ≤ 0.03 overall and equal to 0.04 over polar regions. During daytime, all HKSS’s exceed 0.8 with a daytime global mean of 0.868 for the ~318 thousand 0/100 daytime pixels.

Introducing another 62,000 pixels to obtain the 50/50 dataset, shown in parentheses, degrades all of the daytime metrics. Globally, FC decreases by ~0.05 compared to the 0/100 cases, while the bias drops by 0.002. The HKSS values are all less than 0.8, but the hit rates remain above 90%. The relatively minor change in the bias and greater changes in the other parameters suggest that using the daytime 50/50 criterion affects the random error but not the mean error in the comparison. This is not surprising given the matching uncertainties and the inhomogeneous nature of the 50/50 pixels. The adjusted global averages for both the 0/100 and 50/50 cases are minimally different from the equal-pixel means during the daytime.

Overall, the nocturnal CF_V is 0.026 less than CF_C for the 0/100 matches. HR remains high for nonpolar areas but drops to 0.809 for polar regions. In absolute terms, the nocturnal biases are greater than their daytime counterparts, particularly over polar surfaces. All of the statistics are worse at night for both the 0/100 and 50/50 cases, especially for the polar pixels, which have corresponding FC values of 0.804 and 0.764, respectively. The smallest reduction in FC occurs over the ocean surfaces where the surface skin temperature is more accurately characterized than for other surfaces (not shown). All of the HKSS values are less than 0.8 at night, reaching as low as 0.551 (0.476) for 0/100 (50/50) polar areas. At night, the adjusted global averages for Table 2 represent a significant improvement over their unadjusted counterparts, mainly because the extremely different polar values have reduced influence on the global means. Unlike the daytime case, the inclusion of the non-overcast pixels at night worsens the global bias, as well as all of the other parameters.

The results presented above are for HA ≤ 5 km. At lower HA distances, the CALIOP picks up fewer optically thin clouds increasing all of the FCM parameters, while the opposite is true for longer HA lengths. Figure 3 plots values for several of the variables in Table 2 for HA distances ranging from 1 km to 80 km. The CALIOP resolution increases from left right. The parameters are computed separately for snow and ice free (SIF) land and ocean and for snow and ice-covered surfaces (SIC). FC (Figure 3a) and the bias (Figure 3b), are greatest for daytime ocean, and least for clouds over snow and ice at night. They also decrease with decreasing resolution. For daytime ocean, FC drops from ~0.96 at HA ≤ 1 km to ~0.925 for HA ≤ 80 km, and over nocturnal SIC surfaces it decreases from 0.78 to 0.73. The bias falls from +0.01 to −0.05 for daytime ocean. The bias drop is more precipitous for nighttime SIC, ranging from about −0.01 to −0.16, presumably because the very thin clouds detected at longer HA distances are either more frequent or harder to detect using only thermal channels. Conversely, the FAR values are greatest for nighttime SIC and least for daytime ocean surfaces (Figure 3c). Like the other two parameters, FAR rises as the HA lengths decrease. The hit rate variations with CALIOP resolution (Figure 3d) are similar to their FC counterparts but have generally higher values.

3.2. Cloud Phase

Cloud phase is validated by comparing the phase selections from CV1S to those from CALIOP in various ways. Using all of the collocated data, Figure 4 shows the mean zonal water and ice cloud amounts from CV1S and CALIOP for day and night. Except for areas south of 65°S and between 15°S and 30°S, CV1S tends to identify more pixels as liquid during the day than CALIOP (Figure 4a). On average, CF_VW is 0.035 greater than CF_CW for HA ≤ 5 km. For ice clouds, CF_VI is less than CF_CI everywhere except south of 65°S (Figure 4b) with a global difference of −0.073 for HA ≤ 5 km. Although the overall daytime averages roughly equal those from CM4A, there are some differences in certain zones (see Figure 1 of [19]). At night, the average CF_VW is very close to CF_CW between 15°S and 45°N, but is substantially lower than CF_CW in all other zones (Figure 4c), more so than for CM4A (Figure 2, [19]). The latter is 0.047 less than CF_CW, compared to 0.072 for CV1S. The reason for this drop in CF_W for CV1S, discussed by [4], is mainly due a change in the nocturnal cloud detection algorithm, which was designed to minimize false cirrus detection. Instead, it missed low cumulus clouds. Better agreement is found for ice cloud fraction (Figure 4d). CF_VI is slightly less than or relatively close to CF_CI between 45°S and 45°N. Poleward of 45°, CV1S finds more ice clouds than CALIOP for HA ≤ 5 km. This is particularly true over the austral and boreal ice sheets (not shown). Globally, CF_VI exceeds CF_CI by 0.005 at night.

To explore the phase differences in more detail, the FCM parameters are summarized in Table 3 for all overcast (0/100 dataset) matches having a single cloud phase throughout the observed profile. Profiles having multiple cloud layers were included, but it was required that all detected layers have the same phase or at least a predominant phase. For example, some cloud layers, particularly lower layers, may contain some amount of “unknown” phase, but the phase can be inferred from adjacent layers or profiles. Over non-polar snow/ice-free land and ocean, FC is 0.888 and 0.954, respectively, during the day. Over snow-covered surfaces, FC is similar to that over land, though the ice false alarm rate (FAR) is greater, while the water FAR is larger for snow-free land. Globally, daytime ice and water FARs differ by only 0.001, however, the water cloud amount is nearly double the ice amount, so the amount of false liquid clouds will almost be twice that for ice. The impact of the phase errors is evident in the zonal means plotted in Figure 4. For example, the daytime biases in general have negative values, indicating an overabundance of liquid cloud identifications.

At night, FC values are about 0.05 lower than for day, and biases and ice FARs are significantly higher. Figure 4d indicates that ice cloud fraction is overestimated in the polar regions at night, and Table 3 shows that the polar regions indeed have the largest biases and ice FARs. CV1S ice cloud amount is slightly less than CALIPSO in the tropics, and nearly equal in the midlatitudes. CV1S water cloud fraction is underestimated at night in the midlatitudes and polar regions, but equal to that of CALIPSO in the tropics (Figure 4c). Overall, the CV1S liquid amount is 0.062 less than CALIPSO for HA ≤ 5 km. This underestimation of liquid cloud fraction results in lower water FAR for night compared to day (Table 3).

To determine the sources of the differences, the classifications of clear scenes and ice and liquid clouds were determined for a variety of conditions defined by the CALIPSO profiles. The results are shown in Figure 5. Each bar represents the number of pixels having a given CALIPSO classification, while each color in the bar represents a CV1S classification as a fraction of all pixels in the category. For example, the CALIPSO clear classification during the day (Figure 5a) had 95,844 pixels. Of those, CV1S classified 94% as clear, 3% as water cloud and 3% as ice cloud. At night (Figure 5b), the fraction of correct clear pixels drops to 85%, while the false ice cloud amount is twice the false liquid amount. Considering all of the blue portions of the last five bars of Figure 5a reveals that the bulk (69%) of the ice-cloud underestimate relative to CALIPSO during the day is due to non-opaque cirrus over liquid cloud scenes, while at night, the ice cloud overestimate is due to misclassification of SL water clouds and clear pixels. As a greater part of the overestimate occurs in the polar regions at night, the result is not surprising. Most of the polar water clouds are supercooled and it is difficult to distinguish clear from cloudy scenes in low thermal contrast situations. These results are similar to those for CM4A.

3.3. Cloud Top Height

Cloud-top comparisons were performed for two sets of liquid and ice clouds: (1) single-layer clouds as indicated by asterisks in Figure 5, and (2) all liquid and all ice clouds as indicated by the brackets. Furthermore, the CV1S and CALIOP cloud-top phases must be the same unless otherwise noted. The CALIOP cloud-top, CTH_C, is taken as the top of the highest layer in the atmospheric column.

3.3.1. Liquid Clouds

Figure 6 plots matched CV1S and CALIPSO CTH values for overcast 0/100 liquid water clouds. The CV1S heights, CTH_V, for all matches having liquid cloud tops (categories within the blue bracket in Figure 5) are too low, on average, by 0.35 and 0.10 km compared to CTH_C during the day (Figure 6a) and night (Figure 6b), respectively. For SLP liquid-water clouds, the respective biases are 0.02 km (Figure 6c) and 0.17 km (Figure 6d). The correlation coefficients (R) are greater and the SDDs are much lower for the SLP clouds than for all liquid clouds.

The differences, CTH_V − CTH_C, for liquid clouds are broken down by surface type in Table 4 for the 0/100 cases. During the day over snow/ice (SIC) regions, the bias for SLP clouds is 0.09 ± 0.87 km. The SLP bias and SDD are smallest for snow-free (SIF) ocean regions, while SDD is greatest over snow-free SIF land surfaces. For all liquid clouds, the magnitudes of the bias and SDD are least for SIC areas, but the correlation is also lowest. Since SIF water surfaces predominate, the global statistics are closest to those for ocean. At night, CTH_V overestimates CTH_C by 0.16 km and 0.18 km over SIF ocean and land, respectively, for SLP clouds. The bias is greatest and correlation is smallest over SIC areas. The nocturnal SLP SDD values are similar to their daytime counterparts. If all liquid clouds are considered, the absolute biases at night are less than the daytime biases with global averages of −0.35 km and −0.10 km for day and night, respectively. Despite having the lowest correlation, the mean height biases and SDDs are lowest over SIC regions. The changes between SLP and all liquid CTHs are somewhat greater during the day than at night. Overall, the SLP and all-liquid cloud biases and SDDs in Table 4 are similar to those found for CM4A (Yost et al. 2021) during the day and at night.

Figure 7 maps the regional mean differences, CTH_V − CTH_C, for SLP and the liquid dominant clouds. Despite using a 30 min window (±15 min) for collocation, the results are somewhat noisy but sufficient to discern some patterns. Except for some high-latitude regions, the daytime SLP differences (Figure 7a) are mostly between −0.5 km and 0.5 km. At night (Figure 7c), areas with differences exceeding 0.5 km are much more numerous, particularly south of 30°S. When all types of water clouds are included, there is little change over the cores of the subtropical marine stratocumulus areas during the day (Figure 7b), but elsewhere, the differences are mainly negative resulting in the mean difference of −0.35 km listed in Table 4. A similar change is seen in the nocturnal differences (Figure 7d), which include many more negative values and fewer means above 0.5 km. The most negative areas are primarily found in the tropics, while the most persistent positive areas at night include the Antarctic ice shelf and the Arctic Ocean north of Europe.

3.3.2. Ice Clouds

Comparisons are performed for non-opaque and opaque cloud columns separately. The former includes all clouds having a dominant ice phase top for which a return signal from the surface is observed in the CALIOP profile. This category can include single and multi-layer thin cirrus clouds and thin cirrus over thin low, liquid clouds. If a surface return is not observed, then the cloud is designated an opaque cloud. In addition to opaque ice clouds, it can include thin cirrus over opaque water clouds.

Single-layer CTHs from CV1S are plotted against their CALIOP counterparts in Figure 8 for SL, 0/100 non-opaque and opaque ice clouds. The opaque clouds are well-correlated during the day (Figure 8a) and at night (Figure 8c) with R = 0.87 and 0.84, respectively. The corresponding biases are −0.81 and −0.48, km. For non-opaque clouds, the scatter and the SDDs are greater both day (Figure 8b) and night (Figure 8d), while the correlations are smaller than their opaque counterparts. The magnitudes of the biases noticeably increased also. Most of the points far below the line for CTH_C~10 km are from polar regions, while some are from the midlatitudes. Horizontal features are common for CTH_V > 12 km in most of the plots.

Table 5 lists the mean differences and associated statistics for non-opaque ice clouds. For SL clouds over the globe during the day, CV1S underestimates CTH_C by 1.22 km, a 0.7 km drop from the CM4A difference. Over SIC surfaces, the difference decreased by ~0.9 km relative to the CM4A-CALIPSO analysis of [19]. The SDDs overall are also smaller for the CV1S retrievals, 2.2 km down from 2.6 km. The reason for this improvement is not entirely clear, but is likely due to the use of the brightness temperature difference between the VIIRS 11 and 12 µm channels instead of the MODIS 13.4 µm channel to adjust ice cloud heights when the infrared estimate of cloud effective height is significantly greater than the value from the VISST retrieval. For all ice clouds and surfaces, CTH_V − CTH_C = −1.33 ± 2.23 km, a value only slightly greater in magnitude than that for SL clouds. This difference can be compared to the mean of −1.87 ± 2.54 km from CM4A. Note, the correlation coefficients vary from 0.46 to 0.59 for SL clouds when the different surfaces are considered separately. When combined, R increases to 0.69, presumably because the combined data provide a greater range of heights. A similar increase was found for all non-opaque cirrus clouds.

The magnitudes of the differences in the CTH_V non-opaque ice cloud top heights are smaller at night (Table 5, bottom), with an overall mean of −0.75 ± 2.10 km. This represents an improvement of 0.2 km in the bias over the CM4A results. If all ice clouds are considered, the differences are −0.78 ± 2.34 km, only slightly greater in magnitude than for the SL case. The CM4A difference reported in [19] is −1.31 ± 3.07 km. Unlike the daytime cases, the global average at night is more impacted by results over the SIC regions, presumably because of the dominating presence of ice clouds during the polar night. The biases for SL clouds over snow free surfaces are only −0.26 km compared to −1.4 km for SIC areas. An area-based averaging such as that in Table 2 would yield a difference of −0.4 km. A similar, but less dramatic effect is seen for the results for all ice clouds. It can be concluded that the CV1S top heights for non-opaque ice clouds are more accurate than their CM4A counterparts for both day and night. Again, the improvement relative to CM4A is likely due to the brightness temperature difference method used in CV1S to adjust cloud heights.

Opaque ice cloud-top height differences are listed in Table 6. For SL clouds, |CTH_V − CTH_C| < 1.00 km for all surfaces day and night. On average, it is −0.81 ± 1.28 km during the day and −0.48 ± 1.40 km at night. For all matches with ice top layers, the underestimates are larger, −1.24 ± 1.60 km and −0.89 ± 1.84 km during the day and night, respectively. The biases are greatest over ocean during the day and smallest at night. However, the correlations are greater for the opaque clouds than for their optically thin counterparts. Although the nocturnal magnitudes are less than the CM4A differences, the daytime biases are similar to those from CM4A (corrected CTH in Table IV of [19]). Surprisingly, the opaque ice CTH biases are not too different from those for the non-opaque clouds, except for daytime SLP clouds which see a reduction of ~0.1 km in the bias. The nocturnal, non-opaque height underestimates are actually smaller than the corresponding opaque cloud values at night for the SLP and all-layering cases. This is especially true for the SIF areas. Despite the relatively small differences in the biases, the non-opaque SDDs are significantly larger than their opaque counterparts for both SL-only and all ice cloud cases.

In the comparisons above, only matched data having the same cloud-top phase were considered. Thus, for the cases when ice clouds were mistakenly identified as liquid water clouds by CV1S, and vice versa, the retrieved cloud-top heights are likely subject to greater errors. To determine the uncertainty in CTH for all clouds regardless of phase accuracy, CTH_V − CTH_C was computed for all matched VIIRS and CALIPSO cloudy pixels. Histograms of the differences and the relevant statistics are given in Figure 9 for clouds over all surfaces. Both 50/50 and 0/100 cases are included for completeness. The latter is a subset of the former. Three probability distributions are shown in each graph: all clouds (gray), clouds with CTH_C ≤ 5 km (blue curve), and clouds with CTH_C > 5 km (red).

For the lower clouds, the average daytime differences are 0.13 ± 1.75 km and −0.01 ± 1.35 km for the 50/50 (Figure 9a) and 0/100 (Figure 9b) cases, respectively. The corresponding biases for the higher clouds are much greater in magnitude: −2.48 ± 3.33 km and −2.62 ± 3.33 km. These larger biases and SDDs are primarily due to the inclusion of clouds with mismatched phases and other constraints. For example, the results given for daytime all-layer clouds in Table 4, Table 5 and Table 6 account for only 65% of the pixel matches in Figure 9b. Despite the large standard deviations, the biases remain small for low clouds. The daytime CV1S results for 0/100 are slightly improved from their CM4A counterparts, while 50/50 results are marginally worse. Overall, the CV1S and CM4A CTH errors are quite consistent.

At night, the lower cloud heights are overestimated by 0.67 ± 2.04 km and 0.42 ± 1.53 km for the 50/50 (Figure 9c) and 0/100 (Figure 9d) constraints, respectively. The greater uncertainty at night is expected because of the reduced amount of information available compared to that during daytime. These can be compared to 0.55 ± 1.84 km and 0.26 ± 1.29 km for CM4A, indicating a decrease in accuracy at night. For the higher clouds, the corresponding values of CV1S ∆Z_t are −1.44 ± 3.21 km and −1.49 ± 3.15 km. They are considerably improved from their CM4A counterparts [19]. Overall, the CV1S nocturnal CTHs are more accurate than those from CM4A.

Because of similarities in the CV1S and CM4A optical depths [4], the VIIRS cloud thickness uncertainties are expected to be much the same as the those from [19], who used matched CERES Ed4 MODIS and CALIPSO and CloudSat products for the assessment. The ice-cloud base heights, however, may be biased lower than those from CM4A since the CV1S cloud-top heights are greater than their CM4A counterparts.

4. Discussion

4.1. VZA and Time Window Dependence

The results in Table 2 are much like those presented by [20] for the CM4A-CALIOP comparisons. For example, mean global FC during the day (night) is 0.943 (0.881) in Table 2 compared to 0.936 (0.881) for CM4A. The magnitudes of the CV1S bias and FARs are roughly the same as those for CM4, during the day, but at night, the CV1S parameters over polar regions are slightly worse than their CM4 counterparts. Part of that polar difference could be due to the lack of two critical channels in VIIRS complement, as noted in [4], but also to some differences in VZA. To better understand the effects of VZA on the comparisons, the dependencies of the metrics on matching times and VZA were computed since the VZA range is affected by the collocation time windows.

Figure 10 plots FC as a function of the time difference ∆t between the 0/100 CV1S and CALIOP data for two VZA ranges: 0–30° (black line) and 30–80° (magenta line). Also plotted are the daytime sampling densities as functions of ∆t and VZA (right column). They are very similar to their nighttime counterparts (not shown). Dashed vertical lines denote ∆t = ±15 min. During the day (left column), FC reaches 0.9 or greater for all surfaces for |∆t| < 2.5 min, if only VZAs between 0° and 30° are considered (black line). Except for a few outlying, poorly sampled time bins, FC decreases gradually with increasing |∆t|. Over oceans (Figure 10a), near-nadir FC peaks at ~0.97, while the off-nadir maximum is ~0.95 at ∆t = 0 min, when the sampling density is greatest between 10° and 45° (Figure 10c). Notably, FC is smaller for the off-nadir pixels at ∆t = 0 min. FC values over SIF land (Figure 10d) and SIC areas (Figure 10g) are roughly 0.93 and 0.90, respectively. At ∆t = 0 min, the sampling is concentrated mostly between 20° and 35° over SIF land (Figure 10f) and for VZA < 20° (Figure 10i). The sampling becomes more uniform with VZA between 0° and 45° when results for all surfaces are combined (Figure 10j). Overall, the greatest daytime average for both near-nadir and off-nadir matches coincides closely to ∆t = 0 min, as expected. Enlarging the collocation window will increase the number of samples but will reduce FC.

At night (middle column), the near-nadir FC means also peak around ∆t = 0 min. Over ocean (Figure 10b), the FC maximum of ~0.95 is straddled by values that drop to 0.90 around |∆t| = 25 min and tend to remain nearly steady at greater time differences. The mean land FC of ~0.89 (Figure 10f) around ∆t = 0 min decreases precipitously, reaching almost 0.65 at |∆t| = 60 min. The ∆t dependence is not quite as strong over SIC areas (Figure 10h), where the maximum of ~0.77 diminishes to ~0.70 at |∆t| = 60 min. The overall mean (Figure 10k) peaks at ~0.87 and falls off to 0.70, under the influence of the land and SIC areas. That maximum is still 0.01 less than that found for CM4A and likely arises from the causes reported by [4].

An interesting feature of the nocturnal matches in Figure 10 is the greater FC value over SIF and SIC surfaces off-nadir compared to those at VZA < 30°, regardless of |∆t|. Over oceans, there is minimal difference between the low and high angle values. Thin cirrus or low-level clouds that present very small thermal contrast with the underlying surface should appear colder at higher VZAs. Compared to those over ocean, land and SIC surface temperatures are relatively uncertain. Perhaps, a decreasing cloud temperature with VZA due to increasing optical path yields a greater temperature difference and hence better detectability over these surfaces. If only near-nadir cases were included in the Table 2 statistics, FC would be reduced overall by 0.02 or more for the land SIF and SIC cases.

In addition to revealing the impact of VZA on FC, the plots in Figure 10 clearly show the importance of minimizing the time window for matching the CALIOP and VIIRS data. For cloudy scenes, TPR also depends on |∆t|, with a maximum near zero minutes (not shown). While FC and TPR tend to decrease as |∆t| increases, the bias tends remain relatively constant with |∆t| for a window of 1 h or less during the day and decrease slightly at night (not shown). The bias is closer to 0.0 for the greater VZA ranges, an unsurprising result given the increase of CV1S cloud amounts with VZA [4].

While this in-depth analysis was performed only for the cloud mask, other parameter means were computed for a 30 min window corresponding to Table 3, Table 4, Table 5 and Table 6. The 30 min window, indicated by the vertical dashed lines in Figure 10f, increases the VZA range out to 55° and raises the number of samples by a factor of ~6 relative to the 5 min window. Negligible differences were found in the various cloud phase selection statistics for 0/100 data taken from 5 min and 30 min windows. The liquid cloud height biases for the 30 min matching are within 20–30 m of those in Table 4, while SDDs increased slightly. For both non-opaque and opaque ice cloud CTH, the 30 min biases are within ±100 m and the SDDs are essentially unchanged from those in Table 5 and Table 6. These relatively small differences suggest that the 5 min sampling is sufficient for this study.

4.2. Cloud Fraction: Comparison with Other Results

A variety of cloud detection algorithms have been developed for application to VIIRS data. Among others, these include the operational NOAA Enterprise Cloud Mask (ECM, [28]) and the MODIS-VIIRS Continuity Mask (MVCM, [16]), as well as newer machine-learning based methods such as the neural network cloud mask (NNCM, [26]) and the random forest cloud mask (RFCM, [29]). It is of interest to know how similar the CV1S accuracy metrics are to their counterparts from other VIIRS analysis algorithms. While one-to-one comparisons are possible in some instances, the criteria used in selecting CALIPSO and VIIRS matches are not always the same among the published studies. Therefore, in comparing the CV1S metrics with the other VIIRS evaluations it is important to recognize differences in selection criteria.

The criteria used to train and evaluate the RFCM are too different from those used here, so no attempt is made here to compare with those results for cloud detection. However, the NNCM, ECM, and MVCM results were compared by [26] to 1 km CALIOP data taken within 2.5 min using two groupings: unfiltered, that is, all of the matched data, and filtered, which includes only those collocated data for which five consecutive 1 km CALIOP profiles agree. Presumably, that agreement means all five pixels are either clear or cloudy, a condition similar to the 0/100 case. The unfiltered data would nominally correspond to the 50/50 case.

As a means for comparing with the results of [26], the same parameters, BACC, TPR, and TNR, were computed for the same categories used in [26]. These are presented in Table 7 for day and night using the 0/100 data (top) and the 50/50 data (bottom), which, respectively, correspond to Tables 5 and 6 in [26]. The black numbers denote those CV1S parameters that exceed their ECM and MVCM counterparts in the tables of [26], whereas the bold numbers indicate the CV1S value is greater than the NNCM result. Red and light blue numbers denote that ECM and MVCM values, respectively, exceed their CV1S counterparts. Purple signifies that both ECM and MVCM are greater than CV1S.

In all cases but one, the NNCM values exceed their CV1S counterparts. The mean global BACC differences between CV1S and NNCM for 0/100 are −0.015 and −0.056 for day and night, respectively. For the 50/50 cases, the corresponding differences are −0.018 and −0.055. Those differences are similar in magnitude to the mean cloud fraction differences between CV1S and CALIPSO in Figure 3. The mean global day/night BACC differences between CV1S and ECM are 0.015/0.067 for 0/100 data and 0.008/0.016 for 50/50 data. For CV1S and MVCM, those same differences are 0.043/0.002 and 0.026/−0.006. Thus, except for nighttime 50/50 data, the CV1S BACC values all fall between the NNCM and the operational algorithms. The lower BAM metrics for CV1S relative to the NNCM are no surprise given that the latter is a machine learning code trained with collocated CALIOP data using 16 VIIRS channels from nine pixels centered on each VIIRS pixel of interest.

Figure 11 plots the variation of TPR with the CALIPSO 5 km cloud optical depths for all HA ≤ 1 km, 0/100 cases having a valid optical depth for the entire column regardless of phase. Globally (Figure 11a), TPR rises from ~0.86 for clouds having COD_C < 0.2 up ~0.94 for COD_C = 1 during the daytime (solid curve). TPR drops substantially at night (dashed) with a mean of 0.65 for COD_C < 0.2 and only reaches 0.86 at COD_C = 1. It then increases to 0.90 at COD_C = 2, then decreases and rises again for increasing COD_C. The reason for the nocturnal dip at COD_C = 2.3 is related to the reduction of the lidar ratio within the CALIOP extinction retrieval algorithm in order to achieve a convergent solution [25]. Over nonpolar areas (Figure 11b), TPR moves from 0.87 for COD_C < 0.2 during the day to nearly 0.97 at COD_C = 1, a noticeable improvement over the global daytime case. Eliminating the polar regions yields a more dramatic change in TPR for the nocturnal cases. It jumps to 0.78 for COD_C < 0.2 and 0.96 for COD_C = 1, remaining above 0.93 for greater optical depths. Although not shown, TPR exceeds 0.97 at night for CODc > 5.

The plots in Figure 11a can be compared to the curves for the filtered results in Figure 4c,d of [26]. During the day, the CV1S TPR values are below the NNCM values for all optical depths, but are above those from the MVCM and ECM. The MVCM TPR equal to 0.88 for COD_C < 0.2 represents the sole exception to that generalization. At night, the TPR from CV1S exceeds that from the MVCM for all CODc values. It is slightly greater than or equal to that from the ECM for COD_C < 2, but falls below the ECM curve for larger COD_C values. The NNCM TPRs are higher than the CV1S values for all optical depths. The ordering of the TPR curves from the various methods further supports the results in Table 7 and demonstrates the high quality of the CERES VIIRS cloud mask relative to other operational techniques. It also illuminates areas for improvement, particularly over the polar regions at night for clouds having optical depths below 5.

4.3. Cloud Phase

From Figure 5, it is clear that misclassification of the cloud-top phase is a significant problem for multi-layer (ML) systems with nonopaque ice clouds above a low liquid cloud. That cloud type accounts for roughly 60% and 37% of the total phase misclassifications for day and night, respectively (Figure 5). As seen in [19], the probability of classifying the system as an ice cloud rises as the optical depth of the upper cirrus cloud, COD_u, increases. Figure 12 plots the CV1S classification of CALIOP pixels that include non-opaque or transparent cirrus over a lower water cloud (ML clouds). Unlike the results in Figure 5, the plots are based on data collocated in a 30 min window and use all horizontal averaging results, i.e., HA ≤ 80 km), because some of these cases are detected as clouds in the CV1S cloud mask. During the daytime (Figure 12a), roughly 14% of these ML cases having COD_u ≤ 0.1 are classified as clear and 14% are identified as ice clouds. The remainder are liquid water clouds according to CV1S. As COD_u increases, the clear and water cloud fractions decrease, while the ice cloud fraction rises. The breakeven point occurs at COD_u = 0.93. Of the 17% remaining matched pixels having COD_u > 0.93, the fraction of pixels classified as ice does not exceed 75%. At night (Figure 12b), the breakeven point drops to COD_u = 0.16. For CM4A, the daytime break-even COD_u is ~0.95 and at night it is ~0.20 for CM4A [19]. The day–night discrepancy in the breakeven point is the result of the spectral channel complement differences used for the day and night algorithms. During the day, the optical depth of the lower cloud significantly contributes to the total COD since it is based on the column visible reflectance. The resulting adjustment of the thermal channel brightness temperature to account for the COD in the retrieval process is diminished when COD > COD_u. In these cases, CET > 233 K and more likely will be classified as liquid. At night, COD is based on the thermal contrast between the surface and the cloud temperature. For these ML systems, the surface and lower cloud temperatures are small relative to the cirrus temperature, so there a much smaller impact from the low cloud on the retrieval of CET. Hence, significantly more ML clouds are classified as water during the day than at night.

Another difficult situation for determining cloud phase is the presence of supercooled water clouds (SWC), those clouds having temperatures between 233 K and 273 K. Determining the presence of SWC is important for cloud modeling and for various applications. Figure 13 shows the phase selection percentages of 0/100 CV1S pixels that had a uniform phase and have 233 K < CET < 273 K for three phase classifications from CALIOP pixels collocated within 10 min of the VIIRS pixels. The three CALIOP categories are liquid, ice, and mixed. The last class includes matched pixels having both phases detected somewhere in the column. The other two require horizontal and vertical homogeneity in the phase classification. During the day (Figure 13a) and at night (Figure 13b), the CV1S detects 96% and 75%, respectively, of the SWC that do not occur in mixed phase conditions. Of the pure ice clouds having CET in the supercooled temperature range, 89% are correctly classified as ice by the CVS1 regardless of time of day. The majority of the mixed phase clouds are thin cirrus over liquid clouds and roughly 2/3 of them are classified as liquid during the day and as ice during the night. This day–night phase difference is not surprising based on the previous discussion. The results suggest that some improvement is needed for the nocturnal phase selection algorithm in the SWC temperature range.

With a few caveats, these results can be compared those from [30], who assessed the cloud phase from their algorithm using the 2B-CLDCLASS-LIDAR RO5 product [31], which combines CALIOP and Cloudsat data to define the profile of cloud hydrometeors and their phase. Using only matched pixels having the same phase as the surrounding 3 × 3 array of pixels taken within 10 min, ref. [30] found that they correctly identified 91% of the pure SWC clouds and 85% of the pure ice clouds during the day. Additionally, ref. [30] found that their daytime cloud phase algorithm outperformed the corresponding product from the SNPP VIIRS climate data record continuity cloud properties (CLDPROP) algorithms [32] in detecting SWC. Assuming that the CALIOP and CLDCLASS provide similar cloud phase information and the matching criteria are not too dissimilar, the CV1S SWC detection appears to be more successful than its counterparts from [30,32].

The machine-learning based RFCM was also used to determine cloud phase and validated using CALIOP data from 2017 [29]. Defining ice-phase outcomes as positive events and liquid-phase outcomes as negative events, they evaluated the RFCM and VIIRS CLDPROP operational cloud phase products by computing TPR and FPR for each product for a variety of surface conditions. For comparison, a similar analysis was performed using the CV1S 0/100 SLP data. It should be noted that the comparison is not straightforward due to constraints placed on the data by cloud mask, phase outcomes, and data sampling. For example, the results here are not constrained by the outcomes of the RFCM and CLDPROP cloud mask and phase algorithms, or their use of aerosol-free pixels. However, the cloudy pixels used to evaluate phase in the analyses of [29] used ~27.5% of the available collocated pixels compared to ~26% here. For daytime conditions over all surface types, CV1S yields TPR values of 0.926 and 0.915 for 1 km and 5 km averaged CALIOP data, respectively. The corresponding FPR values are 0.033 and 0.033, respectively. These values are quite similar to the results show in Figure 8 of [29], but it is difficult to precisely compare the values to those plotted in the aforementioned figure. For nighttime conditions over all surface types CV1S has TPR and FPR of 0.967 and 0.161, respectively. This TPR is similar to the RFCM TPR (Figure 9 of [29]). While the CV1S FPR is higher than for the RFCM, it is very similar to that of CLDPROP.

4.4. Cloud Top Heights

Before discussing the CTH results, it is instructive to review the retrieval process. The determination of CTH is relatively straightforward. First, COD, phase, and CER are estimated together. Based on those parameters, CET is essentially computed by solving the simplified form of Equation (12) of [33], which includes all of the atmospheric corrections. The observed 11 µm brightness temperature is T₁₁; T_s and ε_s are the surface skin temperature and emissivity, respectively; and ε_c (COD, CER, phase) is the cloud emissivity. Once CET is determined, it is matched with the lowest altitude, Z_min, in the provided temperature profile corresponding to CET, yielding the cloud effective height, CEH. CTH and CTT are then estimated from CEH by applying one of several empirical corrections that depend on cloud phase, temperature, and COD [4], otherwise it is capped at the tropopause.

B(T₁₁) = ε_c B(CET) − (1 − ε_c) ε_s B(T_s),

(1)

The profile, T(Z), is constructed from a numerical weather reanalysis (see [4]) and a low-altitude lapse rate based on matched SL CALIOP and MODIS pixels [34]. The lapse rate is anchored to T_s and is used exclusively for all altitudes below the level, Z₁, corresponding to a pressure between 750 and 827 hPa depending on surface type and latitude. Between Z₁ and the level, Z₂, corresponding to a pressure between 650 hPa and 750 hPa, T(Z) is a blend of the lapse rate and reanalysis profile. Above Z₂, T(Z) is provided by the reanalysis profile, which is from version 5.4 of the Global Modeling Assimilation Office (GMAO) Global Earth Observing System Model Version 5.41 (GMAO-G541) [35]. For most clouds, CTT is constrained to be equal to or less than the tropopause temperature. Hence, CTH is only allowed to be above the tropopause for overshooting convective tops, and other exceptions noted in [36].

4.4.1. Liquid Water Cloud Heights

Because the results in Table 3 are so much like those from the earlier CM4A comparisons [19], it can be concluded that the biases are systemic and not due to any peculiarities of the VIIRS data. Focusing on SIF ocean cases, it is possible to eliminate a few of the variables that could be major contributors to the biases. Sea surface temperature and emissivity are known fairly accurately, so that ε_c is the most likely source of uncertainty for determining CET in Equation (1). The remaining source of error is in the determination of CEH from CET using T(Z).

Assuming the phase is correct, ε_c is primarily dependent on COD, which is estimated using the visible channel during the day and infrared channels at night. The latter limits the range of retrieved values to roughly 0–8, but more practically to a much smaller range because the uncertainties grow substantially at the higher end of the COD interval. To illustrate the day–night differences, Figure 14 plots histograms of CV1S liquid water COD_V separately for 2012 between 30°S and 60°S. During daylight over ocean (Figure 14a), COD_V occurs most frequently, ~23% of the time, between 8 and 16. Between COD_V = 2 and 4, the maritime frequency is ~15% compared to ~20% between 4 and 8. At night (Figure 14b), for 4 ≤ COD_V < 8, the frequency is ~31% and ~25% for 2 ≤ COD_V < 4. Both are much greater than their daytime counterparts. For COD_V < 2, the frequencies are nearly the same for day and night. The frequencies are significantly reduced at night for COD_V > 8. These results, much like those found globally, indicate that clouds that are optically black in the infrared are being interpreted as gray clouds with emissivities less than unity. This would result in an underestimate of CET and an overestimate of CEH and CTH for those clouds at night.

While the CODs might explain the day–night difference in the CTH biases, the lapse rates employed in assembling T(Z) used CM4A CODs having almost the same characteristics as those from CV1S [34]. Thus, the tendency for over-correcting CET at night should have been taken into account in the empirical lapse rates. The lapse rates were constructed from cloudy, liquid phase CM4A pixels matched with 5 km HA pixels from CALIOP. To increase the sample size, only two out of three subpixels had to be cloudy and be single-layered to qualify as overcast. Thus, the definition for overcast SL pixels used here and for the lapse rate development are slightly different, possibly introducing some discrepancies. It is not clear that this would only be a problem at night. The lapse rate development also used only those MODIS pixels classified as liquid water clouds. At night, 21% of liquid clouds were misclassified as ice, compared to 6% during the day. The day–night phase error difference is even starker for SWC clouds (Figure 13). Again, the data used for the lapse rate construction had approximately the same day–night difference in phase selection, so the day–night bias discrepancy is not explained.

To further explore the biases in liquid CTH, the mean 0/100 CTH differences, ∆Z_T, are plotted in Figure 15 as functions of the 11–12 µm channel brightness temperature differences, BTD, and COD. During the day (Figure 15a), mean ∆Z_T is close to zero or positive for 90% of the data, corresponding to BTD < 1.5 K. As BTD increases, the differences become more negative, reaching approximately −1 km at BTD = 8 K. Those negative values occur primarily for COD < 2 (Figure 15b), suggesting COD is overestimated for those cases. Additionally, they are likely to be midlevel clouds with small droplet sizes, because BTD increases with the cloud-surface temperature contrast and decreasing droplet size (e.g., [37]. Except for those thin clouds, ∆Z_T is near zero or positive for COD > 2. At night (Figure 15c), the mean bias is positive for all BTDs and CODs.

This consistency across the board suggests that the lapse rate method should be revisited as it apparently cannot account for all conditions. The basic assumption behind its use is that there is a connection between the surface and the low cloud height or the inversion that caps the cloud top. That may work well for marine stratus under a strong high pressure, but is less viable when the surface and clouds may be decoupled, for example in baroclinic systems (e.g., [38]) or where the boundary layer inversion is weak or nonexistent in the stratocumulus transition regions. Those are the areas where the nocturnal bias is strongest in Figure 7c. Thus, a different approach may be needed to eliminate the biases in those regions.

4.4.2. Ice Cloud Heights

Matching CET with T(Z) is a process fraught with error sources for all types of clouds. The horizontal features in the scatterplots of Figure 8 appear to cause a significant portion of the underestimates of ice cloud top heights. Few CTH_V values exceed 16 km, while many of the CALIOP measurements yield CTH_C values exceeding 17 km. When compared with spaceborne lidar measurements, CERES Edition-2 cloud effective heights were underestimated by up to 5 km in the tropics, but were around 2 km lower at other latitudes [39]. Since the highest clouds are found in the tropics, it is possible that much of the bias is found only in the tropics.

To examine that idea further, the nighttime 0/100 single-layer ice CTH values from CV1S are plotted in Figure 16 against those from CALIOP separately for mid-latitude and tropical areas. Data matched to within ±15 min were used to increase the sample sizes. The horizontal features seen in Figure 8 dominate the plots for both opaque (Figure 16a) and non-opaque (Figure 16c) clouds in the tropics. Some evidence of the features remains in the midlatitude scatterplots, but CTH_V and CTH_C are mostly balanced at the upper end of the range. Hence, the midlatitude CTH_V biases are −0.6 km for opaque clouds (Figure 16b) and −0.1 km for non-opaque clouds (Figure 16d), dramatically lower than their tropical counterparts. The SDD values are much higher for tropical clouds, while R is significantly greater for mid-latitude clouds. Similar differences between tropical and midlatitude cloud heights were also found using daytime data (not shown).

The most obvious culprit for causing the horizontal features, the tropopause height limitation to CTH noted earlier, is applied at all latitudes. However, the determination of the tropopause altitude, Z_p, in tropical regions is more difficult because of the existence of the tropical tropopause layer (TTL), which is roughly between 14 and 18.5 km (150 and 70 hPa) according to [40]. Figure 17 illustrates the TTL with example soundings from a station on the Atlantic coast of Brazil, Fernando de Noronha at 3.85°S, 32.41°W (Figure 17a). The radiosonde profile (solid red curve) taken at 1200 UTC, 15 July 2022 reveals the coldest temperature is found near 17.1 km among several other relative minima in T(Z) within the approximate TTL altitude range (green lines in Figure 17b). The G541 profile (dashed blue line), the product of assimilation and smoothing at a much lower vertical resolution, shows considerably less detail and misses the coldest temperatures. As indicated by the narrow horizontal dashed line in Figure 17, Z_p from G541 is 15.8 km, well below Z_min in the radiosonde profile. It corresponds to a pressure of ~115 HPa. Other relevant temperature discrepancies between the numerical model and radiosonde profiles are evident between 5 and 8 km.

The value of Z_p in this example is in the upper range of the tropical tropopause altitudes in the G541 data. At 12 UTC, 15 July 2022, the reanalysis produces tropopause pressures that vary from around 200 hPa to 110 hPa. A small cloudless area over the subtropical high near California yields a pressure of ~85 hPa. Thus, the range of tropical tropopause G541 generally falls between ~12.4 and 16 km with occasional anomalies up to 17.5 km. That range corresponds well with the bulk of the horizontal features in Figure 15a,c. It is well known that thin clouds and deep convective tops are commonly found at different levels within the TTL [40]. So, it is not surprising that the CALIOP cloud-top heights reach 18.5 km in the tropics. The horizontal features in Figure 16a are clearly due to the low altitudes of Z_p in the reanalysis and the constraint that limits the maximum CTH to Z_p in the CERES algorithms. Correcting for that shortcoming in the input data and algorithm, a task for future research, should reduce the biases in the ice CTHs. The remaining biases can be addressed by using an improved ice crystal scattering model (e.g., [41]) and a revised CEH to CTH adjustment, as noted by [19].

4.4.3. All Cloud Heights

Except for Figure 9, the height comparisons focused on clouds that are matched in phase and mostly single-layered. Those clouds comprise only a subset of the total. To further explore the nature of the errors in CTH determination when all clouds are considered, both 50/50 and 0/100 CTH_V and CTHc pairs similar those used in the right side of Figure 9 were grouped according to one of the eight CALIPSO VFM cloud types: opaque (OP) and transparent (NO) low overcast; transition stratocumulus (Sc); low broken cumulus (Cu); transparent altocumulus (Ac) and cirrus (Ci); and opaque altostratus (As) and deep convective clouds. Instead of using the results from Figure 9, each 1 km HA CALIOP pixel is matched to the nearest VIIRS pixel within a 2.5 km radius. The higher resolution CALIOP and matching approach are employed to facilitate comparisons with other datasets. The results include matched pixels that agree or disagree in phase selection and have both single and multi-layered clouds. From Figure 9, it is expected that the CTH difference statistics should very close for the 50/50 and 0/100 data, so the 0/100 results can be used to represent all clouds. To increase sampling, data with |∆t| ≤ 15 min are used.

Figure 18 plots the CV1S phase breakdown (Figure 18a) and scatterplots of matched CTH_V and CTH_C values (Figure 18b–h) for daytime 0/100 matches. Except for the transition stratocumulus, low clouds are almost completely classified as liquid, while significant fractions of midlevel clouds are identified as ice. Of the transparent cirrus, 39% of the pixels are classified as water, suggesting the presence of low-level water clouds. Even 10% of the deep convective opaque clouds are classified as liquid. Those cases are most likely composed of cirrus with COD_C ~3 over thick, low clouds or are unglaciated deep convective clouds, as CALIOP classifies some as liquid also (shown below).

Misclassification of low cloud phase as ice would tend to cause an overestimate of CTH_V for clouds having COD_V < 6. This effect is likely responsible for many of the CTH overestimates seen in Figure 18b–d,f–i. Low overcast transparent or non-opaque (NO) clouds in Figure 18a have the smallest bias and number of pixels with CTH_V > 4 km. The larger biases for opaque (OP) low clouds (Figure 18c) and transition Sc (Figure 18f) are accompanied by more values above 4 km, producing biases of ~0.2 km. The broken Cu clouds (Figure 18g) yield the greatest biases. In this instance and possibly for the transition Sc, holes in clouds interpreted as overcast by CV1S, would diminish COD_V and raise CTH_V.

Midlevel cloud CTHs are affected by several factors including the wrong phase. Low-level clouds underneath NO Ac clouds result in underestimation of CTH as seen around CTH_V = 2 km in Figure 6a,b and Figure 18d. Those plots clearly show the need to account for liquid-over-liquid ML clouds in cloud CTH retrievals. The ML cloud effect is less evident for opaque As clouds; however, a different error source is likely responsible for the vertical feature seen near CTH_C = 3.5 km in Figure 18h. The transition from the low-level lapse rate and reanalysis typically occurs between 3 and 4 km. For areas where the T_s-based lapse rate is much steeper than the reanalysis in the transition pressures, CET may correspond to a lower altitude than found in the reanalysis profile (e.g., Figure 5 of [34]), resulting in underestimates of CTH in the transition layer. This effect is also apparent in the Ac plot (Figure 18d), which suggests that COD is overestimated because the bulk of the scatter occurs below the 1:1 line, unlike that for As.

At night (Figure 19), the results are somewhat different. As seen in Figure 19a, more clouds are placed in the clear category and more ice clouds are placed in the low cloud categories, compared to the daytime, as expected from Figure 5. Conversely, fewer cirrus and deep convective clouds are mistaken as liquid water clouds. In a reversal of their daytime biases, nocturnal transparent Ac (Figure 19d) and As (Figure 19h) CTHs are overestimated, on average, by 0.4–0.5 km. The cirrus (Figure 19e) and deep convective (Figure 19i) underestimates are significantly reduced compared to daytime. Biases for all of the other cloud types are larger at night, especially for broken cumulus clouds (Figure 19g). The rise in the bias for those clouds is mainly due to the increased proportion of liquid clouds being identified as ice, which will result in a smaller COD_V and lower CET than if the liquid model were selected for the retrieval.

The impact of selecting the wrong phase on the estimation of CTH can be quantified by dividing the results from Figure 18 and Figure 19 into four different phase-pair groups, denoted by CALIOP/CV1S phase types water (W) and ice (I). Figure 20 plots the bias (top), ∆Z_T, for each phase pair as a function of VFM cloud type. Each bias is accompanied by the fraction of the total number of pixels for a given cloud type (bottom) represented by a given phase pair. When both CALIOP and CV1S agree on liquid water (W/W, blue line), ∆Z_T is very close to zero for low clouds both day (Figure 20a) and night (Figure 20b), but becomes negative for higher clouds, especially for Ci. For low clouds, this represents more than 95% of the pixels during the day (Figure 20c) and more than 85% at night (Figure 20d). When both CALIOP and CV1S agree that the highest cloud is ice (I/I, gray line), the low-cloud biases are between 2 and 4 km during the day and 1 and 5 km at night. Except for Cu during the night, the percent of cases is very small. The biases are between 0 and 1 km for I/I midlevel clouds and account for ~10% of pixels in daylight and ~20% at night. For I/I Ci, ∆Z_T near −1.7 km and comprises ~55% and ~80% of the total during day and night, respectively. Deep convective I/I clouds are less biased, with ∆Z_T between −0.53 and −0.75 km. They account for more than 90% of the total both day and night. For cases with phase agreement, the biases are still relatively large for the midlevel and Ci clouds. The impact of clouds below these higher cloud types can have a significant influence on the CET retrieval, when COD_U < 6.

When the phase selections disagree, |∆Z_T| is larger in all cases, except for low clouds when CALIOP selects ice and CV1S phase is water (I/W, red line). In that instance, the biases are comparable in magnitude, I/W pixels are sparse for low clouds. For higher I/W clouds, ∆Z_T is extremely negative and accounts for significant fractions of the Ac and Ci pixels, especially during the day. Most of the pixels are likely to be ML clouds with low-level water clouds (e.g., Figure 12). Pixels with CALIOP water and CV1S ice phases (W/I, green line) have a strong positive bias and primarily affect the midlevel clouds during the day and low and midlevel clouds at night. Thus, there is some balancing effect of the phase errors on the overall biases for midlevel clouds. The nocturnal bias for low clouds is driven upwards by the CV1S selection of ice instead of the correct water phase. It is quite evident that, among other factors, determining the correct phase for more pixels will result in improved CTH accuracy.

Even with the relatively large errors seen above, the overall accuracy of the CV1S CTH product is in line with other operational CTH retrievals. Retrievals of CTH from three passive methods were compared with CALIOP data by [12]. The three techniques include an experimental neural network (HNN) method developed by [12], the MODIS Collection 6 (MODC6) algorithm [42], and the Polar Platform System (PPS) 2014 version [43]. The HNN uses matched 1 km CALIOP and Aqua MODIS pixels from 4 days in 2010 simulating the VIIRS data using 3 MODIS infrared channels, while results from MODC6 and PPS were also matched with the 1 km CALIOP data for the same dates. Table 8 lists the biases and mean absolute errors (MAE) for the three retrieval algorithms as reported by [12] employing combined day and night results. According to [12], MAE represents a more balanced measure of precision than SDD for non-Gaussian distributions. The statistics in the first 3 rows of Table 8 were taken from Tables 6–9 of [12]. The low, midlevel, and high clouds in [12] are defined by pressures 680 hPa and 440 hPa, the same definition used for the CALIOP cloud typing. Biases and MAEs were computed for the three cloud levels using combined CV1S pixel data from the appropriate cloud types, as in Figure 18 and Figure 19. It is also not clear if the 1 km CALIOP pixels used by [12] are overcast, so both 0/100 (CV1S₁₀₀) and 50/50 (CVS₅₀) were computed. Those results, derived from ~2.7 and ~2.9 million collocations, respectively, comprise the bottom row in Table 8.

In all but two cases, the low and midlevel cloud biases, the experimental HNN produces the smallest error in every category. The CV1S values are the second lowest for all parameters, except the midlevel MAE and midlevel ∆Z_T. PPS yields the second smallest value for the former case and CV1S is lowest result for the midlevel bias. Overall, CV1S and MOD C6 are quite close, with the latter simultaneously yielding the smallest bias and largest MAE for low clouds. The CV1S₅₀ low cloud height bias is the largest, mainly as a result of the phase misclassification discussed above. The CV1S₁₀₀ low-cloud MAE is the smallest of the operational methods, while those for PPS and CV1S are nearly identical. High-cloud CTH errors for the CV1S results are the least for the non-HNN techniques. Of the three considered operational algorithms, the PPS has the largest errors overall. There is minimal difference between the CV1S₅₀ and CV1S₁₀₀ errors. It should be noted that there are some differences between the data used by [12] and those employed here. The latter comprise 750 m VIIRS pixels taken at various VZAs with |∆t| ≤ 15 min, while the former are near-nadir, 1 km MODIS pixels with |∆t| ≤ 3 min. As noted earlier, the time and VZA differences raise the CV1S SDD and hence its MAE, but have a small impact on the bias except for transparent cirrus, which has a nocturnal negative bias that decreases with VZA. While the HNN produces a superior result in general, it is reasonable to conclude that overall the CV1S CTH accuracy is as good as or better than at least two often-used operational CTH products.

CERES is a project devoted to understanding the interactions of clouds and aerosols on the Earth radiation budget, not just at the top of the atmosphere, but within the atmosphere and at the surface. Thus, accurate vertical placement of clouds is an important component in measuring and modeling the radiation field for a given atmospheric column. A reliable estimate of CTH for the highest cloud in the column is a significant step in describing the cloud structure. Cloud base height is another. Those two parameters plus COD and CER, as well as their vertical distribution determine the impact on the emitted and reflected radiation, particularly for the former. That impact can be summarized as the cloud effective height, which, if the radiation from the surface and other atmospheric components is correctly represented, corresponds to the brightness temperature of the cloud contribution to the outgoing infrared flux. It corresponds to some distance below the top of single-layer clouds. That distance depends on the cloud microphysics. For ice clouds it can be 1 km or more (e.g., [44,45]), while it may only be a few tens of meters or less for liquid water clouds (e.g., [46]). For multilayer clouds, it can be located at some altitude between the layers unless the top layer is optically thick. Moreover, if CTH is correctly determined for the ice cloud in a ML system, but the bulk of the COD is from the low-level cloud, there is a mismatch of the cloud phase and the water content, unless the cloud is explicitly interpreted as a ML clouds system (e.g., [47]). Retrieving the bulk parameters needed to more accurately characterize the radiation field than presently possible will therefore require much more than simply finding CTH.

5. Conclusions

Several CV1S cloud products have been compared to CALIOP data taken at various resolutions, collocation time differences, and VIIRS VZAs in order to estimate the CV1S cloud retrieval accuracies. A number of findings result from the ensuing analyses.

-: The accuracies of the CV1S cloud fraction are slightly better than those for CM4A during the daytime and about the same as those at night. CV1S cloud phase accuracy is slightly worse than CM4A during the day and night. Sensitivity of phase selection to ice cloud optical depth in multi-layered clouds is very consistent between CV1S and CM4A. The cloud-top height comparisons are very consistent with their CM4A counterparts with the exception of reduced ice cloud top height biases for CV1S, likely the result of using different channels for backup retrievals.
-: The time window for matching CALIOP and imager data is very important for assessing instantaneous cloud amount, but is not as critical for determining cloud amount bias. Using a larger collocation time window than 5 min would have produced less consistency with CM4A for fraction correct. Imager VZA also has some impact on cloud fraction bias, particularly at night. VZA and time windows are less important for cloud phase and height assessment, except for thin cirrus at night. As seen in [19], the CALIOP detection resolution affects the bias.
-: Daytime CV1S cloud detection is as good or better than several other operational algorithms. At night, the accuracies are comparable overall with the other methods. None of the operational approaches are as accurate as a new machine learning technique for cloud detection.
-: Supercooled liquid water clouds are properly diagnosed 96% of the time during the daytime. At night, they are correctly identified in 75% of the cases. CV1S cloud phase accuracy overall is comparable to that from several operational methods but is slightly less than that from a new neural-net based method.
-: Liquid water cloud top heights are less biased during the daytime than at night. For single-layer clouds, the nocturnal bias is 0.2 km. Further research is need to assess that day–night difference. The transition of a surface-anchored lapse rate to a reanalysis temperature profile in assigning height to a given cloud temperature is responsible for underestimating cloud top height for many altostratus clouds.
-: Ice cloud top heights are underestimated in the tropics partially because CV1S confines the top to the tropopause level. That level is poorly determined in the tropics, and is set near the bottom of the tropical transition layer. Retrievals that initially might place the cloud higher are overwritten with the tropopause altitude, underestimating the top altitude of many clouds above 15 km. Other factors affecting the height retrieval, especially during daytime, are low-level clouds underneath both midlevel and high clouds.
-: Because cloud optical depth, effective temperature, effective hydrometeor radius, and phase are determined simultaneously, selecting the correct phase impacts the effective temperature. Inaccurate phase selection thus affects cloud-top height estimates, even in the absence of multilayered, multiphase clouds. Cloud altitude is overestimated significantly when liquid clouds are interpreted as ice clouds. The opposite effect is found for ice clouds classified as water. There is minimal dependence of this effect on the time of day.
-: CV1S cloud top height uncertainty overall is very similar to or better than several operational algorithms, but again, fails to match the accuracy of an experimental machine learning technique.

The CERES VIIRS Edition 1 cloud parameters can be used with the same level of confidence as their CERES MODIS predecessors, providing a reliable climate data record. Yet, the accuracy of that record can be enhanced. This study has elucidated several areas for improvement in the algorithms. Reducing uncertainties in some parameters, such as CTH, can be accomplished by altering some of the physical and empirical components of the retrieval code. Further steps forward for other parameters may require a different approach. The comparisons here have confirmed that artificial intelligence techniques can also dramatically decrease errors in cloud detection, phase selection, and height determination. Combining physical and machine learning retrievals in the future may be optimal overall for substantially advancing the characterization of clouds from passive satellite imagery for radiation analyses and other applications.

Author Contributions

Conceptualization, C.R.Y. and P.M.; methodology, C.R.Y.; software, C.R.Y.; formal analysis, P.M. and C.R.Y.; investigation, P.M. and C.R.Y.; resources, C.R.Y., S.S.-M. and Q.Z.T.; data curation, C.R.Y., S.S.-M. and Q.Z.T.; writing—original draft preparation, P.M.; writing—review and editing, C.R.Y., W.L.S.J. and P.M.; visualization, C.R.Y. and P.M.; supervision, W.L.S.J.; project administration, P.M. and W.L.S.J. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the National Aeronautics and Space Administration through the CERES Project.

Data Availability Statement

CERES and CALIPSO data products are publicly available from the NASA Langley Research Center Atmospheric Science Data Center at https://asdc.larc.nasa.gov (accessed on 1 December 2022).

Acknowledgments

The tropical temperature profiles were obtained at the Wyoming Weather Web (http://weather.uwyo.edu; accessed on 6 October 2022) provided by the University of Wyoming.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

Smith, G.L.; Priestley, K.J.; Loeb, N.G.; Wielicki, B.A.; Charlock, T.P.; Minnis, P.; Doelling, D.R.; Rutan, D.A. Clouds and the Earth’s Radiant Energy System (CERES), a review: Past, present, and future. Adv. Space Res. 2011, 48, 254–263. [Google Scholar] [CrossRef]
Loeb, N.G.; Su, W.; Doelling, D.R.; Wong, T.; Minnis, P.; Thomas, S.; Miller, W.F. 5.03—Earth’s top-of-atmosphere radiation budget. In Comprehensive Remote Sensing; Elsevier Ltd.: Oxford, UK, 2018; pp. 67–84. [Google Scholar] [CrossRef]
Hillger, D.; Kopp, T.; Lee, T.; Lindsey, D.; Seaman, C.; Miller, S.; Solberg, J.; Kidder, S.; Bachmeier, S.; Jasmin, T.; et al. First-light imagery from Suomi NPP VIIRS. Bull. Am. Meteorol. Soc. 2013, 93, 1019–1029. [Google Scholar] [CrossRef]
Minnis, P.; Sun-Mack, S.; Smith, W.L., Jr.; Trepte, Q.Z.; Hong, G.; Chen, Y.; Yost, C.R.; Chang, F.-L.; Smith, R.A.; Heck, P.W.; et al. VIIRS Edition 1 cloud properties for CERES. Part 1: Algorithm adjustments and results. Remote Sens. 2023, 15, 578. [Google Scholar] [CrossRef]
Winker, D.M.; Vaughan, M.A.; Omar, A.; Hu, Y.; Powell, K.A.; Liu, Z.; Hunt, W.H.; Young, S.A. Overview of the CALIPSO mission and CALIOP data processing algorithms. J. Atmos. Ocean. Technol. 2009, 26, 2310–2323. [Google Scholar] [CrossRef]
Sun-Mack, S.; Wielicki, B.A.; Minnis, P.; Gibson, S.; Chen, Y. Integrated cloud-aerosol-radiation product using CERES, MODIS, CALIPSO, and CloudSat data. Proc. SPIE Remote Sens. Clouds Atmos. XII 2007, 6745, 674513. [Google Scholar] [CrossRef]
Minnis, P.; Yost, C.R.; Sun-Mack, S.; Chen, Y. Estimating the physical top altitude of optically thick ice clouds from thermal infrared satellite observations using CALIPSO data. Geophys. Res. Lett. 2008, 35, L12801. [Google Scholar] [CrossRef] [Green Version]
Holz, R.E.; Ackerman, S.A.; Nagle, F.W.; Frey, R.; Dutcher, S.; Kuehn, R.E.; Vaughan, M.A.; Baum, B. Global moderate resolution imaging spectroradiometer (MODIS) cloud detection and height evaluation using CALIOP. J. Geophys. Res. 2008, 113, D00A19. [Google Scholar] [CrossRef] [Green Version]
Karlsson, K.-G.; Johansson, E. On the optimal method for evaluating cloud products from passive satellite imagery using CALIPSO-CALIOP data: Example investigating the CM SAF CLARA-A1 dataset. Atmos. Meas. Tech. 2013, 6, 1271–1286. [Google Scholar] [CrossRef] [Green Version]
Kox, S.; Bugliaro, L.; Ostler, A. Retrieval of cirrus cloud optical thickness and top altitude from geostationary remote sensing. Atmos. Meas. Tech. 2014, 7, 3233–3246. [Google Scholar] [CrossRef] [Green Version]
Marchant, B.; Platnick, S.; Meyer, K.; Arnold, G.T.; Riedi, J. MODIS Collection 6 shortwave-derived cloud phase classification algorithm and comparisons with CALIOP. Atmos. Meas. Tech. 2016, 9, 1587–1599. [Google Scholar] [CrossRef] [Green Version]
Håkansson, N.; Adok, C.; Thoss, A.; Scheirer, R.; Hörnquist, S. Neural network cloud top pressure and height for MODIS. Atmos. Meas. Tech. 2018, 11, 3177–3196. [Google Scholar] [CrossRef] [Green Version]
Karlsson, K.-G.; Håkansson, N. Characterization of AVHRR global cloud detection sensitivity based on CALIPSO-CALIOP cloud optical thickness information: Demonstration of results based on the CM SAF CLARA-A2 climate data record. Atmos. Meas. Tech. 2018, 11, 633–649. [Google Scholar] [CrossRef] [Green Version]
Wang, C.; Platnick, S.; Fauchez, T.; Meyer, K.; Zhang, Z.; Iwabuchi, H.; Kahn, B.H. An assessment of the impacts of cloud vertical heterogeneity on global ice cloud data records from passive satellite retrievals. J. Geophys. Res. 2019, 124, 1578–1595. [Google Scholar] [CrossRef]
Marchant, B.; Platnick, S.; Meyer, K.; Wind, G. Evaluation of the Aqua MODIS Collection 6.1 multilayer cloud detection algorithm through comparisons with CloudSat CPR and CALIPSO CALIOP products. Atmos. Meas. Tech. 2020, 13, 3263–3275. [Google Scholar] [CrossRef]
Frey, R.A.; Ackerman, S.A.; Holz, R.E.; Dutcher, S.; Griffith, Z. The continuity MODIS-VIIRS cloud mask. Remote Sens. 2020, 12, 3334. [Google Scholar] [CrossRef]
Jiménez, P.A. Assessment of the GOES-16 clear sky mask product over the contiguous USA using CALIPSO retrievals. Remote Sens. 2020, 12, 1630. [Google Scholar] [CrossRef]
Li, Y.; Baum, B.A.; Heidinger, A.K.; Menzel, W.P.; Weise, E. Improvement in cloud retrievals from VIIRS through use of infrared absorption channels constructed from VIIRS+CrIS data fusion. Atmos. Meas. Tech. 2020, 13, 4035–4049. [Google Scholar] [CrossRef]
Yost, C.; Minnis, P.; Sun-Mack, S.; Chen, Y.; Smith, W.L., Jr. CERES MODIS cloud product retrievals for Edition 4, Part II: Comparisons to CloudSat and CALIPSO. IEEE Trans. Geosci. Remote Sens. 2021, 59, 3695–3724. [Google Scholar] [CrossRef]
Trepte, Q.Z.; Minnis, P.; Sun-Mack, S.; Yost, C.R.; Chen, C.R.; Jin, Z.; Chang, F.-L.; Smith, W.L., Jr.; Bedka, K.M.; Chee, T.L. Global cloud detection for CERES Edition 4 using Terra and Aqua MODIS data. IEEE Trans. Geosci. Remote Sens. 2019, 57, 9410–9449. [Google Scholar] [CrossRef]
Stephens, G.L.; Vane, D.G.; Tanelli, S.; Im, E.; Durden, S.; Rokey, M.; Reinke, D.; Partain, P.; Mace, G.G.; Austin, R.; et al. CloudSat mission: Performance and early science after the first year of operation. J. Geophys. Res. 2008, 113, D00A18. [Google Scholar] [CrossRef]
Vaughan, M.A.; Powell, K.A.; Kuehn, R.E.; Young, S.A.; Winker, D.M.; Hostetler, C.A.; Hunt, W.H.; Liu, Z.; McGill, M.J.; Getzewitch, B.J. Fully automated detection of cloud and aerosol layers in the CALIPSO lidar measurements. J. Atmos. Ocean. Technol. 2009, 26, 2034–2050. [Google Scholar] [CrossRef]
Vaughan, M.A.; Pitts, M.; Trepte, C.; Winker, D.; Detweiler, P.; Garnier, A.; Getzewich, B.; Hunt, W.; Lambeth, J.; Lee, K.-P.; et al. Cloud-Aerosol LIDAR Infrared Pathfinder Satellite Observations (CALIPSO) Data Management System Data Products Catalog, Release 4.10; NASA Langley Research Center Document PC-SCI-503; NASA Langley Research Center: Hampton, VA, USA, 2016. [Google Scholar]
Hu, Y.; Winker, D.; Vaughan, M.; Lin, B.; Omar, A.; Trepte, C.; Flittner, D.; Yang, P.; Baum, B.; Sun, W.; et al. CALIPSO/CALIOP cloud phase discrimination algorithm. J. Atmos. Ocean. Technol 2009, 26, 2293–2309. [Google Scholar] [CrossRef] [Green Version]
Young, S.A.; Vaughan, M.A.; Tackett, J.L.; Garnier, A.; Lambeth, J.B.; Powell, K.A. Extinction and optical depth retrievals for CALIPSO’s Version 4 data release. Atmos. Meas. Tech. 2018, 11, 5701–5727. [Google Scholar] [CrossRef] [Green Version]
White, C.H.; Heidinger, A.K.; Ackerman, S.A. Evaluation of Visible Infrared Imaging Radiometer Suite (VIIRS) neural network cloud detection against current operational cloud masks. Atmos. Meas. Tech. 2021, 14, 3371–3394. [Google Scholar] [CrossRef]
Woodcock, F. The evaluation of yes/no forecasts for scientific and administrative purposes. Mon. Weather Rev. 1976, 104, 1209–1214. [Google Scholar] [CrossRef]
Heidinger, A.; Botambekov, D.; Walther, A. A Naïve Bayesian Cloud Mask Delivered to NOAA Enterprise—Version 1.2; Tech. Rep.; NOAA NESDIS Center for Satellite Applications and Research: Silver Spring, MD, USA, 2016. Available online: https://www.star.nesdis.noaa.gov/goesr/documents/ATBDs/Enterprise/ATBD_Enterprise_Cloud_Mask_v1.2_Oct2016.pdf (accessed on 21 July 2020).
Wang, C.; Platnick, S.; Meyer, K.; Zhang, Z.; Zhou, Y. A machine-learning-based cloud detection and thermodynamic-phase classification algorithm using passive spectral observations. Atmos. Meas. Tech. 2020, 13, 2257–2277. [Google Scholar] [CrossRef]
Zhou, G.; Wang, J.; Yin, Y.; Hu, X.; Letu, H.; Sohn, B.-J.; Chao, L. Detecting supercooled water clouds using passive radiometer measurements. Geophys. Res. Lett. 2022, 49, e2021GL096111. [Google Scholar] [CrossRef]
Sassen, K.; Wang, Z.; Liu, D. Global distribution of cirrus clouds from CloudSat/Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) measurements. J. Geophys. Res. 2008, 113, D00A12. [Google Scholar] [CrossRef]
Platnick, S.; Meyer, K.; Wind, G.; Holz, R.E.; Amarasinghe, N.; Hubanks, P.A.; Marchant, B.; Dutcher, S.; Veglio, P. The NASA MODIS-VIIRS continuity cloud optical properties products. Remote Sens. 2021, 13, 2. [Google Scholar] [CrossRef]
Minnis, P.; Sun-Mack, S.; Young, D.F.; Heck, P.W.; Garber, D.P.; Chen, Y.; Spangenberg, D.A.; Arduini, R.F.; Trepte, Q.Z.; Smith, W.L., Jr.; et al. CERES Edition-2 cloud property retrievals using TRMM VIRS and Terra and Aqua MODIS data, Part I: Algorithms. IEEE Trans. Geosci. Remote Sens. 2011, 49, 4374–4400. [Google Scholar] [CrossRef]
Sun-Mack, S.; Minnis, P.; Chen, Y.; Kato, S.; Yi, Y.; Gibson, S.; Heck, P.W.; Winker, D. Regional apparent boundary layer lapse rates determined from CALIPSO and MODIS data for cloud height determination. J. Appl. Meteorol. Climatol. 2014, 53, 990–1011. [Google Scholar] [CrossRef]
Rienecker, M.M.; Suarez, M.J.; Todling, R.; Bacmeister, S.; Takacs, L.; Liu, H.-C.; Gu, W.; Sienkiewicz, M.; Koster, R.D.; Gelaro, R.; et al. The GEOS-5 Data Assimilation System—Documentation of Versions 5.0.1, 5.1.0, and 5.2.0; Technical Report Series on Global Modeling and Data Assimilation, Report No. NASA/TM-2008-104606-VOL-27; NASA: Washington, DC, USA, 2008; Volume 27, 118p. [Google Scholar]
Minnis, P.; Sun-Mack, S.; Yost, C.R.; Chen, Y.; Smith, W.L., Jr.; Chang, F.-L.; Heck, P.W.; Arduini, R.F.; Trepte, Q.Z.; Ayers, K.; et al. CERES MODIS cloud product retrievals for Edition 4, Part I: Algorithm changes to CERES MODIS. IEEE Trans. Geosci. Remote Sens. 2021, 59, 2744–2780. [Google Scholar] [CrossRef]
Baum, B.A.; Arduini, R.F.; Wielicki, B.A.; Minnis, P.; Tsay, S.-C. Multilevel cloud retrieval using multispectral HIRS and AVHRR data: Nighttime oceanic analysis. J. Geophys. Res. 1994, 99, 5499–5514. [Google Scholar] [CrossRef]
Zheng, Y.; Li, Z. Episodes of warm air advection causing cloud-surface decoupling during MARCUS. J. Geophys. Res. Atmos. 2019, 124, 12227–12443. [Google Scholar] [CrossRef]
Minnis, P.; Sun-Mack, S.; Chen, Y.; Khaiyer, M.M.; Yi, Y.; Ayers, J.K.; Brown, R.R.; Dong, X.; Gibson, S.C.; Heck, P.W.; et al. CERES Edition-2 cloud property retrievals using TRMM VIRS and Terra and Aqua MODIS data, Part II: Examples of average results and comparisons with other data. IEEE Trans. Geosci. Remote Sens. 2011, 49, 4401–4430. [Google Scholar] [CrossRef]
Fueglistaler, S.; Dessler, A.E.; Dunkerton, T.J.; Folkins, I.; Fu, Q.; Mote, P.W. Tropical tropopause layer. Rev. Geophys. 2009, 47, RG1004. [Google Scholar] [CrossRef]
Loeb, N.; Yang, P.; Rose, F.G.; Hong, G.; Sun-Mack, S.; Minnis, P.; Kato, S.; Ham, S.-H.; Smith, W.L., Jr. Impact of ice cloud microphysics on satellite retrievals and broadband flux radiative transfer calculations. J. Clim. 2018, 31, 1851–1864. [Google Scholar] [CrossRef]
Baum, B.A.; Menzel, W.P.; Frey, R.A.; Tobin, D.C.; Holz, R.E.; Ackerman, S.A.; Heidinger, A.K.; Yang, P. MODIS cloud-top property refinements for Collection 6. J. Appl. Meteorol. Climatol. 2012, 51, 1145–1163. [Google Scholar] [CrossRef]
Stengel, M.; Stapelberg, S.; Sus, O.; Schlundt, C.; Poulsen, C.; Thomas, G.; Christensen, M.; Carbajal Henken, C.; Preusker, R.; Fischer, J.; et al. Cloud property datasets retrieved from AVHRR, MODIS, AATSR and MERIS in the framework of the Cloud_cci project. Earth Syst. Sci. Data 2017, 9, 881–904. [Google Scholar] [CrossRef] [Green Version]
Minnis, P.; Young, D.F.; Sassen, K.; Alvarez, J.M.; Grund, C.J. The 27-28 October 1986 FIRE IFO Case Study: Cirrus parameter relationships derived from satellite and lidar data. Mon. Weather Rev. 1990, 118, 2402–2425. [Google Scholar] [CrossRef]
Sherwood, S.C.; Chae, J.-H.; Minnis, P.; McGill, M. Underestimation of deep convective cloud tops by thermal imagery. Geophys. Res. Lett. 2004, 31, L11102. [Google Scholar] [CrossRef] [Green Version]
Nakajima, T.Y.; Ishida, H.; Nagao, T.M.; Hori, M.; Letu, H.; Higuchi, R.; Tamaru, N.; Imoto, N.; Yamazaki, A. Theoretical basis of the algorithms and early phase results of the GCOM-C (Shikisai) SGLI cloud products. Prog. Earth Planet. Sci. 2019, 6, 52. [Google Scholar] [CrossRef] [Green Version]
Minnis, P.; Sun-Mack, S.; Smith, W.L., Jr.; Hong, G.; Chen, Y. Advances in neural network detection and retrieval of multilayer clouds for CERES using multispectral satellite data. Proc. SPIE Remote Sens. Clouds and the Atmos. XXIV 2019, 11152, 1115202. [Google Scholar] [CrossRef]

Figure 1. Gridded (5° × 5°) mean daytime (a–c) and nighttime (d–f) cloud fractions from CV1S (a,d) and CALIPSO (HA ≤ 5 km; (b,e)) and their differences (c,f) using all matched data from JAJO 2015–2016.

Figure 2. Mean zonal (a) day and (b) night cloud fractions from CV1S and CALIOP for four horizontal averaging scales, JAJO 2015–2016, using all matched data.

Figure 3. Dependence of CV1S cloud detection (a) fraction correct, (b) bias, (c) false alarm rate, and (d) hit rate on CALIOP horizontal averaging, JAJO, 2015–2016, for 0/100 matched pixels.

Figure 4. Average (a,b) daytime and (c,d) nighttime (a,c) liquid and (b,d) ice cloud amounts for CV1S and CALIOP pixels collocated within ±15 min, JAJO 2015–2016.

Figure 5. CV1S 0/100 (a) day and (b) night scene type fraction for various classifications as determined from CALIOP HA ≤ 5 km data for collocations with ±2.5 min during JAJO 2015–2016. * denotes single-layer, single phase clouds. Brackets indicate all CALIOP classifications treated as liquid-phase (blue) and ice-phase (red) clouds in height and COD discussion.

Figure 6. CV1S versus CALIOP cloud-top heights for 0/100 liquid clouds, JAJO 2015–2016, (a,b): All liquid-dominant clouds, (c,d): Single-layer liquid clouds only. (a,c): day, (b,d): night.

Figure 7. Regional mean cloud-top height differences between CV1S (CTH_V) and CALIOP (CTH_C) for 0/100 liquid water clouds, JAJO 2015–2016. (a,c): Single-layer (SL), (b,d): all with liquid tops, (a,b): day, (c,d): night.

Figure 8. CV1S versus CALIOP cloud-top heights for all 0/100 single-layer ice clouds, JAJO 2015–2016, (a,b): Daytime, (c,d): Nighttime. (a,c): opaque, (b,d): non-opaque.

Figure 9. Probability distributions of cloud-top height differences, CTH_V − CTH_C, regardless of phase, JAJO 2015–2016. (a,b): day, (c,d): night, (a,c): all matches, (b,d): overcast matches.

Figure 10. Cloud-clear detection FC as function of collocation time window and VIIRS VZA for 0/100 cases, JAJO 2015–2016. Mean values are shown for low VZA (0–30°) in black and high VZA (30–80°) in magenta for day (a,d,g,j) and night (b,e,h,k). Sampling density (c,f,i,l) is presented as function of scan time difference and VZA in the (c,f,i,l).

Figure 11. Dependence of CV1S true positive rate on COD_C for 0/100 ice clouds, HA ≤ 1 km, JAJO 2015–2016. Solid (hashed) bars indicate fraction of cloud profiles for (a) day and (b) night observations.

Figure 12. Variation of CV1S (a) day and (b) night scene identification and phase for multilayer non-opaque cirrus over low clouds as a function of CALIOP cirrus optical depth (COD_u) using HA ≤ 80 km data collocated within 15 min. Number of samples used as ratio to total CALIOP cases is indicated in the blue box. The asterisks denote the COD_u of the median and 90th percentile.

Figure 13. CV1S classification of (a) day and (b) night clouds having effective temperatures between 0.0 °C and −40 °C for three different categories liquid, ice, and mixed phase clouds as determined from CALIOP profiles using the method of [30]. Percentages of each CALIOP category classified by CV1S as liquid or ice cloud are indicated in blue and red, respectively.

Figure 14. Frequency of occurrence of CV1S COD for liquid water clouds, 2012, 30°S–60°S, (a) day and (b) night.

Figure 15. Histograms of SIF ocean CTH differences as a function of the brightness temperature differences (BTD) between the VIIRS 11 and 12 µm channels (a,c) and the CV1S cloud optical depth (b,d) from daytime (a,b) and nighttime (c,d) data for 0/100 SL overcast liquid water clouds, JAJO 2015–2016. Fractional frequencies of occurrence are provided in the bottom portion of each plot. Mean differences are indicated by horizontal lines in each bar. Shaded area indicates the range of data for each bar.

Figure 16. Scatterplots of 0/100 CALIOP and CV1S nighttime, single-layer, ice cloud top heights for the (a,c) tropics (latitude ≤ 30°) and (b,d) mid-latitudes (30° < |latitude| ≤ 60°) for opaque (a,b) and non-opaque (c,d) clouds, JAJO 2015–2016, time window of ±15 min.

Figure 17. Temperature profiles near Fernando de Noronha, Brazil, 12 UTC, 15 July 2022 at 3.85°S, 32.41°W. (a) full profile to 28 km, (b) closeup of profile between 10 and 20 km. Thick dashed blue line is from G541 used by CERES and the solid red line is from a radiosonde. Z_p and the thin dashed line mark the tropopause height from G541. Approximate boundaries of the tropical tropopause layer shown as green lines.

Figure 18. (a) CV1S daytime cloud phase selection for different CALIOP cloud types and (b–i) scatterplots of 0/100 CTH_C and CTH_V pairs for each cloud type. Data are from JAJO 2015–2016 and were constrained to CALIOP/CV1S observation time differences of |∆t| ≤ 15 min. CALIOP HA ≤ 5 km.

Figure 19. Same as Figure 18, except for nighttime data.

Figure 20. Cloud-top height biases, ∆Z, as a function of CALIOP/CV1S cloud-top phase agreement for CALIOP VFM cloud types, JAJO 2015–2016, |∆t| ≤ 15 min. W and I represent water and ice, respectively. (a,b): Biases, (c,d): Percent of each phase pair comprising the cloud type samples. (a,c): Day. (b,d): Night.

Table 1. Confusion matrix for evaluating cloud fraction (phase).

VIIRS	CALIPSO
	Clear (Liquid)	Cloudy (Ice)	Sum
Clear (liquid)	Tn	Fn	n
Cloudy (ice)	Fp	Tp	p

Table 2. Cloud detection metrics determined from confusion matrices of matched CERES SNPP VIIRS Ed1a and CALIOP (HA ≤ 5 km) pixels, JAJO 2015 and 2016 for 0/100 cases and 50/50 cases, shown in parentheses.

Day	FC	HR	Bias	FAR	HKSS	N × 10³
Nonpolar	0.952 (0.896)	0.959 (0.914)	−0.011 (−0.012)	0.026 (0.069)	0.885 (0.765)	220 (272)
Polar	0.923 (0.899)	0.923 (0.902)	−0.024 (−0.029)	0.041 (0.057)	0.831 (0.779)	99 (108)
Global, All	0.943 (0.897)	0.948 (0.910)	−0.015 (−0.017)	0.030 (0.066)	0.868 (0.769)	318 (380)
Global, All *	0.948 (0.896)	0.954 (0.912)	−0.013 (−0.014)	0.028 (0.067)	0.878 (0.767)	-
Night
Nonpolar	0.927 (0.870)	0.941 (0.890)	−0.018 (−0.035)	0.037 (0.070)	0.798 (0.697)	213 (271)
Polar	0.804 (0.764)	0.809 (0.777)	−0.080 (−0.068)	0.091 (0.127)	0.551 (0.476)	126 (151)
Global, All	0.881 (0.833)	0.913 (0.846)	−0.041 (−0.050)	0.055 (0.090)	0.698 (0.616)	339 (422)
Global, All *	0.911 (0.856)	0.924 (0.875)	−0.026 (−0.039)	0.044 (0.077)	0.765 (0.669)	-

* Estimated as area-weighted average of polar and nonpolar surfaces, instead of by number of samples.

Table 3. CV1S cloud phase validation metrics for overcast (0/100 dataset) matches having a single phase or predominate phase using HA ≤ 5 km data.

Day	Fraction Correct	Bias	Ice FAR	Water FAR	HKSS	Number Samples (×10³)
Global, All surfaces	0.931	−0.019	0.068	0.069	0.844	132
Nonpolar land, SIF	0.888	−0.079	0.034	0.182	0.791	19
Polar land, SIF	0.920	−0.038	0.073	0.082	0.789	4
Nonpolar ocean, SIF	0.954	−0.005	0.055	0.040	0.900	75
Polar ocean, SIF	0.941	+0.005	0.160	0.034	0.820	10
Global, SIF	0.940	−0.018	0.056	0.062	0.866	108
Global, SIC	0.892	−0.023	0.131	0.097	0.746	25
Nighttime
Global, All surfaces	0.883	0.079	0.185	0.040	0.780	141
Nonpolar land, SIF	0.854	0.025	0.140	0.155	0.690	18
Polar land, SIF	0.869	0.099	0.219	0.033	0.762	3
Nonpolar ocean, SIF	0.909	0.060	0.186	0.026	0.840	72
Polar ocean, SIF	0.883	0.098	0.278	0.015	0.816	10
Global, SIF	0.896	0.059	0.185	0.040	0.809	103
Global, SIC	0.849	0.135	0.186	0.035	0.596	38

Table 4. Differences between CERES SNPP VIIRS Ed1a and CALIPSO cloud top heights for all overcast (0/100) liquid clouds identified by CALIPSO, JAJO 2015 and 2016. Biases are computed as VIIRS—CALIPSO.

	Single Layer Only			All with Liquid Top
Day	Bias (SDD) [km]	R	Number of Matches × 10³	Bias (SDD) [km]	R	Number of Matches × 10³
Land, SIF	0.05 (0.98)	0.86	6.5	−0.41 (1.37)	0.77	14.2
Ocean, SIF	0.00 (0.74)	0.88	42.7	−0.38 (1.27)	0.77	62.4
SIC	0.09 (0.87)	0.85	11.5	−0.24 (1.24)	0.70	19.5
Global, All	0.02 (0.79)	0.88	60.7	−0.35 (1.28)	0.78	96.1
Night
Land, SIF	0.18 (0.94)	0.86	4.9	−0.17 (1.27)	0.79	8.0
Ocean, SIF	0.16 (0.71)	0.80	38.5	−0.10 (1.11)	0.73	53.2
SIC	0.21 (0.81)	0.76	6.3	0.00 (1.02)	0.68	9.3
Global, All	0.17 (0.75)	0.83	49.7	−0.10 (1.12)	0.76	70.5

Table 5. Differences, SNPP VIIRS Ed1 − CALIPSO, cloud top heights for non-opaque ice clouds, JAJO 2015 and 2016.

	Single Layer Ice Only			All with Ice Top
Day	Bias (SDD) [km]	R	Number of Matches × 10⁻³	Bias (SDD) [km]	R	Number of Matches × 10⁻³
Land, SIF	−0.56 (1.96)	0.59	2.6	−0.85 (2.14)	0.67	4.5
Ocean, SIF	−1.44 (2.21)	0.53	6.0	−1.54 (2.28)	0.62	11.1
SIC	−1.35 (2.30)	0.46	2.6	−1.30 (2.14)	0.51	5.6
Global, All	−1.22 (2.21)	0.69	11.2	−1.33 (2.23)	0.74	21.3
Night
Land, SIF	−0.27 (2.01)	0.65	4.8	−0.55 (2.25)	0.71	9.1
Ocean, SIF	−0.26 (1.84)	0.68	8.8	−0.69 (2.28)	0.71	25.0
SIC	−1.40 (2.17)	0.64	10.3	−0.97 (2.43)	0.44	22.4
Global, All	−0.75 (2.10)	0.73	23.9	−0.78 (2.34)	0.74	56.5

Table 6. Differences, SNPP VIIRS Ed1 − CALIPSO, cloud top heights for opaque ice clouds, JAJO 2015 and 2016.

	Single Layer Ice Only			All with Ice Top
Day	Bias (SDD) [km]	R	Number of Matches × 10⁻³	Bias (SDD) [km]	R	Number of Matches × 10⁻³
Land, SIF	−0.47 (1.25)	0.86	5.2	−0.89 (1.53)	0.84	8.8
Ocean, SIF	−0.93 (1.27)	0.86	16.1	−1.35 (1.61)	0.83	29.7
SIC	−0.74 (1.29)	0.71	1.6	−1.13 (1.61)	0.63	4.2
Global, All	−0.81 (1.28)	0.87	22.9	−1.24 (1.60)	0.84	42.7
Night
Land, SIF	−0.38 (1.42)	0.84	3.8	−1.01 (2.01)	0.75	7.5
Ocean, SIF	−0.37 (1.27)	0.86	12.8	−0.83 (1.77)	0.78	27.4
SIC	−0.83 (1.61)	0.62	5.4	−0.96 (1.89)	0.59	11.8
Global, All	−0.48 (1.40)	0.84	21.9	−0.89 (1.84)	0.78	46.7

Table 7. Balanced accuracy method metrics for CV1S detection computed for different surface types using collocated 1 km CALIOP Layers Product. Black: CV1S > ECM and MVCM; red: ECM > CV1S; blue: MVCM > CV1S; purple: ECM and MVCM > CV1S; bold: CV1S > NNCM.

0/100	Day				Night
	BACC	TPR	TNR	N × 10³	BACC	TPR	TNR	N × 10³
Global	0.953	0.970	0.937	297	0.878	0.902	0.853	343
Water	0.958	0.981	0.935	187	0.916	0.939	0.892	206
Land	0.940	0.941	0.939	110	0.825	0.819	0.831	137
Sea Ice	0.959	0.954	0.964	27	0.823	0.838	0.809	43
Perm. snow	0.908	0.908	0.908	32	0.760	0.703	0.817	58
Snow land	0.911	0.913	0.909	44	0.774	0.740	0.808	79
50/50	Day				Night
Global	0.887	0.919	0.855	396	0.824	0.850	0.798	470
Water	0.885	0.935	0.834	250	0.854	0.883	0.826	292
Land	0.878	0.881	0.876	145	0.775	0.776	0.774	178
Sea Ice	0.924	0.926	0.920	33	0.769	0.807	0.731	57
Perm. snow	0.869	0.871	0.867	39	0.713	0.663	0.763	74
Snow land	0.865	0.873	0.857	55	0.724	0.698	0.749	102

Table 8. Error summary for four CTH retrieval products relative to CALIOP VFM product: Bias (∆Z_T) and mean absolute error (MAE). Results for PPS, MOD C6, and HNN adapted from [12]. CV1S results based on 1 km JAJO 2015–2016 data, |∆t| ≤ 15 min.

	All (km)		Low (km)		Midlevel (km)		High (km)
Method	∆Z_T	MAE	∆Z_T	MAE	∆Z_T	MAE	∆Z_T	MAE
PPS	−1.47	2.09	0.31	0.85	−0.35	1.12	−3.46	3.56
MOD C6	−1.15	1.92	0.22	0.95	−0.71	1.76	−2.54	2.85
HNN	−0.41	1.19	0.39	0.53	0.22	0.83	−1.36	1.90
CV1S₁₀₀	−0.99	1.92	0.38	0.80	−0.10	1.58	−2.17	2.75
CV1S₅₀	−0.97	1.95	0.43	0.86	−0.09	1.62	−2.23	2.82

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Yost, C.R.; Minnis, P.; Sun-Mack, S.; Smith, W.L., Jr.; Trepte, Q.Z. VIIRS Edition 1 Cloud Properties for CERES, Part 2: Evaluation with CALIPSO. Remote Sens. 2023, 15, 1349. https://doi.org/10.3390/rs15051349

AMA Style

Yost CR, Minnis P, Sun-Mack S, Smith WL Jr., Trepte QZ. VIIRS Edition 1 Cloud Properties for CERES, Part 2: Evaluation with CALIPSO. Remote Sensing. 2023; 15(5):1349. https://doi.org/10.3390/rs15051349

Chicago/Turabian Style

Yost, Christopher R., Patrick Minnis, Sunny Sun-Mack, William L. Smith, Jr., and Qing Z. Trepte. 2023. "VIIRS Edition 1 Cloud Properties for CERES, Part 2: Evaluation with CALIPSO" Remote Sensing 15, no. 5: 1349. https://doi.org/10.3390/rs15051349

APA Style

Yost, C. R., Minnis, P., Sun-Mack, S., Smith, W. L., Jr., & Trepte, Q. Z. (2023). VIIRS Edition 1 Cloud Properties for CERES, Part 2: Evaluation with CALIPSO. Remote Sensing, 15(5), 1349. https://doi.org/10.3390/rs15051349

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu