Modeling Bland–Altman Limits of Agreement with Fractional Polynomials—An Example with the Agatston Score for Coronary Calcification
<p><b>Top left</b><span class="html-italic">:</span> Bland–Altman plot for inter-rater agreement analysis of left atrium area (cm<sup>2</sup>) measurements in non-contrast computed tomography, n = 140 (reproduced with permission from [<a href="#B20-axioms-12-00884" class="html-bibr">20</a>]). <b>Top right</b><span class="html-italic">:</span> Bland–Altman plot of glomerular filtration rate measured with 51Cr-ethylenediamine tetraacetic acid (EDTA) and 99mTc-diethylenetriamine pentaacetic acid (DTPA), n = 51 (reproduced with permission from [<a href="#B21-axioms-12-00884" class="html-bibr">21</a>]). <b>Bottom left</b><span class="html-italic">:</span> Intra-rater agreement assessment for 2D measurements (cm<sup>2</sup>) for raters 1, 2, 3, and 4, respectively; n = 48 (reproduced with permission from [<a href="#B22-axioms-12-00884" class="html-bibr">22</a>]). <b>Bottom right</b><span class="html-italic">:</span> Distribution of differences between repeated measurements of coronary artery calcium (CAC) as function of average CAC score expressed in Agatston CAC score units; the curve shows 95% repeatability limits which include 98% of differences, n = 2217 (reprinted from ‘Serial electron beam CT measurements of coronary artery calcium: Has your patient’s calcium score actually changed?’, A.B. Sevrukov, J.M. Bland, and G.T. Kondos, the <span class="html-italic">American Journal of Roentgenology</span> 185, Copyright© 2023, copyright owner as specified in the <span class="html-italic">American Journal of Roentgenology</span> [<a href="#B18-axioms-12-00884" class="html-bibr">18</a>]).</p> "> Figure 2
<p>Mean difference plot for inter-rater variation analysis reported in [<a href="#B19-axioms-12-00884" class="html-bibr">19</a>].</p> "> Figure 3
<p>Bland–Altman limits of agreement (<b>left</b>) and nonparametric limits of agreement (<b>right</b>) reported in [<a href="#B19-axioms-12-00884" class="html-bibr">19</a>].</p> "> Figure 4
<p>Examples of some functional forms available with degree-1, degree-2, and degree-3 fractional polynomials with various powers (<span class="html-italic">p<sub>1</sub></span>), (<span class="html-italic">p<sub>1</sub></span>,<span class="html-italic">p<sub>2</sub></span>), and (<span class="html-italic">p<sub>1</sub></span>,<span class="html-italic">p<sub>2</sub></span>,<span class="html-italic">p<sub>3</sub></span>), respectively.</p> "> Figure 5
<p>Stata output for <span class="html-italic">m</span> = 2.</p> "> Figure 6
<p>Limits of agreement based on fractional polynomial regression models. <b>Top left</b>: <math display="inline"><semantics> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mo> </mo> <mi>S</mi> <mo>=</mo> <mfenced close="}" open="{"> <mn>1</mn> </mfenced> </mrow> </semantics></math>, coverage: 84.78%. <b>Top right</b>: <math display="inline"><semantics> <mrow> <mi>m</mi> <mo>=</mo> <mn>2</mn> <mo>,</mo> <mo> </mo> <mi>S</mi> <mo>=</mo> <mfenced close="}" open="{"> <mrow> <mn>0.5</mn> <mo>,</mo> <mn>2</mn> </mrow> </mfenced> </mrow> </semantics></math>, coverage: 91.30%. <b>Bottom left</b>: <math display="inline"><semantics> <mrow> <mi>m</mi> <mo>=</mo> <mn>3</mn> <mo>,</mo> <mi>S</mi> <mo>=</mo> <mfenced close="}" open="{"> <mrow> <mn>0.5</mn> <mo>,</mo> <mn>3</mn> <mo>,</mo> <mn>3</mn> </mrow> </mfenced> </mrow> </semantics></math>, coverage: 91.74%. <b>Bottom right</b><span class="html-italic">:</span> <math display="inline"><semantics> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> <mo>,</mo> <mi>S</mi> <mo>=</mo> <mfenced close="}" open="{"> <mrow> <mn>0.5</mn> </mrow> </mfenced> </mrow> </semantics></math>, coverage: 95.22%.</p> ">
Abstract
:1. Introduction
2. Data
3. Bland–Altman Limits of Agreement and Previously Reported Nonparametric Limits of Agreement
4. Regression of Non-Uniform Differences on the Averages
5. Fractional Polynomials
6. Analysis Strategy of Sevrukov, Bland, and Kondos
7. Reanalysis of Inter-Rater Agreement Reported in [19]
7.1. Degree-2 Fractional Polynomial Models
7.2. Degree-3 Fractional Polynomial Models
7.3. Sevrukov, Bland, and Kondos Model
8. Discussion
8.1. Main Findings
8.2. How Good Is Good Enough?
- The coverage of the observed differences should be roughly around 95%, in line with classical Bland–Altman limits of agreement.
- The limits of agreement fit the data nicely and harmonize with the point cloud of paired differences.
8.3. Strengths and Limitations
8.4. Future Perspectives
9. Concluding Remarks
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Altman, D.G.; Bland, J.M. Measurement in Medicine: The Analysis of Method Comparison Studies. J. R. Stat. Soc. Ser. D Stat. 1983, 32, 307–317. [Google Scholar] [CrossRef]
- 68-95-99.7 Rule. Available online: https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule (accessed on 22 June 2023).
- Bland, J.M.; Altman, D.G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, 1, 307–310. [Google Scholar] [CrossRef] [PubMed]
- Bland, J.M.; Altman, D.G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999, 8, 135–160. [Google Scholar] [CrossRef] [PubMed]
- Ludbrook, J. Confidence in Altman-Bland plots: A critical review of the method of differences. Clin. Exp. Pharmacol. Physiol. 2010, 37, 143–149. [Google Scholar] [CrossRef] [PubMed]
- Bland, J.M.; Altman, D.G. Agreement between methods of measurement with multiple observations per individual. J. Biopharm. Stat. 2007, 17, 571–582. [Google Scholar] [CrossRef] [PubMed]
- Olofsen, E.; Dahan, A.; Borsboom, G.; Drummond, G. Improvements in the application and reporting of advanced Bland-Altman methods of comparison. J. Clin. Monit. Comput. 2015, 29, 127–139. [Google Scholar] [CrossRef] [PubMed]
- Webpage for Bland-Altman Analysis. Available online: https://sec.lumc.nl/method_agreement_analysis/ (accessed on 22 June 2023).
- Carkeet, A. Exact parametric confidence intervals for Bland-Altman limits of agreement. Optom. Vis. Sci. 2015, 92, e71–e80. [Google Scholar] [CrossRef] [PubMed]
- Jan, S.L.; Shieh, G. The Bland-Altman range of agreement: Exact interval procedure and sample size determination. Comput. Biol. Med. 2018, 100, 247–252. [Google Scholar] [CrossRef] [PubMed]
- Shieh, G. The appropriateness of Bland-Altman’s approximate confidence intervals for limits of agreement. BMC Med. Res. Methodol. 2018, 18, 45. [Google Scholar] [CrossRef] [PubMed]
- Shieh, G. Assessing Agreement Between Two Methods of Quantitative Measurements: Exact Test Procedure and Sample Size Calculation. Stat. Biopharm. Res. 2020, 12, 352–359. [Google Scholar] [CrossRef]
- Taffé, P. Effective plots to assess bias and precision in method comparison studies. Stat. Methods Med. Res. 2018, 27, 1650–1660. [Google Scholar] [CrossRef] [PubMed]
- Taffé, P.; Peng, M.; Stagg, V.; Williamson, T. MethodCompare: An R package to assess bias and precision in method comparison studies. Stat. Methods Med. Res. 2019, 28, 2557–2565. [Google Scholar] [CrossRef]
- Taffé, P. Assessing bias, precision, and agreement in method comparison studies. Stat. Methods Med. Res. 2020, 29, 778–796. [Google Scholar] [CrossRef] [PubMed]
- Taffé, P. When can the Bland & Altman limits of agreement method be used and when it should not be used. J. Clin. Epidemiol. 2021, 137, 176–181. [Google Scholar] [CrossRef]
- Gerke, O.; Pedersen, A.K.; Debrabant, B.; Halekoh, U.; Möller, S. Sample size determination in method comparison and observer variability studies. J. Clin. Monit. Comput. 2022, 36, 1241–1243. [Google Scholar] [CrossRef] [PubMed]
- Sevrukov, A.B.; Bland, J.M.; Kondos, G.T. Serial electron beam CT measurements of coronary artery calcium: Has your patient’s calcium score actually changed? Am. J. Roentgenol. 2005, 185, 1546–1553. [Google Scholar] [CrossRef] [PubMed]
- Gerke, O.; Lindholt, J.S.; Abdo, B.H.; Lambrechtsen, J.; Frost, L.; Steffensen, F.H.; Karon, M.; Egstrup, K.; Urbonaviciene, G.; Busk, M.; et al. Prevalence and extent of coronary artery calcification in the middle-aged and elderly population. Eur. J. Prev. Cardiol. 2021, 28, 2048–2055. [Google Scholar] [CrossRef] [PubMed]
- Gerke, O. Reporting Standards for a Bland-Altman Agreement Analysis: A Review of Methodological Reviews. Diagnostics 2020, 10, 334. [Google Scholar] [CrossRef]
- Simonsen, J.A.; Thilsing-Hansen, K.; Høilund-Carlsen, P.F.; Gerke, O.; Andersen, T.L. Glomerular filtration rate: Comparison of simultaneous plasma clearance of 99mTc-DTPA and 51Cr-EDTA revisited. Scand. J. Clin. Lab. Investig. 2020, 80, 408–411. [Google Scholar] [CrossRef] [PubMed]
- Jørgensen, L.B.; Skov-Jeppesen, S.M.; Halekoh, U.; Rasmussen, B.S.; Sørensen, J.A.; Jemec, G.B.E.; Yderstraede, K.B. Validation of three-dimensional wound measurements using a novel 3D-WAM camera. Wound Repair Regen. 2018, 26, 456–462. [Google Scholar] [CrossRef] [PubMed]
- Diederichsen, A.C.; Rasmussen, L.M.; Søgaard, R.; Lambrechtsen, J.; Steffensen, F.H.; Frost, L.; Egstrup, K.; Urbonaviciene, G.; Busk, M.; Olsen, M.H.; et al. The Danish Cardiovascular Screening Trial (DANCAVAS): Study protocol for a randomized controlled trial. Trials 2015, 16, 554. [Google Scholar] [CrossRef] [PubMed]
- Diederichsen, A.C.; Sand, N.P.; Nørgaard, B.; Lambrechtsen, J.; Jensen, J.M.; Munkholm, H.; Aziz, A.; Gerke, O.; Egstrup, K.; Larsen, M.L.; et al. Discrepancy between coronary artery calcium score and HeartScore in middle-aged Danes: The DanRisk study. Eur. J. Prev. Cardiol. 2012, 19, 558–564. [Google Scholar] [CrossRef] [PubMed]
- Alashi, A.; Lang, R.; Seballos, R.; Feinleib, S.; Sukol, R.; Cho, L.; Schoenhagen, P.; Griffin, B.P.; Flamm, S.D.; Desai, M.Y. Reclassification of coronary heart disease risk in a primary prevention setting: Traditional risk factor assessment vs. coronary artery calcium scoring. Cardiovasc. Diagn. Ther. 2019, 9, 214–220. [Google Scholar] [CrossRef] [PubMed]
- Yeboah, J.; McClelland, R.L.; Polonsky, T.S.; Burke, G.L.; Sibley, C.T.; O’Leary, D.; Carr, J.J.; Goff, D.C.; Greenland, P.; Herrington, D.M. Comparison of novel risk markers for improvement in cardiovascular risk assessment in intermediate-risk individuals. JAMA 2012, 308, 788–795. [Google Scholar] [CrossRef] [PubMed]
- Erbel, R.; Möhlenkamp, S.; Moebus, S.; Schmermund, A.; Lehmann, N.; Stang, A.; Dragano, N.; Grönemeyer, D.; Seibel, R.; Kälsch, H.; et al. Coronary risk stratification, discrimination, and reclassification improvement based on quantification of subclinical coronary atherosclerosis: The Heinz Nixdorf Recall study. J. Am. Coll. Cardiol. 2010, 56, 1397–1406. [Google Scholar] [CrossRef] [PubMed]
- Polonsky, T.S.; McClelland, R.L.; Jorgensen, N.W.; Bild, D.E.; Burke, G.L.; Guerci, A.D.; Greenland, P. Coronary artery calcium score and risk classification for coronary heart disease prediction. JAMA 2010, 303, 1610–1616. [Google Scholar] [CrossRef] [PubMed]
- Folsom, A.R.; Kronmal, R.A.; Detrano, R.C.; O’Leary, D.H.; Bild, D.E.; Bluemke, D.A.; Budoff, M.J.; Liu, K.; Shea, S.; Szklo, M.; et al. Coronary artery calcification compared with carotid intima-media thickness in the prediction of cardiovascular disease incidence: The Multi-Ethnic Study of Atherosclerosis (MESA). Arch. Intern. Med. 2008, 168, 1333–1339. [Google Scholar] [CrossRef]
- Agatston, A.S.; Janowitz, W.R.; Hildner, F.J.; Zusmer, N.R.; Viamonte, M., Jr.; Detrano, R. Quantification of coronary artery calcium using ultrafast computed tomography. J. Am. Coll. Cardiol. 1990, 15, 827–832. [Google Scholar] [CrossRef]
- Andersen, K.P.; Gerke, O. Assessing Agreement When Agreement Is Hard to Assess-The Agatston Score for Coronary Calcification. Diagnostics 2022, 12, 2993. [Google Scholar] [CrossRef]
- Bartlett, M.S. The use of transformations. Biometrics 1947, 3, 39–52. [Google Scholar] [CrossRef] [PubMed]
- Tukey, J.W. On the Comparative Anatomy of Transformations. Ann. Math. Statist. 1957, 28, 602–632. [Google Scholar] [CrossRef]
- Gerke, O. Nonparametric Limits of Agreement in Method Comparison Studies: A Simulation Study on Extreme Quantile Estimation. Int. J. Environ. Res. Public Health 2020, 17, 8330. [Google Scholar] [CrossRef]
- Altman, D.G. Construction of age-related reference centiles using absolute residuals. Stat. Med. 1993, 12, 917–924. [Google Scholar] [CrossRef] [PubMed]
- Johnson, N.L.; Kotz, S. Continuous Univariate Distributions; Wiley: New York, NY, USA, 1970; Volume 1, p. 81. [Google Scholar]
- Royston, P.; Altman, D.G. Regression Using Fractional Polynomials of Continuous Covariates: Parsimonious Parametric Modelling. J. Roy. Stat. Soc. Ser. C (Appl. Stat.) 1994, 43, 429–467. [Google Scholar] [CrossRef]
- FP—Fractional Polynomial Regression. Available online: https://www.stata.com/manuals/rfp.pdf (accessed on 22 June 2023).
- Sauerbrei, W.; Meier-Hirmer, C.; Benner, A.; Royston, P. Multivariable regression model building by using fractional polynomials: Description of SAS, STATA and R programs. Comput. Stat. Data Anal. 2006, 50, 3464–3485. [Google Scholar] [CrossRef]
- Royston, P. Model selection for univariable fractional polynomials. Stata J. 2017, 17, 619–629. [Google Scholar] [CrossRef] [PubMed]
Minimum | P5 1 | P25 | Median | P75 | P95 | Maximum |
---|---|---|---|---|---|---|
−538 | −42 | −0.3 | 0 | 0.4 | 15 | 740 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gerke, O.; Möller, S. Modeling Bland–Altman Limits of Agreement with Fractional Polynomials—An Example with the Agatston Score for Coronary Calcification. Axioms 2023, 12, 884. https://doi.org/10.3390/axioms12090884
Gerke O, Möller S. Modeling Bland–Altman Limits of Agreement with Fractional Polynomials—An Example with the Agatston Score for Coronary Calcification. Axioms. 2023; 12(9):884. https://doi.org/10.3390/axioms12090884
Chicago/Turabian StyleGerke, Oke, and Sören Möller. 2023. "Modeling Bland–Altman Limits of Agreement with Fractional Polynomials—An Example with the Agatston Score for Coronary Calcification" Axioms 12, no. 9: 884. https://doi.org/10.3390/axioms12090884