research-article

Many-core compiler fuzzing

Authors:

Christopher Lidbury,

Alastair F. DonaldsonAuthors Info & Claims

ACM SIGPLAN Notices, Volume 50, Issue 6

Pages 65 - 76

https://doi.org/10.1145/2813885.2737986

Published: 03 June 2015 Publication History

Abstract

We address the compiler correctness problem for many-core systems through novel applications of fuzz testing to OpenCL compilers. Focusing on two methods from prior work, random differential testing and testing via equivalence modulo inputs (EMI), we present several strategies for random generation of deterministic, communicating OpenCL kernels, and an injection mechanism that allows EMI testing to be applied to kernels that otherwise exhibit little or no dynamically-dead code. We use these methods to conduct a large, controlled testing campaign with respect to 21 OpenCL (device, compiler) configurations, covering a range of CPU, GPU, accelerator, FPGA and emulator implementations. Our study provides independent validation of claims in prior work related to the effectiveness of random differential testing and EMI testing, proposes novel methods for lifting these techniques to the many-core setting and reveals a significant number of OpenCL compiler bugs in commercial implementations.

References

[1]

A. Aho, M. Lam, R. Sethi, and J. Ullman. Compilers: Principles, Techniques, & Tools. Pearson Education, 2nd edition, 2006.

Digital Library

[2]

E. Bardsley, A. Betts, N. Chong, P. Collingbourne, P. Deligiannis, A. F. Donaldson, J. Ketema, D. Liew, and S. Qadeer. Engineering a static verification tool for GPU kernels. In CAV, pages 226–242, 2014.

Digital Library

[3]

S. Che, M. Boyer, J. Meng, D. Tarjan, J. W. Sheaffer, S. Lee, and K. Skadron. Rodinia: A benchmark suite for heterogeneous computing. In IISWC, pages 44–54, 2009.

Digital Library

[4]

T. Chen, T. Tse, and Z. Zhou. Fault-based testing without the need of oracles. Information and Software Technology, 45(1):1–9, 2003.

Digital Library

[5]

Y. Chen, A. Groce, C. Zhang, W. Wong, X. Fern, E. Eide, and J. Regehr. Taming compiler fuzzers. In PLDI, pages 197–208, 2013.

Digital Library

[6]

N. Chong, A. F. Donaldson, and J. Ketema. A sound and complete abstraction for reasoning about parallel prefix sums. In POPL, pages 397–410, 2014.

Digital Library

[7]

E. Eide and J. Regehr. Volatiles are miscompiled, and what to do about it. In EMSOFT, pages 255–264, 2008.

Digital Library

[8]

C. Holler, K. Herzig, and A. Zeller. Fuzzing with code fragments. In USENIX Security Symposium, pages 445–458, 2012.

Digital Library

[9]

ISO. ISO/IEC 9899:TC2: Programming Languages–C, 1999.

[10]

Khronos. The OpenCL specification, versions 1.0, 1.1, 1.2 and 2.0, 2009–2014.

[11]

Khronos. The OpenCL specification, version 1.2, 2012. Document revision 19.

[12]

V. Le, M. Afshari, and Z. Su. Compiler validation via equivalence modulo inputs. In PLDI, 2014. Article 25.

Digital Library

[13]

W. M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100–107, 1998.

[14]

R. Morisset, P. Pawan, and F. Zappa Nardelli. Compiler testing via a theory of sound optimisations in the C11/C++11 memory model. In PLDI, pages 187–196, 2013.

Digital Library

[15]

J. Price and S. McIntosh-Smith. Oclgrind: An extensible OpenCL device simulator. In IWOCL, 2015. To appear.

Digital Library

[16]

J. Regehr, Y. Chen, P. Cuoq, E. Eide, C. Ellison, and X. Yang. Testcase reduction for C compiler bugs. In PLDI, pages 335–346, 2012.

Digital Library

[17]

F. Sheridan. Practical testing of a C99 compiler using output comparison. Software — Practice and Experience, 2007.

Digital Library

[18]

J. A. Stratton, C. Rodrigues, I. J. Sung, N. Obeid, L. W. Chang, N. Anssari, G. D. Liu, and W. W. Hwu. Parboil: A revised benchmark suite for scientific and commercial throughput computing. Technical Report IMPACT-12-01, UIUC, March 2012.

[19]

Q. Tao, W. Wu, C. Zhao, and W. Shen. An automatic testing approach for compiler based on metamorphic testing technique. In APSEC, pages 270–279, 2010.

Digital Library

[20]

X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in C compilers. In PLDI, pages 283–294, 2011.

Digital Library

[21]

C. Zhao, Y. Xue, Q. Tao, L. Guo, and Z. Wang. Automated test program generation for an industrial optimizing compiler. In AST, pages 36–43, 2009.

Cited By

Jiang HZou PLi XZhou ZZhao XZhang YGuo S(2024)DeLoSo: Detecting Logic Synthesis Optimization Faults Based on Configuration DiversityACM Transactions on Design Automation of Electronic Systems10.1145/370123230:1(1-26)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701232
Wu SHu YWang YGu JMeng JFan LLuan ZWang XZhou Yd'Amorim M(2024)Combating Missed Recalls in E-commerce Search: A CoT-Prompting Testing ApproachCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663842(220-231)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663842
Zhu XJiang MZhang XNie LDing Z(2024)MTAS: A Reference-Free Approach for Evaluating Abstractive Summarization SystemsProceedings of the ACM on Software Engineering10.1145/36608201:FSE(2561-2583)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660820
Show More Cited By

Index Terms

Many-core compiler fuzzing
1. Computing methodologies
  1. Concurrent computing methodologies
    1. Concurrent programming languages
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Compilers
    2. General programming languages
      1. Language types
        Concurrent programming languages

Recommendations

Many-core compiler fuzzing
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation

We address the compiler correctness problem for many-core systems through novel applications of fuzz testing to OpenCL compilers. Focusing on two methods from prior work, random differential testing and testing via equivalence modulo inputs (EMI), we ...
Accelerating the SRP-PHAT algorithm on multi- and many-core platforms using OpenCL

The Steered Response Power with Phase Transform (SRP-PHAT) algorithm is a well-known method for sound source localization due to its robust performance in noisy and reverberant environments. This algorithm is used in a large number of acoustic ...
ParTeCL: parallel testing using OpenCL
ISSTA 2017: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis

With the growing complexity of software, the number of test cases needed for effective validation is extremely large. Executing these large test suites is expensive and time consuming, putting an enormous pressure on the software development cycle. In ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 50, Issue 6

PLDI '15

June 2015

630 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/2813885

Editor:
Andy Gill
University of Kansas, Lawrence, KS

Issue’s Table of Contents

PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation
June 2015
630 pages
ISBN:9781450334686
DOI:10.1145/2737924
General Chair:
David Grove
IBM Research, USA
,
Program Chair:
Steve Blackburn
Australian National University, Australia

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2015

Published in SIGPLAN Volume 50, Issue 6

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

138
Total Citations
View Citations
1,014
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)4

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang HZou PLi XZhou ZZhao XZhang YGuo S(2024)DeLoSo: Detecting Logic Synthesis Optimization Faults Based on Configuration DiversityACM Transactions on Design Automation of Electronic Systems10.1145/370123230:1(1-26)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701232
Wu SHu YWang YGu JMeng JFan LLuan ZWang XZhou Yd'Amorim M(2024)Combating Missed Recalls in E-commerce Search: A CoT-Prompting Testing ApproachCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663842(220-231)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663842
Zhu XJiang MZhang XNie LDing Z(2024)MTAS: A Reference-Free Approach for Evaluating Abstractive Summarization SystemsProceedings of the ACM on Software Engineering10.1145/36608201:FSE(2561-2583)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660820
Li STheodoridis TSu Z(2024)Boosting Compiler Testing by Injecting Real-World CodeProceedings of the ACM on Programming Languages10.1145/36563868:PLDI(223-245)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3656386
Suo CChen JLiu SJiang JZhao YWang JChristakis MPradel M(2024)Fuzzing MLIR Compiler Infrastructure via Operation Dependency AnalysisProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680360(1287-1299)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680360
Xia XFeng YShi QJones JZhang XXu B(2024)Enumerating Valid Non-Alpha-Equivalent Programs for Interpreter TestingACM Transactions on Software Engineering and Methodology10.1145/364799433:5(1-31)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3647994
Xia CPaltenghi MLe Tian JPradel MZhang LRoychoudhury APaiva AAbreu RStorey M(2024)Fuzz4All: Universal Fuzzing with Large Language ModelsProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639121(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639121
Jiang MZheng XChang RZhou YLuo X(2024)Examiner-Pro: Testing Arm Emulators Across Different PrivilegesIEEE Transactions on Software Engineering10.1109/TSE.2024.340690050:11(2786-2806)Online publication date: Nov-2024
https://doi.org/10.1109/TSE.2024.3406900
Jiang HWang ZZhou ZLi XGuo SSun WZhang T(2024)A Testing Program and Pragma Combination Selection Based Framework for High-Level Synthesis Tool Pragma-Related Bug DetectionIEEE Transactions on Software Engineering10.1109/TSE.2024.336855350:4(937-955)Online publication date: 22-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3368553
Li XGuo SCheng HJiang H(2024)Simulink Compiler Testing via Configuration Diversification With Reinforcement LearningIEEE Transactions on Reliability10.1109/TR.2023.331764373:2(1060-1074)Online publication date: Jun-2024
https://doi.org/10.1109/TR.2023.3317643
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents