Plum Hall is the world leader in making compiler validation suites and software for the programmer and software development community.
The 24a release of the Plum Hall Test Suites is available now. This release contains infrastructure improvements, bug fixes and preliminary tests for the proposed C++26 / C26 standards.
C++ tests may now be checked as conforming to 16.4.2.5 Freestanding as opposed to hosted implementations. as well as C++ freestanding tests as denoted in the document ISO/IEC DIS 14882:2023. C++ also now adds tests for undefined behavior, which were previously only available for the C language. The tests are in conform/undeftests/u*.in. Software, especially FREESTANDING, should not contain any constructs which have undefined behavior as per the standards. ISO 26262 in particular has a focus on dealing with undefined or unspecified behavior of C/C++ and on preventing runtime errors. Obeying language standards is recommended by all current safety standards.
This release also adds new test cases for modules and coroutines in the directories t01a.dir and t01b.dir respectively. There are 37 new test cases, documented in "newcases-xvs23a-xvs24a.txt", in multiple directories. These new test cases predominantly pertain to the C++23 and proposed C++26 standards. This version also adds support for CUDA files. These cases must be compiled on CUDA-enabled hardware. See t00a.dir for examples. Initial support for modules tests can be found in t01a.dir. Directory t01b.dir contains new tests for coroutines.
There are 113 new LVS test cases, documented in "newcases-lvs23a-lvs24a.txt", in multiple directories. These new test cases predominantly pertain to the C++23 and proposed C++26 standards. This release also contains initial support for testing freestanding C++ in t161.dir/. Directory t01a.dir/ contains some tests for modules, though more detailed testing is found in xvs24a.
The C++ suite also now adds tests for undefined behavior, which were previously only available for the C language. The tests are in conform/undeftests/u*.in. Software, especially FREESTANDING, should not contain any constructs which have undefined behavior as per the standards. ISO 26262 in particular has a focus on dealing with undefined/unspecified behavior of C/C++ and on preventing runtime errors. Obeying language standards is recommended by all current safety standards.
There are 30 new test cases documented in "newcases-cvs23a-cvs24a.txt", in multiple directories. These new test cases predominantly pertain to the C26 proposed standards. The new test case names are prefixed by c2x_ and c2y_. The 24a release represents 3+ years of test case bug fixing, infrastructure improvements, and new test cases for C17, C20, C23, C26, language and library enhancements. There are many improvements in enhancing the test cases themselves, scripting and enhancing the reporting of the results, through the new html interfaces for reporting coverage, commentary on the intent of the test cases and improved standards conformance reporting.
The file compiler-flags.h contains defines for common compilers. Modify these settings if your compiler is implemented in the list, or add custom settings for your compiler. Also review envsuite in detail. Some suggested settings are included for common compilers. Use these settings to create the appropriate build environment for the version of the compiler that you wish to test.
It is very important that you review envsuite(.bat), flags.h and compiler-flags.h to choose the correct settings for your compiler and the standards version you wish to test against.