ReFrame 4.0.2

Release Notes

New features and enhancements

n/a

Bug fixes

• Add MANIFEST.in to include the README_minimal.md (#2731)

Test library

n/a

Other

• Improve documentation for the RFM_CONFIG_PATH (#2743)

ReFrame 4.0.1

Release Notes

Bug fixes

• Fix perflog_compat crash when performance test fails in sanity (#2718)

ReFrame 4.0.0

Release Notes

Changes since version 3.12.0.

New features and enhancements

• Remove the --strict option (#2711)
• Remove the --force-local option (#2710)
• Control verbosity level of the immediate performance info (#2695)
• Remove the schedulers section from the configuration (#2694)
• Remove deprecated --ignore-check-conflicts CLI option and the associated configuration option (#2706)
• Update temp_environment to use env_vars as an argument (#2707)
• Improve performance logging (#2633)
• Print better error message when pygelf cannot be imported (#2691)
• Build configuration incrementally by allowing to combine multiple configuration files (#2557)
• Relax the value syntax of env_vars (#2649)
• Deprecate variables in favour of the new env_vars (#2648)
• Introduce variable aliases (#2647)
• Change default value of num_gpus_per_node to None (#2638)
• Extend functionality to get the Cray's CDT version to support systems based on the CPE module (#2644)
• Allow tests to require references (#2627)
• Remove the deprecated RFM_GRAYLOG_SERVER environment variable (#2624)
• Improve framework messages and the performance report (#2618)
• Add perflog_ignore attribute in httpjson handler (#2620)
• Allow user defined launchers in the configuration (#2605)
• Add support for the Apptainer container platform (#2613)
• Add support for the Flux scheduler (#2598)
• Stop execution with an error when a cli module fails to load (#2603)
• Revert to generating unique run reports (#2601)
• Change default hostname autodetection parameters (#2583)
• Associate a unique hash with each test (#2578)
• Add command line option to set the execution order of independent tests (#2575)
• Drop the old test naming scheme (#2574)
• Introduce ReFrame 4.0 (#2562)
• Use the test's extra_resources also for the build job (#2547)
• Remove deprecated features for 4.x (#2096)
• Remove deprecated features from 4.x (#1986)
• Drop the @parameterized_test decorator (#1951)

Bug fixes

• Fix cleanup crash for non-performance tests when the perflog_compat option is on (#2716)
• Fix hash clashes for parameterised tests (#2714)
• Restrict reference tuples acceptable types (#2702)
• Properly set display name of tests with join fixtures (#2705)
• Do not require handlers$ in the logging config (#2692)
• Fix nodelist compaction for single digitless nodes (#2689)
• Fix filtering of tests with hashcode (#2688)
• Fix variable assignment when aliases are set in subclasses (#2683)
• Fix injection of env variables for non default craype (#2682)
• Pass explicitly the --cpus-per-task option to srun for Slurm >= 22.05 (#2656)
• Expand the Slurm nodelist using scontrol and not the builtin nodelist_expand() function (#2678)
• Keep updating the node list during the job (#2634)
• Filter out unnecessary fixtures when listing or running tests (#2641)
• Submit jobs using the correct module path when testing scheduler backends (#2611)
• Document missing log record attributes (#2586)
• Log correctly %(check_perfvalues)s attribute (#2588)
• Fix singularity cli option to change to the mounted stage directory with the Singularity container platform (#2581)

Test library

• Use the builtin validation option of OSU benchmarks to validate the results (#2679)

Other

• Use an Ubuntu 20.04 image for Python 3.6 tests (#2712)
• Add support for Python 3.11 and update CI workflow for Python 3.6 (#2703)
• Revert workaround in script for generating release notes (#2680)
• Fix script that auto-generates the release notes (#2594)
• Add a "What's New" page for ReFrame 4.0 (#2709)
• Document the rest of Job's properties (#2713)
• Update link in codecov badge (#2708)
• Fix example in sn.print() docs (#2685)
• Update copyright notice in docs (#2684)
• Remove mailing list from README (#2643)
• Adapt README for dark and light Github modes (#2608)
• Fix typo on the docs (#2599)
• Fix broken links in the docs (#2593)
• Fix documentation of modules test attribute (#2592)
• Add a warning in documentation about the usage of perf_patterns (#2587)
• Add CSCS webinar resources + links (#2582)
• Fix example of httpjson handler usage (#2563)
• Add note about test dependencies in --repeat option docs (#2564)

ReFrame 4.0.0-dev.4

Release Notes

New features and enhancements

• Build configuration incrementally by allowing to combine multiple configuration files (#2557)

Bug fixes

• Fix filtering of tests with hashcode (#2688)
• Fix variable assignment when aliases are set in subclasses (#2683)
• Fix injection of env variables for non default craype (#2682)

Test library

• Use the builtin validation option of OSU benchmarks to validate the results (#2679)

Other

• Revert workaround in script for generating release notes (#2680)
• Fix example in sn.print() docs (#2685)
• Update copyright notice in docs (#2684)

ReFrame 4.0.0-dev.3

Release Notes

New features and enhancements

• Relax the value syntax of env_vars (#2649)
• Deprecate variables in favour of the new env_vars (#2648)

Bug fixes

• Pass explicitly the --cpus-per-task option to srun for Slurm >= 22.05 (#2656)
• Expand the Slurm nodelist using scontrol and not the builtin nodelist_expand() function (#2678)
• Properly capture the node list of a job in the Slurm backend (#2634)

ReFrame 4.0.0-dev.2

Release Notes

New features and enhancements

• Introduce variable aliases (#2647)
• Change default value of num_gpus_per_node to None (#2638)
• Extend functionality to get the Cray's CDT version to support systems based on the CPE module (#2644)
• Allow tests to require references (#2627)
• Remove the deprecated RFM_GRAYLOG_SERVER environment variable (#2624)
• Improve framework messages and the performance report (#2618)

Bug fixes

• Filter out unnecessary fixtures when listing or running tests (#2641)

Test library

<n/a>

Other

• Remove mailing list from README (#2643)
• Adapt README for dark and light Github modes (#2608)

ReFrame 4.0.0-dev.1

Release Notes

New features and enhancements

• Add perflog_ignore attribute in httpjson handler (#2620)
• Allow user defined launchers in the configuration (#2605)
• Add support for the Apptainer container platform (#2613)
• Add support for the Flux scheduler (#2598)
• Stop execution with an error when a cli module fails to load (#2603)
• Revert to generating unique run reports (#2601)

Bug fixes

• Submit jobs using the correct module path when testing scheduler backends (#2611)

Test library

n/a

Other

• Fix script that auto-generates the release notes (#2594)
• Fix typo on the docs (#2599)
• Fix broken links in the docs (#2593)

ReFrame 4.0.0-dev.0

Release Notes

New features and enhancements

• Change default hostname autodetection parameters (#2583)
• Associate a unique hash with each test (#2578)
• Add command line option to set the execution order of independent tests (#2575)
• Drop the old test naming scheme (#2574)
• Introduce ReFrame 4.0 (#2562)
• Use the test's extra_resources also for the build job (#2547)
• Remove deprecated features for 4.x (#1986, #2096)
• Drop the @parameterized_test decorator (#1951)

Bug fixes

• Document missing log record attributes (#2586)
• Log correctly %(check_perfvalues)s attribute (#2588)
• Fix singularity cli option to change to the mounted stage directory with the Singularity container platform (#2581)

Test library

n/a

Other

• Fix documentation of modules test attribute (#2592)
• Add a warning in documentation about the usage of perf_patterns (#2587)
• Add CSCS webinar resources + links (#2582)
• Fix example of httpjson handler usage (#2563)
• Add note about test dependencies in --repeat option docs (#2564)

ReFrame 3.12.0

Release Notes

Changes since 3.11.0

New features and enhancements

• Add a lightweight time profiler in the framework (#2545)
• Optimise the test case generation (#2544)
• Include the loggable variables and parameters in the run report (#2543)
• Add new command line option to repeat selected tests (#2539)
• Do not require to set container_platform for running containerised tests (#2537)
• Add --mode option to GitLab CI pipeline command (#2514)

Bug fixes

• Treat undefined variables correctly in run report (#2546)
• Snooze between polls in serial policy (#2538)
• Improve test selection by their variant id (#2533)
• Reset poll rate every time a test finishes the compilation or run phase (#2534)
• Skip test file when an import error is encountered (#2535)
• Improve detection of ReFrame test files (#2520)
• Check that PBS output is written back to working directory before setting the job as completed (#2519)
• Allow setting fixture variables from the command line (#2515)

Test library

• Fix broken symlinks to the Xdevice header library (#2524)
• Make GPU burn test a library test (#2503)

Other

• Move CSCS tests and configuration out of this repo (#2521)
• Fix documentation of execution modes' target_systems (#2541)

ReFrame 3.11.2

Release Notes

New features and enhancements

n/a

Bug fixes

• Improve test selection by their variant id (#2533)
• Reset poll rate every time a test finishes the compilation or run phase (#2534)
• Skip test file when an import error is encountered (#2535)

Test library

• Fix broken symlinks to the Xdevice header library (#2524)

Other

• Move CSCS tests and configuration out of this repo (#2521)