ReFrame 4.7.3

Release Notes

Enhancements

• Report all failed performance variables in FAILURE INFO (#3358)

Bug fixes

• Fix use of dump_pipeline_progress and RFM_DUMP_PIPELINE_PROGRESS options (#3365)
• Add missing topology_prefix option in the configuration schema (#3364)
• Check response status in the httpjson log handler and retry in case of TOO_MANY_REQUESTS (#3356)
• Treat a node in PLANNED Slurm state as available (#3361)

ReFrame 4.7.2

Release Notes

Enhancements

• Optimize JSON decoding for sessions when applying filtering criteria (#3343)

Bug fixes

• Fix treatment of Slurm constraints in job.options and system partition's access parameter (#3352)

Other

• Update GH actions for Python 3.7 after the move of runners to Ubuntu 24.04 (#3350)

ReFrame 4.7.1

Release Notes

Enhancements

• Prefix special (generated) parameters with . instead of $ (#3330)
• Update formatting of list of talks in the documentation (#3321)
• Remove tip for disabling storage results from documentation (#3319)

Bug fixes

• Exit successfully if --duration limit is reached (#3328)
• Treat correctly Slurm constrains with hyphens when filtering nodes (#3327)
• Fix nodelist abbreviation when last node number is a multiple of 10 (#3339)
• Do not ignore --filter-expr when using --describe-stored-testcases (#3329)
• Fix error in fixture resolution when multiple fixtures of the same class are instantiated with different variables (#3326)
• Fix relative imports in deeply nested tests (#3337)

ReFrame 4.7.0

Release Notes

New features

• Make directory prefix for topology files configurable (#3317)

• Allow partition/environment extras to be used also as feature constraints in valid_systems and valid_prog_environs (#3316)

• Show the last N lines in test failures (#3291)

• New configuration parameter for specifying commands to install reframe remotely for topology detection (#3281)

• Add a check_fail_phase log formatting specifier (#3292)

• Allow users define custom log handlers and attach them to the framework (#3274)

• Add new --retries-threshold option (#3262)

• Introduce permanent result storage and support performance results comparisons between runs (#3227)

  • Add a configuration option to enable/disable results storage (#3295)
  • Support session annotations with a new --session-extras option (#3266)
  • Expose SQLite connection timeout as configuration parameter (#3253)
  • Disable results storage by default (#3313)
  • Update --table-format arguments (#3307)
  • Use builtin json package for decoding testcase data (#3296)
  • Various enhancements to the results DB feature (#3283)
  • Allow setting the file mode of the SQLite database (#3260)
  • Change default values for --performance-report and --list-stored-sessions options (#3258)
  • Fix query error when aggregation groups and presentation columns are disjoint (#3294)
  • Fix DST skew in CMPSPEC period calculations (#3288)
  • Optimize testcase queries over time period (#3256)
  • Ignore records with unset performance when grouping test cases (#3247)

• Add a perflog entry also when sanity fails (#3189)

• Skip by default flexible tests when not enough nodes found (#3173)

For the new feature of permanent results storage and performance analytics, please have a look at the tutorial and the manpage for details.

Enhancements

• Report the undefined parameters if an instantiation of an abstract test or fixture is attempted (#3293)
• Warn instead of failing when a filter expression is invalid for test (#3310)
• Issue a more compact message when JSON schema validation fails (#3314)
• Do not issue warning when skipping tests during the init phase (#3286)
• Reorganize the structure of the manpage (#3267)
• Improve performance of module backend when detecting module conflicts (#3263)
• Improve naming of automatically generated tests (#3265)

Bug fixes

• Add job options from the command-line to processor auto-detection remote job (#3318)
• Handle None cpu model name returned from archspec (#3199)
• Constraint the lxml version on aarch64 (#3193)

Test library

• Fix compilation of gpu burn benchmark with hip (#3232)
• Added MetalWalls scalability test for mw (#3233)

Other

• Enable Python 3.13 in CI + expand Github test workflows (#3309)
• Use local copies of get-pip.py for bootstrapping (#3287)
• Run scheduler unittests with GitHub Actions (#3268)
• Do not upload to Codecov through a GH action (#3188)

ReFrame 4.6.4

Release Notes

Enhancements

• Print test file in the failure info (#3282)

Bug fixes

• Allow RESERVED nodes for Slurm "avail" state (#3290)
• Let Slurm decide partition in flexible node allocation if no default partition is defined (#3280)

ReFrame 4.6.3

Release Notes

Enhancements

• Add information on how to cite ReFrame (#3259)
• Print selected system in [ReFrame Setup] information (#3255)
• Update disclaimer on the support of EOL Python versions (#3238)

Bug fixes

• Fix typo in the --distribute docs (#3239)

ReFrame 4.6.2

Release Notes

Enhancements

• Update description of the configuration in the manpage (#3213)
• Improve description of valid_systems and valid_prog_environs in the tutorial and other minor tutorial fixes (#3208)

Bug fixes

• Fix report schema to properly accept all test results (#3223)
• Fix logging crash when timestamp=True for the file log handler (#3215)

Other

• Fix Tmod32 container due to Centos 7 EOL (#3229)
• Update get-pip.py url for Python 3.7 (#3222)

ReFrame 4.6.1

Release Notes

Enhancements

• Fix broken cross links in the documentation (#3196)
• Display Python imports in the first tutorial example (#3195)
• Use long options in places in the documentation (#3197)
• Add EUM24 slides (#3191)

Bug fixes

• Do not use CLI defaults for options that are backed by a configuration option (#3206)
• Make version hash suffix calculation more robust (#3204)
• Fix perflog header formatting when perflog_compat is used (#3201)
• Fix logging crash when log format is %(check_#ALL)s (#3202)
• Retrieve the test job's nodelist when using the squeue backend (#3184)

Other

• Bump version of requests package (#3205)

ReFrame 4.6.0

Release Notes

New features

• Add support for associating scheduler resources with an environment (#3152)
• Do not require a sanity function for compile-only tests (#3157)
• Include hardware platform info in ProcessorInfo (#3151)
• Make processor model available through ProcessorInfo (#3150)
• Improve state selection criteria in the --distribute and the --flex-alloc-nodes option (#3140)
• Add a model property to the devices configuration (#3138)
• Enable default value type checking and implicit conversions at the variable declaration level (#3115)
• Introduce launcher modifiers replacing LauncherWrapper (#3114)
• Enable multiple inheritance of test variables (#3113)
• Add CPU vendor and model name to topology files (#3107)
• Support relative imports in tests (#3110)
• Include scheduler and launcher as default partition extras (#3108)
• Add nvcc as an environment configuration parameter (#3098)

Enhancements

• Revamp the documentation (#3095)
• Update Job.nodelist description (#3155)

Bug fixes

• Treat OR Slurm constraints in flexible node allocation (#3158)
• Hide disabled hooks from --describe option's output (#3153)

Test library

• Added QuantumESPRESSO scalability test for pw.x (#3134)

Other

• Update versions of GitHub Actions in CI (#3154)
• Stricter install requirements in setup.cfg (#3135)

ReFrame 4.5.2

Release Notes

Enhancements

• Fix spelling mistake in the documentation (#3133)
• Fix missing JQuery dependency in documentation's conf.py (#3121)
• Fix broken link to older versions of the documentation (#3117)

Bug fixes

• Fix logging crash when the check_#ALL placeholder is used (#3142)
• Do not fail if squeue cannot retrieve the reason that a job is pending (#3123)