Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[minor] test messages, Sterling menu filtering, fix #296

Merged
merged 6 commits into from
Feb 10, 2025
Merged

[minor] test messages, Sterling menu filtering, fix #296

merged 6 commits into from
Feb 10, 2025

Conversation

tnelson
Copy link
Owner

@tnelson tnelson commented Feb 9, 2025

Test Failures

Test failures are now highlighted better in the console. A test that fails because a counterexample exists will print a message like this, and then open Sterling to show the counterexample. In such cases, only that failing test run will be displayed in Sterling's run menu; no other runs will be available.

******************** TEST FAILED *******************
Test myTestName failed. Stopping execution.
Test failed due to finding a counterexample, which will be displayed in Sterling.
*****************************************************

If the test failed due to unsatisfiability or inconsistency (which are the same, from the solver's point of view) a similar message will print but Sterling will not open, since it would only be able to say "UNSATISFIABLE".

Fix: live server on test failure

Fixed an issue where a failed test instance would be loaded without the live server, which could result in trouble loading Sterling. (We had previously changed to the live server for security reasons and compatability with modern browsers.)

Sterling menu and tests

As before, users can pick from a menu of Forge commands, each of which corresponds to an instance iterator. However, we've made some changes to reduce clutter in the menu:

  • Tests that have passed, but don't produce an instance in the process (such as assert ... is necessary ..., which passes only if no counterexample is found) will no longer appear in the Sterling menu.
  • Tests that have passed, but do produce an instance (such as assert ... is sat) will continue to appear in the Sterling menu. This includes passing examples, so that a user can visually inspect the example instance if needed.

We will be watching feedback on this feature so we can improve the Sterling menu further.

Non-test commands, such as run, will not actually be run until the user has clicked the "Run" button in Sterling with the command selected. (This should be identical behavior to the prior release.)

Terminal Reporting

Unless the verbosity option is set to 0, passing tests will now print a message to the terminal.

Unless the verbosity options is set to 0, Forge will now print a "Solving for next instance." message to the terminal when "Run" or "Next" is clicked in Sterling.

The terminal message about hitting enter to stop Sterling now includes "or click 'Stop' in VSCode" to avoid confusion.

Minor Changes and Workarounds

Java detection

Added the java_exe_location option, which accepts a path to the Java executable that should be used to run the solver. This option isn't well-tested yet, so we suggest providing absolute paths and using it only if needed, e.g., if you have multiple Java installations.

If you are submitting models to an autograder, we suggest using this option only from the terminal via racket <filename.frg> -O java_exe_location <path>. Technically it can be used within a model file via option java_exe_location <path>, but the path given will not likely match the autograder's setup.

tnelson and others added 6 commits January 28, 2025 09:21
…ents (#291)

* Refactor ABAC language to use a .frg domain model that can be extended into an assignment
none inferred multiplicity is no longer set
* Somewhat improved Froglet errors in narrow cases, re-enabled corresponding test cases
* Fix issue recently introduced where using a Forge internal ID as a sig, field, etc. name would cause an error, but only if the file making the definition was imported into another.
* Minor fixes (e.g., License registration for Racket package server)
@tnelson tnelson merged commit 08da205 into main Feb 10, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant