Add flag to set the maximum error behavior regardless of findings level

[jalseth]

I'd like a flag that lets me set the maximum error behavior, eg treat all errors found as warnings and exit with status code 0. This makes it easier to slowly improve a repo's Rego policies over time without requiring 100% up front effort.

Intended use scenario is to run Regal twice per PR, once in warning-only mode with all checks enabled, once in normal mode with only certain categories enabled. For example:

$ regal lint --warn-only --format github <policy_dir>
$ regal lint --disable-all --enable-category bugs --format github <policy_dir>

Over time, more categories would be treated as blocking rather than warnings.

[anderseknert]

Huh, that's a surprise. I guess we never really used that ourselves. I'll have that fixed.

[anderseknert]

Or wait, no. The --fail-level flag

set level at which to fail with a non-zero exit code (error, warning) (default "error")

So setting it to warning means "treat warnings as errors". If you want warnings to be treated as warnings and not impact the exit code, just leave it at the default error level. Or am I missing something? :)

[jalseth]

Oh, I misunderstood the flag then. I was looking for a flag for "set the maximum error level to warning" without having to maintain a YAML for all of the rules, manually setting them to warning instead of error.

Let's change this to a FR then.

[anderseknert]

You can set the default level to warning to achieve something like that, I think:

rules:
  default:
    level: warning

$ regal lint --format github <policy_dir>
$ regal lint --fail-level warning --disable-all --enable-category bugs --format github <policy_dir>

I'm not opposed to having a flag toggle the default setting outside of conf though.

[jalseth]

That is simple enough, thank you!