Audit/Metadata of Configuration

[atsticks]

In sensitive environments I would like to know the source of each configuration entry. Configuration can change the behaviour of an whole application, so the final source of an entry provided by the configuration system during runtime is a relevant topic. So my questions:

• How the information can be accessed from a configuration about the source (and optionally other metdata) of an entry? From the API side this could be soemthing as
  Map<String,String> getMetadata(String key);
• Do you see alternatives how this requirement can be covered?

Unfortunately the API is the more simple aspect. The question is how the configuration system get's to this information. I see the following options:

1. The ConfigSource is extended with the same API method to provide metadata for an entry. This comes with the disadvantage that the configuration has to reevaluate each access to the API method, since ConfigSources can change and any cached metadata gets invalid if a more significant ConfigSource provides a value for a key later.
2. The ConfigSource returns not only a String, but a more complex value, which also contains the corresponding metadata. This is how it is done in Tamaya with PropertyValue, PropertySource

Other use cases, where this metadata mechanism has shown to be useful is support for collections, where I can provide the expected collection type (set, list, map, subtype mappings) as metadata.

[struberg]

The whole point of Config is that the application (the user code) does not need to know where the configured value is coming from.

I fear this will also add tons of complexity for a very limited benefit.
What are the use cases where you need this meta info?

[atsticks]

all apps with higher sensitivity. Core banking, medical systems, machine control.... Am 31.10.2017 14:27 schrieb "Mark Struberg" <notifications@github.com>:

…

The whole point of Config is that the application (the user code) does *not* need to know where the configured value is coming from. I fear this will also add tons of complexity for a very limited benefit. What are the use cases where you need this meta info? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC5sc9pQkmNblvoya2-d05VGI8ik3C0tks5sxyA3gaJpZM4QMgwD> .

[struberg]

all apps with higher sensitivity. Core banking, medical systems, machine control....

Sorry no, that's simply not true from my experience in working in all those fields!
An application might only need information from a certain location. But then it should not use JSR-382 for this very attribute as it would be overkill. It might simply use Properties#load instead and be done.

The main goal of this very JSR is to completely decouple the configuration consumer from the configuration providers.
We don't win anything if the config consumer (user code) does need to know the specific setup of the ConfigSources, etc.

The other thing is: if you have untrusted code running in your own JVM then you need a SecurityManager and an isolating ClassLoader anyway. In which case you could restrict access in there as well.

Please re-read the initial message to the EG: no overengineering, no feature bloat - just plain straight forward functionality pretty please.

[atsticks]

...seems that you know my employers and customers better than I do ;). That's ok ;(. My experience is that regulations and especially how audits are done highly differs for countries AND companies. I will not argue further on this, ignore this topic or take it seriosly... Am 31.10.2017 17:51 schrieb "Mark Struberg" <notifications@github.com>: all apps with higher sensitivity. Core banking, medical systems, machine control.... Sorry no, that's simply not true from my experience in working in all those fields! An application might only need information from a certain location. But then it should not use JSR-382 for this very attribute as it would be overkill. It might simply use Properties#load instead and be done. The main goal of this very JSR is to completely decouple the configuration consumer from the configuration providers. We don't win anything if the config consumer (user code) does need to know the specific setup of the ConfigSources, etc. The other thing is: if you have untrusted code running in your own JVM then you need a SecurityManager and an isolating ClassLoader anyway. In which case you could restrict access in there as well. Please re-read the initial message to the EG: no overengineering, no feature bloat - just plain straight forward functionality pretty please. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC5sc49RBByNZnHaLusu5-P18QPKJvHyks5sx0_3gaJpZM4QMgwD> .

[atsticks]

BTW etcd for example provides metadata by prefixing it's keys with an underscore, e.g.

foo=bar
_foo.ttl=123456

This would still work without any API extensions. The problem about it is, that considering meta-entries from multiple config sources gets more complex, but not impossible. I have no problem, when we decide not to include any API features for it, but I wanted to have the topic discussed ;-).

[jmesnil]

This is a valid use case to be able to inspect the actual values used by the application but I don't think this requires to be exposed by the API.

As an example in the WildFly implementation of the Config API, I added some management methods (similar to JMX operations) to list the config-sources used by an application/deployment:

[standalone@localhost:9990 /] /deployment=demo.war/subsystem=microprofile-config/configuration=default:list-config-sources
{
    "outcome" => "success",
    "result" => [
        {
            "name" => "SysPropConfigSource",
            "ordinal" => 400
        },
        {
            "name" => "EnvConfigSource",
            "ordinal" => 300
        },
        {
            "name" => "PropertiesConfigSource[source=vfs:/content/demo.war/WEB-INF/classes/META-INF/microprofile-config.properties]",
            "ordinal" => 100
        }
    ]
}

It can also be used to evaluate the actual values that are provided for a given key:

[standalone@localhost:9990 /] /deployment=demo.war/subsystem=microprofile-config/configuration=default:eval-property(name=FOO)
{
    "outcome" => "success",
    "result" => [
        {
            "value" => "comes from the sys props",
            "config-source" => {
                "name" => "SysPropConfigSource",
                "ordinal" => 400
            }
        },
        {
            "value" => "My FOO property comes from the microprofile-config.properties file",
            "config-source" => {
                "name" => "PropertiesConfigSource[source=vfs:/content/demo.war/WEB-INF/classes/META-INF/microprofile-config.properties]",
                "ordinal" => 100
            }
        }
    ]
}

Such management operation are useful and provides additional value but I agree with @struberg that we should not clutter the Config user API with them.
One key aspect of the JSR is to decouple the config from its sources.

We can however mention in the spec that the implementation SHOULD provide some ways to inspect and list config sources actually used by an application.

[lilian-benoit]

I agree with @jmesnil , It is interesting for debug goal

We can however mention in the spec that the implementation SHOULD provide some ways to inspect and list config sources actually used by an application

[atsticks]

We have a Tamaya branch currently in development, where the JSR API is used as a base, so we have a complete standalone and independent JSR implementation. It has shown that metadata can be added in various ways without having to add the feature as part of the API. So IMO if agreed, we can close this issue.

[Emily-Jiang]

@struberg and I met up today and discussed a bit. @jmesnil @atsticks should we revisit this to see whether this needs to be done?