Update configuration from properties file sources on file changes

[sdaschner]

It would be a very helpful feature -- not only for container / orchestration environments -- to trigger updates in ConfigSources that originate from properties files, when these files are updated.

[hendrikebbers]

Already thought about the same topic. Here we can to check how far such a features should be integrated. In addition I think it would be wrong to provide such functionality only by CDI:

Let's say you have a config file with a property foo. When you start your Java application the property has the value A based on the content of the property file.
While the application running you change the content of the file to foo=B. Based on this it would be awesome if the app can do something based on this change. For EE4J / Microprofile you can define this for the next request but I do not think that this is a good solution. I think the best solution would be the observer pattern here: a developer can add a listener to a property to get notified by any changes of the property. This again will be a problem in EE4J / Microprofile since such a listener will never be called in any request and therefore you can not use mostly all of your scope dependent beans (@requestScope, @SessionScope ...). I was already thinking about that topic the last month when trying to create a generic & modern observer pattern API for Java (see https://github.com/guigarage/ObserverPattern if you are interested in that). Since this would be a more or less perfect solution for JavaSE it will become problematic in EE by confusing developers that are not that familiar with thread concepts, how scopes work internally, etc.

[Emily-Jiang]

I have similar thought on this. We need to address dynamic aspect of the config. We either force client to have the up to date config or client decides when to update the value. We talked about this in the past but not settle on any solutions. We can discuss this on the forthcoming hangout meeting.

[TFaga]

Agree on the topic. The application using the config should be able to be notified of any config changes when it happens (or close to when) instead of when the value is naturally looked up. Or even worse the application would need to poll each config value its interested in by itself as perhaps some values are read only when starting the application.

I would however not limit this to only changes in property files as is the original scope of this issue. But instead add a mechanism available to all ConfigSources no matter where they get the values or how the dynamic updating is achieved (file watchers, internal polling, persistent connections, ...) as I described in #9 . We can however close that issue and move the entire discussion here, since the desired end goal I think is the same.

This means adding some sort of mechanism for notifying the application of changes as they happen. CDI events seem like a great fit for this. While as mentioned it will put additional restrictions on how you can consume the events with CDI beans (like the inability to use @RequestScoped or @SessionScoped for the beans that consume them), it would still be very useful if those things are not actually needed (a lot of the times they're not).

However as @hendrikebbers mentioned it should definitely not be the only way and we should also provide a simple API to access the functionality outside CDI for environment where there is no CDI or CDI is simply not used. The observer pattern mentioned would be a good choice, or perhaps something even more simple, like a subscribe/unsubscribe method on the Config class, or something else entirely. Definitely something to discuss.

[struberg]

I think that we have to distinct 2 different topics

1.)
This ticket #8 is about how to immediately pick up values from a file which changed.
That can be implemented in many different ways.
One possibility is to use a file trigger.
At least if it is available on the underlying system.
This is depending on the OS, the file system etc.
The important part here is that all this is purely a matter of the ConfigSource itself.
It can handle all those things internally - the Config system itself doesn't need to provide anything special!

A possible fallback would be to check every second whether the file timestamp did change, and reload in that case.
Or have a background thread running which checks for exactly that each second.
This is btw also the way how a Database based ConfigSource would most likely implement this.

2.)
The topic which would better fit the description in #9 -notifying the application whether a configured value got changed.
This is a bit harder to implement.
a.) Sending an Event is a responsibility of the Config and not of each ConfigSource itself.
The reason is that a value might be overridden in higher ordinal ConfigSources.
If such a ConfigSource would send out any events, then it might even be wrong information.

b.) That means that any ConfigSource which detects a change would need to notify the Config instance.
That might be doable by extending the ConfigSource interface with a method

/**
 * This callback should get invoked if an attribute change got detected
 */
default void setOnAttributeChante(Consumer<Set<String>> reportAttributeChange) {
     // do nothing by default. Just for compat with older ConfigSources.
}

The Config should also pro-actively check all e.g. 2 seconds whether a value got changed.

3.)
How to notify any downstream 'users' of the change?
One possibility would be to send a CDI event.
The other would be a classical Observer/Observable registration. ink that we have to distinct 2 differng things

[hendrikebbers]

I think 1. is not part of the spec since it described how a source implementation will internally check for changes. I think we need observer/event/whatever support in config & config source. Von meinem iPhone gesendet

…

Am 29.10.2017 um 21:41 schrieb Mark Struberg ***@***.***>: I think that we have to distinct 2 different topics 1.) This ticket #8 is about how to immediately pick up values from a file which changed. That can be implemented in many different ways. One possibility is to use a file trigger. At least if it is available on the underlying system. This is depending on the OS, the file system etc. The important part here is that all this is purely a matter of the ConfigSource itself. It can handle all those things internally - the Config system itself doesn't need to provide anything special! A possible fallback would be to check every second whether the file timestamp did change, and reload in that case. Or have a background thread running which checks for exactly that each second. This is btw also the way how a Database based ConfigSource would most likely implement this. 2.) The topic which would better fit the description in #9 -notifying the application whether a configured value got changed. This is a bit harder to implement. a.) Sending an Event is a responsibility of the Config and not of each ConfigSource itself. The reason is that a value might be overridden in higher ordinal ConfigSources. If such a ConfigSource would send out any events, then it might even be wrong information. b.) That means that any ConfigSource which detects a change would need to notify the Config instance. That might be doable by extending the ConfigSource interface with a method /** * This callback should get invoked if an attribute change got detected */ default void setOnAttributeChante(Consumer<Set<String>> reportAttributeChange) { // do nothing by default. Just for compat with older ConfigSources. } The Config should also pro-actively check all e.g. 2 seconds whether a value got changed. 3.) How to notify any downstream 'users' of the change? One possibility would be to send a CDI event. The other would be a classical Observer/Observable registration. ink that we have to distinct 2 differng things — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[struberg]

Indeed 1. is an internal behaviour of each ConfigSource. It might only become a JSR matter if we define this as requirement for our default built-in ConfigSources.

I'm actually not sure whether we need a config change notification system.
So far a straight 'pull' mechanism worked fine as well.

Often it's not a question about one single 'refresh cycle'.
Some configured values do not change at all over the runtime - others might change often.
And some might not change often but any change should get picked up rather quickly or not.

In Apache DeltaSpike we have a TypedResolver to approach this problem.
In ported this over to MP-config where it looks like the following
https://github.com/struberg/javaConfig/blob/configValue/api/src/main/java/javx/config/ConfigValue.java
https://github.com/struberg/javaConfig/blob/configValue/api/src/main/java/javx/config/Config.java#L57

Here is how this is supposed to be used:

ConfigValue<Integer> cfga =
    config.access("some.server.port")
        .as(Integer.class)
        .cacheFor(5, TimeUnit.MINUTES)
        .withDefault(Integer.valueOf(1234));
....
callServer(...cfga.getValue());

[Emily-Jiang]

We can focus on the dynamic aspect in this week's hangout. Let's agree on whether we want client to have a choice whether to pull in the changes as suggested by @TFaga or Client goes back to ask for changes based on each property as demonstrated by @struberg earlier.

[TFaga]

Agreed, we should definitely discuss if a notification system is something to consider adding. Alternatively it could be something that is added in a future version.

I agree that watching for changes in the ConfigSource-s is not something that should be happening by default for every value. Typically only a few config parameters would be worth watching and notifying explicitly for changes. It would make sense for the users of the config to select which ones they want to be notified about and only watch changes for those config values.

[atsticks]

I would like to point you to the work we have done in Tamaya:

• The Tamaya injection API allows injecting DynamicValue instances. This instances are aware of changes and based on a LoadPolicy they can automatically take over changes or only flag that a change has happened. This way applications can explicitly decide if a change should be used or ignored. Additionally DynamicValue allows to register a PropertyChangeListener for listening on changes for a given entry.
• Additionally Tamaya's event module is regularly pulling for config changes and can emit according ConfigChangeEvents . Similarly there is also an event for property source changes.
• The problem of supporting CDI and SE at the same time is solved in Tamaya by defining it's own event manager, which can be backed up by different SPIs, including CDIs, for event distribution.

[hendrikebbers]

I had a chat with @atsticks this morning about the observer pattern in Java and we discussed the idea to create a specific JSR for this topic. My plan is to create an abstract for this topic till the end of the year. From my current view it would make sense to have something like this in the config JSR. Since this is not possible based on time constraints (observer pattern JSR haven't started yet). I would prefer to not support this feature in the first version of the config JSR. Once we have default interface for observer support in Java a support for this interfaces in the config JSR would be a big plus. I will keep you in loop about this topic :)