Skip to content

Commit

Permalink
pkg/cdi: don't crash if we fail to create fsnotify watch.
Browse files Browse the repository at this point in the history
Don't crash in update() if we fail to create an fsnotify watch.
This can happen if we have too many open files. In this case we
now record a failure for all configured spec directories and in
update we always trigger a refresh. If the process if ever able
to create new file descriptors the cache becomes functional but
in a 'always implicitly fully refreshed' mode instead of auto-
refreshed.

It's not entirely clear what is the best option to deal with a
failed watch creation. Being out of file descriptors typically
results in a cascading chain of errors which the process does
not usually survive.

This fix aims for minimal footprint. On failed watch creation
it does not render the cache fully unusable. If the process is
ever able to create new file descriptors again the cache also
becomes functional, but instead of autorefreshed mode it will
be in an 'always implicitly fully refreshed' mode.

Signed-off-by: Krisztian Litkey <krisztian.litkey@intel.com>
  • Loading branch information
klihub committed Feb 22, 2025
1 parent 3a3ea2e commit be9b6f3
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions pkg/cdi/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -579,6 +579,14 @@ func (w *watch) update(dirErrors map[string]error, removed ...string) bool {
update bool
)

// If we failed to create an fsnotify.Watcher we have a nil watcher here
// (but with autoRefresh left on). One known case when this can happen is
// if we have too many open files. In that case we always return true and
// force a refresh.
if w.watcher == nil {
return true
}

for dir, ok = range w.tracked {
if ok {
continue
Expand Down

0 comments on commit be9b6f3

Please sign in to comment.