[GR-65009] MacOS Max Open Files always limited to 10240

[telemenar]

Describe the issue
On MacOS there is no way to allow for more than 10240 open files, no matter what the value of ulimit -Sn or ulimit -Hn or launchctl limit maxfiles. The number of open files is always limits to 10240.

This is a result of the standard practice across many (all?) jvms including GraalVM of setting the current max files limit to the hard max if possible. But on MacOS due to a compatibility note in the man page for setrlimit the max file limit is always set to OPEN_MAX from syslimits.h which has the value 10240.

In jvm's derived from OpenJDK, you can work around this by skipping the calls to setrlimit for max files with -XX:-MaxFDLimit. And in more recent version of OpenJDK they have started ignoring the COMPATIBILITY note from the man page because that note seems to not have been true since ~10.6.

So in GraalVM can something be done to allow for more than 10240 open files?

I'm not strongly opinionated as to how:

• an option to skip setting the limit
• some method to enable it being set to higher than OPEN_MAX

I've included lots of pointers to code/docs/tickets in the More Details section.

This is something I found as part of digging way too deep into why I couldn't use mvnd to build a very large project in this ticket apache/maven-mvnd#710

Steps to reproduce the issue
Please include both build steps as well as run steps

1. Set your local ulimit to higher than 10240
2. Create a java program that opens 10241 files.
3. Run the java program see that it fails with:

Too many open files

(Sorry for the minimal repro steps, hopefully the code links/discussion makeup for it.)

Describe GraalVM and your environment:

• GraalVM version (latest snapshot builds can be found here), or commit id if built from source: [e.g. EE 19.3, CE 20.0, CE 20.1.0-dev-20200304_0848]
• JDK major version: 17 & 21
• OS: macOS Sequoia 15.1.1 (24B91)**
• Architecture: Apple M1 aarch64

More details
setrlimit Man page link:
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/setrlimit.2.html

A somewhat recent 3rd party hosted copy of syslimits.h:
https://github.com/phracker/MacOSX-SDKs/blob/11.3/MacOSX11.3.sdk/usr/include/sys/syslimits.h#L96

The code that does this in GraalVM is here (latest tag as of posting to give a static reference location):
https://github.com/oracle/graal/blob/jdk-25+21/substratevm/src/com.oracle.svm.core.posix/src/com/oracle/svm/core/posix/PosixNativeLibraryFeature.java#L86

The equivalent code from OpenJDK is here (latest jdk 21) which behaves exactly the same as GraalVM does today:
https://github.com/openjdk/jdk/blob/jdk-21%2B35/src/hotspot/os/bsd/os_bsd.cpp#L2005

The current equivalent code from OpenJDK is here (Latest tag as of posting - JDK 25) this no longer tries to enforce OPEN_MAX though it does include it as fallback logic:
https://github.com/openjdk/jdk/blob/jdk-25%2B21/src/hotspot/os/bsd/os_bsd.cpp#L2117

These changes were the result of this ticket (with many related tickets):
https://bugs.openjdk.org/browse/JDK-8324577

The summary is that through testing it has been confirmed that the compatibility note is no longer correct. However, they don't actually set it to RLIM_INFINITY because that ran into issues with certain other processes in the system that would break if RLIMIT_NOFILE was greater than int32 even though the underlying type is actually uint64. Then they limited it down further to match the typical limit in linux of 0x100000 because they discovered several cases where people were iterating over all possible fds trying to close them and with the limit set to int32 max some of these were causing timeouts.

[oubidar-Abderrahim]

GraalVM community is based on OpenJDK and should provide a similar behavior, Do you see a different behavior when running OpenJDK vs GraalVM community?

[telemenar]

OpenJDK differs in two ways.
A) There is an option MaxFDLimit that controls this behavior of setting the max number of open files. It defaults to true, but if disabled for example with -XX:-MaxFDLimit then, the FD Limit is not changed. So if the limit prior to starting to jvm was above 10240 it is not reduced. GraalVM doesn’t have this option so it will always force the limit to be exactly 10240 even if it was higher before the jvm started.

B) In OpenJDK 23+ OpenJDK changed their behavior and they set the limit to min(<hard_limit>, 0x100000) you can find some documentation/history for that static constant in the issue tracke link I included in the more details section above. I link to both the older OpenJDK code for this and the current code in OpenJDK code in the more details section as well.

So to summarize:

• GraalVM behaves the same as OpenJDK if OpenJDK is not provided with an extra option in versions of OpenJDK 22 >
• GraalVM is more restrictive than OpenJDK if OpenJDK was provided with an extra option in OpenJDK 22 >
• GraalVM is more restrictive than OpenJDK 23 <=

[telemenar]

@fniephaus You mentioned in the mvnd ticket that I should tag you on this ticket.

[fniephaus]

Thank you, @telemenar. We are looking into a fix for this.