Occasionally things just go wrong during large builds. One such occurrence is on macOS when compiling a large C++ project.
$ cmake --build build ... In file included from /opt/local/include/boost/spirit/home/x3.hpp:19: In file included from /opt/local/include/boost/spirit/home/x3/operator.hpp:10: In file included from /opt/local/include/boost/spirit/home/x3/operator/sequence.hpp:12: /opt/local/include/boost/spirit/home/x3/operator/detail/sequence.hpp:25:10: fatal error: cannot open file '/opt/local/include/boost/fusion/include/as_deque.hpp': Too many open files #include <boost/fusion/include/as_deque.hpp> ^ 1 error generated.
Too many open files… Really? What decade is this?!
According to Wilson Mar’s article Maximum limits (in macOS file descriptors), this issue is caused by a low default limit for the number of files that can be open simultaneously.
Digging a little bit deeper into the
setrlimit man pages, its important to note this limit is specific to a single process.
What follows are step-by-step instructions for detecting and resolving this issue.
First, check the existing soft and hard limits for the maximum number of open files.
➜ launchctl limit maxfiles maxfiles 256 unlimited
In this case, the 256 file soft limit is the issue. This limit is much too low.
For the running session, remedy the problem by setting higher limits with
➜ sudo launchctl limit maxfiles 65536 2147483647
This sets the soft limit to 65,536 open files and the hard limit to 2,147,483,647 open files.
Don’t exceed the maximum number of 2,147,483,647 for either limit here.
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>limit.maxfiles</string> <key>ProgramArguments</key> <array> <string>launchctl</string> <string>limit</string> <string>maxfiles</string> <string>65536</string> <string>2147483647</string> </array> <key>RunAtLoad</key> <true/> </dict> </plist>
Increasing this limit with
ulimitfrom a startup shell script is quick and do-able. However, such an approach only applies to a single user, doesn’t account for changing shells, and is more likely to accidentally be overwritten or deleted.
Ensure the file is owned by the
rootuser and belongs to the
➜ sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
Set permissions on the file so that it is readable and writeable by the owner and only readable by group members and everyone else.
➜ sudo chmod 644 /Library/LaunchDaemons/limit.maxfiles.plist
➜ sudo launchctl bootstrap system /Library/LaunchDaemons/limit.maxfiles.plist
Set the service to run at boot.
➜ sudo launchctl enable system/limit.maxfiles
Verify that the limits have been updated.
➜ launchctl limit maxfiles maxfiles 65536 2147483647
Restart your terminal application to take advantage of the increased limits.
Check to make sure the changes have taken effect for your shell.
➜ ulimit -S -n 65536
There’s a fair bit of work involved for this fix but it is quite robust.
Not only should file process limits not be an issue for you for the forseeable future, you should have gained some valuable insights into macOS service management with
launchctl limitresult in a hard-limit slightly above the soft-limit?