On MacOS, Python's sitecustomize.py is not called in the virtual environment's site-packages

[flexatone]

Output of brew config

HOMEBREW_VERSION: 4.4.20
ORIGIN: https://github.com/Homebrew/brew
HEAD: 47ef1a3b0b742a78be35c016df6a0279d6035ebb
Last commit: 33 hours ago
Branch: stable
Core tap JSON: 05 Feb 00:34 UTC
Core cask tap JSON: 05 Feb 00:34 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 16
Homebrew Ruby: 3.3.7 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.3.7/bin/ruby
CPU: 16-core 64-bit arm_palma
Clang: 15.0.0 build 1500
Git: 2.39.3 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 8.1.2 => /usr/bin/curl
macOS: 14.1.1-arm64
CLT: 15.0.0.0.1.1694021235
Xcode: 15.2
Rosetta 2: false

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: A newer Command Line Tools release is available.
Update them from Software Update in System Settings.

If that doesn't show you any updates, run:
  sudo rm -rf /Library/Developer/CommandLineTools
  sudo xcode-select --install

Alternatively, manually download them from:
  https://developer.apple.com/download/all/.
You should download the Command Line Tools for Xcode 15.4.


Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae and may need to be deleted.

Unexpected dylibs:
  /usr/local/lib/libPal.dylib
  /usr/local/lib/libTCMsgSvr2.dylib
  /usr/local/lib/libarachnewarp.dylib
  /usr/local/lib/libboost_atomic.dylib
  /usr/local/lib/libboost_filesystem-xgcc40-mt-1_39.dylib
  /usr/local/lib/libboost_iostreams-xgcc40-mt-1_39.dylib
  /usr/local/lib/libboost_locale.dylib
  /usr/local/lib/libboost_program_options-xgcc40-mt-1_39.dylib
  /usr/local/lib/libboost_python.dylib
  /usr/local/lib/libboost_system-xgcc40-mt-1_39.dylib
  /usr/local/lib/libboost_thread-xgcc40-mt-1_39.dylib
  /usr/local/lib/libboost_thread.dylib
  /usr/local/lib/libcdt.4.dylib
  /usr/local/lib/libcgraph.5.dylib
  /usr/local/lib/libfltk.1.1.dylib
  /usr/local/lib/libfltk.dylib
  /usr/local/lib/libfltk_forms.1.1.dylib
  /usr/local/lib/libfltk_forms.dylib
  /usr/local/lib/libfltk_gl.1.1.dylib
  /usr/local/lib/libfltk_gl.dylib
  /usr/local/lib/libfltk_images.1.1.dylib
  /usr/local/lib/libfltk_images.dylib
  /usr/local/lib/libfluidsynth.1.1.1.dylib
  /usr/local/lib/libfluidsynth.1.dylib
  /usr/local/lib/libfomus.dylib
  /usr/local/lib/libgraph.4.dylib
  /usr/local/lib/libgvc.5.dylib
  /usr/local/lib/libgvpr.1.dylib
  /usr/local/lib/libiodbc.2.1.20.dylib
  /usr/local/lib/libiodbcinst.2.1.20.dylib
  /usr/local/lib/libjack.0.0.23.dylib
  /usr/local/lib/libjack.0.dylib
  /usr/local/lib/libjack.dylib
  /usr/local/lib/liblo.0.6.0.dylib
  /usr/local/lib/libmpadec.dylib
  /usr/local/lib/libpathplan.4.dylib
  /usr/local/lib/libpng12.0.dylib
  /usr/local/lib/libportaudio.0.0.19.dylib
  /usr/local/lib/libportaudio.2.0.0.dylib
  /usr/local/lib/libportaudiocpp.0.dylib
  /usr/local/lib/libportmidi.dylib
  /usr/local/lib/libsndfile.1.0.17.dylib
  /usr/local/lib/libsndfile.1.dylib
  /usr/local/lib/libxdot.4.dylib

Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae and may need to be deleted.

Unexpected header files:
  /usr/local/include/csound/*
  /usr/local/include/fomus.h
  /usr/local/include/fomus/*
  /usr/local/include/graphviz/*
  /usr/local/include/lauxlib.h
  /usr/local/include/lua.h
  /usr/local/include/luaconf.h
  /usr/local/include/lualib.h
  /usr/local/include/node/*
  /usr/local/include/pa_asio.h
  /usr/local/include/pa_jack.h
  /usr/local/include/pa_mac_core.h
  /usr/local/include/portaudio.h
  /usr/local/include/sndfile.h
  /usr/local/include/valgrind/*

Warning: Unbrewed '.la' files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae and may need to be deleted.

Unexpected '.la' files:
  /usr/local/lib/libcdt.la
  /usr/local/lib/libcgraph.la
  /usr/local/lib/libcsound.la
  /usr/local/lib/libfluidsynth.la
  /usr/local/lib/libfomus.la
  /usr/local/lib/libgraph.la
  /usr/local/lib/libgvc.la
  /usr/local/lib/libgvpr.la
  /usr/local/lib/liblo.la
  /usr/local/lib/libpathplan.la
  /usr/local/lib/libportaudio.la
  /usr/local/lib/libportaudiocpp.la
  /usr/local/lib/libsndfile.la
  /usr/local/lib/libxdot.la

Warning: Unbrewed '.pc' files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae and may need to be deleted.

Unexpected '.pc' files:
  /usr/local/lib/pkgconfig/libcdt.pc
  /usr/local/lib/pkgconfig/libcgraph.pc
  /usr/local/lib/pkgconfig/libgraph.pc
  /usr/local/lib/pkgconfig/libgvc.pc
  /usr/local/lib/pkgconfig/libgvpr.pc
  /usr/local/lib/pkgconfig/libpathplan.pc
  /usr/local/lib/pkgconfig/libxdot.pc
  /usr/local/lib/pkgconfig/portaudio-2.0.pc
  /usr/local/lib/pkgconfig/portaudiocpp.pc
  /usr/local/lib/pkgconfig/sndfile.pc

Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae and may need to be deleted.

Unexpected static libraries:
  /usr/local/lib/libboost_atomic.a
  /usr/local/lib/libboost_locale.a
  /usr/local/lib/libboost_python.a
  /usr/local/lib/libboost_thread.a
  /usr/local/lib/libcscore.a
  /usr/local/lib/libcsound.a
  /usr/local/lib/libfomus.a
  /usr/local/lib/liblua.a
  /usr/local/lib/libportaudio.a
  /usr/local/lib/libportaudiocpp.a
  /usr/local/lib/libsndfile.a

Warning: Your Xcode (15.2) is outdated.
Please update to Xcode 15.4 (or delete it).
Xcode can be updated from the App Store.

Description of issue

When creating a virtual environment with a python3 sourced from Homebrew, Python's site reports only one site-package directory and that the user site is not enabled:

$ which python3
/Users/ariza/.env313-test/bin/python3
$ python3
Python 3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> site.getsitepackages()
['/Users/ariza/.env313-test/lib/python3.13/site-packages']
>>> site.ENABLE_USER_SITE
False

By Python documentation, a sitecustomize.py module placed in the site-packages directory should be loaded at the start of Python:
https://docs.python.org/3/library/site.html#module-sitecustomize

However, when placing a sitecustomize.py in /Users/ariza/.env313-test/lib/python3.13/site-packages, the code in that file is never executed. This is different than the behavior in Linux.

I see from starting python with python3 -v that something is happening with /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/sitecustomize.py

Processing global site-packages
Adding directory: '/Users/ariza/.env313-test/lib/python3.13/site-packages'
Processing user site-packages
Processing global site-packages
Adding directory: '/Users/ariza/.env313-test/lib/python3.13/site-packages'
# /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/__pycache__/sitecustomize.cpython-313.pyc matches /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/sitecustomize.py

Does this mean that, unlike on Linux, virtual environments created with Homebrew Python will never call sitecustomize.py in the site's site-packages? If so, this seems contrary to the expectations of when sitecustomize.py is called in the Python documentation.

If this is correct, is there anyway around this? Is there anyway to have a virtual environment based on a Homebrew Python call sitecustomize.py in the virtual environment's site-packages?

[p-linnane]

@woodruffw @branchvincent Would one of you mind weighing in here?

[woodruffw]

Hi @flexatone, thanks for the details.

However, when placing a sitecustomize.py in /Users/ariza/.env313-test/lib/python3.13/site-packages, the code in that file is never executed. This is different than the behavior in Linux.

Could you say a bit more about the behavior you're seeing on Linux, and with which Python/how it was installed?

If I had to guess what's happening, I suspect you're not seeing that code execute because Homebrew has its own sitecustomize.py installed, and Python imports whichever one comes first on sys.path (which will always be Homebrew's, since Homebrew's is in the system site-packages).

For example, here's the sitecustomize.py for Python 3.10: https://github.com/Homebrew/homebrew-core/blob/a441dfe7faf1b6a9f43ff3f7fda6883875be3cdf/Formula/p/python%403.10.rb#L390-L444

In other words: Python is working as expected (and as documented) because it is importing and running sitecustomize.py, just not the one you're expecting 🙂

In terms of working around this: you might be able to flip ENABLE_USER_SITE, but I don't know if I would recommend doing that. Another option would be to use pyenv or another non-Homebrewed Python source, in which case Homebrew's existing site customization would not apply.

[flexatone]

Many thanks, @woodruffw , for your detailed response. I had not considered that only the first-encountered sitecustomize.py is loaded, and had incorrectly assumed that sitecustomize.py needed to explicitly reside in a site-packages directory.

Thanks also for your suggestions of workarounds. I have found another: a .pth file can include a single line import statement that is executed on site initialization. By including a pth file that imports another module, I can implement sitecustomize.py-like behavior, specific to a virtual environment.