(This is a reminder to myself, and maybe a help for someone else who might be in the same situation as me. The purpose was to be able to lint documentation I’m trying to update for the pip project work.)

In order to use pyenv, pyenv-virtualenv without conflicting with the native MacOS python we need to add some configuration to our /.zshrc config (for mac os catalina) or your bash config if you are still using bash. It's very imported to maintain the order of the configuration for the loading order.

This applies to installing “the latest” Python on Mac OS X 10.15.6.

MacOS Catalina (released in 2019) still does not have Python 3 installed by default, only Python 2. Therefore, I needed a way to package the whole application into an app bundle and not make it dependent on user’s Python installation. There are several tools that help with that: py2app, briefcase, pyinstaller. I decided to use PyInstaller, it. Install Anaconda (Python 3.7) on Mac OSX Catalina. Nonthakon Jitchiranant. : $ python WARNING: Python 2.7 is not recommended. This version is included in macOS for compatibility with legacy software. Future versions of macOS will not include Python 2.7. Instead, it is recommended that you transition to using 'python3' from within Terminal.

I’ve used this helpful How-TO. Everything worked until the very end where brew couldn’t create a necessary directory:

(no idea why!)

I reran install python and got:

Solution (hack?)

The solution was to create the /usr/local/Framworks directory manually:

Then change the owner and group to mirror other directories (bernard:admin)

And finally run the brew link command from above:

And then I could start what I wanted to actually do. 🙂

Pip broken under Mac OS X Catalina? TLDR;

pip install --global-option=build_ext --global-option='-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/' --global-option='-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/' --prefer-binary -r requirements.txt

As Apple has announced, Mac OS Catalina now runs under its own read only file system:

macOS Catalina runs on a dedicated, read-only system volume called Macintosh HD. This volume is completely separate from all other data to help prevent the accidental overwriting of critical operating system files. [1]

Sounds great. But if you need to change that volume, or if you use software that expects to be able to find files on that read only volume that aren't there and can't be added, you're kind of stuck.

One example is Python's pip which, for some packages, will expect to find header files located in /usr/include. Apple gave developers a get-out-of-jail-free card in Mojave:

As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided. [2]

and in true Apple fashion, the package was taken away. This creates problems if you want to install commonly used, but maybe not so commonly maintained, libraries. One example is Pillow, an image processing library. This will well and truly fail to compile if you just try pip install Pillow. It looks for the headers and libraries, and doesn't find them.

What can we do? Well, for pip there's a few options. The first is simply not to compile at all, but to prefer a binary. This is as simple in some cases as the --prefer-binary option. But what if, like Pillow, there is no precompiled binary for your platform? Or for reasons of information security, you have to compile it yourself?

Fortunately, pip can be directed to look in different places using the 'global-options' flag. To get Pillow to build, use the command at the top of this article, reproduced below:

Uninstall Python Macos Catalina

pip install --global-option=build_ext --global-option='-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/' --global-option='-L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/' --prefer-binary -r requirements.txt

To understand this more fully, take a look at https://stackoverflow.com/questions/18783390/python-pip-specify-a-library-directory-and-an-include-directory.

If you need to do a similar function as part of other install tools, common environment variables are LD_LIBRARY_PATH (deprecated[3], but still commonly used), LIBRARY_PATH and INCLUDE_PATH. Setting these to the library paths (/Library ... /usr/lib) and include paths (/Library ... /usr/include) respectively may help.

Python Idle Macos Catalina

[1]https://support.apple.com/en-gb/HT210650
[2]https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035624
[3]https://stackoverflow.com/questions/18241517/c-include-path-vs-ld-library-path