Hello,
I am trying to build MbedTLS on a Mac, as part of a project to support MbedTLS in the "picoquic" implementation of QUIC. I have a small problem, probably something of my making. I have cloned the repo on an "mbedtls" directory, created a "build" subdirectory, and from there run"cmake ..", which worked fine, and then tried to run "make", which fails in "generating psa_crypto_driver_wrapper.c" when trying the python script "generate_driver_wrappers.py" because it cannot find the python module "jsonschema".
I tried to solve that by installing that module using "pip", and test programs running python3 do find the "jsonschema". Calling print(jsonschema.__file__) shows the module is installed in my user directory: /users/christianhuitema/Library/python3.9/lib/python/...
Is there a simple way to fix that?
-- Christian Huitema
On 8/20/2023 3:23 PM, Christian Huitema via mbed-tls wrote:
Hello,
I am trying to build MbedTLS on a Mac, as part of a project to support MbedTLS in the "picoquic" implementation of QUIC. I have a small problem, probably something of my making. I have cloned the repo on an "mbedtls" directory, created a "build" subdirectory, and from there run"cmake ..", which worked fine, and then tried to run "make", which fails in "generating psa_crypto_driver_wrapper.c" when trying the python script "generate_driver_wrappers.py" because it cannot find the python module "jsonschema".
I tried to solve that by installing that module using "pip", and test programs running python3 do find the "jsonschema". Calling print(jsonschema.__file__) shows the module is installed in my user directory: /users/christianhuitema/Library/python3.9/lib/python/...
Is there a simple way to fix that?
Found it. Instead of running "python3 -m pip install jsonschema" I should run "python3.10 -m pip install jsonschema". Sorry for bothering the list.
-- Christian Huitema
On 21/08/2023 00:30, Christian Huitema via mbed-tls wrote:
On 8/20/2023 3:23 PM, Christian Huitema via mbed-tls wrote:
[snip]
I tried to solve that by installing that module using "pip", and test programs running python3 do find the "jsonschema". Calling print(jsonschema.__file__) shows the module is installed in my user directory: /users/christianhuitema/Library/python3.9/lib/python/...
Is there a simple way to fix that?
Found it. Instead of running "python3 -m pip install jsonschema" I should run "python3.10 -m pip install jsonschema". Sorry for bothering the list.
This is weird. Normally "python3 -m pip ..." is the reliable way to install python modules for our scripts since they start with "#!/usr/bin/env python3". "python3.10 -m pip ..." installs the module for Python 3.10, which may or may not be python3. (Note that each minor version of Python has its own set of installed modules.) There seems to be some inconsistency in your environment, but I don't know what it could be. Some things to check which might hint at what's wrong:
python3 --version type python3 python3 -m pip --version echo $PYTHONPATH python3 -c 'import sys; print(sys.path)' python3 -m pip show jsonschema
(If you need further help, you need python experts, or perhaps expert on whichever packaging system(s) you used to install extra python versions, not mbedtls experts.)
Best regards,
On 8/21/2023 10:10 AM, Gilles Peskine via mbed-tls wrote:
On 21/08/2023 00:30, Christian Huitema via mbed-tls wrote:
On 8/20/2023 3:23 PM, Christian Huitema via mbed-tls wrote:
[snip]
I tried to solve that by installing that module using "pip", and test programs running python3 do find the "jsonschema". Calling print(jsonschema.__file__) shows the module is installed in my user directory: /users/christianhuitema/Library/python3.9/lib/python/...
Is there a simple way to fix that?
Found it. Instead of running "python3 -m pip install jsonschema" I should run "python3.10 -m pip install jsonschema". Sorry for bothering the list.
This is weird. Normally "python3 -m pip ..." is the reliable way to install python modules for our scripts since they start with "#!/usr/bin/env python3". "python3.10 -m pip ..." installs the module for Python 3.10, which may or may not be python3. (Note that each minor version of Python has its own set of installed modules.) There seems to be some inconsistency in your environment, but I don't know what it could be. Some things to check which might hint at what's wrong:
python3 --version
Python 3.9.6
type python3
python is /usr/bin/python3
python3 -m pip --version
pip 23.2.1 /users/christianhuitema/Library/python3.9/lib/python/lib/python/site-packages/pip (python 3.9)
echo $PYTHONPATH
(blank line)
python3 -c 'import sys; print(sys.path)'
['', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python39.zip',
'/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9',
'/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/lib-dynload',
'/Users/christianhuitema/Library/Python/3.9/lib/python/site-packages', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages']
python3 -m pip show jsonschema
Name: jsonschema Version: 3.2.0 Summary: An implementation of JSON Schema validation for Python Home-page: https://github.com/Julian/jsonschema Author: Julian Berman Author-email: Julian@GrayVines.com License: UNKNOWN Location: /Users/christianhuitema/Library/Python/3.9/lib/python/site-packages Requires: attrs, pyrsistent, setuptools, six Required-by:
Just in case, here is also: % uname -a Darwin Christians-iMac.local 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:19 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8103 arm64
(If you need further help, you need python experts, or perhaps expert on whichever packaging system(s) you used to install extra python versions, not mbedtls experts.)
The workaround of adding packages specifically to python3.10 worked. I noticed that the cmake output contained:
-- Found Python3: /opt/homebrew/Frameworks/Python.framework/Versions/3.10/bin/python3.10 (found version "3.10.12") found components: Interpreter
I supposed that cmake includes a script to find the latest available version number, and then directs make to use that version, python3.10. That might explain the issue.
-- Christian Huitema
On 21/08/2023 20:01, Christian Huitema wrote:
The workaround of adding packages specifically to python3.10 worked. I noticed that the cmake output contained:
-- Found Python3: /opt/homebrew/Frameworks/Python.framework/Versions/3.10/bin/python3.10 (found version "3.10.12") found components: Interpreter
I supposed that cmake includes a script to find the latest available version number, and then directs make to use that version, python3.10. That might explain the issue.
Oh. So yes, CMake (more precisely, its find_package command, or maybe specifically about the Python3 package). Experimenting a bit, it seems to traverse $PATH directories in order and take a python3 version in the first $PATH directory that has one. I don't understand how it decides which version when the first directory has more than one: experimentally it isn't the highest version.
I think our goal in CMake was to allow it to find either `python3` or `python`. (Or quite possibly `python2` or `python` at the time we originally wrote this.) It's confusing that it doesn't find the default `python3`! The way we're using CMake seems plausible though, so this looks like a bug or bad design in CMake.
Putting /opt/homebrew/Frameworks/Python.framework/Versions/3.10/bin after /usr/bin in $PATH should resolve this and should generally not matter otherwise, but I don't know if that's easy to arrange (depends how you set up your $PATH).
Best regards,
mbed-tls@lists.trustedfirmware.org
-
Christian Huitema -
Gilles Peskine