Commit graph

11 commits

Author SHA1 Message Date
Jeffery To
c0340f631c python: Use locked for host pip
This also removes PKG_BUILD_PARALLEL:=0 that was added for packages that
use HOST_PYTHON3_PACKAGE_BUILD_DEPENDS.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-08-31 03:48:47 -07:00
Jeffery To
722a5b8efa
python3: Use hash-checking mode when installing host pip packages
In hash-checking mode[1], pip will verify downloaded package archives
(source tarballs in our case) against known SHA256 hashes before
installing the packages.

As a consequence, this requires the use of requirements files[2] and
pinning packages to known versions.

The syntax for package Makefiles has changed slightly;
HOST_PYTHON3_PACKAGE_BUILD_DEPENDS no longer accepts requirement
specifiers like "foo>=1.0", only requirements file names (which are the
same as package names in the most common case).

This also updates affected packages, in particular:

* python-zipp: "setuptools_scm[toml]" has been split into
  "setuptools-scm toml" to reuse the requirements file for
  setuptools-scm (the extra depends installed by "setuptools_scm[toml]"
  is toml).

* python-pycparser: This previously used ply 3.10, whereas the
  requirements file will now install 3.11.

[1]: https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
[2]: https://pip.pypa.io/en/stable/user_guide/#requirements-files

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-08-17 23:05:49 +08:00
Jeffery To
001cd3c571
python3: Add config PYTHON3_HOST_PIP_CACHE_WORLD_READABLE
This adds a config option PYTHON3_HOST_PIP_CACHE_WORLD_READABLE; if
enabled, chmod will be run after pip install to make all
files/directories in the host pip cache world-readable.

Supersedes https://github.com/openwrt/packages/pull/13012.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-08-16 06:09:45 +08:00
Rosen Penev
b40c40151c
python3: fix host compilation with clang
Matched rpath parameter with Makefile.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-08-02 00:49:30 -07:00
Jeffery To
0973d21787
python3: Use default _PYTHON_HOST_PLATFORM
This lets the Python build process set _PYTHON_HOST_PLATFORM instead of
forcing an explicit value.

Also:

* Save the target _PYTHON_HOST_PLATFORM value during Build/InstallDev
  for use when building target Python packages (in python3-package.mk).

* Use the (mostly) default PYTHON_FOR_BUILD value, instead patch
  configure to remove the platform triplet from the sysconfigdata file
  name.

* Remove the "CROSS_COMPILE=yes" make variable (there is no indication
  that this variable is necessary).

* Force host pip to build packages from source instead of downloading
  binary wheels.

  Previously, host pip can download universal (platform-independent)
  wheels but not platform-specific wheels, because of the custom
  _PYTHON_HOST_PLATFORM value. (Packages that do not have universal
  wheels would be compiled from source.)

  With a correct _PYTHON_HOST_PLATFORM, host pip can install
  platform-specific wheels as well. However, the pre-built shared object
  (.so) files in these wheels will have the host's platform triplet in
  their file names. When target Python packages are built (using the
  target's _PYTHON_HOST_PLATFORM), Python will not use these shared
  object files.

  By forcing host pip to build packages from source, the built shared
  object files will not have the platform triplet in their file names.
  (Host Python has been patched to remove the platform triplet from file
  names.) This allows these packages to be used when building target
  Python packages.

  (The net effect of this complete change is that platform-dependent
  packages will continue to be compiled from source, while
  platform-independent packages will now also be compiled from source.)

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-07-08 17:11:42 +08:00
Jeffery To
87b8f45230 python3: Rename canned recipes in python3-host.mk
This changes the recipe name prefix from Build/Compile/HostPy3 to
HostPython3, and clarifies some of the names (RunHost to Run, Mod to
ModSetup).

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-04-19 01:56:11 +08:00
Jeffery To
3642b18441 python3: Remove HostPython3 in python3-host.mk
HostPython3 only adds a few environment variables before running host
Python. It has only two users, Build/Compile/HostPy3RunHost and
Build/Compile/HostPy3RunTarget.

HostPython3 also accesses $(PYTHON3PATH), even though python3-host.mk
does not include python3-package.mk, where the variable is defined.

This removes HostPython3 and has its two users run host Python directly.
This also combines the environment variables of HostPython3 and the two
users into HOST_PYTHON3_VARS and PYTHON3_VARS.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-04-19 01:56:07 +08:00
Jeffery To
2dfdef66a3 python3: Remove include guard for python3-host.mk
Since it only defines variables and canned recipes, it is safe to
include python3-host.mk more than once.

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2020-04-17 23:19:13 +08:00
Jeffery To
ae80ddc7ab python,python3: Update host pip[3] install functions
* Add --cache-dir option to set the pip cache to a directory in
$(DL_DIR), instead of pip's default (build user's ~/.cache/pip),
fixes #9066

* Add --disable-pip-version-check option, since the version check only
prints a message saying a new version is available

* Combine host_python_pip_install and host_python_pip_install_host into
Build/Compile/HostPy[3]PipInstall

* Remove --root and --prefix options, since this function is only used
to install packages to host Python's default site-packages directory
(setting these may serve to confuse pip)

* Pass all of $(HOST_PYTHON[3]_PACKAGE_BUILD_DEPENDS) to the function,
since pip can handle multiple arguments/packages

Signed-off-by: Jeffery To <jeffery.to@gmail.com>
2019-05-29 21:45:16 +08:00
Alexandru Ardelean
995b48121e python,python3: remove --ignore-installed flag for host packages
This was copied over from python-packages, when support for installing
packages host-side (via pip) was added.

Based on the discussion on this commit:
  612c53fc6c
it was mentioned that removing this may add more benefit in terms of
reducing build time, because packages won't get reinstalled every time.

I'm not entirely sure about any potential side-effects of this, but it's
worth trying it out.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
2018-05-14 12:11:00 +03:00
Alexandru Ardelean
ccdc6bc530 python,python3: export mk files outside of python package dirs
Since `lang/python` is it's own folder of Python packages
(for both Python 2 & 3), and these build rules are needed
in a lot of packages [especially Python packages],
putting them here makes sense architecturally,
to be shared.

This also helps get rid of the `include_mk` construct
which relies on OpenWrt core to provide, and seems
like a broken design idea that has persisted for a while.
Reason is: it requires that Python 2/3 be built to provide
these mk files for other Python packages,
which seems like a bad idea.

Long-term, there could be an issue where some other feeds
would require these mk files [e.g. telephony] for
some Python packages.
We'll see how we handle this a bit later.

For now we limit this to this feed.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
2018-01-10 23:01:51 +02:00
Renamed from lang/python/python3/files/python3-host.mk (Browse further)