This required a bit work to get working, compared to other versions. So,
some things have changed a bit more significantly.
Some highlights:
* there is no longer a pgen executable, seems this is now part of
libpython; let's see what this means for us in the future
* blake2 hash (from OpenSSL) detection needs some fixing; will upstream
added patch 002-fix-blake2-detection.patch
* removed all bpo patches; those should be fixed in upstream
* some needed to be manually re-applied as stuff changed:
- 001-enable-zlib.patch - file changed
- 004-do-not-write-bytes-codes.patch - file changed
- 015-abort-on-failed-modules.patch - variable was renamed
cross_compiling -> CROSS_COMPILING
* 017_lib2to3_fix_pyc_search.patch - the code changed, it does not seem to
have the original problem with respect to file-extension, as there
does not seem to be any special extension logic anymore there
* 006-remove-multi-arch-and-local-paths.patch - dropped patch; I can't
remember the full-details of this issue; it was something with
Debian/Ubuntu's multi-arch stuff; it was probably added maybe due to
some overzealous (on my part) thingy caused by some weird reports,
that I could never solve; let's have this patch dropped and see
* make package/python3/refresh to reduce fuzz for the rest
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This adds pypi.mk, which can be included in Python packages that
download their sources from PyPI, to auto-fill various PKG_* variables
based on the value of PYPI_NAME.
This makefile should be included after $(TOPDIR)/rules.mk but before
$(INCLUDE_DIR)/package.mk (and $(INCLUDE_DIR)/host-build.mk).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
These patches address issues:
CVE-2019-16056: email.utils.parseaddr mistakenly parse an email
CVE-2019-16935: A reflected XSS in python/Lib/DocXMLRPCServer.py (for
Python 2.7)
CVE-2019-16935 was fixed for python3 in #10109
Links to Python issues:
https://bugs.python.org/issue34155https://bugs.python.org/issue38243
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
- Remove PKG_BUILD_DEPENDS as it is no longer necessary.
- The Python3 is already included in DEPENDS.
- Remove PKG_BUILD_DIR and PKG_UNPACK was for dual Python version.
- Change TITLE and description
- Add source package
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
And splitting the gunicorn[3] binary/executable away from the
python[3]-gunicorn libraries. This was inspired from Debian packaging.
The gunicorn[3] binaries require the new `python[3]-pkg-resources`
libraries to run, which add ~1.1 MB on the [ram]disk when uncompressed.
For the Python2 variant, the `_gaiohttp.py` is dropped as it fails to
compile, so it would likely be unusable anyway:
```
File "/usr/lib/python2.7/site-packages/gunicorn/workers/_gaiohttp.py", line 84
yield from self.wsgi.close()
^
SyntaxError: invalid syntax
```
People around the web recommend this as well:
https://stackoverflow.com/questions/25611140/syntax-error-installing-gunicornhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=803170https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=803202
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Some packages just install some Python binaries, that may need their
shebang fixed.
This change adds some utilities to help with that and try to centralize the
sed rules a bit.
It also removes the logic from the `python-package-install.sh` into the
`python-package[3].mk` files. This does 2 things:
1. It minimizes the need for the shell script to know the Python
version 2/3
2. Makes the logic re-usable in packages; especially if the install rules
differ a bit
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This package is required by other packages to run some binaries via
`load_entry_point`.
So, this splits this package away from setuptools.
setuptools is pretty big, akd pkg-resources is also big, but not as big.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
As I remember this worked.
But since `set -e` is set, I am a bit paranoid about it. In the sense that
it may fail if `ver` != 3.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The CONTRIBUTING.md requests an (or multiple) SPDX identifier for GPL
licenses. But a lot of packages did use a different, non-SPDX style with a
"+" at the end instead of "-or-later".
Signed-off-by: Sven Eckelmann <sven@narfation.org>
The package on PyPi is named `mysqlclient`.
This should have been named `python-mysqlclient` from the start.
There is a `mysql` package on PyPi already but that's a different
code/package.
Doing this should avoid any future confusion.
There is no good time to do this rename; at least 19.07 has been branched
already and this can go into the next release [in a year or so].
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This is a feature release including improvement to OIDC and security
enhancements, as well as bugfixes.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
* Remove $$$$(pkg-config --static --libs libcrypto libssl) from
HOST_LDFLAGS
Having this leads to an "unknown type name 'u_int'" error on Mac.
Removing it doesn't appear to affect Python's ability to find
buildroot LibreSSL.
* Change -Wl,-rpath=... to -Wl,-rpath,... in HOST_LDFLAGS
The equals sign version is not supported by the Mac linker (appears to
be an GNU extension). The comma version is supported; -rpath and its
argument will be separated by a space when passed to the linker.
* Add ac_cv_header_libintl_h=no to HOST_CONFIGURE_VARS for Mac
Python on Mac doesn't expect to use libintl, but if gettext-full is
compiled for host, it will try, leading to undefined symbol errors
during compilation. This prevents configure from finding libintl.h.
Fixes#7171.
Fixes#9621.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This is a bugfix release. Relevant to openwrt, it removes the binary
flag while opening the dictionary file.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
This changes the python[3]-django dependencies in packages to be
non-selecting, and adds an MDEPENDS line so that the *-src packages get
placed inside the django menu as well.
Added MENU:= to the src-package definitions in python[3]-package.mk,
so it does not import that setting from the binary package.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
They seem to have become popular, so adding src packages for these as well.
Not bumping PKG_RELEASE intentionally, since these packages are new.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Optionally fixes compilation with uClibc-ng.
Based on the surrounding code, this looks like an oversight.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
The librt depends are not needed. EGLIBC is not even in the tree.
Rearranged some stuff for consistency between packages.
Signed-off-by: Rosen Penev <rosenp@gmail.com>
- Improve TITLE and Description
- Remove PKG_UNPACK as it is not necessary
- Correct LICENSE file
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
This change also updates the maintainer email to cotequeiroz@gmail.com, as
requested on a different change.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This change also updates the maintainer email to cotequeiroz@gmail.com, as
requested on a different change.
Also, changing here is the download URL to github's codeload, since that
one offers .tar.gz archives.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This change also updates the maintainer email to cotequeiroz@gmail.com, as
requested on a different change.
Not updating deps here, since that will be done in the next changeset that
will convert packages to the Python[3] packaging format.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The email change was part of a previous request from a different PR series.
In this series, the change is the only thing left to do (from what the
original changeset intended to do).
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
- Update setuptools to 40.8.0
- Update pip to 19.0.3
- Refreshed patches
- Removed 4 patches (2 of them was included in 3.7.3 and other two are
included in this release)
Makefile python3:
- Move PKG_MAINTAINER above PKG_LICENSE
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Shorter TITLE
- The longer one was not shown in make menuconfig.
Reodered stuff in Makefile
Removed PKG_UNPACK and change PKG_BUILD_DIR
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Upstream backport. It seems the holdup is on python-twisted.
Without this, it fails with
SSL_get0_next_proto_negotiated: symbol not found
Signed-off-by: Rosen Penev <rosenp@gmail.com>
This patch, taken from buildroot, avoids the use of host paths when
compiling third-party extensions.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
This clarifies that Python 2-only libraries will have a Python 3 version
added, instead of being replaced by a Python 3 version (before the 2020
mass removal).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
seafile-seahub was the only user of this.
Remove this from the build.
Having it here is a bit messy.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
Build/InstallDev is passed a second argument, a path where host binaries
should be placed (ultimately $(STAGING_DIR)/host).
This change moves python[3]-config to that directory.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
- Change URL of the website and for PKG_SOURCE_URL
- Change TITLE and description
- Remove PKG_BUILD_DEPENDS, PKG_UNPACK as they are not necessary
- Add src package
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
These patches address issue:
CVE-2019-9948: Unnecessary URL scheme exists to allow local_file://
reading file in urllib
Link to Python issue:
https://bugs.python.org/issue35907
Issue 35907 is still currently open, waiting for a decision for
Python 3.5; these patches for Python 2.7 and 3.7 have been merged.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
These patches address issues:
CVE-2019-9740: Python urllib CRLF injection vulnerability
CVE-2019-9947: Header Injection in urllib
Links to Python issues:
https://bugs.python.org/issue36276 (resolved duplicated of 30458)
https://bugs.python.org/issue35906 (resolved duplicated of 30458)
https://bugs.python.org/issue30458
Issue 30458 is still currently open, waiting for a decision for
Python 3.5; these patches for Python 2.7 and 3.7 have been merged.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the "patched" indicator files for host setuptools and pip
to include their PKG_RELEASE values. This also removes host setuptools
and/or pip before host install, if the installed copy does not match the
version (and PKG_RELEASE) of the copy to be installed.
This will allow added or removed patches to affect host setuptools /
pip, since these changes will cause PKG_RELEASE to be incremented.
This also fixes the host install error, when the install tries to patch
an already patched copy of setuptools. (This error occurs because the
existing indicator files do not have version numbers in their file
names, whereas host install expected version numbers to be present.)
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds the current setuptools/pip version numbers to the indicator
files' names, which should allow upgraded versions to be patched.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
python-cryptography's build depends (host cffi, libffi) were transferred
to python-cffi at some point; this corrects the situation.
python-cryptography's host Python build depends is copied from its
setup.py[1].
[1]: https://github.com/pyca/cryptography/blob/2.6.1/setup.py#L47
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This uses two find commands to delete __pycache__ contents then the
__pycache__ directories, rather than a for loop.
The second command omits a -empty test, so that if the first command
doesn't remove all directory contents for some reason, the second
command will return an error (find will not delete a non-empty
directory).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the --prefix option, passed to host pip when "installing"
target setuptools and pip, to /usr, in case the prefix is recorded in
the packages.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds --cache-dir and --disable-pip-version-check options for host
pip, when "installing" target setuptools and pip.
This also changes the pip command to use $(HOST_PYTHON[3]_PIP) from
python[3]-host.mk.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Normally, Python will include the user's site-packages directory
(~/.local/lib/python$(PYTHON_VERSION)/site-packages) in it's internal
search path for modules.
This disables this default inclusion for host Python.
This change is applied during Host/Configure instead of as a patch to
keep this setting unchanged for target Python.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
* Add --cache-dir option to set the pip cache to a directory in
$(DL_DIR), instead of pip's default (build user's ~/.cache/pip),
fixes#9066
* Add --disable-pip-version-check option, since the version check only
prints a message saying a new version is available
* Combine host_python_pip_install and host_python_pip_install_host into
Build/Compile/HostPy[3]PipInstall
* Remove --root and --prefix options, since this function is only used
to install packages to host Python's default site-packages directory
(setting these may serve to confuse pip)
* Pass all of $(HOST_PYTHON[3]_PACKAGE_BUILD_DEPENDS) to the function,
since pip can handle multiple arguments/packages
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
After some thinking over this, documenting this behavior makes sense
versus adding some functionst to handle this.
There is some validity/use-cases where some users may want to reference
a python[3]-package.mk from some other location as well as have the
flexibility to change it (locally). One example can be when the local
`packages` is renamed to something else.
This does not fall on the responsibility of the Python maintainers, but
it can be documented.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This changes --with-ensurepip=install to upgrade, to upgrade host
versions of setuptools and pip to the Python-bundled versions.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The Python 2 and 3 versions of chardet both install a script with the
same name (/usr/bin/chardetect). This is the issue identified in #9006
(https://github.com/openwrt/packages/pull/9006#issuecomment-493709812).
This renames the Python 3 script to chardetect3.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Added a python3 variant, and removed python-cryptography, and pyjwt from
the dependencies. They are required only to run one test, that is not
even being installed.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
This adds the ability to patch setuptools (and pip), and adds 3
reproducibility patches from Debian[1].
(003-PKG-INFO-output-reproducible.patch addresses the issue identified
in #9039.)
The patching is not perfect, in that the patches are applied to
setuptools and pip after they have been installed, since they are
installed from wheels which are already "precompiled".
Also, patching for the host install cannot be updated in place, for
example if a patch is added or removed.
[1]: https://sources.debian.org/patches/python-setuptools/40.8.0-1/
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
The current package does not work, due to missing dependencies, so they
are being added now, along with python3 support.
This versions brings many bugfixes, and the option to use defusedxml if
available, protecting against many xml exploits.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
This is a dependency of the openpyxl package.
The package Makefile was reworked, and a python3 variant was added.
Maintainer was changed to Alexandru Ardelean & Eneas U de Queiroz.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
This is a dependency of the openpyxl package.
The package Makefile was reworked, and a python3 variant was added.
Maintainer was changed to Alexandru Ardelean & Eneas U de Queiroz.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Changed PKG_LICENSE to reflect spdx license tag, and PKG_LICENSE_FILES
to include all lincense-related files applicable to the parts of the
code we are actually using to build and/or distributing. The
Windows-only files, and the python-bundled Tools we're not using have
been left out.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Changed PKG_LICENSE to reflect spdx license tag, and PKG_LICENSE_FILES
to include all lincense-related files applicable to the parts of the
code we are actually using to build and/or distributing. The
Windows-only files, and the python-bundled Tools we're not using have
been left out.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
If a package builds python & python3 variants, then the respective
PACKAGE-python* conditional DEPENDS were added, since circular
dependencies should all be resolved now.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Adding the conditionals to DEPENDS should not cause circular
dependencies any more. This adjusts the text to point out that it used
to be a problem, and if it happens again, one should open an issue.
Also, some spotted trivial errors were fixed.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Added python-rcssmin, and django-appconfig as dependencies, and a note
in the package help text about not having a rjsmin package, so the
jsmin (javascript) filter will not work.
Adjusted the Makefile to conform to current python-package style, and to
display the package title correctly in menuconfig.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
This is a dependency of django-compressor.
The package Makefile was reworked, and a python3 variant was added.
Maintainer was changed to Alexandru Ardelean & Eneas U de Queiroz.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
The defusedxml package contains several Python-only workarounds and
fixes for denial of service and other vulnerabilities in Python's XML
libraries. In order to benefit from the protection you just have to
import and use the listed functions / classes from the right defusedxml
module instead of the original module.
Currently, openpyxl detects, and uses defusedxml, if installed.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
Fixes issue #8978. If libcurl's SSL library is set to an SSL
library other than libmbedtls, compilation fails. This patch
configures python-curl to use the currently selected SSL library
for libcurl.
Signed-off-by: Val Kulkov <val.kulkov@gmail.com>
With pip3.7, `--index-url ""` is different from absence of --index-url
argument. Apply the same for python3 variant
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
This is largely done by suffixing "python" or "py" with "3". The
README.md file is also copied here and we intend to maintain it
independently from its python2 counterpart.
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
The note was note correct when mentioning encodings vs python[3]-codecs.
This change fixes this confusion.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This describes the proposal outlined in #8520, with a few
additions/modifications:
* Describes the handling of the Python 2 interpreter
* Allows for normal updates of Python 2 libraries that share the same
Makefile as their Python 3 version
* Mass removal event has a name
Supersedes #8788.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This change changes the maintainer to
`Alexandru Ardelean <ardeleanalex@gmail.com`
for all Python packages owned by
`Gergely Kiss <mail.gery@gmail.com>`
No functional changes.
Bumping PKG_RELEASE on each package that is updated.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
The python2 host variant is not needed anymore as openvswitch has now
switch to using python3 for building
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
This is a minor cleanup.
Use PYTHON[3]_PKG_SETUP_GLOBAL_ARGS variable to specify extra build args.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
This change removes a series of Python packages that are added to
seafile-seahub and are not needed.
After some investigation into seafile, there are no references for it.
These are some of the low-hanging fruits.
They're not used [not sure when they were], and these packages were born
out of some weird sprints somewhere and forgotten on web and left
un-maintained.
So, remove them.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
- Reordered things in Makefile
- Removed PKG_BUILD_DIR and PKG_UNPACK as it isn't necessary
- Changed TITLE from python3-jinja2 to Jinja2
- Added src package
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
This adds more details on the PYTHON[3]_PKG_SETUP_* variables, the
filespec format, and the build/install processes in general.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
- COPYING as a license is no longer available. It's LICENSE
- Add PKG_CPE_ID
- Add a description
- Add Python sources
- Add me as maintainer as the previous maintainer is inactive in OpenWrt
- Changed URL to Git repository, where you can find more details
- Moved TITLE to Default
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
Some packages (PyYAML,
https://github.com/openwrt/packages/pull/8482#discussion_r270692276)
recognize "global" options to setup.py; these must appear before the
"install" command on the command line.
This adds PYTHON[3]_PKG_SETUP_GLOBAL_ARGS, which let packages set these
global options.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
"python -m compileall" has a default maximum recursion level of 10, i.e.
it will descend up to 10 levels of subdirectories when looking for
source files to compile. This is usually sufficient but there are
packages that include more than 10 levels (botocore,
https://github.com/openwrt/packages/pull/8214#discussion_r270056741).
This adds the "-r" command line option to the call to compileall to
increase the max recursion level (currently set to 20).
This also patches Python 2's compileall.py to add this max recursion
level option. (Python 3's compileall.py already supports this option.)
This also applies some related changes to python-package-install.sh:
* Use the "-delete" option with find instead of exec'ing rm / rmdir. For
the case of removing empty directories (in delete_empty_dirs()), this
has the added benefit of simplifying the code, as the "-delete" option
implies "-depth", and thus find "does the right thing" (removing empty
directories depth-first).
* Remove the backslash in "-name" patterns (for find), as they are not
regular expression but glob patterns.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Currently, all files in usr/bin (presumably all Python scripts) are run
through sed to replace the shebang; sed will overwrite the file whether
or not a match is found. This causes symlinks to be overridden and made
into copies of their targets. python[3]-base and python[3]-dev are
affected by this.
This adds the --follow-symlinks flag to sed, in addition to using
$(SED), so that symlinks are not overridden.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
- Use mbedTLS as default and add patches for it
- Use files.pythonhosted.org in PKG_SOURCE_URL
- Changed description
- Reordered things in Makefile
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
