Compare commits

...

356 Commits
f26 ... master

Author SHA1 Message Date
Miro Hrončok 7baa535f27 Enable https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup on power and arm
https://bugzilla.redhat.com/show_bug.cgi?id=1795575 is "fixed for almost a month"
2020-02-28 18:34:31 +01:00
Miro Hrončok 66b21b9b89 Rebased to 3.8.2 final
All patches applied cleanly.
2020-02-26 15:03:31 +01:00
Miro Hrončok 471a304653 Update the ensurepip module to work with setuptools >= 45
setuptools 45.x is Python 3 only and changed the name of the wheel
2020-02-24 21:49:54 +01:00
Nicolas Chauvet 086f5b2234 Use __isa_bits instead of crafted value
This is easier to read and the appropriate macro to use here

Signed-off-by: Nicolas Chauvet <kwizart@gmail.com>
2020-02-24 19:53:58 +00:00
Marcel Plch b6982963b8 Rebase to v3.8.2rc2 2020-02-24 17:22:04 +01:00
Miro Hrončok e2dce1ef0e Rebased to 3.8.2rc1
Patches 339 and 341 were backports, now included.
No other patches changes required.
2020-02-12 11:40:41 +01:00
Lumir Balhar e90b6d950b Use compileall2 module for byte-compilation in the python3 package 2020-01-31 07:46:43 +00:00
Tomas Orsava 97e152a6da De-hardcode py_SOVERSION in the files section 2020-01-30 20:52:35 +00:00
Miro Hrončok b1c7775af7 Fix test_zipfile.test_add_file_after_2107() 2020-01-30 17:50:12 +01:00
Miro Hrončok a7179c0ac5 Temporarily don't use -fno-semantic-interposition on power and arm
Workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1795575
2020-01-30 17:50:12 +01:00
Victor Stinner bb05cf63a4 Run the test suite with --timeout=1800 2020-01-30 17:50:12 +01:00
Miro Hrončok acf2f58231 ctypes: Disable checks for union types being passed by value
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1794572
Fixed upstream: https://bugs.python.org/issue16575
2020-01-30 17:50:10 +01:00
Fedora Release Engineering 1e4e35f9c2 - Rebuilt for https://fedoraproject.org/wiki/Fedora_32_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2020-01-30 16:28:16 +00:00
Miro Hrončok a262879258 Update to Python 3.8.1
No patches changes since 3.8.1rc1.
2019-12-19 12:21:03 +01:00
Miro Hrončok 8d46a842b6 Rebased to Python 3.8.1rc1
All patches applied cleanly.

Among other things, this contains the necessary fix for gcc 10 support:
 https://bugs.python.org/issue38965
2019-12-10 14:38:19 +01:00
Charalampos Stratakis 141731d4d0 Build Python with -fno-semantic-interposition
The compiler flag has been added to CFLAGS_NODIST and
LDFLAGS_NODIST. This will compile the core interpreter
and the stdlib modules with -fno-semantic-interposition
but will not affect user build and rpm C extension modules
compiled by distutils.

This has the effect of speeding up the interpreter up to
27%, depending on the workload, with the drawback of disabling
the capability of using LD_PRELOAD to override symbols in
libpython.

https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
2019-12-03 19:39:22 +01:00
Miro Hrončok 4b316fdc3f Reintroduce the OPENSSL_CONF=/non-existing-file workaround
Since crypto-policies-20191002-1.gitc93dc99.fc32, the maximum version of TLS
is customized in Fedora as well.

See https://bugzilla.redhat.com/show_bug.cgi?id=1778357
and https://bugs.python.org/issue38815

This reverts commit b33b4a5162.
2019-12-02 18:24:24 +01:00
Miro Hrončok 25c078cff5 Recommend python3-tkinter when tk is installed
The tkinter module is part of the Python's standard library, however it is in
a separate subpackage to save an unwanted dependency on tk.

When tk is installed, we recommend the subpackage to provide more upstream-like
experience, but still provide a way out.
2019-11-28 15:45:05 +01:00
Tomas Orsava a50a780039 New bcond main_python
The bcond flatpackage assumed that if building *without* flatpackage,
that we want to build the main Python for the distribution. However, in
some instances we want a non-flatpackage build of Python that is *not*
the main Python in the distro.
Therefore I have split the main-Python setting into its own bcond.
2019-11-26 15:34:21 +00:00
Miro Hrončok 6e14981d8b Add "downstream only" explanations to various patches
Suggested by Neal Gompa in https://bugzilla.redhat.com/show_bug.cgi?id=1774417
2019-11-25 13:20:27 +01:00
Miro Hrončok 102b2bcc79 Verify upstream sources with GPG
This is now a recommended thing to do:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_source_file_verification

Regardless if it adds actual security, it should prevent problems like this one:
https://mail.python.org/archives/list/python-dev@python.org/message/OYNQS2BZYABXACBRHBHV4RCEPQU5R6EP/
2019-11-01 08:09:41 +01:00
Miro Hrončok 9fa948fc22 Fedora CI: Test -O flags of the debug build
See https://src.fedoraproject.org/tests/python/pull-request/13
2019-10-24 01:50:13 +02:00
Miro Hrončok 10ec9de2ef Update to Python 3.8.0 final 2019-10-15 19:43:42 +02:00
Miro Hrončok bb663e5153 Fedora CI: Run the smoke test on the debug build as well 2019-10-15 19:43:09 +02:00
Miro Hrončok 796e80bd9a Pass -Og to EXTRA_CFLAGS as well as to CFLAGS_NODIST
- to CFLAGS_NODIST to take precedence as CFLAGS_NODIST go after EXTRA_CFLAGS
 - to EXTRA_CFLAGS to "bake it in" and build extension modules with -Og
 - call make regen-all with the same flags, as it invokes gcc as well
2019-10-15 19:42:25 +02:00
Charalampos Stratakis 577a903f29 Make the debug build compile with -Og
Due to Python's handling of CFLAGS and the way we
were passing extra compiler flags through the SPEC
the -Og flag for the debug build was overridden by
the -O2 flag, resulting in various side effects.

See: https://bugs.python.org/issue37631

Resolves: rhbz#1678277
2019-10-03 13:03:23 +00:00
Miro Hrončok 4332c4f431 Update rpmlintrc to accept Python >= 3.10 2019-10-03 13:06:46 +02:00
Miro Hrončok a1d16f719d Update rpmlintrc to reflect 3.8 changes and python-unversioned-command
- python-unversioned-command has dangling symlinks
 - the ABI flags no longer contain "m"
 - new -embed .pc files were added in 3.8
2019-10-03 12:25:30 +02:00
Miro Hrončok 9a14f3e986 Rebased to Python 3.8.0rc1
Rebased patches: 102
Removed patches already merged upstream: 331
2019-10-01 23:05:55 +02:00
Kalev Lember cbb927f346 Avoid using _prefix macro for /usr/lib/debug dir
In case of flatpak module builds, _prefix is redefined as /app.
rpm-build however hardcodes /usr/lib/debug without using the _prefix
macro, so do the same here to match this.

This fixes building Python as a flatpak module.

https://bugzilla.redhat.com/show_bug.cgi?id=1737933#c14
2019-09-25 15:54:58 +02:00
Miro Hrončok e702ad4c2b Rebased to Python 3.8.0b4
Patch 189 rebased for new setuptools/pip version.

Patch 331 added to fix clang: https://bugzilla.redhat.com/show_bug.cgi?id=1715016

importlib.metadata is now a simple module, not a folder.
2019-08-30 17:55:01 +02:00
Miro Hrončok 7b9cec6bb1 Enable Profile-guided optimization for all arches, not just x86
This increases the build time (to ~2 hours on armv7hl Koji),
but should bring more optimized Python to architectures other than x86.

The build time overhead is not so big on Python 3.8,
as only a limited number of tests is used.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1741015
2019-08-19 16:49:13 +02:00
Miro Hrončok c4cab72129 Rebuilt for Python 3.8 2019-08-14 23:07:29 +02:00
Miro Hrončok 8fe9ebffa8 Bootstrap for Python 3.8 2019-08-14 19:46:57 +02:00
Miro Hrončok 1e3b8fac07 Rebase to 3.8.0b3
https://fedoraproject.org/wiki/Changes/Python3.8
2019-08-12 19:10:51 +02:00
Miro Hrončok e9ba67b74b Conditionalize python3-devel runtime dependencies on RPM packages and setuptools
Installing python3-devel currently brings in:

- annobin
- dwz
- efi-srpm-macros
- fpc-srpm-macros
- ghc-srpm-macros
- gnat-srpm-macros
- go-srpm-macros
- nim-srpm-macros
- ocaml-srpm-macros
- openblas-srpm-macros
- perl-srpm-macros
- python-rpm-macros
- python-srpm-macros
- python3-rpm-generators
- python3-rpm-macros
- qt5-srpm-macros
- redhat-rpm-config
- rust-srpm-macros

For somebody who needs python3-devel to build extension modules,
those are all useless, so we conditionalize it on rpm-build.

Based on the reason for setuptools dependency, we do the same here
(but note that python3 already recommends it).
2019-08-12 15:54:51 +02:00
Miro Hrončok e2599943a2 Conditionalize python3-devel runtime dependencies on RPM packages and setuptools
Installing python3-devel currently brings in:

- annobin
- dwz
- efi-srpm-macros
- fpc-srpm-macros
- ghc-srpm-macros
- gnat-srpm-macros
- go-srpm-macros
- nim-srpm-macros
- ocaml-srpm-macros
- openblas-srpm-macros
- perl-srpm-macros
- python-rpm-macros
- python-srpm-macros
- python3-rpm-generators
- python3-rpm-macros
- qt5-srpm-macros
- redhat-rpm-config
- rust-srpm-macros

For somebody who needs python3-devel to build extension modules,
those are all useless, so we conditionalize it on rpm-build.

Based on the reason for setuptools dependency, we do the same here
(but note that python3 already recommends it).
2019-08-11 11:37:21 +02:00
Miro Hrončok 2d970f424c Stop disabling test_asyncio, the problem was fixed 2019-08-09 21:04:49 +02:00
Miro Hrončok 2d19c7ecdc Skip test_gdb everywhere
https://bugzilla.redhat.com/show_bug.cgi?id=1734327

Also remove a note about test_gdb skipped on armv7hl, that one was fixed:

https://bugzilla.redhat.com/show_bug.cgi?id=1196181
2019-08-09 20:59:30 +02:00
Miro Hrončok 29530ba2a8 Update to 3.8.0b3 2019-07-29 23:47:30 +00:00
Petr Viktorin 69c2d11d9a Update python-debug description for Python 3.8 ABI unification 2019-07-29 16:42:16 +02:00
Fedora Release Engineering d35394ed78 - Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2019-07-26 17:27:40 +00:00
Fedora Release Engineering 5c6b2e2b5c - Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2019-07-26 17:22:20 +00:00
Miro Hrončok 46d035d896 Keep the LICENSE.txt file in lib/pythonX.Y dir
The license() builtin tries to read it and virtualenv tries to copy it.

See https://github.com/pypa/virtualenv/issues/1352

Up until now, the license() builtin juts felt back to:

    See https://www.python.org/psf/license/

However it should output the full license text.

Virtualenv ~16.6 warns:

    No LICENSE.txt / LICENSE found in source

Technically, it is probably possible to install the package without
%license files, but that would simply resort to the previous noncritical
behavior.

This fix is not critical and hence it doesn't bump release, for easier
backporting to all our Python packages.
2019-07-22 18:38:17 +02:00
Miro Hrončok ec995d8172 Restore pyc to TIMESTAMP invalidation mode as default in rpmbubild
See https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57#comment-27426
2019-07-15 17:38:09 +02:00
Miro Hrončok ff90d23b8f Keep the LICENSE.txt file in lib/pythonX.Y dir
The license() builtin tries to read it and virtualenv tries to copy it.

See https://github.com/pypa/virtualenv/issues/1352

Up until now, the license() builtin juts felt back to:

    See https://www.python.org/psf/license/

However it should output the full license text.

Virtualenv ~16.6 warns:

    No LICENSE.txt / LICENSE found in source

Technically, it is probably possible to install the package without
%license files, but that would simply resort to the previous noncritical
behavior.

This fix is not critical and hence it doesn't bump release, for easier
backporting to all our Python packages.
2019-07-15 14:27:11 +00:00
Miro Hrončok 9213c03924 Restore pyc to TIMESTAMP invalidation mode as default in rpmbubild
See https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57#comment-27426
2019-07-15 15:38:52 +02:00
Lumir Balhar a664336af9 Move test.support module to python3-test subpackage
More info: https://fedoraproject.org/wiki/Changes/Move_test.support_module_to_python3-test_subpackage
2019-07-15 15:28:54 +02:00
Lumir Balhar 7bf69ee163 Move test.support module to python3-test subpackage
More info: https://fedoraproject.org/wiki/Changes/Move_test.support_module_to_python3-test_subpackage
2019-07-15 15:21:13 +02:00
Miro Hrončok 988896a172 Python now means Python 3
- https://fedoraproject.org/wiki/Changes/Python_means_Python3
- The python-unversioned-command package is no longer Python 2, but 3
- The python, pydoc, python-config, python-debug, idle, pygettext.py and
  msgfmt.py commands are now in python3
2019-07-15 15:17:06 +02:00
Miro Hrončok 337d3348c9 Python now means Python 3
- https://fedoraproject.org/wiki/Changes/Python_means_Python3
- The python-unversioned-command package is no longer Python 2, but 3
- The python, pydoc, python-config, python-debug, idle, pygettext.py and
  msgfmt.py commands are now in python3
2019-07-15 15:06:48 +02:00
Miro Hrončok 0f89ab9113 Remove obsolete obsoletes 2019-07-12 18:41:02 +02:00
Miro Hrončok b9306b1085 Add %python_provide provides
This allows easier compatibility back and forth once (and if)
https://fedoraproject.org/wiki/Changes/Python_means_Python3
happens. This is not the full implementation of the change,
as the python-unversioned package still needs to be added.

This is currently a no-op as %python_provide does nothing
for python3.
2019-07-09 19:59:58 +02:00
Miro Hrončok f7475de5fc Add %python_provide provides
This allows easier compatibility back and forth once (and if)
https://fedoraproject.org/wiki/Changes/Python_means_Python3
happens. This is not the full implementation of the change,
as the python-unversioned package still needs to be added.

This is currently a no-op as %python_provide does nothing
for python3.
2019-07-09 15:42:18 +00:00
Miro Hrončok 663a998388 Update to 3.7.4 2019-07-09 11:00:50 +02:00
Miro Hrončok bc1bef742b Remove the OPENSSL_CONF=/non-existing-file workaround
The test was fixed upstream to expect different config on Fedora.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1618753

(Note that we still keep that on the CI for older Python versions.)
2019-07-08 11:48:41 +02:00
Miro Hrončok f16aa02ebf Remove patch 316
The patch landed in Python in be5bb52f5f2d4da4b9d6f42399f7275ab47910f3

However patch 316 still applied cleanly.
Removing it doesn't make any difference, so not bumping anything.
2019-07-08 09:52:41 +02:00
Miro Hrončok 13257b57b3 Update to 3.8.0b2 2019-07-05 12:10:54 +02:00
Miro Hrončok f585cd7ec1 Update to 3.7.4rc2 2019-07-03 00:37:56 +02:00
Miro Hrončok b6a6e97b8f Don't skip so many tests on the CI
test_wsgiref is tracked in https://bugs.python.org/issue37411
2019-06-28 13:13:03 +02:00
Miro Hrončok 949410ed57 Fedora CI: Provision 3 GiB of RAM 2019-06-28 13:13:03 +02:00
Miro Hrončok 3815a08388 Don't skip so many tests on the CI
test_wsgiref is tracked in https://bugs.python.org/issue37411
2019-06-28 02:46:02 +02:00
Miro Hrončok 9f5808cf53 Don't let RPM set SOURCE_DATE_EPOCH based on the latest %changelog date
It breaks tests with: can't find '__main__' module in .../test_zip.zip

Reported at https://bugs.python.org/issue34022
Tracked at https://bugzilla.redhat.com/show_bug.cgi?id=1724753
2019-06-27 20:17:39 +02:00
Victor Stinner 17d397670a Skip test_distutils if bootstraping, use python3.X for regen-all 2019-06-26 12:08:56 +02:00
Miro Hrončok b33b4a5162 Remove the OPENSSL_CONF=/non-existing-file workaround
The test was fixed upstream to expect different config on Fedora.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1618753

(Note that we still keep that on the CI for older Python versions.)
2019-06-26 10:30:21 +02:00
Miro Hrončok dd5f80b1b8 Update to 3.7.4rc1
- reenable test_asyncio
 - drop merged CVE fixes (patches 320, 324)
2019-06-25 23:16:36 +02:00
Miro Hrončok 9941fbbeb9 CI: Stop running findleaks tests, but run the test suite on debug Python 2019-06-25 15:32:50 +02:00
Victor Stinner 3a6f3f6941 Reenable test_distutils and test_bdist_rpm
Resolves: rhbz#1685557
2019-06-11 15:01:33 +02:00
Miro Hrončok a292838ccc Update to 3.8.0b1 2019-06-05 09:35:21 +02:00
Miro Hrončok 7aab0ddc87 Remove Patch178, merge Patch205 into Patch102
Fixes invalid value of distutils.sysconfig.get_config_var('LIBPL').

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1710767

GitHub PR: https://github.com/fedora-python/cpython/pull/2
2019-05-17 13:28:10 +02:00
Miro Hrončok 9fbe2eee9d rpmlint: Filter out shared-lib-without-dependency-information
Since Python 3.8.0a4, extension modules are no longer linked against
libpython3.8.so.1.0. As a result, some of them are not linked against
anything at all.

And that is fine. They are not used as standard shared libraries.
2019-05-16 10:33:14 +02:00
Miro Hrončok 7e0c125217 Regenerate generated files with python3
Switch to %make_build when touching this

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1377240
2019-05-08 00:12:04 +02:00
Miro Hrončok 285f5549c6 Update to 3.8.0a4
Patches rebased via git, see
https://github.com/fedora-python/cpython/tree/fedora-3.8.0a4

ABIFLAGS changed not to contain m, see https://bugs.python.org/issue36707
2019-05-08 00:11:20 +02:00
Miro Hrončok d3803a6dd6 Security fix for CVE-2019-9740 and CVE-2019-9947
Disallow control chars in http URLs
Fixed upstream: https://bugs.python.org/issue30458
2019-05-07 18:06:05 +02:00
Charalampos Stratakis 1b92cc7981 Fix handling of pre-normalization characters in urlsplit 2019-05-07 15:46:35 +00:00
Patrik Kopkan 084a366e3d add symlink for python3.Xm manpages 2019-04-25 18:10:22 +02:00
Miro Hrončok 0bee54773d Fedora CI: Provision 3 GiB of RAM 2019-04-25 12:59:53 +02:00
Patrik Kopkan 8f86cfe696 add symlink for python3.Xm manpages 2019-04-23 14:52:08 +02:00
Victor Stinner 35551f989b Reenable test_posix.test_fs_holes()
Since 2012, when the test was skipped from Python 3.3.0b1 (commit
3b4dd24c42), Linux now have a good
support for punching holes in filesystems.

Only 00160-disable-test_fs_holes-in-rpm-build.patch and
00163-disable-parts-of-test_socket-in-rpm-build.patch patches used
@unittest._skipInRpmBuild @unittest._expectedFailureInRpmBuild
decorators, but these 2 patches have been removed: remove
00132-add-rpmbuild-hooks-to-unittest.patch and
WITHIN_PYTHON_RPM_BUILD environment variable as well.
2019-04-11 16:51:20 +02:00
Victor Stinner b564dce4fa Reenable RecvmsgGenericStreamTests.testRecvmsgEOF()
Reenable RecvmsgGenericStreamTests.testRecvmsgEOF() of test_socket:
remove 00163-disable-parts-of-test_socket-in-rpm-build.patch.
2019-04-11 16:49:47 +02:00
Victor Stinner 0cd827e77e Reenable test_gdb on most archs except arm and s390x
gdb has been fixed in Rawhide:
https://bugzilla.redhat.com/show_bug.cgi?id=1638798
https://bugzilla.redhat.com/show_bug.cgi?id=1634781

Install also gdb in selftest to no longer skip test_gdb
on the x86_64 CI.

But test_gdb is still skipped on armv7hl:
https://bugzilla.redhat.com/show_bug.cgi?id=1196181

And skipped on s390x:
https://bugzilla.redhat.com/show_bug.cgi?id=1678277
2019-04-11 16:49:35 +02:00
Victor Stinner 41ec7ac424 Reenable test_buffer on ppc64le
The test failed because of a GCC bug which has been fixed in GCC
master branch and Rawhide uses GCC 9 (master branch):

* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88892
* https://bugzilla.redhat.com/show_bug.cgi?id=1540995
2019-04-11 16:25:16 +02:00
Tomas Orsava b02de40d57 Fix arch names for the ppc architecture
Updated patch 00274
2019-04-11 16:23:06 +02:00
Victor Stinner 9733ead195 Reenable test_posix.test_fs_holes()
Since 2012, when the test was skipped from Python 3.3.0b1 (commit
3b4dd24c42), Linux now have a good
support for punching holes in filesystems.

Only 00160-disable-test_fs_holes-in-rpm-build.patch and
00163-disable-parts-of-test_socket-in-rpm-build.patch patches used
@unittest._skipInRpmBuild @unittest._expectedFailureInRpmBuild
decorators, but these 2 patches have been removed: remove
00132-add-rpmbuild-hooks-to-unittest.patch and
WITHIN_PYTHON_RPM_BUILD environment variable as well.
2019-03-29 17:17:41 +01:00
Miro Hrončok 136178652c Temporarily skip test_asyncio to workaround bpo-35998
https://bugs.python.org/issue35998
2019-03-27 12:29:48 +01:00
Miro Hrončok 804ec7c37d Update to 3.8.0a3
Rebased patches: 102
Updated pip/setuptools versions in spec and patch 189
2019-03-27 01:14:40 +01:00
Miro Hrončok 6caee321b2 Update to 3.7.3 2019-03-27 01:03:17 +01:00
Miro Hrončok 45a4368c6e Rebase to 3.7.3rc1
Patches merged upstream: 317, 320
Patches rebased: 189 (pip and setuptools versions only)
2019-03-21 13:10:27 +00:00
Victor Stinner ff4d48f430 Reenable RecvmsgGenericStreamTests.testRecvmsgEOF()
Reenable RecvmsgGenericStreamTests.testRecvmsgEOF() of test_socket:
remove 00163-disable-parts-of-test_socket-in-rpm-build.patch.
2019-03-21 09:20:39 +00:00
Miro Hrončok 774c3f329b Temporarily skip test_asyncio to workaround bpo-35998
https://bugs.python.org/issue35998
2019-03-20 12:31:21 +01:00
Miro Hrončok ad7d856a30 Add bootstrap bcond for Python 3.8 2019-03-19 16:42:06 +01:00
Miro Hrončok 498b8834a0 Security fix for CVE-2019-9636 (#1688543, #1688546) 2019-03-19 09:19:02 +01:00
Miro Hrončok 5e25a87091 Update to 3.8.0a2 2019-02-26 00:51:48 +01:00
Victor Stinner 4b6961f25f Reenable test_gdb on most archs except arm and s390x
gdb has been fixed in Rawhide:
https://bugzilla.redhat.com/show_bug.cgi?id=1638798
https://bugzilla.redhat.com/show_bug.cgi?id=1634781

Install also gdb in selftest to no longer skip test_gdb
on the x86_64 CI.

But test_gdb is still skipped on armv7hl:
https://bugzilla.redhat.com/show_bug.cgi?id=1196181

And skipped on s390x:
https://bugzilla.redhat.com/show_bug.cgi?id=1678277
2019-02-19 16:19:01 +00:00
Miro Hrončok 8449402f77 Reduced default build flags used to build extension modules
Utilize extension_...flags for extension builds.

See https://fedoraproject.org/wiki/Changes/Python_Extension_Flags

Depends on e80fa1344a
2019-02-19 16:14:21 +01:00
Victor Stinner 1e30f555cf Reenable test_buffer on ppc64le
The test failed because of a GCC bug which has been fixed in GCC
master branch and Rawhide uses GCC 9 (master branch):

* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88892
* https://bugzilla.redhat.com/show_bug.cgi?id=1540995
2019-02-18 15:34:12 +00:00
Igor Gnatenko d080918767 Rebuild for readline 8.0 2019-02-17 09:30:52 +01:00
Igor Gnatenko 50afd5bbf5 Rebuild for readline 8.0 2019-02-17 09:30:52 +01:00
Tomas Orsava d8428d8696 Fix arch names for the ppc architecture
Updated patch 00274
2019-02-14 17:37:18 +01:00
Miro Hrončok 8927d3f267 Reduced default build flags used to build extension modules
Utilize extension_...flags for extension builds.

See https://fedoraproject.org/wiki/Changes/Python_Extension_Flags

Depends on e80fa1344a
2019-02-14 15:01:06 +01:00
Miro Hrončok 553fd279ba rpmlint: Python modules don't need to be linked against libc
They are loaded from Python and Python is already linked against libc.

See https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/QGJEDBFN7OXMP7OVTSVBO4DCGTX4K4WI/
2019-02-07 08:28:26 +01:00
Miro Hrončok 4aedf056ed Update Fedora CI tests definitions for 3.8 2019-02-06 17:28:30 +01:00
Miro Hrončok 1f112461d1 Trim the %changelog 2019-02-06 17:24:49 +01:00
Miro Hrončok b796d42367 Merge python38 from review request
https://bugzilla.redhat.com/show_bug.cgi?id=1672833
2019-02-06 17:23:15 +01:00
Gwyn Ciesla 13cea58a5c Added the README 2019-02-06 16:18:54 +00:00
Miro Hrončok a6496be3e2 rpmlint: Python modules don't need to be linked against libc
They are loaded from Python and Python is already linked against libc.

See https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/QGJEDBFN7OXMP7OVTSVBO4DCGTX4K4WI/
2019-02-06 02:25:20 +01:00
Miro Hrončok 3dcd1e8c08 Make flat, rename to python38 2019-02-05 12:35:53 +01:00
Miro Hrončok d051c04cb1 Drop obsoletes for platform-python 2019-02-05 12:33:45 +01:00
Miro Hrončok 1fb3c04b75 Update to 3.8.0a1
Uses tilde for prerelease information,
https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_versioning_prereleases_with_tilde

Patches rebased: 111, 132

Patches merged upstream: 155, 170 (differently), 317

/usr/bin/pyvenv is no more
2019-02-05 11:49:12 +01:00
Fedora Release Engineering 1cdc336f35 - Rebuilt for https://fedoraproject.org/wiki/Fedora_30_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2019-02-02 10:18:25 +00:00
Miro Hrončok 75005c20f6 Clean old cruft, provide pythonX.Y
In diff order:

- gdbm releases in all Fedoras are long above 1:1.13
- the flat package should be called python3.7 but we are lazy to rename
- system-python is long gone, upgrade path works
- python3-enum34 is only required by 2 pkgs [0][1]
- glibc releases in all Fedoras are long above 2.24.90-26

Not removed:

- platform-python obsoletes shall be removed in Fedora 31, they were added in 28

[0] https://src.fedoraproject.org/rpms/python-qt5/pull-request/3
[1] https://src.fedoraproject.org/rpms/python-taskflow/pull-request/1
2019-01-28 12:54:13 +01:00
Miro Hrončok 4e7015b153 Security fix for CVE-2019-5010
https://bugs.python.org/issue35746
https://bugzilla.redhat.com/show_bug.cgi?id=1666519
https://bugzilla.redhat.com/show_bug.cgi?id=1666522
2019-01-16 09:13:16 +01:00
Björn Esser 053863ab41
Rebuilt for libcrypt.so.2 (#1666033) 2019-01-14 18:53:46 +01:00
Miro Hrončok 41f5904a99 No longer revert upstream commit 3b699932e5ac3e7
This was a dirty workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1644936
2019-01-04 23:52:11 +01:00
Miro Hrončok 1385d6e65a Update to 3.7.2
Already upstreamed patches: 313, 315
Patches rebased: 111, 189 (new pip/setuptools versions only)
2018-12-25 13:38:36 +01:00
Miro Hrončok 15d5b5cfc5 Mark bdist_wininst unsupported to skip the tests
See also https://src.fedoraproject.org/rpms/python3/pull-request/74
2018-12-20 12:46:25 +01:00
Victor Stinner 0e0fe4ff87 Remove patches fixed upstream
Remove patches:

* 00291-setup-Link-ctypes-against-dl-explicitly.patch
  Fixed upstream: https://bugs.python.org/issue32647
* 00292-restore-PyExc_RecursionErrorInst-symbol.patch
  Fixed upstream: https://bugs.python.org/issue30697
* 00294-define-TLS-cipher-suite-on-build-time.patch
  Fixed upstream: https://bugs.python.org/issue31429
* 00301-pathfix-add-n-option-for-no-backup.patch
  Fixed upstream: https://bugs.python.org/issue32885
* 00302-fix-multiprocessing-regression-on-newer-glibcs.patch
  Fixed upstream: https://bugs.python.org/issue33329

Note: the ctypes issue has been fixed differently (better fix)
upstream.
2018-12-19 18:06:04 +01:00
Miro Hrončok 1092d478f5 Make sure we don't ship any exe files (not needed an prebuilt)
Such files are only used to create Microsoft Windows installers
and that functionality is broken on Linux anyway.

We can keep the exe files in SRPM, as they are free software as
confirmed by the Python Software Foundation:

> Hello,
>
> Shipping and distributing these files is ok.
>
> These files are stubs, used as the executable part of a self-unzipping
> executable and showing an "installer window" to the user.
> They are original to Python and, while they call win32 functions,
> they don't have Windows SDK code in them.
> The source is at: PC/bdist_wininst/install.c
>
> The binaries are included so that anyone creating a bdist_wininst
> distribution (mostly deprecated, actually) can do so, even when
> cross-compiling.
>
> Thanks,
> Van (Lindberg)

See also https://mail.python.org/pipermail/python-dev/2018-October/155507.html
2018-12-19 17:40:48 +01:00
Miro Hrončok 5258d1d194 Run the testsuite in parallel, --findleaks runs in the CI 2018-12-19 15:58:27 +01:00
Miro Hrončok bc86296869 Run the selftest suite on Fedora CI 2018-12-19 15:58:27 +01:00
Victor Stinner 7be8eebca6 Require glibc-all-langpacks to test locales
Python 3 test suite requires various locales. Install all of them to
build the package to make sure that no test is skipped because of a
missing locale.
2018-12-18 11:27:06 +01:00
Miro Hrončok 4f22584498 Fix mktime() error in test_email
http://bugs.python.org/issue35317

https://bugzilla.redhat.com/show_bug.cgi?id=1652843
2018-11-27 13:36:14 +01:00
Miro Hrončok e21e6a6b22 Show test.pythoninfo, helpful for debugging test failures 2018-11-26 15:52:56 +01:00
Miro Hrončok 31d96372de Make sure the entire test.support module is in python3-libs (#1651245) 2018-11-21 18:21:13 +01:00
Victor Stinner a4e394925c Verify the value of '-s' when execute the CLI of cProfile
Resolves: rhbz#1160640
2018-11-19 10:30:19 +01:00
Miro Hrončok 72bc3f43c4 Temporarily revert upstream commit 3b699932e5ac3e7
This is dirty workaround for
https://bugzilla.redhat.com/show_bug.cgi?id=1644936
2018-11-04 19:26:46 +01:00
Miro Hrončok d533a32456 Update to 3.7.1
Removed patches due to being upstreamed: 307, 308

Rebased patches: 170
2018-10-22 10:15:53 +02:00
Petr Viktorin 091b4b20e9 Compile the debug build with -Og rather than -O0
That is, "optimize for debugging experience" rather than
"don't optimize".
2018-10-02 16:38:43 +02:00
Charalampos Stratakis e354fb5703 Disable test_gdb for now as it fails
with latest gdb.
2018-10-01 20:12:07 +02:00
Miro Hrončok 33cd8ff371 rpmlintrc updates
* put 64 in ()? to support 32bit RPMs
 * add new executables to the manpages ignore list
2018-09-03 14:10:31 +02:00
Petr Šplíchal 704ecff960 Enable basic venv smoke test in the CI
This executes virtual environment smoke test from the shared
python tests repository against Python 3.7.
2018-08-31 11:03:06 +02:00
Miro Hrončok 4da7cff796 Require python3-setuptools from python3-devel to prevent packaging errors (#1623914) 2018-08-30 15:35:14 +02:00
Miro Hrončok 37056bc0f9 Renew check-pyc-timestamps.py
- rename it, the are no pyo files on Python 3
- make sure to check all levels of pyc files optimization
- use path globs
- use comprehensions instead of map + filter
- use f-strings and print() instead of sys.stderr.write()
2018-08-17 16:05:13 +02:00
Miro Hrončok 83a5f52678 Add /usr/bin/pygettext3.py and msgfmt3.py to python3-devel
Resolves: rhbz#1571474
2018-08-17 16:05:04 +02:00
Miro Hrončok 4aa9ebcc03 Backport TLS 1.3 related fixes to fix FTBFS
Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1609291
2018-08-17 15:40:30 +02:00
Miro Hrončok b9e8a46cca Recommend setuptools and pip for developer experience 2018-08-16 12:30:28 +02:00
Miro Hrončok 0b241abda6 Drop the rewheel patch, use RPM built wheels instead 2018-08-15 17:55:47 +02:00
Igor Gnatenko da8bebda6d
Fix wrong requirement on gdbm
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2018-08-10 12:56:24 +02:00
Miro Hrončok a64e7f7e98 Use the %{valgrind_arches} macro instead of hardcoding unsupported arches
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/7GJQDPCQ743DKJ3LAJFDDX3ZD6LOPB5N/
2018-08-04 22:07:21 +02:00
Miro Hrončok 37437e1d44 Allow to call Py_Main() after Py_Initialize()
Resolves: rhbz#1595421
2018-07-20 17:48:24 +02:00
Miro Hrončok d90f9872b6 Replace no longer relevant notes from bootstrapping 3.6 with link to 3.7 2018-07-19 15:30:36 +02:00
Fedora Release Engineering 266317a762 - Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2018-07-14 02:50:11 +00:00
Igor Gnatenko 7a4ecff9a7
Rebuild for new gdbm
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2018-07-12 10:58:16 +02:00
Miro Hrončok dc63bc71b5 Update to 3.7.0 final 2018-06-27 12:45:38 +02:00
Miro Hrončok 838e397e16 Install appdata files to %{_metainfodir}
See: https://www.freedesktop.org/software/appstream/docs/sect-Metadata-Application.html
and: https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html
2018-06-18 23:44:35 +02:00
Miro Hrončok 40d6240272 Remove %defattr(-, root, root)
It is not needed since gods know when. Is's the default.
2018-06-18 23:42:46 +02:00
Miro Hrončok d780a93d16 Don't hardcode pip version, make it dynamic at build time
This will make test_with_pip (test.test_venv.EnsurePipTest) happy.
2018-06-13 20:47:48 +02:00
Miro Hrončok 3ba1d1aadf Finish bootstrapping, enable rewheel, tests, optimizations 2018-06-13 19:14:30 +02:00
Miro Hrončok 2266a26125 Bootstrap, disable tests, optimizations 2018-06-13 15:27:50 +02:00
Miro Hrončok 00bb7bddf2 Bootstrap, disable rewheel 2018-06-13 14:54:46 +02:00
Miro Hrončok cb1167a878 Merge from python37, update to 3.7.0rc1 2018-06-13 14:54:39 +02:00
Miro Hrončok de441a2d3a Fix prerel comment 2018-06-12 12:42:28 +02:00
Miro Hrončok ceff4b8870 Add 3394 magic number to rpmlintrc 2018-06-12 11:47:27 +02:00
Miro Hrončok 63f277fa3b Update to 3.7.0rc1 2018-06-12 11:01:05 +02:00
Miro Hrončok bcf8a5d914 Update to 3.7.0b5 2018-05-31 11:45:12 +02:00
Tomas Orsava 8fcf2dac2d Update the bootstrapping sequence 2018-05-24 18:13:25 +02:00
Tomas Orsava 2d5b84388b Update the bootstrapping sequence 2018-05-24 18:11:21 +02:00
Charalampos Stratakis 29a768cb73 Merge #23 `Rewheel related: rebase the patch and bump the pip bundled version` 2018-05-14 09:58:59 +00:00
Miro Hrončok 611414f20d Bump the bundled pip version to 10.0.1 2018-05-13 13:51:29 +02:00
Miro Hrončok 63266cf9ed Rebase the rewheel patch
It was not rebased before, because it was never applied
2018-05-13 13:51:29 +02:00
Miro Hrončok 8810a59050 Only require python3-rpm-generators when built with rewheel
Without rewheel is used to bootstrap setuptools+pip
python3-rpm-generators needs python3-setuptools, so we cannot have it yet
2018-05-11 23:27:10 +02:00
Miro Hrončok 0c391afb69 Update to 3.7.0b4 2018-05-10 16:58:07 +02:00
Miro Hrončok e324bf2dbb Fix multiprocessing regression on newer glibcs
Enable test_multiprocessing_fork(server) and _spawn again

Resolves: rhbz#1569933
2018-04-23 18:17:32 +02:00
Miro Hrončok 240a1f769d Fix multiprocessing regression on newer glibcs
Enable test_multiprocessing_fork(server) and _spawn again

Resolves: rhbz#1569933
2018-04-23 14:30:11 +02:00
Miro Hrončok 45a046d2f4 Skip test_multiprocessing_fork(server) and _spawn for now
It hangs the build
2018-04-19 22:18:28 +02:00
Miro Hrončok 872e1bafb7 Skip test_multiprocessing_fork(server) and _spawn for now
It hangs the build
2018-04-19 20:27:15 +02:00
Miro Hrončok fc23c477cf Typos and such
From https://src.fedoraproject.org/rpms/python3/pull-request/33
2018-04-18 17:14:16 +02:00
Miro Hrončok 8c67bafd7c Add flatpackage conditionals
This will allow us to create python36 easily once needed [1].
It shoudl also make it easier to merge stuff between python36 and 37.

This more or less backports:

63459d7d2a
7f3ae64d2f
6a6d4a0324

[1] https://fedoraproject.org/wiki/Changes/Python3.7
2018-04-18 17:10:57 +02:00
Miro Hrončok 34ebe3da13 Merge master with f28 2018-04-18 13:36:00 +02:00
Miro Hrončok 8799184fe7 Remove macros.systempython
It was for backward compatibility only, to be removed in F29.
This will only become non-flat package in F29+.
2018-03-29 18:10:26 +02:00
Charalampos Stratakis 9a2e4be589 Update to 3.6.5
Rebased patches: 102, 111, 262

Removed patches due to being upstreamed:
264, 273, 289, 290, 291, 298

Update pip version to 9.0.3
2018-03-29 16:57:50 +02:00
Charalampos Stratakis fb9702fb7f Update to 3.6.5
Rebased patches: 102, 111, 262

Removed patches due to being upstreamed:
264, 273, 289, 290, 291, 298

Update pip version to 9.0.3
2018-03-29 16:56:36 +02:00
Miro Hrončok 3422f8b4a4 Remove macros.pybytecompile3.7
It will live in python-rpm-macros, see:
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/4
2018-03-29 15:03:53 +02:00
Miro Hrončok a50d5ce864 Update to 3.7.0b3 2018-03-29 15:00:59 +02:00
Miro Hrončok 6f038c8ede Update setuptools and pip bundled versions 2018-03-27 13:24:32 +02:00
Miro Hrončok cb0fe74d03 Fix shebangs of the GDB hooks
Also, use -p (preserve timestamp) and -n (don't create backup files)
with pathfix.py.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1560295
2018-03-25 20:55:08 +02:00
Miro Hrončok 6c5992ec4e Fix shebangs of the GDB hooks
Also, use -p (preserve timestamp) and -n (don't create backup files)
with pathfix.py.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1560295
2018-03-25 20:51:42 +02:00
Miro Hrončok 34f9f5ef13 rpmlintrc: Filter macro-in-comment %{_pyconfig(32|64)_h} 2018-03-25 20:32:42 +02:00
Miro Hrončok 3777cb5efa Fix broken macro invocation and broken building of C Python extensions
Revert "Use %% for actual % in spec"

This reverts commit acda1a219a.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1560103
2018-03-25 20:32:13 +02:00
Miro Hrončok 37c970a4db rpmlintrc: Filter macro-in-comment %{_pyconfig(32|64)_h} 2018-03-24 08:06:56 +01:00
Miro Hrončok e8fa184158 Fix broken macro invocation and broken building of C Python extensions
Revert "Use %% for actual % in spec"

This reverts commit 90512a5a1b.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1560103
2018-03-24 08:04:21 +01:00
Miro Hrončok 58c6b177d4 Add rpmlintrc file
Filter all the errors and warnings. This allows us to actually read the rpmlint
output to get new information. From now on, we can rely on this information
when pushing updates.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1548683

Backport of https://src.fedoraproject.org/rpms/python37/pull-request/10
2018-03-20 11:28:40 +01:00
Miro Hrončok 90512a5a1b Use %% for actual % in spec
rpmlint reports this as macro in comment, however it was left here, because it
is not macro and not comment. On the other hand, % shall be escaped using %%.
All this needs to blow is somebody defining a macro called _pyconfig64_h.
2018-03-20 11:28:40 +01:00
Miro Hrončok 545e680253 Add -n option for pathfix.py (#1546990) 2018-03-20 11:13:09 +01:00
Miro Hrončok e5f4159f8f Fix the py_byte_compile macro to work on Python 2
See https://bugzilla.redhat.com/show_bug.cgi?id=1484993

Inspired by Terje Røsten's workaround from that bugzilla
2018-03-15 23:10:56 +01:00
Miro Hrončok 94ba907c0b rpmlintrc: Add stuff from the nonflat package 2018-03-14 12:00:35 +01:00
Miro Hrončok 42c61b3ece Add rpmlintrc file
Filter all the errors and warnings. This allows us to actually read the rpmlint
output to get new information. From now on, we can rely on this information
when pushing updates.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1548683
2018-03-14 11:59:53 +01:00
Charalampos Stratakis c9bb114a1d Do not send IP addresses in SNI TLS extension 2018-03-13 16:58:18 +01:00
Miro Hrončok acda1a219a Use %% for actual % in spec
rpmlint reports this as macro in comment, however it was left here, because it
is not macro and not comment. On the other hand, % shall be escaped using %%.
All this needs to blow is somebody defining a macro called _pyconfig64_h.
2018-03-01 15:19:51 +01:00
Miro Hrončok ef18b4528a Update to 3.7.0b2 2018-02-28 12:18:01 +01:00
Miro Hrončok 6a6a8d45dd Fix the py_byte_compile macro to work on Python 2
See https://bugzilla.redhat.com/show_bug.cgi?id=1484993

Inspired by Terje Røsten's workaround from that bugzilla

This also removes the pybytecompile macro file from the flat package because it
doesn't belong there as pointed out during the review and later forgotten.
2018-02-28 09:48:35 +01:00
Charalampos Stratakis 6b9e1d1d7a Replace patch 291 with the upstream fix. 2018-02-26 17:28:35 +01:00
Florian Weimer 347e968f54 Rebuild with new LDFLAGS from redhat-rpm-config 2018-02-24 18:28:44 +01:00
Iryna Shcherbina f52973f5cb Reenable test_gdb on x86_64
The test was disabled for Python 3.6.4 where it fails currently, but it
completes successfully on Python 3.7. See commit:
d84538b320

Enable also runnig test_gdb on aarch64, as failures are no longer observed.
The bugzilla associated with the reqson why it was disabled is long closed.
See commit: 3b7dcebf39
2018-02-22 17:23:17 +01:00
Iryna Shcherbina a2c6e9ba55 Do not skip test_faulthandler on ppc64le and aarch64 as it no longer fails 2018-02-22 17:23:17 +01:00
Iryna Shcherbina de5e1e875a test_pass_by_value no longer fails on aarch64; patch removed 2018-02-22 17:23:17 +01:00
Miro Hrončok 3f778c43e3 Filter out automatic /usr/bin/python3.X requirement...
...recommend the main package from libs instead.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1547131
2018-02-21 17:28:00 +01:00
Miro Hrončok 06a649c4a5 Filter out automatic /usr/bin/python3.X requirement...
...recommend the main package from libs instead.

Resolves https://bugzilla.redhat.com/show_bug.cgi?id=1547131
2018-02-21 14:36:42 +01:00
Miro Hrončok 6e4c18d0c7 Remove %ldconfig_scriptlets (F28+ only)
%ldconfig_scriptlets does nothing on F28+.

The original idea was to keep is present not to unnecessarily diverge master
and f27, but since that have happened anyway, I'm removing it for spec
simplicity. Not adding a new changelog entry (it's not user facing) nor bumping
the release (it doesn't need to be rebuilt, it changes nothing). Altering the
changelog entry that was added when %ldconfig_scriptlets was added.
2018-02-17 21:01:39 +01:00
Iryna Shcherbina ffaa0f3078 Remove unused Postfix argument from InstallPython
It was used for Systemtap hooks, which were removed recently:
dd3cb47629
2018-02-16 18:32:06 +01:00
Iryna Shcherbina 56c7fd9871 Fix deprecation warning on using imp
In build log fixes the following warning:
/builddir/build/SOURCES/check-pyc-and-pyo-timestamps.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2018-02-16 18:32:06 +01:00
Iryna Shcherbina daeec838fe Remove not used patches
Should have been removed with update to 3.6.3
47069ff417
2018-02-16 18:32:06 +01:00
Iryna Shcherbina 3bbc2a0bdf Fix rpmlint errors: spelling, tabs (bz#1535596) 2018-02-16 18:32:06 +01:00
Michal Cyprian bf7378574e Remove sys.executable check from change-user-install-location patch
Resolves: rhbz#1532287
2018-02-16 15:33:54 +01:00
Iryna Shcherbina d938460ea1 Remove the python3-tools package (bz#1312030)
For more details see https://bugzilla.redhat.com/show_bug.cgi?id=1312030

Scope:

- Remove the python3-tools package and `Tools/` library
- Move /usr/bin/2to3 to python3-devel
- Move /usr/bin/idle and idlelib to python3-idle
- Provide python3-tools from python3-idle to keep upgrades sane
- Remove tests for tools from python3-tests package

Backported from https://src.fedoraproject.org/rpms/python37/pull-request/4
2018-02-15 21:14:03 +01:00
Iryna Shcherbina 9203532359 Remove unused Postfix argument from InstallPython
It was used for Systemtap hooks, which were removed recently:
31d212b3c7
2018-02-15 14:47:38 +01:00
Iryna Shcherbina f3f5f60ac0 Fix deprecation warning on using imp
In build log fixes the following warning:
/builddir/build/SOURCES/check-pyc-and-pyo-timestamps.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2018-02-15 14:47:38 +01:00
Iryna Shcherbina 627aaf656c Remove not used patches
Should have been removed with update to 3.6.3
47069ff417
2018-02-15 14:47:38 +01:00
Iryna Shcherbina ac19ab7c07 Fix rpmlint errors: spelling, tabs (bz#1535596) 2018-02-15 14:47:21 +01:00
Miro Hrončok 2bcebb2b16 Fix the name macro in the description 2 2018-02-15 14:35:08 +01:00
Miro Hrončok 654b8b4665 Fix the name macro in the description 2 2018-02-15 14:34:07 +01:00
Charalampos Stratakis 1161a4660d Fix the name macro in the description 2018-02-15 14:32:54 +01:00
Miro Hrončok 0679427e97 Readd ldconfig scriplets via macro 2018-02-14 18:48:02 +01:00
Miro Hrončok cc54a40057 3.7.0b1 sources 2018-02-12 19:44:15 +01:00
Charalampos Stratakis 02bfb4dc2a Define TLS cipher suite on build time 2018-02-12 12:43:21 +01:00
Tomas Orsava d84538b320 Disable test_gdb for all arches and test_buffer for ppc64le in anticipation of the F28 mass rebuild
- Re-enable these tests after the mass rebuild when they can be properly
  addressed
2018-02-12 12:43:21 +01:00
Iryna Shcherbina cea6d60615 Update to 3.7.0b1
- Rebase patches 102, 170
- Remove patches 273, 289, 290
2018-02-12 12:43:21 +01:00
Charalampos Stratakis 10ba826cb6 Synced patch for libxcrypt with upstream 2018-02-12 12:41:06 +01:00
Björn Esser d3af3ef92c Add patch to explicitly link _ctypes module with -ldl (#1537489)
Add patch to explicitly link _ctypes module with -ldl (#1537489)
Refactored patch for libxcrypt
Re-enable strict symbol checks in the link editor
2018-02-12 12:41:06 +01:00
Charalampos Stratakis 221978fced Disable strict symbol checks in the link editor. 2018-02-12 12:41:06 +01:00
Björn Esser 212f3d7eb8 Add patch for libxcrypt 2018-02-12 12:41:06 +01:00
Igor Gnatenko 8ab3c20b1a R: gdbm-devel → R: gdbm for python3-libs
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2018-02-12 12:41:06 +01:00
Miro Hrončok 60eff5e207 Require large enough gdbm (fixup for previous commit) 2018-02-12 12:41:06 +01:00
Charalampos Stratakis 13f7a0acb9 Rebuild for reverted gdbm 1:1.13 on F27 2018-02-12 12:41:06 +01:00
Charalampos Stratakis 0b33e0cf27 Remove a ppc64 segfault workaround which provided a larger stack for that arch, as it doesn't seem to affect the build anymore. 2018-02-12 12:41:06 +01:00
Charalampos Stratakis 3dcb34959e Masc two macros in comments that were expanded.
Remove the commented out file for the time shared library.
2018-02-12 12:41:06 +01:00
Charalampos Stratakis 6f9a91b642 Remove python-gdb.py source file as it now gets installed from the upstream sources 2018-02-12 12:41:06 +01:00
Charalampos Stratakis 31d212b3c7 Remove our downstream systemtap instrumentation as now upstream provides us with dtrace hooks. 2018-02-12 12:41:06 +01:00
Fedora Release Engineering 72d368458c - Rebuilt for https://fedoraproject.org/wiki/Fedora_28_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2018-02-09 12:01:54 +00:00
Igor Gnatenko 40b8f9ece2
Escape macros in %changelog
Reference: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/Y2ZUKK2B7T2IKXPMODNF6HB2O5T5TS6H/
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2018-02-09 09:05:39 +01:00
Charalampos Stratakis af1dc2628d Fix the name macro in the description 2018-02-08 14:54:08 +01:00
Michal Cyprian 08ffd081b5 Remove sys.executable check from change-user-install-location patch
Resolves: rhbz#1532287
2018-02-07 14:37:45 +01:00
Charalampos Stratakis c86891d488 Remove obsolete scriptlets that are now automatic. 2018-02-05 17:59:56 +01:00
Charalampos Stratakis 14deb52d02 Remove obsolete scriptlets that are now automatic. 2018-02-05 17:50:53 +01:00
Charalampos Stratakis 969d51434e Define TLS cipher suite on build time 2018-02-01 11:20:15 +01:00
Tomas Orsava be04920913 Disable test_gdb for all arches and test_buffer for ppc64le in anticipation of the F28 mass rebuild
- Re-enable these tests after the mass rebuild when they can be properly
  addressed
2018-01-31 18:55:27 +01:00
Iryna Shcherbina dc669abc1d Remove the python3-tools package (bz#1312030)
For more details see https://bugzilla.redhat.com/show_bug.cgi?id=1312030

Scope:

- Remove the python3-tools package and `Tools/` library
- Move /usr/bin/2to3 to python3-devel
- Move /usr/bin/idle and idlelib to python3-idle
- Provide python3-tools from python3-idle to keep upgrades sane
- Remove tests for tools from python3-tests package

Merges https://src.fedoraproject.org/rpms/python37/pull-request/4
2018-01-30 19:48:12 +01:00
Charalampos Stratakis d623e929ca Restore the PyExc_RecursionErrorInst public symbol 2018-01-23 18:09:30 +01:00
Charalampos Stratakis 434d469259 Synced patch for libxcrypt with upstream 2018-01-23 18:09:27 +01:00
Björn Esser ce6ec4f16a
Add patch to explicitly link _ctypes module with -ldl (#1537489)
Refactored patch for libxcrypt
Re-enable strict symbol checks in the link editor
2018-01-23 15:39:07 +01:00
Charalampos Stratakis 5a8bf97593 Disable strict symbol checks in the link editor. 2018-01-23 13:22:12 +01:00
Björn Esser 5ea7bf417d
Add patch for libxcrypt 2018-01-22 21:10:32 +01:00
Björn Esser 08148d40f8
Rebuilt for switch to libxcrypt 2018-01-20 23:07:44 +01:00
Björn Esser 085ae328d4
Rebuilt for switch to libxcrypt 2018-01-20 23:07:42 +01:00
Charalampos Stratakis 40bb48b584 Properly add patch 273 2018-01-19 17:44:40 +01:00
Charalampos Stratakis 26024771df Fix localeconv() encoding for LC_NUMERIC 2018-01-19 17:36:05 +01:00
Igor Gnatenko 57ec20127d
R: gdbm-devel → R: gdbm for python3-libs
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
2018-01-18 19:41:19 +01:00
Miro Hrončok 6a6d4a0324 Move install stuff from %check to %install, where it belongs (oops) 2018-01-18 16:03:21 +01:00
Miro Hrončok 7f3ae64d2f Remove /usr/bin/python37 (#1526129) 2018-01-18 16:02:54 +01:00
Miro Hrončok c9349a73c7 Update to 3.7.0 alpha 4 2018-01-17 13:54:10 +01:00
Miro Hrončok 1a43a65185 Require large enough gdbm (fixup for previous commit) 2018-01-17 11:50:59 +01:00
Charalampos Stratakis 6e38667b8a Rebuild for reverted gdbm 1:1.13 on F27 2018-01-16 20:36:17 +01:00
Charalampos Stratakis 0077a19780 Update to version 3.6.4
Rebased patches: 189, 262

Dropped patches due to being upstreamed: 277, 279
2018-01-15 15:52:17 +01:00
Charalampos Stratakis 6ff303ee10 Fix the compilation of the nis module.
The headers required to build the nis module were removed
from glibc and they are now provided by the libtirpc and libnsl2
packages, thus adding their respective -devel subpackages as
BuildRequires and the upstream fix to search for the headers
and the shared libraries at the appropriate locations.
2018-01-15 15:31:28 +01:00
Charalampos Stratakis ec5f432a6e Remove a ppc64 segfault workaround which provided a larger stack for that
arch, as it doesn't seem to affect the build anymore.
2017-12-04 17:53:50 +01:00
Charalampos Stratakis 15a050eb4b Masc two macros in comments that were expanded.
Remove the commented out file for the time shared library.
2017-12-04 14:41:46 +01:00
Charalampos Stratakis 522ee16d1a Remove python-gdb.py source file as it now gets installed from the upstream sources 2017-12-04 14:40:50 +01:00
Charalampos Stratakis dd3cb47629 Remove our downstream systemtap instrumentation as now upstream provides us
with dtrace hooks.
2017-12-04 14:40:33 +01:00
Miro Hrončok d673becd53 Ignore stuff 2017-12-01 13:57:42 +01:00
Miro Hrončok 2f82270763 Add sources for 3.7.0a2 2017-12-01 13:55:24 +01:00
Miro Hrončok c93495efdf Purge %changelog 2017-12-01 13:53:36 +01:00
Miro Hrončok 1da74de71e Rename the specfile to fit package name 2017-12-01 13:52:53 +01:00
Miro Hrončok cf16d06001 Merge python37 from churchyard/py37 2017-12-01 13:52:03 +01:00
Gwyn Ciesla 33e1cb6f54 Added the README 2017-12-01 12:28:37 +00:00
Miro Hrončok 08796195b5 Remove /usr/bin/2to3-3.7 2017-11-30 16:47:53 +01:00
Miro Hrončok 63459d7d2a Add flatpackage conditional for python37 2017-11-30 16:40:32 +01:00
Miro Hrončok d668c1e89c Python 3.7 (fedpkg prep ok, builds ok without debug) 2017-11-30 09:25:27 +01:00
Miro Hrončok 1f528dac57 Raise the release of platform-python obsoletes for better maintainability 2017-11-21 11:04:48 +01:00
Miro Hrončok e522c84bc5 Obsolete platform-python and it's subpackages 2017-11-15 16:05:21 +01:00
Charalampos Stratakis a8c86d3aed Apply the patch 279 correctly. 2017-10-09 13:21:53 +02:00
Charalampos Stratakis 22b8f2b10b Fix memory corruption due to allocator mix (rhbz#1498207) 2017-10-09 13:10:17 +02:00
Charalampos Stratakis 47069ff417 Update to Python 3.6.3
Patches 270, 271, 272 were removed as they have been upstreamed.

Patch 277 was added to fix two hanging tests from test_subprocess.

Added gdb at the build dependencies, since gdb is not anymore in the
buildroot, and that made test_gdb to be skipped.
2017-10-06 13:30:25 +02:00
Miro Hrončok 71718b673e Make the -devel package require redhat-rpm-config
https://bugzilla.redhat.com/show_bug.cgi?id=1217376
https://bugzilla.redhat.com/show_bug.cgi?id=1496757
https://bugzilla.redhat.com/show_bug.cgi?id=1218294

Later should be changed to a specific subpackage.
2017-09-29 12:30:12 +02:00
Miro Hrončok 0652b06063 Move pathfix.py to bindir
See https://github.com/fedora-python/python-rpm-porting/issues/24
2017-09-20 14:16:59 +02:00
Iryna Shcherbina 685fb7bc99 Fix /usr/bin/env dependency from python3-tools (rhbz#1482118) 2017-09-14 15:30:27 +02:00
Iryna Shcherbina 87e35ed7a5 Include -g in LDFLAGS sent to the linker (rhbz#1483222)
Include -g flag for both compilation and linking so that the debugging
information is correctly generated.
2017-09-06 17:19:02 +02:00
Petr Viktorin 7d6b083b83 Add changelog entry for this round of cleanups 2017-09-05 17:31:28 +02:00
Petr Viktorin 15a5ab1694 Make the main description also applicable to the SRPM
Also, add a comment to highlight its dual purpose
2017-09-05 11:54:13 +02:00
Petr Viktorin 7c72174fbd Snipe a typo 2017-09-05 11:54:06 +02:00
Petr Viktorin 4ced9a188e Check %{_lib} rather than hardcode all 64-bit arches
Checking %{lib} isn't guaranteed to determine the word size
correctly, but it works for current arches and is more likely
to stay correct when a new arch are added.
2017-09-05 11:43:32 +02:00
Petr Viktorin 4f6b427d2e Include audiotest.au again
These files were removed because historically they included
a short sound clip from Monty Python shows, which might fall
under copyright.
In 2010 this was replaced upstream by a recording of Guido
van Rossum saying something silly in Dutch. So, there's no
reason to exclude the file anymore.

The bug mentioned in the removed comment is private due to
corporate policy from 2004. Sorry for that!
The problem itself is discussed at: http://bugs.python.org/issue7755
2017-09-05 11:33:00 +02:00
Petr Viktorin 39c4234d48 Remove the %clean section 2017-09-04 16:06:13 +02:00
Petr Viktorin 64593af0dd Move systemtap tapfile installation into the per-build function 2017-09-04 16:06:13 +02:00
Petr Viktorin 6c5169565e Move multilib support for pyconfig.h to the per-build section 2017-09-04 16:06:13 +02:00
Petr Viktorin 474cc9f637 install: Move renaming -devel scripts to the per-build function 2017-09-04 16:06:13 +02:00
Petr Viktorin ec8375ccf7 Move packaging checks from %install to %check 2017-09-04 16:06:13 +02:00
Petr Viktorin d54a775b46 Remove some unneeded old cleanups in %install 2017-09-04 16:06:13 +02:00
Petr Viktorin f86b3d1707 Move creation of site-packages/__pycache__ dirs together 2017-09-04 16:06:13 +02:00
Petr Viktorin 208332b1c9 Clean up the start of the %install section
- Rewrite comments
- Move common part of GDB hook install out of the per-build function
2017-09-04 16:06:13 +02:00
Petr Viktorin b3fc51be32 Replace ${RPM_BUILD_ROOT} by %{buildroot} 2017-09-04 16:06:13 +02:00
Petr Viktorin db2797c8b7 Clean up the %build section
- Add more explanatory comments, remove historical notes
- Put configuration options in %%global
- Shell: use $() instead of backticks
- Remove unused aruments to BuildPython()! (fun fact: most were *never* used!)
2017-09-04 16:06:13 +02:00
Petr Viktorin e441743e28 Don't remove the buildroot in %install 2017-09-04 16:06:13 +02:00
Miro Hrončok 1acd7dfa33 Remove %{pylibdir}/Tools/scripts/2to3
* the file has wrong shebang (#!/usr/bin/env python)
    * it is not covered by the shebang correction because it is not
      named with .py extension
    * a note has been added to the shebang correction code about this
 * ignoring shebang, the file is identical to /usr/bin/2to3
   * thus redundant
2017-09-04 10:54:33 +02:00
Miro Hrončok 5f24f39d01 For consistency, require /usr/sbin/ifconfig instead of net-tools 2017-09-01 11:49:17 +02:00
Miro Hrončok 51501f9a0b Conditionalize systemtap-devel BuildRequires 2017-09-01 11:45:01 +02:00
Miro Hrončok 33e1101626 Expat >= 2.1.0 is everywhere, remove explicit requires
(Except EL6)
2017-09-01 11:04:02 +02:00
Miro Hrončok 357f40b6c4 Move python2-tools conflicts to tools subpackage
They were previously wrongly placed in tkinter subpackage
2017-09-01 10:43:39 +02:00
Petr Viktorin 1151a3e1a4 Remove commented-out prep code for hashlib OpenSSL/FIPS patch
A task is open in Bugzilla for adding this back:
https://bugzilla.redhat.com/show_bug.cgi?id=1486212
2017-08-31 14:26:42 +02:00
Petr Viktorin 3a884e4af3 Make module removal (unbundling) more strict
Failures from removing bundled expat or zlib should not be silent;
if they occur they should be investigated.

Also, reword the comments for conciseness
2017-08-31 14:26:42 +02:00
Petr Viktorin 9faeaef40a Remove fix for version-specific documentation URLs in pydoc
This was fixed upstream some time ago. The removed sed invocation
did nothing.
2017-08-31 14:26:42 +02:00
Petr Viktorin 28aab8e5db Remove the BuildRoot tag
According to current guidelines, this SHOULD NOT be used.
2017-08-31 14:26:42 +02:00
Petr Viktorin 9423d36fb4 Consolidate architecture-naming patches in one
Patches 180, 206, 243 and 5001 all changed architecture names
from Debian style (which upstream uses) to what Fedora uses.
Consolidate them into a single patch, 274.

Delete changes to `configure` from patch 243 without replacement;
that file is generated from `configure.ac`.
2017-08-31 14:26:42 +02:00
Petr Viktorin ac06deac65 Remove patch 188, a temporary workaround for hashlib tests 2017-08-31 14:26:42 +02:00
Petr Viktorin d8970d7383 Remove patch 186, which added a duplicate test
The actual change is upstream; the patch as it was just added an
inferior version of upstream's test_bad_coding.
2017-08-31 14:26:42 +02:00
Petr Viktorin 720c2ade20 Update comment for patch 170 2017-08-31 14:26:42 +02:00
Petr Viktorin 61c2d437a2 Remove patch 157, which contained test changes left over after upstreaming 2017-08-31 14:26:42 +02:00
Petr Viktorin c4a5733df6 Use full URLs for bugzilla bugs
Seasoned packagers have pattern-matching circuits for "rhbz"
embedded in their brains, but for newcomers it's one more
opaque, non-googlable string.

The original comment for patch 00111 had a wrong bug number.
Correct it.
2017-08-31 14:26:42 +02:00
Petr Viktorin bbf2225353 Skip test_bdist_rpm using test config rather than a patch
Remove patch 137
2017-08-31 14:26:42 +02:00
Petr Viktorin 2c6abcc9a3 Merge lib64 patches into one (104 into 102)
It doean't make sense for these to be separate.
2017-08-31 14:26:42 +02:00
Petr Viktorin 96a7bd540e Run autotools to generate the configure script before building
Partially fixes https://bugzilla.redhat.com/show_bug.cgi?id=1377240
2017-08-31 14:26:42 +02:00
Petr Viktorin d27ccf6033 Rename patch files to start with the number
A consistent naming scheme makes managing patch files easier.
2017-08-31 14:26:42 +02:00
Michal Cyprian eef42fd7ea Use python3 style of calling super() without arguments in rpath patch 2017-08-31 14:22:22 +02:00
Tomas Orsava b7855d3644 Add a warning about updating python3-docs 2017-08-28 15:20:28 +02:00
Petr Viktorin 37ed19d2c9 Add forgotten patch file 2017-08-28 13:39:39 +02:00
Petr Viktorin 74f6a3d59e Skip failing test_float_with_comma
I am unable to reproduce this bug outside of a build environment.
Skip the test to make Python build again.
Bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=1484497
2017-08-23 23:42:09 +02:00
Petr Viktorin 094ccc055a Reword comments for descriptions and subpackage metadata 2017-08-23 12:01:07 +02:00
Petr Viktorin 44bb068a8e Top-level metadata: move URL, reword Summary 2017-08-23 12:01:07 +02:00
Petr Viktorin 737c23e807 Remove unused patches 2017-08-23 12:01:07 +02:00
Petr Viktorin f50a7d31c3 Reword and expand comment on Provides/Obsoletes of pythonXY 2017-08-22 15:53:24 +02:00
Petr Viktorin 1824b3ec27 Use configuration macro instead of workaround to disable byte-compilation 2017-08-22 15:32:48 +02:00
Petr Viktorin 66c621f01a Re-alphabetize the BuidRequires section 2017-08-22 14:33:54 +02:00
Petr Viktorin 88b30e0657 Remove duplicate globals 2017-08-22 14:21:01 +02:00
Petr Viktorin c2f8719035 Reword package summaries and descriptions
Also, remove Group tags
2017-08-22 00:02:34 +02:00
Petr Viktorin 59c11e6706 Revise the bootstrapping notes 2017-08-22 00:02:34 +02:00
Petr Viktorin 0c74e11532 Remove historical information from the global macros section 2017-08-22 00:02:34 +02:00
Petr Viktorin f63e273cfd Document configuration options 2017-08-22 00:02:33 +02:00
Petr Viktorin a7a170bba6 Add bcond for --without optimizations 2017-08-22 00:02:06 +02:00
Miro Hrončok 11efc8ea59 Remove system-python 2017-08-21 13:36:10 +02:00
Petr Viktorin 043c57400a Use bconds for configuring the build
This makes configuring the build much easier, at least with
rpmbuild and mock.

Also, reorganize the initial sections, where the config options
were scattered.
2017-08-16 17:05:00 +02:00
Miro Hrončok b0aea52b40 2to3 fixes
* Have /usr/bin/2to3 (rhbz#1111275)
 * Provide 2to3 and idle3, list them in summary and description (rhbz#1076401)
2017-08-16 15:36:09 +02:00
Michal Cyprian 58f92d897b Revert "Add --executable option to install.py command"
This enhancement is currently not needed and it can possibly
collide with `pip --editable`option

Reverts commit ef5c6fe479.
2017-08-11 13:02:07 +02:00
Iryna Shcherbina 2c3b2229eb Fix the "urllib FTP protocol stream injection" vulnerability (rhbz#1478916) 2017-08-07 18:26:47 +02:00
Tomas Orsava 7ce2b35305 Dropped BuildRequires on db4-devel
- it was useful for Python 2 (module bsddb), however, no longer needod for
  Python 3
- Tested building Python 3 with and without the dependency, all tests pass and
  filelists of resulting RPMs are identical
2017-08-01 16:59:32 +02:00
Charalampos Stratakis e49a729d35 Make debuginfo packages parallel installable 2017-07-31 14:02:39 +02:00
Florian Weimer c6fe5c680b Do not generate debuginfo subpackages (#1476593)
Rebuild with binutils fix for ppc64le (#1475636)
2017-07-30 17:13:49 +02:00
Fedora Release Engineering 4cc9b2e33c - Rebuilt for https://fedoraproject.org/wiki/Fedora_27_Mass_Rebuild 2017-07-27 12:26:00 +00:00
Charalampos Stratakis 3a226ae7d7 Make sure that we get the default signal handler for test_asyncio,
The parent process may have decided to ignore SIGHUP, and signal handlers
are inherited which can cause the test to hang in koji.
2017-07-25 18:03:55 +02:00
Charalampos Stratakis 5b6a3e0e34 Replace the "--verbose" flag with "-wW" when invoking the test suite, so that the tests are run in non-verbose mode by default and failed tests are rerun in verbose mode.
Also add the "--slowest" flag when invoking the test suite to
list the ten slowest tests at the end of the test run.
2017-07-20 17:50:38 +02:00
Charalampos Stratakis a94422417a Remove unused patches 2017-07-19 14:25:35 +02:00
Charalampos Stratakis f13050e2f3 Update the PEP 538 backport 2017-07-18 16:48:45 +02:00
Charalampos Stratakis 4ff21139d7 Update to Python 3.6.2
Rebased patch 180.

Dropped patches 249, 258, 269 as they have been upstreamed.
2017-07-18 16:40:02 +02:00
Charalampos Stratakis 46034c8139 Provide additional references for patches 251 and 252 2017-06-28 15:47:26 +02:00
Tomas Orsava 7e7918aadd Fix for a build failure when with_debug_build=0 2017-06-27 18:02:44 +02:00
Charalampos Stratakis 252962389c Update to the latest implementation of PEP 538.
Remove patch 261 as it was only required for the previous implementation
of PEP 538.
2017-06-27 15:38:30 +02:00
Michal Cyprian 332b947dfc Make pip and distutils in user environment install into separate location 2017-06-26 16:32:56 +02:00
Charalampos Stratakis b5403ba5eb Fix test_alpn_protocols from test_ssl 2017-06-24 17:24:42 +02:00
Charalampos Stratakis 3f280d64ad setuptools from version 36.0.0 rebundled its dependencies so we don't have to
require them in order for rewheel to work, so dropping patch 260 as well.
2017-06-19 11:14:11 +02:00
Tomas Orsava bb9ca6b393 Add dependency to the devel subpackage on python3-rpm-generators
... which have been excised out of rpm-build.

Involves: rhbz#1410631, rhbz#1444925
2017-05-16 11:15:05 +02:00
Tomas Orsava afb0016f9d Polishing the Notes on bootstrapping Python 2017-05-16 11:10:00 +02:00
Charalampos Stratakis 2287f0c54b Fix conditional macro 2017-05-09 17:05:44 +02:00
Charalampos Stratakis 27c558eb66 Added patch for fixing recompilation of Python when using common
build commands and the --enable-optimizations flag.

Enable profile guided optimizations for x86_64 and i686 architectures
2017-05-09 16:57:18 +02:00
Charalampos Stratakis 51bb7c4cc2 Update to a newer implementation of PEP 538 2017-05-09 16:40:05 +02:00
Charalampos Stratakis 6245861480 Change description, to reflect that python3 is now the default Python
Also enhance verbosity for the descriptions of subpackages
2017-05-09 16:38:49 +02:00
Charalampos Stratakis 31fe33b583 Update PEP 538 to the latest upstream implementation 2017-05-06 23:27:11 +02:00
Charalampos Stratakis bbbc440d2c system-python-libs subpackage was shipping some windows executables
which are used as templates, for creating windows installers for python
modules. These executables are moved to the devel subpackage.

Resolves: rhbz#1426257
2017-04-18 13:51:54 +02:00
Charalampos Stratakis 3f69bcd628 Enable link time optimizations 2017-04-18 13:48:56 +02:00
53 changed files with 13462 additions and 5918 deletions

2
.gitignore vendored
View File

@ -1 +1,3 @@
/*.tar.*
/*.src.rpm
/results_python3*

33
00001-rpath.patch Normal file
View File

@ -0,0 +1,33 @@
From bf01d6c367d9cb8f6594afa87c16f0498ae7321f Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Wed, 13 Jan 2010 21:25:18 +0000
Subject: [PATCH] 00001: Fixup distutils/unixccompiler.py to remove standard
library path from rpath Was Patch0 in ivazquez' python3000 specfile
---
Lib/distutils/unixccompiler.py | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/Lib/distutils/unixccompiler.py b/Lib/distutils/unixccompiler.py
index d10a78da31..4df4b67810 100644
--- a/Lib/distutils/unixccompiler.py
+++ b/Lib/distutils/unixccompiler.py
@@ -82,6 +82,15 @@ class UnixCCompiler(CCompiler):
if sys.platform == "cygwin":
exe_extension = ".exe"
+ def _fix_lib_args(self, libraries, library_dirs, runtime_library_dirs):
+ """Remove standard library path from rpath"""
+ libraries, library_dirs, runtime_library_dirs = super()._fix_lib_args(
+ libraries, library_dirs, runtime_library_dirs)
+ libdir = sysconfig.get_config_var('LIBDIR')
+ if runtime_library_dirs and (libdir in runtime_library_dirs):
+ runtime_library_dirs.remove(libdir)
+ return libraries, library_dirs, runtime_library_dirs
+
def preprocess(self, source, output_file=None, macros=None,
include_dirs=None, extra_preargs=None, extra_postargs=None):
fixed_args = self._fix_compile_args(None, macros, include_dirs)
--
2.24.1

View File

@ -1,822 +0,0 @@
diff -up Python-3.3.0rc2/configure.ac.systemtap Python-3.3.0rc2/configure.ac
--- Python-3.3.0rc2/configure.ac.systemtap 2012-09-09 05:11:14.000000000 -0400
+++ Python-3.3.0rc2/configure.ac 2012-09-10 09:17:21.114511781 -0400
@@ -2678,6 +2678,23 @@ if test "$with_valgrind" != no; then
OPT="-DDYNAMIC_ANNOTATIONS_ENABLED=1 $OPT"
fi
+# Check for systemtap support
+# On Linux, /usr/bin/dtrace is in fact a shim to SystemTap
+AC_MSG_CHECKING([for --with-systemtap])
+AC_ARG_WITH([systemtap],
+ AC_HELP_STRING([--with(out)-systemtap], [disable/enable SystemTap support]),,
+ with_systemtap=no)
+AC_MSG_RESULT([$with_systemtap])
+if test "$with_systemtap" != no; then
+ AC_DEFINE(WITH_SYSTEMTAP, 1,
+ [Define if you want to compile in SystemTap support])
+ SYSTEMTAPOBJS="Python/pysystemtap.o"
+ SYSTEMTAPDEPS="\$(srcdir)/Python/pysystemtap.h"
+fi
+
+AC_SUBST(SYSTEMTAPOBJS)
+AC_SUBST(SYSTEMTAPDEPS)
+
# -I${DLINCLDIR} is added to the compile rule for importdl.o
AC_SUBST(DLINCLDIR)
DLINCLDIR=.
diff -up Python-3.3.0rc2/configure.systemtap Python-3.3.0rc2/configure
--- Python-3.3.0rc2/configure.systemtap 2012-09-09 05:11:14.000000000 -0400
+++ Python-3.3.0rc2/configure 2012-09-10 09:17:21.116511780 -0400
@@ -618,6 +618,8 @@ TRUE
MACHDEP_OBJS
DYNLOADFILE
DLINCLDIR
+SYSTEMTAPDEPS
+SYSTEMTAPOBJS
THREADOBJ
LDLAST
USE_THREAD_MODULE
@@ -779,6 +781,7 @@ with_doc_strings
with_tsc
with_pymalloc
with_valgrind
+with_systemtap
with_fpectl
with_libm
with_libc
@@ -1456,6 +1459,7 @@ Optional Packages:
--with(out)-tsc enable/disable timestamp counter profile
--with(out)-pymalloc disable/enable specialized mallocs
--with-valgrind Enable Valgrind support
+ --with(out)-systemtap disable/enable SystemTap support
--with-fpectl enable SIGFPE catching
--with-libm=STRING math library
--with-libc=STRING C library
@@ -10065,6 +10069,31 @@ fi
OPT="-DDYNAMIC_ANNOTATIONS_ENABLED=1 $OPT"
fi
+# Check for systemtap support
+# On Linux, /usr/bin/dtrace is in fact a shim to SystemTap
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-systemtap" >&5
+$as_echo_n "checking for --with-systemtap... " >&6; }
+
+# Check whether --with-systemtap was given.
+if test "${with_systemtap+set}" = set; then :
+ withval=$with_systemtap;
+else
+ with_systemtap=no
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_systemtap" >&5
+$as_echo "$with_systemtap" >&6; }
+if test "$with_systemtap" != no; then
+
+$as_echo "#define WITH_SYSTEMTAP 1" >>confdefs.h
+
+ SYSTEMTAPOBJS="Python/pysystemtap.o"
+ SYSTEMTAPDEPS="\$(srcdir)/Python/pysystemtap.h"
+fi
+
+
+
+
# -I${DLINCLDIR} is added to the compile rule for importdl.o
DLINCLDIR=.
diff -up Python-3.3.0rc2/Doc/howto/index.rst.systemtap Python-3.3.0rc2/Doc/howto/index.rst
--- Python-3.3.0rc2/Doc/howto/index.rst.systemtap 2012-09-09 05:10:51.000000000 -0400
+++ Python-3.3.0rc2/Doc/howto/index.rst 2012-09-10 09:17:21.117511779 -0400
@@ -29,4 +29,5 @@ Currently, the HOWTOs are:
argparse.rst
ipaddress.rst
clinic.rst
+ instrumentation.rst
diff -up Python-3.3.0rc2/Doc/howto/instrumentation.rst.systemtap Python-3.3.0rc2/Doc/howto/instrumentation.rst
--- Python-3.3.0rc2/Doc/howto/instrumentation.rst.systemtap 2012-09-10 09:17:21.117511779 -0400
+++ Python-3.3.0rc2/Doc/howto/instrumentation.rst 2012-09-10 09:17:21.117511779 -0400
@@ -0,0 +1,295 @@
+.. _instrumentation:
+
+====================================
+Instrumenting CPython with SystemTap
+====================================
+
+:author: David Malcolm <dmalcolm@redhat.com>
+
+DTrace and SystemTap are monitoring tools, each providing a way to inspect
+what the processes on a computer system are doing. They both use
+domain-specific languages allowing a user to write scripts which:
+
+ - filter which processes are to be observed
+ - gather data from the processes of interest
+ - generate reports on the data
+
+As of Python 3.3, CPython can be built with embedded "markers" that can be
+observed by a SystemTap script, making it easier to monitor what the CPython
+processes on a system are doing.
+
+.. Potentially this document could be expanded to also cover DTrace markers.
+ However, I'm not a DTrace expert.
+
+.. I'm using ".. code-block:: c" for SystemTap scripts, as "c" is syntactically
+ the closest match that Sphinx supports
+
+
+Enabling the static markers
+---------------------------
+
+In order to build CPython with the embedded markers for SystemTap, the
+SystemTap development tools must be installed.
+
+On a Fedora or Red Hat Enterprise Linux machine, this can be done via::
+
+ yum install systemtap-sdt-devel
+
+CPython must then be configured `--with-systemtap`::
+
+ checking for --with-systemtap... yes
+
+You can verify if the SystemTap static markers are present in the built
+binary by seeing if it contains a ".note.stapsdt" section.
+
+.. code-block:: bash
+
+ $ eu-readelf -S ./python | grep .note.stapsdt
+ [29] .note.stapsdt NOTE 0000000000000000 00308d78 000000b8 0 0 0 4
+
+If you've built python as a shared library (with --enable-shared), you need
+to look instead within the shared library. For example:
+
+.. code-block:: bash
+
+ $ eu-readelf -S libpython3.3dm.so.1.0 | grep .note.stapsdt
+ [28] .note.stapsdt NOTE 0000000000000000 00365b68 000000b8 0 0 0 4
+
+Earlier versions of SystemTap stored the markers in a ".probes" section.
+
+For the curious, you can see the metadata for the static markers using this
+invocation.
+
+.. code-block:: bash
+
+ $ eu-readelf -x .note.stapsdt ./python
+
+ Hex dump of section [29] '.note.stapsdt', 184 bytes at offset 0x308d78:
+ 0x00000000 08000000 45000000 03000000 73746170 ....E.......stap
+ 0x00000010 73647400 d4664b00 00000000 4fc36600 sdt..fK.....O.f.
+ 0x00000020 00000000 488d9000 00000000 70797468 ....H.......pyth
+ 0x00000030 6f6e0066 756e6374 696f6e5f 5f656e74 on.function__ent
+ 0x00000040 72790038 40257261 78203840 25726478 ry.8@%rax 8@%rdx
+ 0x00000050 202d3440 25656378 00000000 08000000 -4@%ecx........
+ 0x00000060 46000000 03000000 73746170 73647400 F.......stapsdt.
+ 0x00000070 0d674b00 00000000 4fc36600 00000000 .gK.....O.f.....
+ 0x00000080 4a8d9000 00000000 70797468 6f6e0066 J.......python.f
+ 0x00000090 756e6374 696f6e5f 5f726574 75726e00 unction__return.
+ 0x000000a0 38402572 61782038 40257264 78202d34 8@%rax 8@%rdx -4
+ 0x000000b0 40256563 78000000 @%ecx...
+
+and a sufficiently modern eu-readelf can print the metadata:
+
+.. code-block:: bash
+
+ $ eu-readelf -n ./python
+
+ Note section [ 1] '.note.gnu.build-id' of 36 bytes at offset 0x190:
+ Owner Data size Type
+ GNU 20 GNU_BUILD_ID
+ Build ID: a28f8db1b224530b0d38ad7b82a249cf7c3f18d6
+
+ Note section [27] '.note.stapsdt' of 184 bytes at offset 0x1ae884:
+ Owner Data size Type
+ stapsdt 70 Version: 3
+ PC: 0xe0d3a, Base: 0x14b150, Semaphore: 0x3ae882
+ Provider: python, Name: function__return, Args: '8@%rbx 8@%r13 -4@%eax'
+ stapsdt 69 Version: 3
+ PC: 0xe0f37, Base: 0x14b150, Semaphore: 0x3ae880
+ Provider: python, Name: function__entry, Args: '8@%rbx 8@%r13 -4@%eax'
+
+The above metadata contains information for SystemTap describing how it can
+patch strategically-placed machine code instructions to enable the tracing
+hooks used by a SystemTap script.
+
+
+Static markers
+--------------
+
+The low-level way to use the SystemTap integration is to use the static
+markers directly. This requires you to explicitly state the binary file
+containing them.
+
+For example, this script can be used to show the call/return hierarchy of a
+Python script:
+
+.. code-block:: c
+
+ probe process('python').mark("function__entry") {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+
+ printf("%s => %s in %s:%d\\n",
+ thread_indent(1), funcname, filename, lineno);
+ }
+
+ probe process('python').mark("function__return") {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+
+ printf("%s <= %s in %s:%d\\n",
+ thread_indent(-1), funcname, filename, lineno);
+ }
+
+It can be invoked like this:
+
+.. code-block:: bash
+
+ $ stap \
+ show-call-hierarchy.stp \
+ -c ./python test.py
+
+The output looks like this::
+
+ 11408 python(8274): => __contains__ in Lib/_abcoll.py:362
+ 11414 python(8274): => __getitem__ in Lib/os.py:425
+ 11418 python(8274): => encode in Lib/os.py:490
+ 11424 python(8274): <= encode in Lib/os.py:493
+ 11428 python(8274): <= __getitem__ in Lib/os.py:426
+ 11433 python(8274): <= __contains__ in Lib/_abcoll.py:366
+
+where the columns are:
+
+ - time in microseconds since start of script
+
+ - name of executable
+
+ - PID of process
+
+and the remainder indicates the call/return hierarchy as the script executes.
+
+For a `--enable-shared` build of CPython, the markers are contained within the
+libpython shared library, and the probe's dotted path needs to reflect this. For
+example, this line from the above example::
+
+ probe process('python').mark("function__entry") {
+
+should instead read::
+
+ probe process('python').library("libpython3.3dm.so.1.0").mark("function__entry") {
+
+(assuming a debug build of CPython 3.3)
+
+.. I'm reusing the "c:function" type for markers
+
+.. c:function:: function__entry(str filename, str funcname, int lineno)
+
+ This marker indicates that execution of a Python function has begun. It is
+ only triggered for pure-python (bytecode) functions.
+
+ The filename, function name, and line number are provided back to the
+ tracing script as positional arguments, which must be accessed using
+ `$arg1`, `$arg2`:
+
+ * `$arg1` : `(const char *)` filename, accessible using `user_string($arg1)`
+
+ * `$arg2` : `(const char *)` function name, accessible using
+ `user_string($arg2)`
+
+ * `$arg3` : `int` line number
+
+ * `$arg4` : `(PyFrameObject *)`, the frame being executed
+
+.. c:function:: function__return(str filename, str funcname, int lineno)
+
+ This marker is the converse of `function__entry`, and indicates that
+ execution of a Python function has ended (either via ``return``, or via an
+ exception). It is only triggered for pure-python (bytecode) functions.
+
+ The arguments are the same as for `function__entry`
+
+
+Tapsets
+-------
+
+The higher-level way to use the SystemTap integration is to use a "tapset":
+SystemTap's equivalent of a library, which hides some of the lower-level
+details of the static markers.
+
+Here is a tapset file, based on a non-shared build of CPython:
+
+.. code-block:: c
+
+ /*
+ Provide a higher-level wrapping around the function__entry and
+ function__return markers:
+ */
+ probe python.function.entry = process("python").mark("function__entry")
+ {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+ frameptr = $arg4
+ }
+ probe python.function.return = process("python").mark("function__return")
+ {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+ frameptr = $arg4
+ }
+
+If this file is installed in SystemTap's tapset directory (e.g.
+`/usr/share/systemtap/tapset`), then these additional probepoints become
+available:
+
+.. c:function:: python.function.entry(str filename, str funcname, int lineno, frameptr)
+
+ This probe point indicates that execution of a Python function has begun.
+ It is only triggered for pure-python (bytecode) functions.
+
+.. c:function:: python.function.return(str filename, str funcname, int lineno, frameptr)
+
+ This probe point is the converse of `python.function.return`, and indicates
+ that execution of a Python function has ended (either via ``return``, or
+ via an exception). It is only triggered for pure-python (bytecode) functions.
+
+
+Examples
+--------
+This SystemTap script uses the tapset above to more cleanly implement the
+example given above of tracing the Python function-call hierarchy, without
+needing to directly name the static markers:
+
+.. code-block:: c
+
+ probe python.function.entry
+ {
+ printf("%s => %s in %s:%d\n",
+ thread_indent(1), funcname, filename, lineno);
+ }
+
+ probe python.function.return
+ {
+ printf("%s <= %s in %s:%d\n",
+ thread_indent(-1), funcname, filename, lineno);
+ }
+
+
+The following script uses the tapset above to provide a top-like view of all
+running CPython code, showing the top 20 most frequently-entered bytecode
+frames, each second, across the whole system:
+
+.. code-block:: c
+
+ global fn_calls;
+
+ probe python.function.entry
+ {
+ fn_calls[pid(), filename, funcname, lineno] += 1;
+ }
+
+ probe timer.ms(1000) {
+ printf("\033[2J\033[1;1H") /* clear screen */
+ printf("%6s %80s %6s %30s %6s\n",
+ "PID", "FILENAME", "LINE", "FUNCTION", "CALLS")
+ foreach ([pid, filename, funcname, lineno] in fn_calls- limit 20) {
+ printf("%6d %80s %6d %30s %6d\n",
+ pid, filename, lineno, funcname,
+ fn_calls[pid, filename, funcname, lineno]);
+ }
+ delete fn_calls;
+ }
+
diff -up Python-3.3.0rc2/Lib/test/test_systemtap.py.systemtap Python-3.3.0rc2/Lib/test/test_systemtap.py
--- Python-3.3.0rc2/Lib/test/test_systemtap.py.systemtap 2012-09-10 09:17:21.117511779 -0400
+++ Python-3.3.0rc2/Lib/test/test_systemtap.py 2012-09-10 09:17:21.117511779 -0400
@@ -0,0 +1,234 @@
+# Verify that systemtap static probes work
+#
+import subprocess
+import sys
+import sysconfig
+import os
+import unittest
+
+from test.support import run_unittest, TESTFN, unlink
+
+if '--with-systemtap' not in sysconfig.get_config_var('CONFIG_ARGS'):
+ raise unittest.SkipTest("Python was not configured --with-systemtap")
+
+try:
+ _, stap_version = subprocess.Popen(["stap", "-V"],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ ).communicate()
+except OSError:
+ # This is what "no stap" looks like. There may, however, be other
+ # errors that manifest this way too.
+ raise unittest.SkipTest("Couldn't find stap on the path")
+
+def invoke_systemtap_script(script, cmd):
+ # Start a child process, probing with the given systemtap script
+ # (passed as stdin to the "stap" tool)
+ # The script should be a bytes instance
+ # Return (stdout, stderr) pair
+
+ p = subprocess.Popen(["stap", "-", '-vv', '-c', cmd],
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ out, err = p.communicate(input=script)
+ return out, err
+
+# Verify that stap can run a simple "hello world"-style script
+# This can fail for various reasons:
+# - missing kernel headers
+# - permissions (a non-root user needs to be in the "stapdev" group)
+TRIVIAL_STAP_SCRIPT = b'probe begin { println("hello world") exit () }'
+
+out, err = invoke_systemtap_script(TRIVIAL_STAP_SCRIPT, 'true')
+if out != b'hello world\n':
+ raise unittest.SkipTest("Test systemtap script did not run; stderr was: %s" % err)
+
+# We don't expect stderr to be empty, since we're invoking stap with "-vv": stap
+# will (we hope) generate debugging output on stderr.
+
+def invoke_python_under_systemtap(script, pythoncode=None, pythonfile=None):
+ # Start a child python process, probing with the given systemtap script
+ # (passed as stdin to the "stap" tool)
+ # The script should be a bytes instance
+ # Return (stdout, stderr) pair
+
+ if pythonfile:
+ pythoncmd = '%s %s' % (sys.executable, pythonfile)
+ else:
+ pythoncmd = '%s -c %r' % (sys.executable, pythoncode)
+
+ # The process tree of a stap invocation of a command goes through
+ # something like this:
+ # stap ->fork/exec(staprun; exec stapio ->f/e(-c cmd); exec staprun -r)
+ # and this trip through setuid leads to LD_LIBRARY_PATH being dropped,
+ # which would lead to an --enable-shared build of python failing to be
+ # find its libpython, with an error like:
+ # error while loading shared libraries: libpython3.3dm.so.1.0: cannot
+ # open shared object file: No such file or directory
+ # Hence we need to jump through some hoops to expose LD_LIBRARY_PATH to
+ # the invoked python process:
+ LD_LIBRARY_PATH = os.environ.get('LD_LIBRARY_PATH', '')
+ if LD_LIBRARY_PATH:
+ pythoncmd = 'env LD_LIBRARY_PATH=%s ' % LD_LIBRARY_PATH + pythoncmd
+
+ return invoke_systemtap_script(script, pythoncmd)
+
+# When using the static markers, we need to supply the prefix of a systemtap
+# dotted probe point that containing the marker.
+# See http://sourceware.org/systemtap/langref/Probe_points.html
+#
+# We need to determine if this is a shared-library build
+#
+# Note that sysconfig can get this wrong; see:
+# http://bugs.python.org/issue14774
+#
+if '--enable-shared' in sysconfig.get_config_var('CONFIG_ARGS'):
+ # For a shared-library build, the markers are in library(INSTSONAME):
+ INSTSONAME = sysconfig.get_config_var('INSTSONAME')
+ probe_prefix = 'process("%s").library("%s")' % (sys.executable, INSTSONAME)
+else:
+ # For a non-shared-library build, we can simply use sys.executable:
+ probe_prefix = 'process("%s")' % sys.executable
+
+# The following script ought to generate lots of lines showing recursive
+# function entry and return, of the form:
+# 11408 python(8274): => __contains__ in Lib/_abcoll.py:362
+# 11414 python(8274): => __getitem__ in Lib/os.py:425
+# 11418 python(8274): => encode in Lib/os.py:490
+# 11424 python(8274): <= encode in Lib/os.py:493
+# 11428 python(8274): <= __getitem__ in Lib/os.py:426
+# 11433 python(8274): <= __contains__ in Lib/_abcoll.py:366
+# where the column are:
+# - time in microseconds since start of script
+# - name of executable
+# - PID of process
+# and the remainder indicates the call/return hierarchy
+
+hierarchy_script = ('''
+probe %s.mark("function__entry") {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+
+ printf("%%s => %%s in %%s:%%d\\n", thread_indent(1), funcname, filename, lineno);
+}
+
+probe %s.mark("function__return") {
+ filename = user_string($arg1);
+ funcname = user_string($arg2);
+ lineno = $arg3;
+
+ printf("%%s <= %%s in %%s:%%d\\n", thread_indent(-1), funcname, filename, lineno);
+}
+''' % (probe_prefix, probe_prefix)).encode('utf-8')
+
+
+class ErrorDumper:
+ # A context manager that dumps extra information if an exception is raised,
+ # to help track down why the problem occurred
+ def __init__(self, out, err):
+ self.out = out
+ self.err = err
+
+ def __enter__(self):
+ pass
+
+ def __exit__(self, type_, value, traceback):
+ if type_:
+ # an exception is being raised:
+ print('stdout: %s' % out.decode())
+ print('stderr: %s' % err.decode())
+
+class SystemtapTests(unittest.TestCase):
+
+ def test_invoking_python(self):
+ # Ensure that we can invoke python under stap, with a trivial stap
+ # script:
+ out, err = invoke_python_under_systemtap(
+ b'probe begin { println("hello from stap") exit () }',
+ pythoncode="print('hello from python')")
+ with ErrorDumper(out, err):
+ self.assertIn(b'hello from stap', out)
+ self.assertIn(b'hello from python', out)
+
+ def test_function_entry(self):
+ # Ensure that the function_entry static marker works
+ out, err = invoke_python_under_systemtap(hierarchy_script)
+ # stdout ought to contain various lines showing recursive function
+ # entry and return (see above)
+
+ # Uncomment this for debugging purposes:
+ # print(out.decode('utf-8'))
+
+ # Executing the cmdline-supplied "pass":
+ # 0 python(8274): => <module> in <string>:1
+ # 5 python(8274): <= <module> in <string>:1
+ with ErrorDumper(out, err):
+ self.assertIn(b'=> <module> in <string>:1', out,
+ msg="stdout: %s\nstderr: %s\n" % (out, err))
+
+ def test_function_encoding(self):
+ # Ensure that function names containing non-Latin 1 code
+ # points are handled:
+ pythonfile = TESTFN
+ try:
+ unlink(pythonfile)
+ f = open(pythonfile, "wb")
+ f.write("""
+# Sample script with non-ASCII filename, for use by test_systemtap.py
+# Implicitly UTF-8
+
+def 文字化け():
+ '''Function with non-ASCII identifier; I believe this reads "mojibake"'''
+ print("hello world!")
+
+文字化け()
+""".encode('utf-8'))
+ f.close()
+
+ out, err = invoke_python_under_systemtap(hierarchy_script,
+ pythonfile=pythonfile)
+ out_utf8 = out.decode('utf-8')
+ with ErrorDumper(out, err):
+ self.assertIn('=> <module> in %s:5' % pythonfile, out_utf8)
+ self.assertIn(' => 文字化け in %s:5' % pythonfile, out_utf8)
+ self.assertIn(' <= 文字化け in %s:7' % pythonfile, out_utf8)
+ self.assertIn('<= <module> in %s:9' % pythonfile, out_utf8)
+ finally:
+ unlink(pythonfile)
+
+ @unittest.skipIf(sys.getfilesystemencoding() == 'ascii',
+ 'the test filename is not encodable with ASCII')
+ def test_filename_encoding(self):
+ # Ensure that scripts names containing non-Latin 1 code
+ # points are handled:
+ pythonfile = TESTFN + '_☠.py'
+ try:
+ unlink(pythonfile)
+ f = open(pythonfile, "wb")
+ f.write("""
+def foo():
+ '''Function with non-ASCII identifier; I believe this reads "mojibake"'''
+ print("hello world!")
+
+foo()
+""".encode('utf-8'))
+ f.close()
+
+ out, err = invoke_python_under_systemtap(hierarchy_script,
+ pythonfile=pythonfile)
+ out_utf8 = out.decode('utf-8')
+ with ErrorDumper(out, err):
+ self.assertIn('=> <module> in %s:2' % pythonfile, out_utf8)
+ self.assertIn(' => foo in %s:2' % pythonfile, out_utf8)
+ self.assertIn(' <= foo in %s:4' % pythonfile, out_utf8)
+ self.assertIn('<= <module> in %s:6' % pythonfile, out_utf8)
+ finally:
+ unlink(pythonfile)
+
+def test_main():
+ run_unittest(SystemtapTests)
+
+if __name__ == "__main__":
+ test_main()
diff -up Python-3.3.0rc2/Makefile.pre.in.systemtap Python-3.3.0rc2/Makefile.pre.in
--- Python-3.3.0rc2/Makefile.pre.in.systemtap 2012-09-09 05:11:05.000000000 -0400
+++ Python-3.3.0rc2/Makefile.pre.in 2012-09-10 09:19:51.195501518 -0400
@@ -363,6 +363,7 @@ PYTHON_OBJS= \
Python/formatter_unicode.o \
Python/fileutils.o \
Python/$(DYNLOADFILE) \
+ @SYSTEMTAPOBJS@ \
$(LIBOBJS) \
$(MACHDEP_OBJS) \
$(THREADOBJ)
@@ -713,7 +714,8 @@ Objects/setobject.o: $(srcdir)/Objects/s
$(OPCODETARGETS_H): $(OPCODETARGETGEN_FILES)
$(OPCODETARGETGEN) $(OPCODETARGETS_H)
-Python/ceval.o: $(OPCODETARGETS_H) $(srcdir)/Python/ceval_gil.h
+Python/ceval.o: $(OPCODETARGETS_H) $(srcdir)/Python/ceval_gil.h \
+ $(srcdir)/Python/ceval_systemtap.h @SYSTEMTAPDEPS@
Python/frozen.o: Python/importlib.h Python/importlib_external.h
@@ -724,6 +726,13 @@ Objects/typeobject.o: $(srcdir)/Objects/
Objects/typeslots.inc: $(srcdir)/Include/typeslots.h $(srcdir)/Objects/typeslots.py
$(PYTHON) $(srcdir)/Objects/typeslots.py < $(srcdir)/Include/typeslots.h > Objects/typeslots.inc
+# Only needed with --with-systemtap; not a public header:
+$(srcdir)/Python/pysystemtap.h: $(srcdir)/Python/pysystemtap.d
+ dtrace -o $@ $(DFLAGS) -C -h -s $(srcdir)/Python/pysystemtap.d
+
+Python/pysystemtap.o: $(srcdir)/Python/pysystemtap.d Python/ceval.o
+ dtrace -o $@ $(DFLAGS) -C -G -s $(srcdir)/Python/pysystemtap.d Python/ceval.o
+
############################################################################
# Header files
@@ -1345,6 +1354,7 @@ clean: pycremoval
-rm -f Lib/lib2to3/*Grammar*.pickle
-rm -f Programs/_testembed Programs/_freeze_importlib
-rm -rf build
+ -rm -f $(srcdir)/Python/pysystemtap.h
profile-removal:
find . -name '*.gc??' -exec rm -f {} ';'
diff -up Python-3.3.0rc2/pyconfig.h.in.systemtap Python-3.3.0rc2/pyconfig.h.in
--- Python-3.3.0rc2/pyconfig.h.in.systemtap 2012-09-09 05:11:14.000000000 -0400
+++ Python-3.3.0rc2/pyconfig.h.in 2012-09-10 09:17:21.120511781 -0400
@@ -1306,6 +1306,9 @@
/* Define if you want to compile in Python-specific mallocs */
#undef WITH_PYMALLOC
+/* Define if you want to compile in SystemTap support */
+#undef WITH_SYSTEMTAP
+
/* Define if you want to compile in rudimentary thread support */
#undef WITH_THREAD
diff -up Python-3.3.0rc2/Python/ceval.c.systemtap Python-3.3.0rc2/Python/ceval.c
--- Python-3.3.0rc2/Python/ceval.c.systemtap 2012-09-09 05:11:12.000000000 -0400
+++ Python-3.3.0rc2/Python/ceval.c 2012-09-10 09:17:21.122511781 -0400
@@ -18,6 +18,8 @@
#include <ctype.h>
+#include "ceval_systemtap.h"
+
#ifndef WITH_TSC
#define READ_TIMESTAMP(var)
@@ -1160,6 +1162,10 @@ PyEval_EvalFrameEx(PyFrameObject *f, int
}
}
+ if (PYTHON_FUNCTION_ENTRY_ENABLED()) {
+ systemtap_function_entry(f);
+ }
+
co = f->f_code;
names = co->co_names;
consts = co->co_consts;
@@ -3077,6 +3083,11 @@ fast_yield:
/* pop frame */
exit_eval_frame:
+
+ if (PYTHON_FUNCTION_RETURN_ENABLED()) {
+ systemtap_function_return(f);
+ }
+
Py_LeaveRecursiveCall();
f->f_executing = 0;
tstate->frame = f->f_back;
diff -up Python-3.3.0rc2/Python/ceval_systemtap.h.systemtap Python-3.3.0rc2/Python/ceval_systemtap.h
--- Python-3.3.0rc2/Python/ceval_systemtap.h.systemtap 2012-09-10 09:17:21.122511781 -0400
+++ Python-3.3.0rc2/Python/ceval_systemtap.h 2012-09-10 09:17:21.122511781 -0400
@@ -0,0 +1,86 @@
+/*
+ Support for SystemTap static markers
+*/
+
+#ifdef WITH_SYSTEMTAP
+
+#include "pysystemtap.h"
+
+/*
+ A struct to hold all of the information gathered when one of the traceable
+ markers is triggered
+*/
+struct frame_marker_info
+{
+ PyObject *filename_obj;
+ PyObject *funcname_obj;
+ const char *filename;
+ const char *funcname;
+ int lineno;
+};
+
+static void
+get_frame_marker_info(PyFrameObject *f, struct frame_marker_info *fmi)
+{
+ PyObject *ptype;
+ PyObject *pvalue;
+ PyObject *ptraceback;
+
+ PyErr_Fetch(&ptype, &pvalue, &ptraceback);
+
+ fmi->filename_obj = PyUnicode_EncodeFSDefault(f->f_code->co_filename);
+ if (fmi->filename_obj) {
+ fmi->filename = PyBytes_AsString(fmi->filename_obj);
+ } else {
+ fmi->filename = NULL;
+ }
+
+ fmi->funcname_obj = PyUnicode_AsUTF8String(f->f_code->co_name);
+ if (fmi->funcname_obj) {
+ fmi->funcname = PyBytes_AsString(fmi->funcname_obj);
+ } else {
+ fmi->funcname = NULL;
+ }
+
+ fmi->lineno = PyCode_Addr2Line(f->f_code, f->f_lasti);
+
+ PyErr_Restore(ptype, pvalue, ptraceback);
+
+}
+
+static void
+release_frame_marker_info(struct frame_marker_info *fmi)
+{
+ Py_XDECREF(fmi->filename_obj);
+ Py_XDECREF(fmi->funcname_obj);
+}
+
+static void
+systemtap_function_entry(PyFrameObject *f)
+{
+ struct frame_marker_info fmi;
+ get_frame_marker_info(f, &fmi);
+ PYTHON_FUNCTION_ENTRY(fmi.filename, fmi.funcname, fmi.lineno, f);
+ release_frame_marker_info(&fmi);
+}
+
+static void
+systemtap_function_return(PyFrameObject *f)
+{
+ struct frame_marker_info fmi;
+ get_frame_marker_info(f, &fmi);
+ PYTHON_FUNCTION_RETURN(fmi.filename, fmi.funcname, fmi.lineno, f);
+ release_frame_marker_info(&fmi);
+}
+
+#else /* #ifdef WITH_SYSTEMTAP */
+
+/*
+ When configured --without-systemtap, everything compiles away to nothing:
+*/
+#define PYTHON_FUNCTION_ENTRY_ENABLED() 0
+#define PYTHON_FUNCTION_RETURN_ENABLED() 0
+#define systemtap_function_entry(f)
+#define systemtap_function_return(f)
+
+#endif
diff -up Python-3.3.0rc2/Python/pysystemtap.d.systemtap Python-3.3.0rc2/Python/pysystemtap.d
--- Python-3.3.0rc2/Python/pysystemtap.d.systemtap 2012-09-10 09:17:21.122511781 -0400
+++ Python-3.3.0rc2/Python/pysystemtap.d 2012-09-10 09:17:21.122511781 -0400
@@ -0,0 +1,4 @@
+provider python {
+ probe function__entry(const char *, const char *, int, PyFrameObject *);
+ probe function__return(const char *, const char *, int, PyFrameObject *);
+};

View File

@ -1,5 +1,39 @@
From 96580364051672475607c88cdb31ec875cea6e97 Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Wed, 13 Jan 2010 21:25:18 +0000
Subject: [PATCH] 00102: Change the various install paths to use /usr/lib64/
instead or /usr/lib/
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Only used when "%{_lib}" == "lib64".
Co-authored-by: David Malcolm <dmalcolm@redhat.com>
Co-authored-by: Thomas Spura <tomspur@fedoraproject.org>
Co-authored-by: Slavek Kabrda <bkabrda@redhat.com>
Co-authored-by: Matej Stuchlik <mstuchli@redhat.com>
Co-authored-by: Tomas Orsava <torsava@redhat.com>
Co-authored-by: Charalampos Stratakis <cstratak@redhat.com>
Co-authored-by: Petr Viktorin <pviktori@redhat.com>
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Iryna Shcherbina <shcherbina.iryna@gmail.com>
---
Lib/distutils/command/install.py | 4 ++--
Lib/distutils/sysconfig.py | 6 +++++-
Lib/distutils/tests/test_install.py | 3 ++-
Lib/site.py | 4 ++++
Lib/sysconfig.py | 12 ++++++------
Lib/test/test_site.py | 4 ++--
Makefile.pre.in | 2 +-
Modules/getpath.c | 6 +++---
configure | 4 ++--
configure.ac | 4 ++--
setup.py | 6 +++---
11 files changed, 32 insertions(+), 23 deletions(-)
diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py
index 9474e9c..c0ce4c6 100644
index c625c95bf7..ae4f915669 100644
--- a/Lib/distutils/command/install.py
+++ b/Lib/distutils/command/install.py
@@ -30,14 +30,14 @@ WINDOWS_SCHEME = {
@ -20,10 +54,10 @@ index 9474e9c..c0ce4c6 100644
'scripts': '$base/bin',
'data' : '$base',
diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
index 026cca7..6d3e077 100644
index b51629eb94..9a4892a737 100644
--- a/Lib/distutils/sysconfig.py
+++ b/Lib/distutils/sysconfig.py
@@ -132,8 +132,12 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
@@ -146,8 +146,12 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
prefix = plat_specific and EXEC_PREFIX or PREFIX
if os.name == "posix":
@ -37,11 +71,26 @@ index 026cca7..6d3e077 100644
if standard_lib:
return libpython
else:
diff --git a/Lib/distutils/tests/test_install.py b/Lib/distutils/tests/test_install.py
index 287ab1989e..d4c05e0ab1 100644
--- a/Lib/distutils/tests/test_install.py
+++ b/Lib/distutils/tests/test_install.py
@@ -57,8 +57,9 @@ class InstallTestCase(support.TempdirManager,
self.assertEqual(got, expected)
libdir = os.path.join(destination, "lib", "python")
+ platlibdir = os.path.join(destination, "lib64", "python")
check_path(cmd.install_lib, libdir)
- check_path(cmd.install_platlib, libdir)
+ check_path(cmd.install_platlib, platlibdir)
check_path(cmd.install_purelib, libdir)
check_path(cmd.install_headers,
os.path.join(destination, "include", "python", "foopkg"))
diff --git a/Lib/site.py b/Lib/site.py
index a84e3bb..ba0d3ea 100644
index a065ab0b5d..22d53fa562 100644
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -303,11 +303,15 @@ def getsitepackages(prefixes=None):
@@ -335,11 +335,15 @@ def getsitepackages(prefixes=None):
seen.add(prefix)
if os.sep == '/':
@ -55,10 +104,10 @@ index a84e3bb..ba0d3ea 100644
sitepackages.append(prefix)
+ sitepackages.append(os.path.join(prefix, "lib64", "site-packages"))
sitepackages.append(os.path.join(prefix, "lib", "site-packages"))
if sys.platform == "darwin":
# for framework builds *only* we add the standard Apple
return sitepackages
diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
index b9bbfe5..2a5f29c 100644
index b9e2fafbc0..0ae6d35b69 100644
--- a/Lib/sysconfig.py
+++ b/Lib/sysconfig.py
@@ -20,10 +20,10 @@ __all__ = [
@ -75,7 +124,7 @@ index b9bbfe5..2a5f29c 100644
'include':
'{installed_base}/include/python{py_version_short}{abiflags}',
'platinclude':
@@ -61,10 +61,10 @@ _INSTALL_SCHEMES = {
@@ -62,10 +62,10 @@ _INSTALL_SCHEMES = {
'data': '{userbase}',
},
'posix_user': {
@ -90,13 +139,13 @@ index b9bbfe5..2a5f29c 100644
'scripts': '{userbase}/bin',
'data': '{userbase}',
diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py
index f698927..bc977b5 100644
index 41c4229919..543c88432a 100644
--- a/Lib/test/test_site.py
+++ b/Lib/test/test_site.py
@@ -248,8 +248,8 @@ class HelperFunctionsTests(unittest.TestCase):
self.assertEqual(dirs[1], wanted)
elif os.sep == '/':
# OS X non-framwework builds, Linux, FreeBSD, etc
@@ -266,8 +266,8 @@ class HelperFunctionsTests(unittest.TestCase):
dirs = site.getsitepackages()
if os.sep == '/':
# OS X, Linux, FreeBSD, etc
- self.assertEqual(len(dirs), 1)
- wanted = os.path.join('xoxo', 'lib',
+ self.assertEqual(len(dirs), 2)
@ -105,10 +154,10 @@ index f698927..bc977b5 100644
'site-packages')
self.assertEqual(dirs[0], wanted)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 8fa7934..a693917 100644
index a914a9c70f..406a441082 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -126,7 +126,7 @@ LIBDIR= @libdir@
@@ -143,7 +143,7 @@ LIBDIR= @libdir@
MANDIR= @mandir@
INCLUDEDIR= @includedir@
CONFINCLUDEDIR= $(exec_prefix)/include
@ -118,71 +167,95 @@ index 8fa7934..a693917 100644
# Detailed destination directories
diff --git a/Modules/getpath.c b/Modules/getpath.c
index 65b47a3..eaa756c 100644
index b727f66953..a0c5fb6139 100644
--- a/Modules/getpath.c
+++ b/Modules/getpath.c
@@ -494,7 +494,7 @@ calculate_path(void)
_pythonpath = Py_DecodeLocale(PYTHONPATH, NULL);
_prefix = Py_DecodeLocale(PREFIX, NULL);
_exec_prefix = Py_DecodeLocale(EXEC_PREFIX, NULL);
- lib_python = Py_DecodeLocale("lib/python" VERSION, NULL);
+ lib_python = Py_DecodeLocale("lib64/python" VERSION, NULL);
@@ -730,7 +730,7 @@ calculate_exec_prefix(PyCalculatePath *calculate, _PyPathConfig *pathconfig,
if (safe_wcscpy(exec_prefix, calculate->exec_prefix, exec_prefix_len) < 0) {
return PATHLEN_ERR();
}
- status = joinpath(exec_prefix, L"lib/lib-dynload", exec_prefix_len);
+ status = joinpath(exec_prefix, L"lib64/lib-dynload", exec_prefix_len);
if (_PyStatus_EXCEPTION(status)) {
return status;
}
@@ -1067,7 +1067,7 @@ calculate_zip_path(PyCalculatePath *calculate, const wchar_t *prefix,
return PATHLEN_ERR();
}
}
- status = joinpath(zip_path, L"lib/python00.zip", zip_path_len);
+ status = joinpath(zip_path, L"lib64/python00.zip", zip_path_len);
if (_PyStatus_EXCEPTION(status)) {
return status;
}
@@ -1197,7 +1197,7 @@ calculate_init(PyCalculatePath *calculate, const PyConfig *config)
if (!calculate->exec_prefix) {
return DECODE_LOCALE_ERR("EXEC_PREFIX define", len);
}
- calculate->lib_python = Py_DecodeLocale("lib/python" VERSION, &len);
+ calculate->lib_python = Py_DecodeLocale("lib64/python" VERSION, &len);
if (!calculate->lib_python) {
return DECODE_LOCALE_ERR("EXEC_PREFIX define", len);
}
diff --git a/configure b/configure
index a979363acf..b89ae1be3c 100755
--- a/configure
+++ b/configure
@@ -15188,9 +15188,9 @@ fi
if (!_pythonpath || !_prefix || !_exec_prefix || !lib_python) {
Py_FatalError(
@@ -683,7 +683,7 @@ calculate_path(void)
}
else
wcsncpy(zip_path, _prefix, MAXPATHLEN);
- joinpath(zip_path, L"lib/python00.zip");
+ joinpath(zip_path, L"lib64/python00.zip");
bufsz = wcslen(zip_path); /* Replace "00" with version */
zip_path[bufsz - 6] = VERSION[0];
zip_path[bufsz - 5] = VERSION[2];
@@ -695,7 +695,7 @@ calculate_path(void)
fprintf(stderr,
"Could not find platform dependent libraries <exec_prefix>\n");
wcsncpy(exec_prefix, _exec_prefix, MAXPATHLEN);
- joinpath(exec_prefix, L"lib/lib-dynload");
+ joinpath(exec_prefix, L"lib64/lib-dynload");
}
/* If we found EXEC_PREFIX do *not* reduce it! (Yet.) */
if test x$PLATFORM_TRIPLET = x; then
- LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
+ LIBPL='$(prefix)'"/lib64/python${VERSION}/config-${LDVERSION}"
else
- LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"
+ LIBPL='$(prefix)'"/lib64/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"
fi
diff --git a/configure.ac b/configure.ac
index e57ef7c38b..c59cbc223f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -4674,9 +4674,9 @@ fi
dnl define LIBPL after ABIFLAGS and LDVERSION is defined.
AC_SUBST(PY_ENABLE_SHARED)
if test x$PLATFORM_TRIPLET = x; then
- LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"
+ LIBPL='$(prefix)'"/lib64/python${VERSION}/config-${LDVERSION}"
else
- LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"
+ LIBPL='$(prefix)'"/lib64/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"
fi
AC_SUBST(LIBPL)
diff --git a/setup.py b/setup.py
index 0f2dfc4..da37896 100644
index 20d7f35652..024a1035c0 100644
--- a/setup.py
+++ b/setup.py
@@ -492,7 +492,7 @@ class PyBuildExt(build_ext):
@@ -649,7 +649,7 @@ class PyBuildExt(build_ext):
# directories (i.e. '.' and 'Include') must be first. See issue
# 10520.
if not cross_compiling:
if not CROSS_COMPILING:
- add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib')
+ add_dir_to_list(self.compiler.library_dirs, '/usr/local/lib64')
add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
# only change this for cross builds for 3.3, issues on Mageia
if cross_compiling:
@@ -780,11 +780,11 @@ class PyBuildExt(build_ext):
if CROSS_COMPILING:
@@ -953,11 +953,11 @@ class PyBuildExt(build_ext):
elif curses_library:
readline_libs.append(curses_library)
elif self.compiler.find_library_file(lib_dirs +
elif self.compiler.find_library_file(self.lib_dirs +
- ['/usr/lib/termcap'],
+ ['/usr/lib64/termcap'],
'termcap'):
readline_libs.append('termcap')
exts.append( Extension('readline', ['readline.c'],
- library_dirs=['/usr/lib/termcap'],
+ library_dirs=['/usr/lib64/termcap'],
extra_link_args=readline_extra_link_args,
libraries=readline_libs) )
self.add(Extension('readline', ['readline.c'],
- library_dirs=['/usr/lib/termcap'],
+ library_dirs=['/usr/lib64/termcap'],
extra_link_args=readline_extra_link_args,
libraries=readline_libs))
else:
@@ -821,8 +821,8 @@ class PyBuildExt(build_ext):
if krb5_h:
ssl_incs += krb5_h
ssl_libs = find_library_file(self.compiler, 'ssl',lib_dirs,
- ['/usr/local/ssl/lib',
- '/usr/contrib/ssl/lib/'
+ ['/usr/local/ssl/lib64',
+ '/usr/contrib/ssl/lib64/'
] )
if (ssl_incs is not None and
--
2.24.1

View File

@ -1,13 +0,0 @@
--- Python-2.7.2/Lib/distutils/tests/test_install.py.lib64 2011-09-08 17:51:57.851405376 -0400
+++ Python-2.7.2/Lib/distutils/tests/test_install.py 2011-09-08 18:40:46.754205096 -0400
@@ -41,8 +41,9 @@ class InstallTestCase(support.TempdirMan
self.assertEqual(got, expected)
libdir = os.path.join(destination, "lib", "python")
+ platlibdir = os.path.join(destination, "lib64", "python")
check_path(cmd.install_lib, libdir)
- check_path(cmd.install_platlib, libdir)
+ check_path(cmd.install_platlib, platlibdir)
check_path(cmd.install_purelib, libdir)
check_path(cmd.install_headers,
os.path.join(destination, "include", "python", "foopkg"))

View File

@ -1,45 +1,61 @@
From fb93392b0f4975a02775a608611dc9ceb20c06ad Mon Sep 17 00:00:00 2001
From: David Malcolm <dmalcolm@redhat.com>
Date: Mon, 18 Jan 2010 17:59:07 +0000
Subject: [PATCH] 00111: Don't try to build a libpythonMAJOR.MINOR.a
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Downstream only: not appropriate for upstream.
See https://bugzilla.redhat.com/show_bug.cgi?id=556092
Co-authored-by: David Malcolm <dmalcolm@redhat.com>
Co-authored-by: Bohuslav Kabrda <bkabrda@redhat.com>
Co-authored-by: Matej Stuchlik <mstuchli@redhat.com>
Co-authored-by: Robert Kuska <rkuska@redhat.com>
Co-authored-by: Charalampos Stratakis <cstratak@redhat.com>
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
---
Makefile.pre.in | 21 ++-------------------
1 file changed, 2 insertions(+), 19 deletions(-)
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 70e5927..04c8e3d 100644
index 406a441082..917303dd92 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -556,7 +556,7 @@ clinic: $(BUILDPYTHON) $(srcdir)/Modules/_blake2/blake2s_impl.c
$(RUNSHARED) $(PYTHON_FOR_BUILD) ./Tools/clinic/clinic.py --make
@@ -562,7 +562,7 @@ clinic: check-clean-src $(srcdir)/Modules/_blake2/blake2s_impl.c
$(PYTHON_FOR_REGEN) $(srcdir)/Tools/clinic/clinic.py --make --srcdir $(srcdir)
# Build the interpreter
-$(BUILDPYTHON): Programs/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
+$(BUILDPYTHON): Programs/python.o $(LDLIBRARY) $(PY3LIBRARY)
$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS)
platform: $(BUILDPYTHON) pybuilddir.txt
@@ -601,18 +601,6 @@ sharedmods: $(BUILDPYTHON) pybuilddir.txt Modules/_math.o
@@ -610,12 +610,6 @@ sharedmods: $(BUILDPYTHON) pybuilddir.txt Modules/_math.o
_TCLTK_INCLUDES='$(TCLTK_INCLUDES)' _TCLTK_LIBS='$(TCLTK_LIBS)' \
$(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build
-
-# Build static library
-# avoid long command lines, same as LIBRARY_OBJS
-$(LIBRARY): $(LIBRARY_OBJS)
- -rm -f $@
- $(AR) $(ARFLAGS) $@ Modules/getbuildinfo.o
- $(AR) $(ARFLAGS) $@ $(PARSER_OBJS)
- $(AR) $(ARFLAGS) $@ $(OBJECT_OBJS)
- $(AR) $(ARFLAGS) $@ $(PYTHON_OBJS) Python/frozen.o
- $(AR) $(ARFLAGS) $@ $(MODULE_OBJS)
- $(AR) $(ARFLAGS) $@ $(MODOBJS)
- $(RANLIB) $@
- $(AR) $(ARFLAGS) $@ $(LIBRARY_OBJS)
-
libpython$(LDVERSION).so: $(LIBRARY_OBJS)
libpython$(LDVERSION).so: $(LIBRARY_OBJS) $(DTRACE_OBJS)
if test $(INSTSONAME) != $(LDLIBRARY); then \
$(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
@@ -702,7 +690,7 @@ Modules/Setup: $(srcdir)/Modules/Setup.dist
echo "-----------------------------------------------"; \
fi
$(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM); \
@@ -693,7 +687,7 @@ Makefile Modules/config.c: Makefile.pre \
@echo "The Makefile was updated, you may need to re-run make."
-Programs/_testembed: Programs/_testembed.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
+Programs/_testembed: Programs/_testembed.o $(LDLIBRARY) $(PY3LIBRARY)
$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS)
############################################################################
@@ -1382,18 +1370,6 @@ libainstall: all python-config
@@ -1557,17 +1551,6 @@ libainstall: @DEF_MAKE_RULE@ python-config
else true; \
fi; \
done
@ -49,7 +65,6 @@ index 70e5927..04c8e3d 100644
- $(INSTALL_DATA) $(LDLIBRARY) $(DESTDIR)$(LIBPL) ; \
- else \
- $(INSTALL_DATA) $(LIBRARY) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \
- $(RANLIB) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \
- fi; \
- else \
- echo Skip install of $(LIBRARY) - use make frameworkinstall; \
@ -58,3 +73,6 @@ index 70e5927..04c8e3d 100644
$(INSTALL_DATA) Modules/config.c $(DESTDIR)$(LIBPL)/config.c
$(INSTALL_DATA) Programs/python.o $(DESTDIR)$(LIBPL)/python.o
$(INSTALL_DATA) $(srcdir)/Modules/config.c.in $(DESTDIR)$(LIBPL)/config.c.in
--
2.24.1

View File

@ -1,46 +0,0 @@
diff -up Python-3.2.2/Lib/unittest/case.py.add-rpmbuild-hooks-to-unittest Python-3.2.2/Lib/unittest/case.py
--- Python-3.2.2/Lib/unittest/case.py.add-rpmbuild-hooks-to-unittest 2011-09-03 12:16:44.000000000 -0400
+++ Python-3.2.2/Lib/unittest/case.py 2011-09-09 06:35:16.365568382 -0400
@@ -3,6 +3,7 @@
import sys
import functools
import difflib
+import os
import logging
import pprint
import re
@@ -101,5 +102,21 @@ def expectedFailure(func):
raise self.test_case.failureException(msg)
+# Non-standard/downstream-only hooks for handling issues with specific test
+# cases:
+
+def _skipInRpmBuild(reason):
+ """
+ Non-standard/downstream-only decorator for marking a specific unit test
+ to be skipped when run within the %check of an rpmbuild.
+
+ Specifically, this takes effect when WITHIN_PYTHON_RPM_BUILD is set within
+ the environment, and has no effect otherwise.
+ """
+ if 'WITHIN_PYTHON_RPM_BUILD' in os.environ:
+ return skip(reason)
+ else:
+ return _id
+
class _AssertRaisesBaseContext(_BaseTestCaseContext):
def __init__(self, expected, test_case, expected_regex=None):
diff -up Python-3.2.2/Lib/unittest/__init__.py.add-rpmbuild-hooks-to-unittest Python-3.2.2/Lib/unittest/__init__.py
--- Python-3.2.2/Lib/unittest/__init__.py.add-rpmbuild-hooks-to-unittest 2011-09-03 12:16:44.000000000 -0400
+++ Python-3.2.2/Lib/unittest/__init__.py 2011-09-09 06:35:16.366568382 -0400
@@ -57,7 +57,8 @@ __unittest = True
from .result import TestResult
from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf,
- skipUnless, expectedFailure)
+ skipUnless, expectedFailure,
+ _skipInRpmBuild)
from .suite import BaseTestSuite, TestSuite
from .loader import (TestLoader, defaultTestLoader, makeSuite, getTestCaseNames,
findTestCases)

View File

@ -1,12 +0,0 @@
diff -up Python-3.2.2/Lib/distutils/tests/test_bdist_rpm.py.skip-distutils-tests-that-fail-in-rpmbuild Python-3.2.2/Lib/distutils/tests/test_bdist_rpm.py
--- Python-3.2.2/Lib/distutils/tests/test_bdist_rpm.py.skip-distutils-tests-that-fail-in-rpmbuild 2011-09-03 12:16:40.000000000 -0400
+++ Python-3.2.2/Lib/distutils/tests/test_bdist_rpm.py 2011-09-10 05:04:56.328852558 -0400
@@ -23,6 +23,7 @@ setup(name='foo', version='0.1', py_modu
"""
+@unittest._skipInRpmBuild("don't try to nest one rpm build inside another rpm build")
class BuildRpmTestCase(support.TempdirManager,
support.EnvironGuard,
support.LoggingSilencer,
diff -up Python-3.2.2/Lib/distutils/tests/test_build_ext.py.skip-distutils-tests-that-fail-in-rpmbuild Python-3.2.2/Lib/distutils/tests/test_build_ext.py

View File

@ -1,640 +0,0 @@
diff --git a/Lib/hashlib.py b/Lib/hashlib.py
index 316cece..b7ad879 100644
--- a/Lib/hashlib.py
+++ b/Lib/hashlib.py
@@ -23,6 +23,16 @@ the zlib module.
Choose your hash function wisely. Some have known collision weaknesses.
sha384 and sha512 will be slow on 32 bit platforms.
+If the underlying implementation supports "FIPS mode", and this is enabled, it
+may restrict the available hashes to only those that are compliant with FIPS
+regulations. For example, it may deny the use of MD5, on the grounds that this
+is not secure for uses such as authentication, system integrity checking, or
+digital signatures. If you need to use such a hash for non-security purposes
+(such as indexing into a data structure for speed), you can override the keyword
+argument "usedforsecurity" from True to False to signify that your code is not
+relying on the hash for security purposes, and this will allow the hash to be
+usable even in FIPS mode.
+
Hash objects have these methods:
- update(arg): Update the hash object with the bytes in arg. Repeated calls
are equivalent to a single call with the concatenation of all
@@ -62,6 +72,18 @@ algorithms_available = set(__always_supported)
__all__ = __always_supported + ('new', 'algorithms_guaranteed',
'algorithms_available', 'pbkdf2_hmac')
+import functools
+def __ignore_usedforsecurity(func):
+ """Used for sha3_* functions. Until OpenSSL implements them, we want
+ to use them from Python _sha3 module, but we want them to accept
+ usedforsecurity argument too."""
+ # TODO: remove this function when OpenSSL implements sha3
+ @functools.wraps(func)
+ def inner(*args, **kwargs):
+ if 'usedforsecurity' in kwargs:
+ kwargs.pop('usedforsecurity')
+ return func(*args, **kwargs)
+ return inner
__builtin_constructor_cache = {}
@@ -100,31 +122,39 @@ def __get_openssl_constructor(name):
f = getattr(_hashlib, 'openssl_' + name)
# Allow the C module to raise ValueError. The function will be
# defined but the hash not actually available thanks to OpenSSL.
- f()
+ # We pass "usedforsecurity=False" to disable FIPS-based restrictions:
+ # at this stage we're merely seeing if the function is callable,
+ # rather than using it for actual work.
+ f(usedforsecurity=False)
# Use the C function directly (very fast)
return f
except (AttributeError, ValueError):
+ # TODO: We want to just raise here when OpenSSL implements sha3
+ # because we want to make sure that Fedora uses everything from OpenSSL
return __get_builtin_constructor(name)
-def __py_new(name, data=b''):
- """new(name, data=b'') - Return a new hashing object using the named algorithm;
- optionally initialized with data (which must be bytes).
+def __py_new(name, data=b'', usedforsecurity=True):
+ """new(name, data=b'', usedforsecurity=True) - Return a new hashing object using
+ the named algorithm; optionally initialized with data (which must be bytes).
+ The 'usedforsecurity' keyword argument does nothing, and is for compatibilty
+ with the OpenSSL implementation
"""
return __get_builtin_constructor(name)(data)
-def __hash_new(name, data=b''):
- """new(name, data=b'') - Return a new hashing object using the named algorithm;
- optionally initialized with data (which must be bytes).
+def __hash_new(name, data=b'', usedforsecurity=True):
+ """new(name, data=b'', usedforsecurity=True) - Return a new hashing object using
+ the named algorithm; optionally initialized with data (which must be bytes).
+
+ Override 'usedforsecurity' to False when using for non-security purposes in
+ a FIPS environment
"""
try:
- return _hashlib.new(name, data)
+ return _hashlib.new(name, data, usedforsecurity)
except ValueError:
- # If the _hashlib module (OpenSSL) doesn't support the named
- # hash, try using our builtin implementations.
- # This allows for SHA224/256 and SHA384/512 support even though
- # the OpenSSL library prior to 0.9.8 doesn't provide them.
+ # TODO: We want to just raise here when OpenSSL implements sha3
+ # because we want to make sure that Fedora uses everything from OpenSSL
return __get_builtin_constructor(name)(data)
@@ -207,7 +237,10 @@ for __func_name in __always_supported:
# try them all, some may not work due to the OpenSSL
# version not supporting that algorithm.
try:
- globals()[__func_name] = __get_hash(__func_name)
+ func = __get_hash(__func_name)
+ if 'sha3_' in __func_name:
+ func = __ignore_usedforsecurity(func)
+ globals()[__func_name] = func
except ValueError:
import logging
logging.exception('code for hash %s was not found.', __func_name)
@@ -215,3 +248,4 @@ for __func_name in __always_supported:
# Cleanup locals()
del __always_supported, __func_name, __get_hash
del __py_new, __hash_new, __get_openssl_constructor
+del __ignore_usedforsecurity
\ No newline at end of file
diff --git a/Lib/test/test_hashlib.py b/Lib/test/test_hashlib.py
index c9b113e..60e2392 100644
--- a/Lib/test/test_hashlib.py
+++ b/Lib/test/test_hashlib.py
@@ -24,7 +24,22 @@ from test.support import _4G, bigmemtest, import_fresh_module
COMPILED_WITH_PYDEBUG = hasattr(sys, 'gettotalrefcount')
c_hashlib = import_fresh_module('hashlib', fresh=['_hashlib'])
-py_hashlib = import_fresh_module('hashlib', blocked=['_hashlib'])
+# skipped on Fedora, since we always use OpenSSL implementation
+# py_hashlib = import_fresh_module('hashlib', blocked=['_hashlib'])
+
+def openssl_enforces_fips():
+ # Use the "openssl" command (if present) to try to determine if the local
+ # OpenSSL is configured to enforce FIPS
+ from subprocess import Popen, PIPE
+ try:
+ p = Popen(['openssl', 'md5'],
+ stdin=PIPE, stdout=PIPE, stderr=PIPE)
+ except OSError:
+ # "openssl" command not found
+ return False
+ stdout, stderr = p.communicate(input=b'abc')
+ return b'unknown cipher' in stderr
+OPENSSL_ENFORCES_FIPS = openssl_enforces_fips()
def hexstr(s):
assert isinstance(s, bytes), repr(s)
@@ -34,6 +49,16 @@ def hexstr(s):
r += h[(i >> 4) & 0xF] + h[i & 0xF]
return r
+# hashlib and _hashlib-based functions support a "usedforsecurity" keyword
+# argument, and FIPS mode requires that it be used overridden with a False
+# value for these selftests to work. Other cryptographic code within Python
+# doesn't support this keyword.
+# Modify a function to one in which "usedforsecurity=False" is added to the
+# keyword arguments:
+def suppress_fips(f):
+ def g(*args, **kwargs):
+ return f(*args, usedforsecurity=False, **kwargs)
+ return g
class HashLibTestCase(unittest.TestCase):
supported_hash_names = ( 'md5', 'MD5', 'sha1', 'SHA1',
@@ -63,11 +88,11 @@ class HashLibTestCase(unittest.TestCase):
# For each algorithm, test the direct constructor and the use
# of hashlib.new given the algorithm name.
for algorithm, constructors in self.constructors_to_test.items():
- constructors.add(getattr(hashlib, algorithm))
+ constructors.add(suppress_fips(getattr(hashlib, algorithm)))
def _test_algorithm_via_hashlib_new(data=None, _alg=algorithm):
if data is None:
- return hashlib.new(_alg)
- return hashlib.new(_alg, data)
+ return suppress_fips(hashlib.new)(_alg)
+ return suppress_fips(hashlib.new)(_alg, data)
constructors.add(_test_algorithm_via_hashlib_new)
_hashlib = self._conditional_import_module('_hashlib')
@@ -79,27 +104,12 @@ class HashLibTestCase(unittest.TestCase):
for algorithm, constructors in self.constructors_to_test.items():
constructor = getattr(_hashlib, 'openssl_'+algorithm, None)
if constructor:
- constructors.add(constructor)
+ constructors.add(suppress_fips(constructor))
def add_builtin_constructor(name):
constructor = getattr(hashlib, "__get_builtin_constructor")(name)
self.constructors_to_test[name].add(constructor)
- _md5 = self._conditional_import_module('_md5')
- if _md5:
- add_builtin_constructor('md5')
- _sha1 = self._conditional_import_module('_sha1')
- if _sha1:
- add_builtin_constructor('sha1')
- _sha256 = self._conditional_import_module('_sha256')
- if _sha256:
- add_builtin_constructor('sha224')
- add_builtin_constructor('sha256')
- _sha512 = self._conditional_import_module('_sha512')
- if _sha512:
- add_builtin_constructor('sha384')
- add_builtin_constructor('sha512')
-
super(HashLibTestCase, self).__init__(*args, **kwargs)
@property
@@ -148,9 +158,6 @@ class HashLibTestCase(unittest.TestCase):
else:
del sys.modules['_md5']
self.assertRaises(TypeError, get_builtin_constructor, 3)
- constructor = get_builtin_constructor('md5')
- self.assertIs(constructor, _md5.md5)
- self.assertEqual(sorted(builtin_constructor_cache), ['MD5', 'md5'])
def test_hexdigest(self):
for cons in self.hash_constructors:
@@ -433,6 +440,64 @@ class HashLibTestCase(unittest.TestCase):
self.assertEqual(expected_hash, hasher.hexdigest())
+ def test_issue9146(self):
+ # Ensure that various ways to use "MD5" from "hashlib" don't segfault:
+ m = hashlib.md5(usedforsecurity=False)
+ m.update(b'abc\n')
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = hashlib.new('md5', usedforsecurity=False)
+ m.update(b'abc\n')
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = hashlib.md5(b'abc\n', usedforsecurity=False)
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = hashlib.new('md5', b'abc\n', usedforsecurity=False)
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ @unittest.skipUnless(OPENSSL_ENFORCES_FIPS,
+ 'FIPS enforcement required for this test.')
+ def test_hashlib_fips_mode(self):
+ # Ensure that we raise a ValueError on vanilla attempts to use MD5
+ # in hashlib in a FIPS-enforced setting:
+ with self.assertRaisesRegexp(ValueError, '.*unknown cipher'):
+ m = hashlib.md5()
+
+ if not self._conditional_import_module('_md5'):
+ with self.assertRaisesRegexp(ValueError, '.*unknown cipher'):
+ m = hashlib.new('md5')
+
+ @unittest.skipUnless(OPENSSL_ENFORCES_FIPS,
+ 'FIPS enforcement required for this test.')
+ def test_hashopenssl_fips_mode(self):
+ # Verify the _hashlib module's handling of md5:
+ _hashlib = self._conditional_import_module('_hashlib')
+ if _hashlib:
+ assert hasattr(_hashlib, 'openssl_md5')
+
+ # Ensure that _hashlib raises a ValueError on vanilla attempts to
+ # use MD5 in a FIPS-enforced setting:
+ with self.assertRaisesRegexp(ValueError, '.*unknown cipher'):
+ m = _hashlib.openssl_md5()
+ with self.assertRaisesRegexp(ValueError, '.*unknown cipher'):
+ m = _hashlib.new('md5')
+
+ # Ensure that in such a setting we can whitelist a callsite with
+ # usedforsecurity=False and have it succeed:
+ m = _hashlib.openssl_md5(usedforsecurity=False)
+ m.update(b'abc\n')
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = _hashlib.new('md5', usedforsecurity=False)
+ m.update(b'abc\n')
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = _hashlib.openssl_md5(b'abc\n', usedforsecurity=False)
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
+
+ m = _hashlib.new('md5', b'abc\n', usedforsecurity=False)
+ self.assertEquals(m.hexdigest(), "0bee89b07a248e27c83fc3d5951213c1")
class KDFTests(unittest.TestCase):
@@ -516,7 +581,7 @@ class KDFTests(unittest.TestCase):
out = pbkdf2(hash_name='sha1', password=b'password', salt=b'salt',
iterations=1, dklen=None)
self.assertEqual(out, self.pbkdf2_results['sha1'][0][0])
-
+ @unittest.skip('skipped on Fedora, as we always use OpenSSL pbkdf2_hmac')
def test_pbkdf2_hmac_py(self):
self._test_pbkdf2_hmac(py_hashlib.pbkdf2_hmac)
diff --git a/Modules/_hashopenssl.c b/Modules/_hashopenssl.c
index 44765ac..b8cf490 100644
--- a/Modules/_hashopenssl.c
+++ b/Modules/_hashopenssl.c
@@ -20,6 +20,8 @@
/* EVP is the preferred interface to hashing in OpenSSL */
+#include <openssl/ssl.h>
+#include <openssl/err.h>
#include <openssl/evp.h>
#include <openssl/hmac.h>
/* We use the object interface to discover what hashes OpenSSL supports. */
@@ -45,11 +47,19 @@ typedef struct {
static PyTypeObject EVPtype;
+/* Struct to hold all the cached information we need on a specific algorithm.
+ We have one of these per algorithm */
+typedef struct {
+ PyObject *name_obj;
+ EVP_MD_CTX ctxs[2];
+ /* ctx_ptrs will point to ctxs unless an error occurred, when it will
+ be NULL: */
+ EVP_MD_CTX *ctx_ptrs[2];
+ PyObject *error_msgs[2];
+} EVPCachedInfo;
-#define DEFINE_CONSTS_FOR_NEW(Name) \
- static PyObject *CONST_ ## Name ## _name_obj = NULL; \
- static EVP_MD_CTX CONST_new_ ## Name ## _ctx; \
- static EVP_MD_CTX *CONST_new_ ## Name ## _ctx_p = NULL;
+#define DEFINE_CONSTS_FOR_NEW(Name) \
+ static EVPCachedInfo cached_info_ ##Name;
DEFINE_CONSTS_FOR_NEW(md5)
DEFINE_CONSTS_FOR_NEW(sha1)
@@ -92,6 +102,48 @@ EVP_hash(EVPobject *self, const void *vp, Py_ssize_t len)
}
}
+static void
+mc_ctx_init(EVP_MD_CTX *ctx, int usedforsecurity)
+{
+ EVP_MD_CTX_init(ctx);
+
+ /*
+ If the user has declared that this digest is being used in a
+ non-security role (e.g. indexing into a data structure), set
+ the exception flag for openssl to allow it
+ */
+ if (!usedforsecurity) {
+#ifdef EVP_MD_CTX_FLAG_NON_FIPS_ALLOW
+ EVP_MD_CTX_set_flags(ctx,
+ EVP_MD_CTX_FLAG_NON_FIPS_ALLOW);
+#endif
+ }
+}
+
+/* Get an error msg for the last error as a PyObject */
+static PyObject *
+error_msg_for_last_error(void)
+{
+ char *errstr;
+
+ errstr = ERR_error_string(ERR_peek_last_error(), NULL);
+ ERR_clear_error();
+
+ return PyUnicode_FromString(errstr); /* Can be NULL */
+}
+
+static void
+set_evp_exception(void)
+{
+ char *errstr;
+
+ errstr = ERR_error_string(ERR_peek_last_error(), NULL);
+ ERR_clear_error();
+
+ PyErr_SetString(PyExc_ValueError, errstr);
+}
+
+
/* Internal methods for a hash object */
static void
@@ -259,15 +311,16 @@ EVP_repr(EVPobject *self)
static int
EVP_tp_init(EVPobject *self, PyObject *args, PyObject *kwds)
{
- static char *kwlist[] = {"name", "string", NULL};
+ static char *kwlist[] = {"name", "string", "usedforsecurity", NULL};
PyObject *name_obj = NULL;
PyObject *data_obj = NULL;
+ int usedforsecurity = 1;
Py_buffer view;
char *nameStr;
const EVP_MD *digest;
- if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|O:HASH", kwlist,
- &name_obj, &data_obj)) {
+ if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|Oi:HASH", kwlist,
+ &name_obj, &data_obj, &usedforsecurity)) {
return -1;
}
@@ -288,7 +341,12 @@ EVP_tp_init(EVPobject *self, PyObject *args, PyObject *kwds)
PyBuffer_Release(&view);
return -1;
}
- EVP_DigestInit(&self->ctx, digest);
+ mc_ctx_init(&self->ctx, usedforsecurity);
+ if (!EVP_DigestInit_ex(&self->ctx, digest, NULL)) {
+ set_evp_exception();
+ PyBuffer_Release(&view);
+ return -1;
+ }
self->name = name_obj;
Py_INCREF(self->name);
@@ -372,7 +430,8 @@ static PyTypeObject EVPtype = {
static PyObject *
EVPnew(PyObject *name_obj,
const EVP_MD *digest, const EVP_MD_CTX *initial_ctx,
- const unsigned char *cp, Py_ssize_t len)
+ const unsigned char *cp, Py_ssize_t len,
+ int usedforsecurity)
{
EVPobject *self;
@@ -387,7 +446,12 @@ EVPnew(PyObject *name_obj,
if (initial_ctx) {
EVP_MD_CTX_copy(&self->ctx, initial_ctx);
} else {
- EVP_DigestInit(&self->ctx, digest);
+ mc_ctx_init(&self->ctx, usedforsecurity);
+ if (!EVP_DigestInit_ex(&self->ctx, digest, NULL)) {
+ set_evp_exception();
+ Py_DECREF(self);
+ return NULL;
+ }
}
if (cp && len) {
@@ -411,21 +475,29 @@ PyDoc_STRVAR(EVP_new__doc__,
An optional string argument may be provided and will be\n\
automatically hashed.\n\
\n\
-The MD5 and SHA1 algorithms are always supported.\n");
+The MD5 and SHA1 algorithms are always supported.\n\
+\n\
+An optional \"usedforsecurity=True\" keyword argument is provided for use in\n\
+environments that enforce FIPS-based restrictions. Some implementations of\n\
+OpenSSL can be configured to prevent the usage of non-secure algorithms (such\n\
+as MD5). If you have a non-security use for these algorithms (e.g. a hash\n\
+table), you can override this argument by marking the callsite as\n\
+\"usedforsecurity=False\".");
static PyObject *
EVP_new(PyObject *self, PyObject *args, PyObject *kwdict)
{
- static char *kwlist[] = {"name", "string", NULL};
+ static char *kwlist[] = {"name", "string", "usedforsecurity", NULL};
PyObject *name_obj = NULL;
PyObject *data_obj = NULL;
+ int usedforsecurity = 1;
Py_buffer view = { 0 };
PyObject *ret_obj;
char *name;
const EVP_MD *digest;
- if (!PyArg_ParseTupleAndKeywords(args, kwdict, "O|O:new", kwlist,
- &name_obj, &data_obj)) {
+ if (!PyArg_ParseTupleAndKeywords(args, kwdict, "O|Oi:new", kwlist,
+ &name_obj, &data_obj, &usedforsecurity)) {
return NULL;
}
@@ -439,7 +511,8 @@ EVP_new(PyObject *self, PyObject *args, PyObject *kwdict)
digest = EVP_get_digestbyname(name);
- ret_obj = EVPnew(name_obj, digest, NULL, (unsigned char*)view.buf, view.len);
+ ret_obj = EVPnew(name_obj, digest, NULL, (unsigned char*)view.buf, view.len,
+ usedforsecurity);
if (data_obj)
PyBuffer_Release(&view);
@@ -722,57 +795,114 @@ generate_hash_name_list(void)
/*
- * This macro generates constructor function definitions for specific
- * hash algorithms. These constructors are much faster than calling
- * the generic one passing it a python string and are noticably
- * faster than calling a python new() wrapper. Thats important for
+ * This macro and function generates a family of constructor function
+ * definitions for specific hash algorithms. These constructors are much
+ * faster than calling the generic one passing it a python string and are
+ * noticably faster than calling a python new() wrapper. That's important for
* code that wants to make hashes of a bunch of small strings.
*/
#define GEN_CONSTRUCTOR(NAME) \
static PyObject * \
- EVP_new_ ## NAME (PyObject *self, PyObject *args) \
+ EVP_new_ ## NAME (PyObject *self, PyObject *args, PyObject *kwdict) \
{ \
- PyObject *data_obj = NULL; \
- Py_buffer view = { 0 }; \
- PyObject *ret_obj; \
- \
- if (!PyArg_ParseTuple(args, "|O:" #NAME , &data_obj)) { \
- return NULL; \
- } \
- \
- if (data_obj) \
- GET_BUFFER_VIEW_OR_ERROUT(data_obj, &view); \
- \
- ret_obj = EVPnew( \
- CONST_ ## NAME ## _name_obj, \
- NULL, \
- CONST_new_ ## NAME ## _ctx_p, \
- (unsigned char*)view.buf, \
- view.len); \
- \
- if (data_obj) \
- PyBuffer_Release(&view); \
- return ret_obj; \
+ return implement_specific_EVP_new(self, args, kwdict, \
+ "|Oi:" #NAME, \
+ &cached_info_ ## NAME ); \
}
+static PyObject *
+implement_specific_EVP_new(PyObject *self, PyObject *args, PyObject *kwdict,
+ const char *format,
+ EVPCachedInfo *cached_info)
+{
+ static char *kwlist[] = {"string", "usedforsecurity", NULL};
+ PyObject *data_obj = NULL;
+ Py_buffer view = { 0 };
+ int usedforsecurity = 1;
+ int idx;
+ PyObject *ret_obj = NULL;
+
+ assert(cached_info);
+
+ if (!PyArg_ParseTupleAndKeywords(args, kwdict, format, kwlist,
+ &data_obj, &usedforsecurity)) {
+ return NULL;
+ }
+
+ if (data_obj)
+ GET_BUFFER_VIEW_OR_ERROUT(data_obj, &view);
+
+ idx = usedforsecurity ? 1 : 0;
+
+ /*
+ * If an error occurred during creation of the global content, the ctx_ptr
+ * will be NULL, and the error_msg will hopefully be non-NULL:
+ */
+ if (cached_info->ctx_ptrs[idx]) {
+ /* We successfully initialized this context; copy it: */
+ ret_obj = EVPnew(cached_info->name_obj,
+ NULL,
+ cached_info->ctx_ptrs[idx],
+ (unsigned char*)view.buf, view.len,
+ usedforsecurity);
+ } else {
+ /* Some kind of error happened initializing the global context for
+ this (digest, usedforsecurity) pair.
+ Raise an exception with the saved error message: */
+ if (cached_info->error_msgs[idx]) {
+ PyErr_SetObject(PyExc_ValueError, cached_info->error_msgs[idx]);
+ } else {
+ PyErr_SetString(PyExc_ValueError, "Error initializing hash");
+ }
+ }
+
+ if (data_obj)
+ PyBuffer_Release(&view);
+
+ return ret_obj;
+}
+
/* a PyMethodDef structure for the constructor */
#define CONSTRUCTOR_METH_DEF(NAME) \
- {"openssl_" #NAME, (PyCFunction)EVP_new_ ## NAME, METH_VARARGS, \
+ {"openssl_" #NAME, (PyCFunction)EVP_new_ ## NAME, \
+ METH_VARARGS|METH_KEYWORDS, \
PyDoc_STR("Returns a " #NAME \
" hash object; optionally initialized with a string") \
}
-/* used in the init function to setup a constructor: initialize OpenSSL
- constructor constants if they haven't been initialized already. */
-#define INIT_CONSTRUCTOR_CONSTANTS(NAME) do { \
- if (CONST_ ## NAME ## _name_obj == NULL) { \
- CONST_ ## NAME ## _name_obj = PyUnicode_FromString(#NAME); \
- if (EVP_get_digestbyname(#NAME)) { \
- CONST_new_ ## NAME ## _ctx_p = &CONST_new_ ## NAME ## _ctx; \
- EVP_DigestInit(CONST_new_ ## NAME ## _ctx_p, EVP_get_digestbyname(#NAME)); \
- } \
- } \
+/*
+ Macro/function pair to set up the constructors.
+
+ Try to initialize a context for each hash twice, once with
+ EVP_MD_CTX_FLAG_NON_FIPS_ALLOW and once without.
+
+ Any that have errors during initialization will end up with a NULL ctx_ptrs
+ entry, and err_msgs will be set (unless we're very low on memory)
+*/
+#define INIT_CONSTRUCTOR_CONSTANTS(NAME) do { \
+ init_constructor_constant(&cached_info_ ## NAME, #NAME); \
} while (0);
+static void
+init_constructor_constant(EVPCachedInfo *cached_info, const char *name)
+{
+ assert(cached_info);
+ cached_info->name_obj = PyUnicode_FromString(name);
+ if (EVP_get_digestbyname(name)) {
+ int i;
+ for (i=0; i<2; i++) {
+ mc_ctx_init(&cached_info->ctxs[i], i);
+ if (EVP_DigestInit_ex(&cached_info->ctxs[i],
+ EVP_get_digestbyname(name), NULL)) {
+ /* Success: */
+ cached_info->ctx_ptrs[i] = &cached_info->ctxs[i];
+ } else {
+ /* Failure: */
+ cached_info->ctx_ptrs[i] = NULL;
+ cached_info->error_msgs[i] = error_msg_for_last_error();
+ }
+ }
+ }
+}
GEN_CONSTRUCTOR(md5)
GEN_CONSTRUCTOR(sha1)
@@ -819,13 +949,10 @@ PyInit__hashlib(void)
{
PyObject *m, *openssl_md_meth_names;
- OpenSSL_add_all_digests();
- ERR_load_crypto_strings();
+ SSL_load_error_strings();
+ SSL_library_init();
- /* TODO build EVP_functions openssl_* entries dynamically based
- * on what hashes are supported rather than listing many
- * but having some be unsupported. Only init appropriate
- * constants. */
+ OpenSSL_add_all_digests();
Py_TYPE(&EVPtype) = &PyType_Type;
if (PyType_Ready(&EVPtype) < 0)

View File

@ -1,15 +0,0 @@
diff -up Python-3.2.3/Lib/ctypes/__init__.py.rhbz814391 Python-3.2.3/Lib/ctypes/__init__.py
--- Python-3.2.3/Lib/ctypes/__init__.py.rhbz814391 2012-04-20 15:12:49.017867692 -0400
+++ Python-3.2.3/Lib/ctypes/__init__.py 2012-04-20 15:15:09.501111408 -0400
@@ -275,11 +275,6 @@ def _reset_cache():
# _SimpleCData.c_char_p_from_param
POINTER(c_char).from_param = c_char_p.from_param
_pointer_type_cache[None] = c_void_p
- # XXX for whatever reasons, creating the first instance of a callback
- # function is needed for the unittests on Win64 to succeed. This MAY
- # be a compiler bug, since the problem occurs only when _ctypes is
- # compiled with the MS SDK compiler. Or an uninitialized variable?
- CFUNCTYPE(c_int)(lambda: None)
def create_unicode_buffer(init, size=None):
"""create_unicode_buffer(aString) -> character array

View File

@ -1,68 +0,0 @@
diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
index e9fdb07..ea60e6e 100644
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -1723,30 +1723,36 @@ class PosixUidGidTests(unittest.TestCase):
def test_setuid(self):
if os.getuid() != 0:
self.assertRaises(OSError, os.setuid, 0)
+ self.assertRaises(TypeError, os.setuid, 'not an int')
self.assertRaises(OverflowError, os.setuid, 1<<32)
@unittest.skipUnless(hasattr(os, 'setgid'), 'test needs os.setgid()')
def test_setgid(self):
if os.getuid() != 0 and not HAVE_WHEEL_GROUP:
self.assertRaises(OSError, os.setgid, 0)
+ self.assertRaises(TypeError, os.setgid, 'not an int')
self.assertRaises(OverflowError, os.setgid, 1<<32)
@unittest.skipUnless(hasattr(os, 'seteuid'), 'test needs os.seteuid()')
def test_seteuid(self):
if os.getuid() != 0:
self.assertRaises(OSError, os.seteuid, 0)
+ self.assertRaises(TypeError, os.seteuid, 'not an int')
self.assertRaises(OverflowError, os.seteuid, 1<<32)
@unittest.skipUnless(hasattr(os, 'setegid'), 'test needs os.setegid()')
def test_setegid(self):
if os.getuid() != 0 and not HAVE_WHEEL_GROUP:
self.assertRaises(OSError, os.setegid, 0)
+ self.assertRaises(TypeError, os.setegid, 'not an int')
self.assertRaises(OverflowError, os.setegid, 1<<32)
@unittest.skipUnless(hasattr(os, 'setreuid'), 'test needs os.setreuid()')
def test_setreuid(self):
if os.getuid() != 0:
self.assertRaises(OSError, os.setreuid, 0, 0)
+ self.assertRaises(TypeError, os.setreuid, 'not an int', 0)
+ self.assertRaises(TypeError, os.setreuid, 0, 'not an int')
self.assertRaises(OverflowError, os.setreuid, 1<<32, 0)
self.assertRaises(OverflowError, os.setreuid, 0, 1<<32)
@@ -1762,6 +1768,8 @@ class PosixUidGidTests(unittest.TestCase):
def test_setregid(self):
if os.getuid() != 0 and not HAVE_WHEEL_GROUP:
self.assertRaises(OSError, os.setregid, 0, 0)
+ self.assertRaises(TypeError, os.setregid, 'not an int', 0)
+ self.assertRaises(TypeError, os.setregid, 0, 'not an int')
self.assertRaises(OverflowError, os.setregid, 1<<32, 0)
self.assertRaises(OverflowError, os.setregid, 0, 1<<32)
diff --git a/Lib/test/test_pwd.py b/Lib/test/test_pwd.py
index ac9cff7..db98159 100644
--- a/Lib/test/test_pwd.py
+++ b/Lib/test/test_pwd.py
@@ -104,11 +104,11 @@ class PwdTest(unittest.TestCase):
# In some cases, byuids isn't a complete list of all users in the
# system, so if we try to pick a value not in byuids (via a perturbing
# loop, say), pwd.getpwuid() might still be able to find data for that
- # uid. Using sys.maxint may provoke the same problems, but hopefully
+ # uid. Using 2**32 - 2 may provoke the same problems, but hopefully
# it will be a more repeatable failure.
# Android accepts a very large span of uids including sys.maxsize and
# -1; it raises KeyError with 1 or 2 for example.
- fakeuid = sys.maxsize
+ fakeuid = 2**32 - 2
self.assertNotIn(fakeuid, byuids)
if not support.is_android:
self.assertRaises(KeyError, pwd.getpwuid, fakeuid)

View File

@ -1,11 +0,0 @@
diff -up cpython-59223da36dec/Lib/test/test_posix.py.disable-test_fs_holes-in-rpm-build cpython-59223da36dec/Lib/test/test_posix.py
--- cpython-59223da36dec/Lib/test/test_posix.py.disable-test_fs_holes-in-rpm-build 2012-08-07 17:15:59.000000000 -0400
+++ cpython-59223da36dec/Lib/test/test_posix.py 2012-08-07 17:16:53.528330330 -0400
@@ -973,6 +973,7 @@ class PosixTester(unittest.TestCase):
posix.RTLD_GLOBAL
posix.RTLD_LOCAL
+ @unittest._skipInRpmBuild('running kernel may not match kernel in chroot')
@unittest.skipUnless(hasattr(os, 'SEEK_HOLE'),
"test needs an OS that reports file holes")
def test_fs_holes(self):

View File

@ -1,11 +0,0 @@
diff -up Python-3.3.0b1/Lib/test/test_socket.py.disable-test_socket-in-rpm-builds Python-3.3.0b1/Lib/test/test_socket.py
--- Python-3.3.0b1/Lib/test/test_socket.py.disable-test_socket-in-rpm-builds 2012-07-24 15:02:30.823355067 -0400
+++ Python-3.3.0b1/Lib/test/test_socket.py 2012-07-24 15:08:13.021354999 -0400
@@ -2188,6 +2188,7 @@ class RecvmsgGenericStreamTests(RecvmsgG
# Tests which require a stream socket and can use either recvmsg()
# or recvmsg_into().
+ @unittest._skipInRpmBuild('fails intermittently when run within Koji')
def testRecvmsgEOF(self):
# Receive end-of-stream indicator (b"", peer socket closed).
msg, ancdata, flags, addr = self.doRecvmsg(self.serv_sock, 1024)

View File

@ -1,310 +0,0 @@
diff --git a/Include/object.h b/Include/object.h
index 0c88603..e3413e8 100644
--- a/Include/object.h
+++ b/Include/object.h
@@ -1059,6 +1059,49 @@ PyAPI_FUNC(void)
_PyObject_DebugTypeStats(FILE *out);
#endif /* ifndef Py_LIMITED_API */
+/*
+ Define a pair of assertion macros.
+
+ These work like the regular C assert(), in that they will abort the
+ process with a message on stderr if the given condition fails to hold,
+ but compile away to nothing if NDEBUG is defined.
+
+ However, before aborting, Python will also try to call _PyObject_Dump() on
+ the given object. This may be of use when investigating bugs in which a
+ particular object is corrupt (e.g. buggy a tp_visit method in an extension
+ module breaking the garbage collector), to help locate the broken objects.
+
+ The WITH_MSG variant allows you to supply an additional message that Python
+ will attempt to print to stderr, after the object dump.
+*/
+#ifdef NDEBUG
+/* No debugging: compile away the assertions: */
+#define PyObject_ASSERT_WITH_MSG(obj, expr, msg) ((void)0)
+#else
+/* With debugging: generate checks: */
+#define PyObject_ASSERT_WITH_MSG(obj, expr, msg) \
+ ((expr) \
+ ? (void)(0) \
+ : _PyObject_AssertFailed((obj), \
+ (msg), \
+ (__STRING(expr)), \
+ (__FILE__), \
+ (__LINE__), \
+ (__PRETTY_FUNCTION__)))
+#endif
+
+#define PyObject_ASSERT(obj, expr) \
+ PyObject_ASSERT_WITH_MSG(obj, expr, NULL)
+
+/*
+ Declare and define the entrypoint even when NDEBUG is defined, to avoid
+ causing compiler/linker errors when building extensions without NDEBUG
+ against a Python built with NDEBUG defined
+*/
+PyAPI_FUNC(void) _PyObject_AssertFailed(PyObject *, const char *,
+ const char *, const char *, int,
+ const char *);
+
#ifdef __cplusplus
}
#endif
diff --git a/Lib/test/test_gc.py b/Lib/test/test_gc.py
index e727499..6efcafb 100644
--- a/Lib/test/test_gc.py
+++ b/Lib/test/test_gc.py
@@ -1,10 +1,11 @@
import unittest
from test.support import (verbose, refcount_test, run_unittest,
strip_python_stderr, cpython_only, start_threads,
- temp_dir, requires_type_collecting)
+ temp_dir, import_module, requires_type_collecting)
from test.support.script_helper import assert_python_ok, make_script
import sys
+import sysconfig
import time
import gc
import weakref
@@ -50,6 +51,8 @@ class GC_Detector(object):
# gc collects it.
self.wr = weakref.ref(C1055820(666), it_happened)
+BUILD_WITH_NDEBUG = ('-DNDEBUG' in sysconfig.get_config_vars()['PY_CFLAGS'])
+
@with_tp_del
class Uncollectable(object):
"""Create a reference cycle with multiple __del__ methods.
@@ -862,6 +865,50 @@ class GCCallbackTests(unittest.TestCase):
self.assertEqual(len(gc.garbage), 0)
+ @unittest.skipIf(BUILD_WITH_NDEBUG,
+ 'built with -NDEBUG')
+ def test_refcount_errors(self):
+ self.preclean()
+ # Verify the "handling" of objects with broken refcounts
+ import_module("ctypes") #skip if not supported
+
+ import subprocess
+ code = '''if 1:
+ a = []
+ b = [a]
+
+ # Simulate the refcount of "a" being too low (compared to the
+ # references held on it by live data), but keeping it above zero
+ # (to avoid deallocating it):
+ import ctypes
+ ctypes.pythonapi.Py_DecRef(ctypes.py_object(a))
+
+ # The garbage collector should now have a fatal error when it reaches
+ # the broken object:
+ import gc
+ gc.collect()
+ '''
+ p = subprocess.Popen([sys.executable, "-c", code],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ stdout, stderr = p.communicate()
+ p.stdout.close()
+ p.stderr.close()
+ # Verify that stderr has a useful error message:
+ self.assertRegex(stderr,
+ b'Modules/gcmodule.c:[0-9]+: visit_decref: Assertion "\(\(gc\)->gc.gc_refs >> \(1\)\) != 0" failed.')
+ self.assertRegex(stderr,
+ b'refcount was too small')
+ self.assertRegex(stderr,
+ b'object : \[\]')
+ self.assertRegex(stderr,
+ b'type : list')
+ self.assertRegex(stderr,
+ b'refcount: 1')
+ self.assertRegex(stderr,
+ b'address : 0x[0-9a-f]+')
+
+
class GCTogglingTests(unittest.TestCase):
def setUp(self):
gc.enable()
diff --git a/Modules/gcmodule.c b/Modules/gcmodule.c
index 0c6f444..87edd5a 100644
--- a/Modules/gcmodule.c
+++ b/Modules/gcmodule.c
@@ -341,7 +341,8 @@ update_refs(PyGC_Head *containers)
{
PyGC_Head *gc = containers->gc.gc_next;
for (; gc != containers; gc = gc->gc.gc_next) {
- assert(_PyGCHead_REFS(gc) == GC_REACHABLE);
+ PyObject_ASSERT(FROM_GC(gc),
+ _PyGCHead_REFS(gc) == GC_REACHABLE);
_PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc)));
/* Python's cyclic gc should never see an incoming refcount
* of 0: if something decref'ed to 0, it should have been
@@ -361,7 +362,8 @@ update_refs(PyGC_Head *containers)
* so serious that maybe this should be a release-build
* check instead of an assert?
*/
- assert(_PyGCHead_REFS(gc) != 0);
+ PyObject_ASSERT(FROM_GC(gc),
+ _PyGCHead_REFS(gc) != 0);
}
}
@@ -376,7 +378,9 @@ visit_decref(PyObject *op, void *data)
* generation being collected, which can be recognized
* because only they have positive gc_refs.
*/
- assert(_PyGCHead_REFS(gc) != 0); /* else refcount was too small */
+ PyObject_ASSERT_WITH_MSG(FROM_GC(gc),
+ _PyGCHead_REFS(gc) != 0,
+ "refcount was too small"); /* else refcount was too small */
if (_PyGCHead_REFS(gc) > 0)
_PyGCHead_DECREF(gc);
}
@@ -436,9 +440,10 @@ visit_reachable(PyObject *op, PyGC_Head *reachable)
* If gc_refs == GC_UNTRACKED, it must be ignored.
*/
else {
- assert(gc_refs > 0
- || gc_refs == GC_REACHABLE
- || gc_refs == GC_UNTRACKED);
+ PyObject_ASSERT(FROM_GC(gc),
+ gc_refs > 0
+ || gc_refs == GC_REACHABLE
+ || gc_refs == GC_UNTRACKED);
}
}
return 0;
@@ -480,7 +485,7 @@ move_unreachable(PyGC_Head *young, PyGC_Head *unreachable)
*/
PyObject *op = FROM_GC(gc);
traverseproc traverse = Py_TYPE(op)->tp_traverse;
- assert(_PyGCHead_REFS(gc) > 0);
+ PyObject_ASSERT(op, _PyGCHead_REFS(gc) > 0);
_PyGCHead_SET_REFS(gc, GC_REACHABLE);
(void) traverse(op,
(visitproc)visit_reachable,
@@ -543,7 +548,7 @@ move_legacy_finalizers(PyGC_Head *unreachable, PyGC_Head *finalizers)
for (gc = unreachable->gc.gc_next; gc != unreachable; gc = next) {
PyObject *op = FROM_GC(gc);
- assert(IS_TENTATIVELY_UNREACHABLE(op));
+ PyObject_ASSERT(op, IS_TENTATIVELY_UNREACHABLE(op));
next = gc->gc.gc_next;
if (has_legacy_finalizer(op)) {
@@ -619,7 +624,7 @@ handle_weakrefs(PyGC_Head *unreachable, PyGC_Head *old)
PyWeakReference **wrlist;
op = FROM_GC(gc);
- assert(IS_TENTATIVELY_UNREACHABLE(op));
+ PyObject_ASSERT(op, IS_TENTATIVELY_UNREACHABLE(op));
next = gc->gc.gc_next;
if (! PyType_SUPPORTS_WEAKREFS(Py_TYPE(op)))
@@ -640,9 +645,9 @@ handle_weakrefs(PyGC_Head *unreachable, PyGC_Head *old)
* the callback pointer intact. Obscure: it also
* changes *wrlist.
*/
- assert(wr->wr_object == op);
+ PyObject_ASSERT(wr->wr_object, wr->wr_object == op);
_PyWeakref_ClearRef(wr);
- assert(wr->wr_object == Py_None);
+ PyObject_ASSERT(wr->wr_object, wr->wr_object == Py_None);
if (wr->wr_callback == NULL)
continue; /* no callback */
@@ -676,7 +681,7 @@ handle_weakrefs(PyGC_Head *unreachable, PyGC_Head *old)
*/
if (IS_TENTATIVELY_UNREACHABLE(wr))
continue;
- assert(IS_REACHABLE(wr));
+ PyObject_ASSERT(op, IS_REACHABLE(wr));
/* Create a new reference so that wr can't go away
* before we can process it again.
@@ -685,7 +690,8 @@ handle_weakrefs(PyGC_Head *unreachable, PyGC_Head *old)
/* Move wr to wrcb_to_call, for the next pass. */
wrasgc = AS_GC(wr);
- assert(wrasgc != next); /* wrasgc is reachable, but
+ PyObject_ASSERT(op, wrasgc != next);
+ /* wrasgc is reachable, but
next isn't, so they can't
be the same */
gc_list_move(wrasgc, &wrcb_to_call);
@@ -701,11 +707,11 @@ handle_weakrefs(PyGC_Head *unreachable, PyGC_Head *old)
gc = wrcb_to_call.gc.gc_next;
op = FROM_GC(gc);
- assert(IS_REACHABLE(op));
- assert(PyWeakref_Check(op));
+ PyObject_ASSERT(op, IS_REACHABLE(op));
+ PyObject_ASSERT(op, PyWeakref_Check(op));
wr = (PyWeakReference *)op;
callback = wr->wr_callback;
- assert(callback != NULL);
+ PyObject_ASSERT(op, callback != NULL);
/* copy-paste of weakrefobject.c's handle_callback() */
temp = PyObject_CallFunctionObjArgs(callback, wr, NULL);
@@ -822,12 +828,14 @@ check_garbage(PyGC_Head *collectable)
for (gc = collectable->gc.gc_next; gc != collectable;
gc = gc->gc.gc_next) {
_PyGCHead_SET_REFS(gc, Py_REFCNT(FROM_GC(gc)));
- assert(_PyGCHead_REFS(gc) != 0);
+ PyObject_ASSERT(FROM_GC(gc),
+ _PyGCHead_REFS(gc) != 0);
}
subtract_refs(collectable);
for (gc = collectable->gc.gc_next; gc != collectable;
gc = gc->gc.gc_next) {
- assert(_PyGCHead_REFS(gc) >= 0);
+ PyObject_ASSERT(FROM_GC(gc),
+ _PyGCHead_REFS(gc) >= 0);
if (_PyGCHead_REFS(gc) != 0)
return -1;
}
diff --git a/Objects/object.c b/Objects/object.c
index 559794f..a47d47f 100644
--- a/Objects/object.c
+++ b/Objects/object.c
@@ -2022,6 +2022,35 @@ _PyTrash_thread_destroy_chain(void)
}
}
+PyAPI_FUNC(void)
+_PyObject_AssertFailed(PyObject *obj, const char *msg, const char *expr,
+ const char *file, int line, const char *function)
+{
+ fprintf(stderr,
+ "%s:%d: %s: Assertion \"%s\" failed.\n",
+ file, line, function, expr);
+ if (msg) {
+ fprintf(stderr, "%s\n", msg);
+ }
+
+ fflush(stderr);
+
+ if (obj) {
+ /* This might succeed or fail, but we're about to abort, so at least
+ try to provide any extra info we can: */
+ _PyObject_Dump(obj);
+ }
+ else {
+ fprintf(stderr, "NULL object\n");
+ }
+
+ fflush(stdout);
+ fflush(stderr);
+
+ /* Terminate the process: */
+ abort();
+}
+
#ifndef Py_TRACE_REFS
/* For Py_LIMITED_API, we need an out-of-line version of _Py_Dealloc.
Define this here, so we can undefine the macro. */

View File

@ -1,30 +0,0 @@
diff -r 39b9b05c3085 Lib/distutils/sysconfig.py
--- a/Lib/distutils/sysconfig.py Wed Apr 10 00:27:23 2013 +0200
+++ b/Lib/distutils/sysconfig.py Wed Apr 10 10:14:18 2013 +0200
@@ -362,7 +362,10 @@
done[n] = item = ""
if found:
after = value[m.end():]
- value = value[:m.start()] + item + after
+ value = value[:m.start()]
+ if item.strip() not in value:
+ value += item
+ value += after
if "$" in after:
notdone[name] = value
else:
diff -r 39b9b05c3085 Lib/sysconfig.py
--- a/Lib/sysconfig.py Wed Apr 10 00:27:23 2013 +0200
+++ b/Lib/sysconfig.py Wed Apr 10 10:14:18 2013 +0200
@@ -296,7 +296,10 @@
if found:
after = value[m.end():]
- value = value[:m.start()] + item + after
+ value = value[:m.start()]
+ if item.strip() not in value:
+ value += item
+ value += after
if "$" in after:
notdone[name] = value
else:

View File

@ -1,12 +0,0 @@
diff -r de35eae9048a config.sub
--- a/config.sub Wed Apr 24 23:33:20 2013 +0200
+++ b/config.sub Thu Apr 25 08:51:00 2013 +0200
@@ -1008,7 +1008,7 @@
;;
ppc64) basic_machine=powerpc64-unknown
;;
- ppc64-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'`
+ ppc64-* | ppc64p7-*) basic_machine=powerpc64-`echo $basic_machine | sed 's/^[^-]*-//'`
;;
ppc64le | powerpc64little | ppc64-le | powerpc64-little)
basic_machine=powerpc64le-unknown

View File

@ -1,12 +0,0 @@
--- Python-3.3.2/setup.py.orig 2013-07-01 15:23:24.377711044 +0200
+++ Python-3.3.2/setup.py 2013-07-01 15:23:34.094676496 +0200
@@ -1882,7 +1882,8 @@
if not line:
ffi_inc = None
break
- if line.startswith('#define LIBFFI_H'):
+ if line.startswith('#define LIBFFI_H') or \
+ line.startswith('#define ffi_wrapper_h'):
break
ffi_lib = None
if ffi_inc is not None:

View File

@ -1,14 +0,0 @@
diff -r 7fa3e824a4ee Lib/test/test_py_compile.py
--- a/Lib/test/test_py_compile.py Tue Oct 29 22:25:06 2013 -0400
+++ b/Lib/test/test_py_compile.py Wed Oct 30 11:08:31 2013 +0100
@@ -54,6 +54,10 @@
self.assertTrue(os.path.exists(self.pyc_path))
self.assertFalse(os.path.exists(self.cache_path))
+ def test_bad_coding(self):
+ bad_coding = os.path.join(os.path.dirname(__file__), 'bad_coding2.py')
+ self.assertIsNone(py_compile.compile(bad_coding, doraise=False))
+
def test_relative_path(self):
py_compile.compile(os.path.relpath(self.source_path),
os.path.relpath(self.pyc_path))

View File

@ -1,11 +0,0 @@
diff -r 28c04e954bb6 Lib/lib2to3/main.py
--- a/Lib/lib2to3/main.py Tue Oct 29 22:25:55 2013 -0400
+++ b/Lib/lib2to3/main.py Wed Nov 06 14:33:07 2013 +0100
@@ -213,6 +213,7 @@
# Set up logging handler
level = logging.DEBUG if options.verbose else logging.INFO
+ logging.root.handlers = []
logging.basicConfig(format='%(name)s: %(message)s', level=level)
logger = logging.getLogger('lib2to3.main')

View File

@ -1,232 +0,0 @@
diff -Nur Python-3.4.1/Lib/ensurepip/__init__.py Python-3.4.1-rewheel/Lib/ensurepip/__init__.py
--- Python-3.4.1/Lib/ensurepip/__init__.py 2014-08-21 10:49:30.792695824 +0200
+++ Python-3.4.1-rewheel/Lib/ensurepip/__init__.py 2014-08-21 10:10:41.958341726 +0200
@@ -1,8 +1,10 @@
import os
import os.path
import pkgutil
+import shutil
import sys
import tempfile
+from ensurepip import rewheel
__all__ = ["version", "bootstrap"]
@@ -38,6 +40,8 @@
# Install the bundled software
import pip
+ if args[0] in ["install", "list", "wheel"]:
+ args.append('--pre')
pip.main(args)
@@ -87,20 +91,39 @@
# omit pip and easy_install
os.environ["ENSUREPIP_OPTIONS"] = "install"
+ whls = []
+ rewheel_dir = None
+ # try to see if we have system-wide versions of _PROJECTS
+ dep_records = rewheel.find_system_records([p[0] for p in _PROJECTS])
+ # TODO: check if system-wide versions are the newest ones
+ # if --upgrade is used?
+ if all(dep_records):
+ # if we have all _PROJECTS installed system-wide, we'll recreate
+ # wheels from them and install those
+ rewheel_dir = tempfile.TemporaryDirectory()
+ for dr in dep_records:
+ new_whl = rewheel.rewheel_from_record(dr, rewheel_dir.name)
+ whls.append(os.path.join(rewheel_dir.name, new_whl))
+ else:
+ # if we don't have all the _PROJECTS installed system-wide,
+ # let's just fall back to bundled wheels
+ for project, version in _PROJECTS:
+ whl = os.path.join(
+ os.path.dirname(__file__),
+ "_bundled",
+ "{}-{}-py2.py3-none-any.whl".format(project, version)
+ )
+ whls.append(whl)
+
with tempfile.TemporaryDirectory() as tmpdir:
# Put our bundled wheels into a temporary directory and construct the
# additional paths that need added to sys.path
additional_paths = []
- for project, version in _PROJECTS:
- wheel_name = "{}-{}-py2.py3-none-any.whl".format(project, version)
- whl = pkgutil.get_data(
- "ensurepip",
- "_bundled/{}".format(wheel_name),
- )
- with open(os.path.join(tmpdir, wheel_name), "wb") as fp:
- fp.write(whl)
-
- additional_paths.append(os.path.join(tmpdir, wheel_name))
+ for whl in whls:
+ shutil.copy(whl, tmpdir)
+ additional_paths.append(os.path.join(tmpdir, os.path.basename(whl)))
+ if rewheel_dir:
+ rewheel_dir.cleanup()
# Construct the arguments to be passed to the pip command
args = ["install", "--no-index", "--find-links", tmpdir]
diff -Nur Python-3.4.1/Lib/ensurepip/rewheel/__init__.py Python-3.4.1-rewheel/Lib/ensurepip/rewheel/__init__.py
--- Python-3.4.1/Lib/ensurepip/rewheel/__init__.py 1970-01-01 01:00:00.000000000 +0100
+++ Python-3.4.1-rewheel/Lib/ensurepip/rewheel/__init__.py 2014-08-21 10:11:22.560320121 +0200
@@ -0,0 +1,143 @@
+import argparse
+import codecs
+import csv
+import email.parser
+import os
+import io
+import re
+import site
+import subprocess
+import sys
+import zipfile
+
+def run():
+ parser = argparse.ArgumentParser(description='Recreate wheel of package with given RECORD.')
+ parser.add_argument('record_path',
+ help='Path to RECORD file')
+ parser.add_argument('-o', '--output-dir',
+ help='Dir where to place the wheel, defaults to current working dir.',
+ dest='outdir',
+ default=os.path.curdir)
+
+ ns = parser.parse_args()
+ retcode = 0
+ try:
+ print(rewheel_from_record(**vars(ns)))
+ except BaseException as e:
+ print('Failed: {}'.format(e))
+ retcode = 1
+ sys.exit(1)
+
+def find_system_records(projects):
+ """Return list of paths to RECORD files for system-installed projects.
+
+ If a project is not installed, the resulting list contains None instead
+ of a path to its RECORD
+ """
+ records = []
+ # get system site-packages dirs
+ sys_sitepack = site.getsitepackages([sys.base_prefix, sys.base_exec_prefix])
+ sys_sitepack = [sp for sp in sys_sitepack if os.path.exists(sp)]
+ # try to find all projects in all system site-packages
+ for project in projects:
+ path = None
+ for sp in sys_sitepack:
+ dist_info_re = os.path.join(sp, project) + r'-[^\{0}]+\.dist-info'.format(os.sep)
+ candidates = [os.path.join(sp, p) for p in os.listdir(sp)]
+ # filter out candidate dirs based on the above regexp
+ filtered = [c for c in candidates if re.match(dist_info_re, c)]
+ # if we have 0 or 2 or more dirs, something is wrong...
+ if len(filtered) == 1:
+ path = filtered[0]
+ if path is not None:
+ records.append(os.path.join(path, 'RECORD'))
+ else:
+ records.append(None)
+ return records
+
+def rewheel_from_record(record_path, outdir):
+ """Recreates a whee of package with given record_path and returns path
+ to the newly created wheel."""
+ site_dir = os.path.dirname(os.path.dirname(record_path))
+ record_relpath = record_path[len(site_dir):].strip(os.path.sep)
+ to_write, to_omit = get_records_to_pack(site_dir, record_relpath)
+ new_wheel_name = get_wheel_name(record_path)
+ new_wheel_path = os.path.join(outdir, new_wheel_name + '.whl')
+
+ new_wheel = zipfile.ZipFile(new_wheel_path, mode='w', compression=zipfile.ZIP_DEFLATED)
+ # we need to write a new record with just the files that we will write,
+ # e.g. not binaries and *.pyc/*.pyo files
+ new_record = io.StringIO()
+ writer = csv.writer(new_record)
+
+ # handle files that we can write straight away
+ for f, sha_hash, size in to_write:
+ new_wheel.write(os.path.join(site_dir, f), arcname=f)
+ writer.writerow([f, sha_hash,size])
+
+ # rewrite the old wheel file with a new computed one
+ writer.writerow([record_relpath, '', ''])
+ new_wheel.writestr(record_relpath, new_record.getvalue())
+
+ new_wheel.close()
+
+ return new_wheel.filename
+
+def get_wheel_name(record_path):
+ """Return proper name of the wheel, without .whl."""
+
+ wheel_info_path = os.path.join(os.path.dirname(record_path), 'WHEEL')
+ with codecs.open(wheel_info_path, encoding='utf-8') as wheel_info_file:
+ wheel_info = email.parser.Parser().parsestr(wheel_info_file.read())
+
+ metadata_path = os.path.join(os.path.dirname(record_path), 'METADATA')
+ with codecs.open(metadata_path, encoding='utf-8') as metadata_file:
+ metadata = email.parser.Parser().parsestr(metadata_file.read())
+
+ # construct name parts according to wheel spec
+ distribution = metadata.get('Name')
+ version = metadata.get('Version')
+ build_tag = '' # nothing for now
+ lang_tag = []
+ for t in wheel_info.get_all('Tag'):
+ lang_tag.append(t.split('-')[0])
+ lang_tag = '.'.join(lang_tag)
+ abi_tag, plat_tag = wheel_info.get('Tag').split('-')[1:3]
+ # leave out build tag, if it is empty
+ to_join = filter(None, [distribution, version, build_tag, lang_tag, abi_tag, plat_tag])
+ return '-'.join(list(to_join))
+
+def get_records_to_pack(site_dir, record_relpath):
+ """Accepts path of sitedir and path of RECORD file relative to it.
+ Returns two lists:
+ - list of files that can be written to new RECORD straight away
+ - list of files that shouldn't be written or need some processing
+ (pyc and pyo files, scripts)
+ """
+ record_file_path = os.path.join(site_dir, record_relpath)
+ with codecs.open(record_file_path, encoding='utf-8') as record_file:
+ record_contents = record_file.read()
+ # temporary fix for https://github.com/pypa/pip/issues/1376
+ # we need to ignore files under ".data" directory
+ data_dir = os.path.dirname(record_relpath).strip(os.path.sep)
+ data_dir = data_dir[:-len('dist-info')] + 'data'
+
+ to_write = []
+ to_omit = []
+ for l in record_contents.splitlines():
+ spl = l.split(',')
+ if len(spl) == 3:
+ # new record will omit (or write differently):
+ # - abs paths, paths with ".." (entry points),
+ # - pyc+pyo files
+ # - the old RECORD file
+ # TODO: is there any better way to recognize an entry point?
+ if os.path.isabs(spl[0]) or spl[0].startswith('..') or \
+ spl[0].endswith('.pyc') or spl[0].endswith('.pyo') or \
+ spl[0] == record_relpath or spl[0].startswith(data_dir):
+ to_omit.append(spl)
+ else:
+ to_write.append(spl)
+ else:
+ pass # bad RECORD or empty line
+ return to_write, to_omit
diff -Nur Python-3.4.1/Makefile.pre.in Python-3.4.1-rewheel/Makefile.pre.in
--- Python-3.4.1/Makefile.pre.in 2014-08-21 10:49:31.512695040 +0200
+++ Python-3.4.1-rewheel/Makefile.pre.in 2014-08-21 10:10:41.961341722 +0200
@@ -1145,7 +1145,7 @@
test/test_asyncio \
collections concurrent concurrent/futures encodings \
email email/mime test/test_email test/test_email/data \
- ensurepip ensurepip/_bundled \
+ ensurepip ensurepip/_bundled ensurepip/rewheel \
html json test/test_json http dbm xmlrpc \
sqlite3 sqlite3/test \
logging csv wsgiref urllib \

View File

@ -0,0 +1,70 @@
From 72d6cb277804f58b660bf96d8f5efff78d88491c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miro=20Hron=C4=8Dok?= <miro@hroncok.cz>
Date: Wed, 15 Aug 2018 15:36:29 +0200
Subject: [PATCH] 00189: Instead of bundled wheels, use our RPM packaged wheels
We keep them in /usr/share/python-wheels
---
Lib/ensurepip/__init__.py | 32 ++++++++++++++++++++++----------
1 file changed, 22 insertions(+), 10 deletions(-)
diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index fc0edec6e3..731817a3f0 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -1,16 +1,31 @@
+import distutils.version
+import glob
import os
import os.path
-import pkgutil
import sys
import tempfile
__all__ = ["version", "bootstrap"]
+_WHEEL_DIR = "/usr/share/python-wheels/"
-_SETUPTOOLS_VERSION = "41.2.0"
+_wheels = {}
-_PIP_VERSION = "19.2.3"
+def _get_most_recent_wheel_version(pkg):
+ prefix = os.path.join(_WHEEL_DIR, "{}-".format(pkg))
+ _wheels[pkg] = {}
+ for suffix in "-py2.py3-none-any.whl", "-py3-none-any.whl":
+ pattern = "{}*{}".format(prefix, suffix)
+ for path in glob.glob(pattern):
+ version_str = path[len(prefix):-len(suffix)]
+ _wheels[pkg][version_str] = os.path.basename(path)
+ return str(max(_wheels[pkg], key=distutils.version.LooseVersion))
+
+
+_SETUPTOOLS_VERSION = _get_most_recent_wheel_version("setuptools")
+
+_PIP_VERSION = _get_most_recent_wheel_version("pip")
_PROJECTS = [
("setuptools", _SETUPTOOLS_VERSION),
@@ -95,13 +110,10 @@ def _bootstrap(*, root=None, upgrade=False, user=False,
# additional paths that need added to sys.path
additional_paths = []
for project, version in _PROJECTS:
- wheel_name = "{}-{}-py2.py3-none-any.whl".format(project, version)
- whl = pkgutil.get_data(
- "ensurepip",
- "_bundled/{}".format(wheel_name),
- )
- with open(os.path.join(tmpdir, wheel_name), "wb") as fp:
- fp.write(whl)
+ wheel_name = _wheels[project][version]
+ with open(os.path.join(_WHEEL_DIR, wheel_name), "rb") as sfp:
+ with open(os.path.join(tmpdir, wheel_name), "wb") as fp:
+ fp.write(sfp.read())
additional_paths.append(os.path.join(tmpdir, wheel_name))
--
2.24.1

View File

@ -1,30 +0,0 @@
Subject: python3.test gdb match addr before builtin
From: Michel Normand <normand@linux.vnet.ibm.com>
For ppc64le archi and python3... and gdb... versions
the test_gdb.py need a change of re.match to handle address before the builtin_id word.
Of course there is no error if this substring is not present.
===
...
#0 0x00003fffb7dd0898 in builtin_id (self=<module at remote 0x3fffb7901440>, v=<unknown at remote 0x3fffb7f68910>) at /builddir/build/BUILD/Python-3.3.2/Python/bltinmodule.c:966
....xxxxxxxxxxxxxxxxxxxxxx <= added regexp
===
Signed-off-by: Michel Normand <normand@linux.vnet.ibm.com>
---
Lib/test/test_gdb.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Index: Python-3.3.2/Lib/test/test_gdb.py
===================================================================
--- Python-3.3.2.orig/Lib/test/test_gdb.py
+++ Python-3.3.2/Lib/test/test_gdb.py
@@ -230,7 +230,7 @@ class DebuggerTests(unittest.TestCase):
# gdb can insert additional '\n' and space characters in various places
# in its output, depending on the width of the terminal it's connected
# to (using its "wrap_here" function)
- m = re.match('.*#0\s+builtin_id\s+\(self\=.*,\s+v=\s*(.*?)\)\s+at\s+\S*Python/bltinmodule.c.*',
+ m = re.match('.*#0\s+(?: 0x[0-9a-f]+\s+in\s+)?builtin_id\s+\(self\=.*,\s+v=\s*(.*?)\)\s+at\s+\S*Python/bltinmodule.c.*',
gdb_output, re.DOTALL)
if not m:
self.fail('Unexpected gdb output: %r\n%s' % (gdb_output, gdb_output))

View File

@ -1,12 +0,0 @@
diff -up Python-3.5.0/Makefile.pre.in.lib Python-3.5.0/Makefile.pre.in
--- Python-3.5.0/Makefile.pre.in.lib 2015-09-21 15:39:47.928286620 +0200
+++ Python-3.5.0/Makefile.pre.in 2015-09-21 15:42:58.004042762 +0200
@@ -1340,7 +1340,7 @@ inclinstall:
# Install the library and miscellaneous stuff needed for extending/embedding
# This goes into $(exec_prefix)
-LIBPL= @LIBPL@
+LIBPL= $(LIBDEST)/config-$(LDVERSION)-$(MULTIARCH)
# pkgconfig directory
LIBPC= $(LIBDIR)/pkgconfig

View File

@ -1,15 +0,0 @@
diff -up Python-3.5.0/configure.ac.eabi Python-3.5.0/configure.ac
--- Python-3.5.0/configure.eabi 2015-09-23 13:52:20.756909744 +0200
+++ Python-3.5.0/configure 2015-09-23 13:52:46.859163629 +0200
@@ -762,9 +762,9 @@ cat >> conftest.c <<EOF
alpha-linux-gnu
# elif defined(__ARM_EABI__) && defined(__ARM_PCS_VFP)
# if defined(__ARMEL__)
- arm-linux-gnueabihf
+ arm-linux-gnueabi
# else
- armeb-linux-gnueabihf
+ armeb-linux-gnueabi
# endif
# elif defined(__ARM_EABI__) && !defined(__ARM_PCS_VFP)
# if defined(__ARMEL__)

View File

@ -1,42 +0,0 @@
diff -urp Python-3.5.0/configure p/configure
--- Python-3.5.0/configure 2016-02-25 16:12:12.615184011 +0000
+++ p/configure 2016-02-25 16:13:01.293412517 +0000
@@ -5133,7 +5133,7 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64el-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64el-linux-gnuabi64
+ mips64el-linux-gnu
# else
# error unknown platform triplet
# endif
@@ -5143,7 +5143,7 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64-linux-gnuabi64
+ mips64-linux-gnu
# else
# error unknown platform triplet
# endif
diff -urp Python-3.5.0/configure.ac p/configure.ac
--- Python-3.5.0/configure.ac 2016-02-25 16:12:11.663159985 +0000
+++ p/configure.ac 2016-02-25 16:13:18.814854710 +0000
@@ -784,7 +784,7 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64el-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64el-linux-gnuabi64
+ mips64el-linux-gnu
# else
# error unknown platform triplet
# endif
@@ -794,7 +794,7 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64-linux-gnuabi64
+ mips64-linux-gnu
# else
# error unknown platform triplet
# endif

View File

@ -1,12 +0,0 @@
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 28df0e1..42f811c 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -867,6 +867,7 @@ Python/frozen.o: Python/importlib.h Python/importlib_external.h
# follow our naming conventions. dtrace(1) uses the output filename to generate
# an include guard, so we can't use a pipeline to transform its output.
Include/pydtrace_probes.h: $(srcdir)/Include/pydtrace.d
+ @$(MKDIR_P) Include
$(DTRACE) $(DFLAGS) -o $@ -h -s $<
: sed in-place edit with POSIX-only tools
sed 's/PYTHON_/PyDTrace_/' $@ > $@.tmp

View File

@ -0,0 +1,64 @@
From a1f0ea8fae6fb87cdc9d9c16bc0898e8f66fa907 Mon Sep 17 00:00:00 2001
From: Michal Cyprian <m.cyprian@gmail.com>
Date: Mon, 26 Jun 2017 16:32:56 +0200
Subject: [PATCH] 00251: Change user install location
Set values of prefix and exec_prefix in distutils install command
to /usr/local if executable is /usr/bin/python* and RPM build
is not detected to make pip and distutils install into separate location.
Fedora Change: https://fedoraproject.org/wiki/Changes/Making_sudo_pip_safe
---
Lib/distutils/command/install.py | 15 +++++++++++++--
Lib/site.py | 9 ++++++++-
2 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py
index ae4f915669..0e4fd5b74a 100644
--- a/Lib/distutils/command/install.py
+++ b/Lib/distutils/command/install.py
@@ -418,8 +418,19 @@ class install(Command):
raise DistutilsOptionError(
"must not supply exec-prefix without prefix")
- self.prefix = os.path.normpath(sys.prefix)
- self.exec_prefix = os.path.normpath(sys.exec_prefix)
+ # self.prefix is set to sys.prefix + /local/
+ # if neither RPM build nor virtual environment is
+ # detected to make pip and distutils install packages
+ # into the separate location.
+ if (not (hasattr(sys, 'real_prefix') or
+ sys.prefix != sys.base_prefix) and
+ 'RPM_BUILD_ROOT' not in os.environ):
+ addition = "/local"
+ else:
+ addition = ""
+
+ self.prefix = os.path.normpath(sys.prefix) + addition
+ self.exec_prefix = os.path.normpath(sys.exec_prefix) + addition
else:
if self.exec_prefix is None:
diff --git a/Lib/site.py b/Lib/site.py
index 22d53fa562..9513526109 100644
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -348,7 +348,14 @@ def getsitepackages(prefixes=None):
return sitepackages
def addsitepackages(known_paths, prefixes=None):
- """Add site-packages to sys.path"""
+ """Add site-packages to sys.path
+
+ '/usr/local' is included in PREFIXES if RPM build is not detected
+ to make packages installed into this location visible.
+
+ """
+ if ENABLE_USER_SITE and 'RPM_BUILD_ROOT' not in os.environ:
+ PREFIXES.insert(0, "/usr/local")
for sitedir in getsitepackages(prefixes):
if os.path.isdir(sitedir):
addsitedir(sitedir, known_paths)
--
2.24.1

View File

@ -1,45 +0,0 @@
diff --git a/Lib/distutils/cmd.py b/Lib/distutils/cmd.py
index c89d5ef..dd61621 100644
--- a/Lib/distutils/cmd.py
+++ b/Lib/distutils/cmd.py
@@ -296,7 +296,8 @@ class Command:
finalized command object.
"""
cmd_obj = self.distribution.get_command_obj(command, create)
- cmd_obj.ensure_finalized()
+ if cmd_obj is not None:
+ cmd_obj.ensure_finalized()
return cmd_obj
# XXX rename to 'get_reinitialized_command()'? (should do the
diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py
index 8174192..30ca739 100644
--- a/Lib/distutils/command/install.py
+++ b/Lib/distutils/command/install.py
@@ -122,6 +122,8 @@ class install(Command):
"force installation (overwrite any existing files)"),
('skip-build', None,
"skip rebuilding everything (for testing/debugging)"),
+ ('executable=', 'e',
+ "specify final destination interpreter path (install.py)"),
# Where to install documentation (eventually!)
#('doc-format=', None, "format of documentation to generate"),
@@ -194,6 +196,7 @@ class install(Command):
# directory not in sys.path.
self.force = 0
self.skip_build = 0
+ self.executable = None
self.warn_dir = 1
# These are only here as a conduit from the 'build' command to the
@@ -365,6 +368,9 @@ class install(Command):
('build_base', 'build_base'),
('build_lib', 'build_lib'))
+ if self.executable is None:
+ self.executable = os.path.normpath(sys.executable)
+
# Punt on doc directories for now -- after all, we're punting on
# documentation completely!

View File

@ -1,12 +0,0 @@
diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
index 8992a01..74170f4 100644
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -5458,6 +5458,7 @@ class LinuxKernelCryptoAPI(unittest.TestCase):
self.assertEqual(dec, msg * multiplier)
@support.requires_linux_version(4, 3) # see test_aes_cbc
+ @unittest.skip('Failure on Kernel 4.9+')
def test_aead_aes_gcm(self):
key = bytes.fromhex('c939cc13397c1d37de6ae0e1cb7c423c')
iv = bytes.fromhex('b3d8cc017cbb89b39e0f67e2')

View File

@ -1,183 +0,0 @@
diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index 9f5d151..4dfa8a1 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -8,13 +8,25 @@ import tempfile
__all__ = ["version", "bootstrap"]
-_SETUPTOOLS_VERSION = "28.8.0"
+_SETUPTOOLS_VERSION = "34.2.0"
_PIP_VERSION = "9.0.1"
+_SIX_VERSION = "1.10.0"
+
+_APPDIRS_VERSION = "1.4.0"
+
+_PACKAGING_VERSION = "16.8"
+
+_PYPARSING_VERSION = "2.1.10"
+
_PROJECTS = [
- ("setuptools", _SETUPTOOLS_VERSION),
- ("pip", _PIP_VERSION),
+ ("setuptools", _SETUPTOOLS_VERSION),
+ ("pip", _PIP_VERSION),
+ ("six", _SIX_VERSION),
+ ("appdirs", _APPDIRS_VERSION),
+ ("packaging", _PACKAGING_VERSION),
+ ("pyparsing", _PYPARSING_VERSION)
]
diff --git a/Lib/test/test_ensurepip.py b/Lib/test/test_ensurepip.py
index 9b04c18..23664c4 100644
--- a/Lib/test/test_ensurepip.py
+++ b/Lib/test/test_ensurepip.py
@@ -40,13 +40,14 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "setuptools", "pip",
+ unittest.mock.ANY,
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
additional_paths = self.run_pip.call_args[0][1]
- self.assertEqual(len(additional_paths), 2)
+ self.assertEqual(len(additional_paths), 6)
def test_bootstrapping_with_root(self):
ensurepip.bootstrap(root="/foo/bar/")
@@ -55,7 +56,7 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
[
"install", "--no-index", "--find-links",
unittest.mock.ANY, "--root", "/foo/bar/",
- "setuptools", "pip",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -66,7 +67,8 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "--user", "setuptools", "pip",
+ unittest.mock.ANY, "--user",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -77,7 +79,8 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "--upgrade", "setuptools", "pip",
+ unittest.mock.ANY, "--upgrade",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -88,7 +91,8 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "-v", "setuptools", "pip",
+ unittest.mock.ANY, "-v",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -99,7 +103,8 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "-vv", "setuptools", "pip",
+ unittest.mock.ANY, "-vv",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -110,7 +115,8 @@ class TestBootstrap(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "-vvv", "setuptools", "pip",
+ unittest.mock.ANY, "-vvv",
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
@@ -186,8 +192,8 @@ class TestUninstall(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
- "uninstall", "-y", "--disable-pip-version-check", "pip",
- "setuptools",
+ "uninstall", "-y", "--disable-pip-version-check",
+ "pyparsing", "packaging", "appdirs", "six", "pip", "setuptools",
]
)
@@ -197,8 +203,8 @@ class TestUninstall(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
- "uninstall", "-y", "--disable-pip-version-check", "-v", "pip",
- "setuptools",
+ "uninstall", "-y", "--disable-pip-version-check", "-v",
+ "pyparsing", "packaging", "appdirs", "six", "pip", "setuptools",
]
)
@@ -208,8 +214,8 @@ class TestUninstall(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
- "uninstall", "-y", "--disable-pip-version-check", "-vv", "pip",
- "setuptools",
+ "uninstall", "-y", "--disable-pip-version-check", "-vv",
+ "pyparsing", "packaging", "appdirs", "six", "pip", "setuptools",
]
)
@@ -220,7 +226,7 @@ class TestUninstall(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"uninstall", "-y", "--disable-pip-version-check", "-vvv",
- "pip", "setuptools",
+ "pyparsing", "packaging", "appdirs", "six", "pip", "setuptools",
]
)
@@ -260,13 +266,14 @@ class TestBootstrappingMainFunction(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
"install", "--no-index", "--find-links",
- unittest.mock.ANY, "setuptools", "pip",
+ unittest.mock.ANY,
+ "setuptools", "pip", "six", "appdirs", "packaging", "pyparsing",
],
unittest.mock.ANY,
)
additional_paths = self.run_pip.call_args[0][1]
- self.assertEqual(len(additional_paths), 2)
+ self.assertEqual(len(additional_paths), 6)
class TestUninstallationMainFunction(EnsurepipMixin, unittest.TestCase):
@@ -284,8 +291,8 @@ class TestUninstallationMainFunction(EnsurepipMixin, unittest.TestCase):
self.run_pip.assert_called_once_with(
[
- "uninstall", "-y", "--disable-pip-version-check", "pip",
- "setuptools",
+ "uninstall", "-y", "--disable-pip-version-check", "pyparsing", "packaging",
+ "appdirs", "six", "pip", "setuptools",
]
)

View File

@ -1,104 +0,0 @@
diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py
index 6c3625d..2a53f3d 100644
--- a/Lib/test/test_capi.py
+++ b/Lib/test/test_capi.py
@@ -385,7 +385,7 @@ class EmbeddingTests(unittest.TestCase):
def test_subinterps(self):
# This is just a "don't crash" test
- out, err = self.run_embedded_interpreter()
+ out, err = self.run_embedded_interpreter("repeated_init_and_subinterpreters")
if support.verbose:
print()
print(out)
diff --git a/Programs/_testembed.c b/Programs/_testembed.c
index 3968399..a68d4fa 100644
--- a/Programs/_testembed.c
+++ b/Programs/_testembed.c
@@ -33,7 +33,7 @@ static void print_subinterp(void)
);
}
-static void test_repeated_init_and_subinterpreters(void)
+static int test_repeated_init_and_subinterpreters(void)
{
PyThreadState *mainstate, *substate;
#ifdef WITH_THREAD
@@ -70,6 +70,7 @@ static void test_repeated_init_and_subinterpreters(void)
PyEval_RestoreThread(mainstate);
Py_Finalize();
}
+ return 0;
}
/*****************************************************
@@ -103,7 +104,7 @@ static void check_stdio_details(const char *encoding, const char * errors)
Py_Finalize();
}
-static void test_forced_io_encoding(void)
+static int test_forced_io_encoding(void)
{
/* Check various combinations */
printf("--- Use defaults ---\n");
@@ -122,19 +123,51 @@ static void test_forced_io_encoding(void)
printf("Unexpected success calling Py_SetStandardStreamEncoding");
}
Py_Finalize();
+ return 0;
}
-/* Different embedding tests */
-int main(int argc, char *argv[])
+/* *********************************************************
+ * List of test cases and the function that implements it.
+ *
+ * Names are compared case-sensitively with the first
+ * argument. If no match is found, or no first argument was
+ * provided, the names of all test cases are printed and
+ * the exit code will be -1.
+ *
+ * The int returned from test functions is used as the exit
+ * code, and test_capi treats all non-zero exit codes as a
+ * failed test.
+ *********************************************************/
+struct TestCase
{
+ const char *name;
+ int (*func)(void);
+};
+
+static struct TestCase TestCases[] = {
+ { "forced_io_encoding", test_forced_io_encoding },
+ { "repeated_init_and_subinterpreters", test_repeated_init_and_subinterpreters },
+ { NULL, NULL }
+};
- /* TODO: Check the argument string to allow for more test cases */
+int main(int argc, char *argv[])
+{
if (argc > 1) {
- /* For now: assume "forced_io_encoding */
- test_forced_io_encoding();
- } else {
- /* Run the original embedding test case by default */
- test_repeated_init_and_subinterpreters();
+ for (struct TestCase *tc = TestCases; tc && tc->name; tc++) {
+ if (strcmp(argv[1], tc->name) == 0)
+ return (*tc->func)();
+ }
}
- return 0;
+
+ /* No match found, or no test name provided, so display usage */
+ printf("Python " PY_VERSION " _testembed executable for embedded interpreter tests\n"
+ "Normally executed via 'EmbeddingTests' in Lib/test/test_capi.py\n\n"
+ "Usage: %s TESTNAME\n\nAll available tests:\n", argv[0]);
+ for (struct TestCase *tc = TestCases; tc && tc->name; tc++) {
+ printf(" %s\n", tc->name);
+ }
+
+ /* Non-zero exit code will cause test_capi.py tests to fail.
+ This is intentional. */
+ return -1;
}

View File

@ -1,575 +0,0 @@
diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
index c0e64d6..0bb28da 100644
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -711,6 +711,35 @@ conflict.
.. versionadded:: 3.6
+
+.. envvar:: PYTHONCOERCECLOCALE
+
+ If set to a non-empty string, causes the main Python command line application
+ to skip coercing the legacy ASCII-based C locale to a more capable UTF-8
+ based alternative. Note that this setting is checked even when the
+ :option:`-E` or :option:`-I` options are used, as it is handled prior to
+ the processing of command line options.
+
+ If this variable is *not* set, and the current locale reported for the
+ ``LC_CTYPE`` category is the default ``C`` locale, then the Python CLI will
+ attempt to configure one of the following locales for the given locale
+ categories before loading the interpreter runtime:
+
+ * ``C.UTF-8` (``LC_ALL``)
+ * ``C.utf8` (``LC_ALL``)
+ * ``UTF-8` (``LC_CTYPE``)
+
+ If setting one of these locale categories succeeds, then the matching
+ environment variables will be set (both ``LC_ALL` and ``LANG`` for the
+ ``LC_ALL`` category, and ``LC_CTYPE`` for the ``LC_CTYPE`` category),
+ and (if not already set to a non-empty string) :envvar:`PYTHONIOENCODING`
+ will be set to ``utf-8:surrogateescape``.
+
+ Availability: \*nix
+
+ .. versionadded:: 3.7
+ See :pep:`538` for more details.
+
Debug-mode variables
~~~~~~~~~~~~~~~~~~~~
diff --git a/Lib/test/support/script_helper.py b/Lib/test/support/script_helper.py
index 80889b1..1a1a862 100644
--- a/Lib/test/support/script_helper.py
+++ b/Lib/test/support/script_helper.py
@@ -51,8 +51,35 @@ def interpreter_requires_environment():
return __cached_interp_requires_environment
-_PythonRunResult = collections.namedtuple("_PythonRunResult",
- ("rc", "out", "err"))
+class _PythonRunResult(collections.namedtuple("_PythonRunResult",
+ ("rc", "out", "err"))):
+ """Helper for reporting Python subprocess run results"""
+ def fail(self, cmd_line):
+ """Provide helpful details about failed subcommand runs"""
+ # Limit to 80 lines to ASCII characters
+ maxlen = 80 * 100
+ out, err = self.out, self.err
+ if len(out) > maxlen:
+ out = b'(... truncated stdout ...)' + out[-maxlen:]
+ if len(err) > maxlen:
+ err = b'(... truncated stderr ...)' + err[-maxlen:]
+ out = out.decode('ascii', 'replace').rstrip()
+ err = err.decode('ascii', 'replace').rstrip()
+ raise AssertionError("Process return code is %d\n"
+ "command line: %r\n"
+ "\n"
+ "stdout:\n"
+ "---\n"
+ "%s\n"
+ "---\n"
+ "\n"
+ "stderr:\n"
+ "---\n"
+ "%s\n"
+ "---"
+ % (self.rc, cmd_line,
+ out,
+ err))
# Executing the interpreter in a subprocess
@@ -99,30 +126,7 @@ def run_python_until_end(*args, **env_vars):
def _assert_python(expected_success, *args, **env_vars):
res, cmd_line = run_python_until_end(*args, **env_vars)
if (res.rc and expected_success) or (not res.rc and not expected_success):
- # Limit to 80 lines to ASCII characters
- maxlen = 80 * 100
- out, err = res.out, res.err
- if len(out) > maxlen:
- out = b'(... truncated stdout ...)' + out[-maxlen:]
- if len(err) > maxlen:
- err = b'(... truncated stderr ...)' + err[-maxlen:]
- out = out.decode('ascii', 'replace').rstrip()
- err = err.decode('ascii', 'replace').rstrip()
- raise AssertionError("Process return code is %d\n"
- "command line: %r\n"
- "\n"
- "stdout:\n"
- "---\n"
- "%s\n"
- "---\n"
- "\n"
- "stderr:\n"
- "---\n"
- "%s\n"
- "---"
- % (res.rc, cmd_line,
- out,
- err))
+ res.fail(cmd_line)
return res
def assert_python_ok(*args, **env_vars):
diff --git a/Lib/test/test_capi.py b/Lib/test/test_capi.py
index 2a53f3d..ece84af 100644
--- a/Lib/test/test_capi.py
+++ b/Lib/test/test_capi.py
@@ -386,7 +386,7 @@ class EmbeddingTests(unittest.TestCase):
def test_subinterps(self):
# This is just a "don't crash" test
out, err = self.run_embedded_interpreter("repeated_init_and_subinterpreters")
- if support.verbose:
+ if support.verbose > 1:
print()
print(out)
print(err)
@@ -404,14 +404,15 @@ class EmbeddingTests(unittest.TestCase):
def test_forced_io_encoding(self):
# Checks forced configuration of embedded interpreter IO streams
out, err = self.run_embedded_interpreter("forced_io_encoding")
- if support.verbose:
+ if support.verbose > 1:
print()
print(out)
print(err)
- expected_errors = sys.__stdout__.errors
- expected_stdin_encoding = sys.__stdin__.encoding
+ expected_errors = "surrogateescape"
+ expected_stdin_encoding = "UTF-8"
expected_pipe_encoding = self._get_default_pipe_encoding()
expected_output = '\n'.join([
+ "Setting PYTHONIOENCODING=UTF-8:surrogateescape",
"--- Use defaults ---",
"Expected encoding: default",
"Expected errors: default",
diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py
index b71bb9f..56867fc 100644
--- a/Lib/test/test_cmd_line.py
+++ b/Lib/test/test_cmd_line.py
@@ -9,8 +9,9 @@ import sys
import subprocess
import tempfile
from test.support import script_helper, is_android
-from test.support.script_helper import (spawn_python, kill_python, assert_python_ok,
- assert_python_failure)
+from test.support.script_helper import (
+ spawn_python, kill_python, assert_python_ok, assert_python_failure
+)
# XXX (ncoghlan): Move to script_helper and make consistent with run_python
@@ -151,6 +152,7 @@ class CmdLineTest(unittest.TestCase):
env = os.environ.copy()
# Use C locale to get ascii for the locale encoding
env['LC_ALL'] = 'C'
+ env['PYTHONCOERCECLOCALE'] = '0'
code = (
b'import locale; '
b'print(ascii("' + undecodable + b'"), '
diff --git a/Lib/test/test_sys.py b/Lib/test/test_sys.py
index df9ebd4..63145e4 100644
--- a/Lib/test/test_sys.py
+++ b/Lib/test/test_sys.py
@@ -680,6 +680,7 @@ class SysModuleTest(unittest.TestCase):
# Force the POSIX locale
env = os.environ.copy()
env["LC_ALL"] = "C"
+ env["PYTHONCOERCECLOCALE"] = "0"
code = '\n'.join((
'import sys',
'def dump(name):',
diff --git a/Programs/_testembed.c b/Programs/_testembed.c
index a68d4fa..1494452 100644
--- a/Programs/_testembed.c
+++ b/Programs/_testembed.c
@@ -1,4 +1,5 @@
-#include <Python.h>
+#include "Python.h"
+#include "pyconfig.h"
#include <stdio.h>
/*********************************************************
@@ -106,6 +107,9 @@ static void check_stdio_details(const char *encoding, const char * errors)
static int test_forced_io_encoding(void)
{
+ /* Ensure consistent "defaults" */
+ printf("Setting PYTHONIOENCODING=UTF-8:surrogateescape\n");
+ setenv("PYTHONIOENCODING", "UTF-8:surrogateescape", 1);
/* Check various combinations */
printf("--- Use defaults ---\n");
check_stdio_details(NULL, NULL);
@@ -126,6 +130,20 @@ static int test_forced_io_encoding(void)
return 0;
}
+static int test_c_locale_warning(void)
+{
+#ifdef PY_WARN_ON_C_LOCALE
+ /* Force use of the C locale */
+ setenv("LC_ALL", "C", 1);
+
+ _testembed_Py_Initialize();
+ Py_Finalize();
+#else
+ printf("C locale compatibility warning disabled at compile time\n");
+#endif
+ return 0;
+}
+
/* *********************************************************
* List of test cases and the function that implements it.
*
@@ -147,6 +165,7 @@ struct TestCase
static struct TestCase TestCases[] = {
{ "forced_io_encoding", test_forced_io_encoding },
{ "repeated_init_and_subinterpreters", test_repeated_init_and_subinterpreters },
+ { "c_locale_warning", test_c_locale_warning },
{ NULL, NULL }
};
diff --git a/Programs/python.c b/Programs/python.c
index a7afbc7..b5edebb 100644
--- a/Programs/python.c
+++ b/Programs/python.c
@@ -15,6 +15,110 @@ wmain(int argc, wchar_t **argv)
}
#else
+/* Helpers to better handle the legacy C locale
+ *
+ * The legacy C locale assumes ASCII as the default text encoding, which
+ * causes problems not only for the CPython runtime, but also other
+ * components like GNU readline.
+ *
+ * Accordingly, when the CLI detects it, it attempts to coerce it to a
+ * more capable UTF-8 based alternative.
+ *
+ * See the documentation of the PYTHONCOERCECLOCALE setting for more details.
+ *
+ */
+
+#ifdef PY_COERCE_C_LOCALE
+static const char *_C_LOCALE_COERCION_WARNING =
+ "Python detected LC_CTYPE=C: %.20s coerced to %.20s (set another locale "
+ "or PYTHONCOERCECLOCALE=0 to disable this locale coercion behaviour).\n";
+
+typedef struct _CandidateLocale {
+ const char *locale_name;
+ int category;
+} _LocaleCoercionTarget;
+
+static _LocaleCoercionTarget _TARGET_LOCALES[] = {
+ { "C.UTF-8", LC_ALL },
+ { "C.utf8", LC_ALL },
+ { "UTF-8", LC_CTYPE },
+ { NULL, 0 }
+};
+
+void
+_coerce_default_locale_settings(const _LocaleCoercionTarget *target)
+{
+ const char *newloc = target->locale_name;
+ int category = target->category;
+
+ /* Reset locale back to currently configured defaults */
+ setlocale(LC_ALL, "");
+
+ /* Set the relevant locale environment variables */
+ if (category == LC_ALL) {
+ const char *env_vars_updated = "LC_ALL & LANG";
+ if (setenv("LC_ALL", newloc, 1)) {
+ fprintf(stderr,
+ "Error setting LC_ALL, skipping C locale coercion\n");
+ return;
+ }
+ if (setenv("LANG", newloc, 1)) {
+ fprintf(stderr,
+ "Error setting LANG during C locale coercion\n");
+ env_vars_updated = "LC_ALL";
+ }
+ fprintf(stderr, _C_LOCALE_COERCION_WARNING, env_vars_updated, newloc);
+ } else if (category == LC_CTYPE) {
+ if (setenv("LC_CTYPE", newloc, 1)) {
+ fprintf(stderr,
+ "Error setting LC_CTYPE, skipping C locale coercion\n");
+ return;
+ }
+ fprintf(stderr, _C_LOCALE_COERCION_WARNING, "LC_CTYPE", newloc);
+ } else {
+ fprintf(stderr, "Locale coercion must target LC_ALL or LC_CTYPE\n");
+ return;
+ }
+
+ /* Set PYTHONIOENCODING if not already set */
+ if (setenv("PYTHONIOENCODING", "utf-8:surrogateescape", 0)) {
+ fprintf(stderr,
+ "Error setting PYTHONIOENCODING during C locale coercion\n");
+ }
+
+ /* Reconfigure with the overridden environment variables */
+ setlocale(LC_ALL, "");
+}
+
+void
+_handle_legacy_c_locale(void)
+{
+ const char *coerce_c_locale = getenv("PYTHONCOERCECLOCALE");
+ /* We ignore the Python -E and -I flags here, as we need to sort out
+ * the locale settings *before* we try to do anything with the command
+ * line arguments. For cross-platform debugging purposes, we also need
+ * to give end users a way to force even scripts that are otherwise
+ * isolated from their environment to use the legacy ASCII-centric C
+ * locale.
+ */
+ if (coerce_c_locale == NULL || strncmp(coerce_c_locale, "0", 2) != 0) {
+ /* PYTHONCOERCECLOCALE is not set, or is not set to exactly "0" */
+ const _LocaleCoercionTarget *target = NULL;
+ for (target = _TARGET_LOCALES; target->locale_name; target++) {
+ const char *reconfigured_locale = setlocale(target->category,
+ target->locale_name);
+ if (reconfigured_locale != NULL) {
+ /* Successfully configured locale, so make it the default */
+ _coerce_default_locale_settings(target);
+ return;
+ }
+ }
+
+ }
+ /* No C locale warning here, as Py_Initialize will emit one later */
+}
+#endif
+
int
main(int argc, char **argv)
{
@@ -49,7 +153,26 @@ main(int argc, char **argv)
return 1;
}
+#ifdef __ANDROID__
+ /* Passing "" to setlocale() on Android requests the C locale rather
+ * than checking environment variables, so request C.UTF-8 explicitly
+ */
+ setlocale(LC_ALL, "C.UTF-8");
+#else
+ /* Reconfigure the locale to the default for this process */
setlocale(LC_ALL, "");
+#endif
+
+#ifdef PY_COERCE_C_LOCALE
+ /* When the LC_CTYPE category still claims to be using the C locale,
+ assume configuration error and try for a UTF-8 based locale instead */
+ const char *ctype_loc = setlocale(LC_CTYPE, NULL);
+ if (ctype_loc != NULL && strcmp(ctype_loc, "C") == 0) {
+ _handle_legacy_c_locale();
+ }
+#endif
+
+ /* Convert from char to wchar_t based on the locale settings */
for (i = 0; i < argc; i++) {
argv_copy[i] = Py_DecodeLocale(argv[i], NULL);
if (!argv_copy[i]) {
diff --git a/Python/pylifecycle.c b/Python/pylifecycle.c
index a4f7f82..dd58dc9 100644
--- a/Python/pylifecycle.c
+++ b/Python/pylifecycle.c
@@ -301,6 +301,31 @@ import_init(PyInterpreterState *interp, PyObject *sysmod)
}
+#ifdef PY_WARN_ON_C_LOCALE
+static const char *_C_LOCALE_WARNING =
+ "Python runtime initialized with LC_CTYPE=C (a locale with default ASCII "
+ "encoding), which may cause Unicode compatibility problems. Using C.UTF-8, "
+ "C.utf8, or UTF-8 (if available) as alternative Unicode-compatible "
+ "locales is recommended.\n";
+
+static void
+_emit_stderr_warning_for_c_locale(void)
+{
+ const char *coerce_c_locale = getenv("PYTHONCOERCECLOCALE");
+ /* We don't emit a warning if locale coercion has been explicitly disabled.
+ *
+ * For consistency with the corresponding check in Programs/python.c
+ * we ignore the Python -E and -I flags here.
+ */
+ if (coerce_c_locale == NULL || strncmp(coerce_c_locale, "0", 2) != 0) {
+ const char *ctype_loc = setlocale(LC_CTYPE, NULL);
+ if (ctype_loc != NULL && strcmp(ctype_loc, "C") == 0) {
+ fprintf(stderr, "%s", _C_LOCALE_WARNING);
+ }
+ }
+}
+#endif
+
void
_Py_InitializeEx_Private(int install_sigs, int install_importlib)
{
@@ -315,11 +340,19 @@ _Py_InitializeEx_Private(int install_sigs, int install_importlib)
initialized = 1;
_Py_Finalizing = NULL;
-#ifdef HAVE_SETLOCALE
+#ifdef __ANDROID__
+ /* Passing "" to setlocale() on Android requests the C locale rather
+ * than checking environment variables, so request C.UTF-8 explicitly
+ */
+ setlocale(LC_CTYPE, "C.UTF-8");
+#else
/* Set up the LC_CTYPE locale, so we can obtain
the locale's charset without having to switch
locales. */
setlocale(LC_CTYPE, "");
+#ifdef PY_WARN_ON_C_LOCALE
+ _emit_stderr_warning_for_c_locale();
+#endif
#endif
if ((p = Py_GETENV("PYTHONDEBUG")) && *p != '\0')
diff --git a/configure b/configure
index 6bcddb7..13052d6 100755
--- a/configure
+++ b/configure
@@ -834,6 +834,8 @@ with_thread
enable_ipv6
with_doc_strings
with_pymalloc
+with_c_locale_coercion
+with_c_locale_warning
with_valgrind
with_dtrace
with_fpectl
@@ -1527,6 +1529,12 @@ Optional Packages:
deprecated; use --with(out)-threads
--with(out)-doc-strings disable/enable documentation strings
--with(out)-pymalloc disable/enable specialized mallocs
+ --with(out)-c-locale-coercion
+ disable/enable C locale coercion to a UTF-8 based
+ locale
+ --with(out)-c-locale-warning
+ disable/enable locale compatibility warning in the C
+ locale
--with-valgrind Enable Valgrind support
--with(out)-dtrace disable/enable DTrace support
--with-fpectl enable SIGFPE catching
@@ -11016,6 +11024,52 @@ fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_pymalloc" >&5
$as_echo "$with_pymalloc" >&6; }
+# Check for --with-c-locale-coercion
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-c-locale-coercion" >&5
+$as_echo_n "checking for --with-c-locale-coercion... " >&6; }
+
+# Check whether --with-c-locale-coercion was given.
+if test "${with_c_locale_coercion+set}" = set; then :
+ withval=$with_c_locale_coercion;
+fi
+
+
+if test -z "$with_c_locale_coercion"
+then
+ with_c_locale_coercion="yes"
+fi
+if test "$with_c_locale_coercion" != "no"
+then
+
+$as_echo "#define PY_COERCE_C_LOCALE 1" >>confdefs.h
+
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_c_locale_coercion" >&5
+$as_echo "$with_c_locale_coercion" >&6; }
+
+# Check for --with-c-locale-warning
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-c-locale-warning" >&5
+$as_echo_n "checking for --with-c-locale-warning... " >&6; }
+
+# Check whether --with-c-locale-warning was given.
+if test "${with_c_locale_warning+set}" = set; then :
+ withval=$with_c_locale_warning;
+fi
+
+
+if test -z "$with_c_locale_warning"
+then
+ with_c_locale_warning="yes"
+fi
+if test "$with_c_locale_warning" != "no"
+then
+
+$as_echo "#define PY_WARN_ON_C_LOCALE 1" >>confdefs.h
+
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_c_locale_warning" >&5
+$as_echo "$with_c_locale_warning" >&6; }
+
# Check for Valgrind support
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for --with-valgrind" >&5
$as_echo_n "checking for --with-valgrind... " >&6; }
diff --git a/configure.ac b/configure.ac
index e222c21..a1653e7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3287,6 +3287,40 @@ then
fi
AC_MSG_RESULT($with_pymalloc)
+# Check for --with-c-locale-coercion
+AC_MSG_CHECKING(for --with-c-locale-coercion)
+AC_ARG_WITH(c-locale-coercion,
+ AS_HELP_STRING([--with(out)-c-locale-coercion],
+ [disable/enable C locale coercion to a UTF-8 based locale]))
+
+if test -z "$with_c_locale_coercion"
+then
+ with_c_locale_coercion="yes"
+fi
+if test "$with_c_locale_coercion" != "no"
+then
+ AC_DEFINE(PY_COERCE_C_LOCALE, 1,
+ [Define if you want to coerce the C locale to a UTF-8 based locale])
+fi
+AC_MSG_RESULT($with_c_locale_coercion)
+
+# Check for --with-c-locale-warning
+AC_MSG_CHECKING(for --with-c-locale-warning)
+AC_ARG_WITH(c-locale-warning,
+ AS_HELP_STRING([--with(out)-c-locale-warning],
+ [disable/enable locale compatibility warning in the C locale]))
+
+if test -z "$with_c_locale_warning"
+then
+ with_c_locale_warning="yes"
+fi
+if test "$with_c_locale_warning" != "no"
+then
+ AC_DEFINE(PY_WARN_ON_C_LOCALE, 1,
+ [Define to emit a locale compatibility warning in the C locale])
+fi
+AC_MSG_RESULT($with_c_locale_warning)
+
# Check for Valgrind support
AC_MSG_CHECKING([for --with-valgrind])
AC_ARG_WITH([valgrind],
diff --git a/pyconfig.h.in b/pyconfig.h.in
index e7a836c..11e0798 100644
--- a/pyconfig.h.in
+++ b/pyconfig.h.in
@@ -1241,9 +1241,15 @@
/* Define as the preferred size in bits of long digits */
#undef PYLONG_BITS_IN_DIGIT
+/* Define if you want to coerce the C locale to a UTF-8 based locale */
+#undef PY_COERCE_C_LOCALE
+
/* Define to printf format modifier for Py_ssize_t */
#undef PY_FORMAT_SIZE_T
+/* Define to emit a locale compatibility warning in the C locale */
+#undef PY_WARN_ON_C_LOCALE
+
/* Define if you want to build an interpreter with many run-time checks. */
#undef Py_DEBUG

View File

@ -1,12 +0,0 @@
diff --git a/Lib/ctypes/test/test_structures.py b/Lib/ctypes/test/test_structures.py
index 3eded77..ad7859a 100644
--- a/Lib/ctypes/test/test_structures.py
+++ b/Lib/ctypes/test/test_structures.py
@@ -392,6 +392,7 @@ class StructureTestCase(unittest.TestCase):
(1, 0, 0, 0, 0, 0))
self.assertRaises(TypeError, lambda: Z(1, 2, 3, 4, 5, 6, 7))
+ @unittest.skip('Fails on aarch64: http://bugs.python.org/issue29804')
def test_pass_by_value(self):
# This should mirror the structure in Modules/_ctypes/_ctypes_test.c
class X(Structure):

View File

@ -0,0 +1,86 @@
From b60a8fee7e91e36b48a2ea27d1bb9f42642c3eb2 Mon Sep 17 00:00:00 2001
From: Petr Viktorin <pviktori@redhat.com>
Date: Mon, 28 Aug 2017 17:16:46 +0200
Subject: [PATCH] 00274: Upstream uses Debian-style architecture naming, change
to match Fedora
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Co-authored-by: Petr Viktorin <pviktori@redhat.com>
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
Co-authored-by: Tomas Orsava <torsava@redhat.com>
---
config.sub | 2 +-
configure.ac | 16 ++++++++--------
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/config.sub b/config.sub
index ba37cf99e2..52a9ec6662 100755
--- a/config.sub
+++ b/config.sub
@@ -1042,7 +1042,7 @@ case $basic_machine in
;;
ppc64) basic_machine=powerpc64-unknown
;;
- ppc64-*) basic_machine=powerpc64-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+ ppc64-* | ppc64p7-*) basic_machine=powerpc64-`echo "$basic_machine" | sed 's/^[^-]*-//'`
;;
ppc64le | powerpc64little)
basic_machine=powerpc64le-unknown
diff --git a/configure.ac b/configure.ac
index c59cbc223f..a3e1c04e1b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -747,9 +747,9 @@ cat >> conftest.c <<EOF
alpha-linux-gnu
# elif defined(__ARM_EABI__) && defined(__ARM_PCS_VFP)
# if defined(__ARMEL__)
- arm-linux-gnueabihf
+ arm-linux-gnueabi
# else
- armeb-linux-gnueabihf
+ armeb-linux-gnueabi
# endif
# elif defined(__ARM_EABI__) && !defined(__ARM_PCS_VFP)
# if defined(__ARMEL__)
@@ -789,7 +789,7 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64el-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64el-linux-gnuabi64
+ mips64el-linux-gnu
# else
# error unknown platform triplet
# endif
@@ -799,22 +799,22 @@ cat >> conftest.c <<EOF
# elif _MIPS_SIM == _ABIN32
mips64-linux-gnuabin32
# elif _MIPS_SIM == _ABI64
- mips64-linux-gnuabi64
+ mips64-linux-gnu
# else
# error unknown platform triplet
# endif
# elif defined(__or1k__)
or1k-linux-gnu
# elif defined(__powerpc__) && defined(__SPE__)
- powerpc-linux-gnuspe
+ ppc-linux-gnuspe
# elif defined(__powerpc64__)
# if defined(__LITTLE_ENDIAN__)
- powerpc64le-linux-gnu
+ ppc64le-linux-gnu
# else
- powerpc64-linux-gnu
+ ppc64-linux-gnu
# endif
# elif defined(__powerpc__)
- powerpc-linux-gnu
+ ppc-linux-gnu
# elif defined(__s390x__)
s390x-linux-gnu
# elif defined(__s390__)
--
2.24.1

View File

@ -0,0 +1,55 @@
From 0d41a311e805af08637e3f6dc0fb6fae32e508ab Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miro=20Hron=C4=8Dok?= <miro@hroncok.cz>
Date: Thu, 11 Jul 2019 13:44:13 +0200
Subject: [PATCH] 00328: Restore pyc to TIMESTAMP invalidation mode as default
in rpmbuild
Since Fedora 31, the $SOURCE_DATE_EPOCH is set in rpmbuild to the latest
%changelog date. This makes Python default to the CHECKED_HASH pyc
invalidation mode, bringing more reproducible builds traded for an import
performance decrease. To avoid that, we don't default to CHECKED_HASH
when $RPM_BUILD_ROOT is set (i.e. when we are building RPM packages).
See https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/57#comment-27426
---
Lib/py_compile.py | 3 ++-
Lib/test/test_py_compile.py | 2 ++
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/Lib/py_compile.py b/Lib/py_compile.py
index 21736896af..310bed5620 100644
--- a/Lib/py_compile.py
+++ b/Lib/py_compile.py
@@ -70,7 +70,8 @@ class PycInvalidationMode(enum.Enum):
def _get_default_invalidation_mode():
- if os.environ.get('SOURCE_DATE_EPOCH'):
+ if (os.environ.get('SOURCE_DATE_EPOCH') and not
+ os.environ.get('RPM_BUILD_ROOT')):
return PycInvalidationMode.CHECKED_HASH
else:
return PycInvalidationMode.TIMESTAMP
diff --git a/Lib/test/test_py_compile.py b/Lib/test/test_py_compile.py
index d4a68c9320..ed09874023 100644
--- a/Lib/test/test_py_compile.py
+++ b/Lib/test/test_py_compile.py
@@ -17,6 +17,7 @@ def without_source_date_epoch(fxn):
def wrapper(*args, **kwargs):
with support.EnvironmentVarGuard() as env:
env.unset('SOURCE_DATE_EPOCH')
+ env.unset('RPM_BUILD_ROOT')
return fxn(*args, **kwargs)
return wrapper
@@ -27,6 +28,7 @@ def with_source_date_epoch(fxn):
def wrapper(*args, **kwargs):
with support.EnvironmentVarGuard() as env:
env['SOURCE_DATE_EPOCH'] = '123456789'
+ env.unset('RPM_BUILD_ROOT')
return fxn(*args, **kwargs)
return wrapper
--
2.24.1

View File

@ -1,20 +0,0 @@
diff -up Python-3.1.1/Lib/distutils/unixccompiler.py.rpath Python-3.1.1/Lib/distutils/unixccompiler.py
--- Python-3.1.1/Lib/distutils/unixccompiler.py.rpath 2009-09-04 17:29:34.000000000 -0400
+++ Python-3.1.1/Lib/distutils/unixccompiler.py 2009-09-04 17:49:54.000000000 -0400
@@ -141,6 +141,16 @@ class UnixCCompiler(CCompiler):
if sys.platform == "cygwin":
exe_extension = ".exe"
+ def _fix_lib_args(self, libraries, library_dirs, runtime_library_dirs):
+ """Remove standard library path from rpath"""
+ libraries, library_dirs, runtime_library_dirs = super(
+ self.__class__, self)._fix_lib_args(libraries, library_dirs,
+ runtime_library_dirs)
+ libdir = sysconfig.get_config_var('LIBDIR')
+ if runtime_library_dirs and (libdir in runtime_library_dirs):
+ runtime_library_dirs.remove(libdir)
+ return libraries, library_dirs, runtime_library_dirs
+
def preprocess(self, source, output_file=None, macros=None,
include_dirs=None, extra_preargs=None, extra_postargs=None):
fixed_args = self._fix_compile_args(None, macros, include_dirs)

View File

@ -1,59 +0,0 @@
"""Checks if all *.pyc and *.pyo files have later mtime than their *.py files."""
import imp
import os
import sys
# list of test and other files that we expect not to have bytecode
not_compiled = [
'test/bad_coding.py',
'test/bad_coding2.py',
'test/badsyntax_3131.py',
'test/badsyntax_future3.py',
'test/badsyntax_future4.py',
'test/badsyntax_future5.py',
'test/badsyntax_future6.py',
'test/badsyntax_future7.py',
'test/badsyntax_future8.py',
'test/badsyntax_future9.py',
'test/badsyntax_future10.py',
'test/badsyntax_async1.py',
'test/badsyntax_async2.py',
'test/badsyntax_async3.py',
'test/badsyntax_async4.py',
'test/badsyntax_async5.py',
'test/badsyntax_async6.py',
'test/badsyntax_async7.py',
'test/badsyntax_async8.py',
'test/badsyntax_async9.py',
'test/badsyntax_pep3120.py',
'lib2to3/tests/data/bom.py',
'lib2to3/tests/data/crlf.py',
'lib2to3/tests/data/different_encoding.py',
'lib2to3/tests/data/false_encoding.py',
'lib2to3/tests/data/py2_test_grammar.py',
'.debug-gdb.py',
]
failed = 0
def bytecode_expected(source):
for f in not_compiled:
if source.endswith(f):
return False
return True
compiled = filter(lambda f: bytecode_expected(f), sys.argv[1:])
for f in compiled:
# check both pyo and pyc
to_check = map(lambda b: imp.cache_from_source(f, b), (True, False))
f_mtime = os.path.getmtime(f)
for c in to_check:
c_mtime = os.path.getmtime(c)
if c_mtime < f_mtime:
sys.stderr.write('Failed bytecompilation timestamps check: ')
sys.stderr.write('Bytecode file {} is older than source file {}.\n'.format(c, f))
failed += 1
if failed:
sys.stderr.write('\n{} files failed bytecompilation timestamps check.\n'.format(failed))
sys.exit(1)

55
check-pyc-timestamps.py Normal file
View File

@ -0,0 +1,55 @@
"""Checks if all *.pyc files have later mtime than their *.py files."""
import os
import sys
from importlib.util import cache_from_source
from pathlib import Path
RPM_BUILD_ROOT = os.environ.get('RPM_BUILD_ROOT', '')
# ...cpython-3X.pyc
# ...cpython-3X.opt-1.pyc
# ...cpython-3X.opt-2.pyc
LEVELS = (None, 1, 2)
# list of globs of test and other files that we expect not to have bytecode
not_compiled = [
'/usr/bin/*',
'*/test/bad_coding.py',
'*/test/bad_coding2.py',
'*/test/badsyntax_*.py',
'*/lib2to3/tests/data/bom.py',
'*/lib2to3/tests/data/crlf.py',
'*/lib2to3/tests/data/different_encoding.py',
'*/lib2to3/tests/data/false_encoding.py',
'*/lib2to3/tests/data/py2_test_grammar.py',
'*.debug-gdb.py',
]
def bytecode_expected(path):
path = Path(path[len(RPM_BUILD_ROOT):])
for glob in not_compiled:
if path.match(glob):
return False
return True
failed = 0
compiled = (path for path in sys.argv[1:] if bytecode_expected(path))
for path in compiled:
to_check = (cache_from_source(path, optimization=opt) for opt in LEVELS)
f_mtime = os.path.getmtime(path)
for pyc in to_check:
c_mtime = os.path.getmtime(pyc)
if c_mtime < f_mtime:
print('Failed bytecompilation timestamps check: '
f'Bytecode file {pyc} is older than source file {path}',
file=sys.stderr)
failed += 1
if failed:
print(f'\n{failed} files failed bytecompilation timestamps check.',
file=sys.stderr)
sys.exit(1)

View File

@ -1,17 +0,0 @@
/* Systemtap tapset to make it easier to trace Python */
/*
Define python.function.entry/return:
*/
probe python.function.entry = process("python3").library("LIBRARY_PATH").mark("function__entry")
{
filename = user_string($arg1);
funcname = user_string($arg2);
lineno = $arg3;
}
probe python.function.return = process("python3").library("LIBRARY_PATH").mark("function__return")
{
filename = user_string($arg1);
funcname = user_string($arg2);
lineno = $arg3;
}

View File

@ -1,10 +0,0 @@
# Note that the path could itself be a python file, or a directory
# Python's compile_all module only works on directories, and requires a max
# recursion depth
%py_byte_compile()\
python_binary="%1"\
bytecode_compilation_path="%2"\
find $bytecode_compilation_path -type f -a -name "*.py" -print0 | xargs -0 $python_binary -O -c 'import py_compile, sys; [py_compile.compile(f, dfile=f.partition("$RPM_BUILD_ROOT")[2], optimize=opt) for opt in range(2) for f in sys.argv[1:]]' || :\
%{nil}

View File

@ -1,4 +0,0 @@
%system_python_abi %{expand: \
%global __requires_exclude ^python\\\\(abi\\\\) = 3\\\\..$
Requires: system-python(abi) = %{python3_version}
}

11542
pubkeys.txt Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,21 +0,0 @@
#!/usr/bin/stap
global fn_calls;
probe python.function.entry
{
fn_calls[pid(), filename, funcname, lineno] += 1;
}
probe timer.ms(1000) {
printf("\033[2J\033[1;1H") /* clear screen */
printf("%6s %80s %6s %30s %6s\n",
"PID", "FILENAME", "LINE", "FUNCTION", "CALLS")
foreach ([pid, filename, funcname, lineno] in fn_calls- limit 20) {
printf("%6d %80s %6d %30s %6d\n",
pid, filename, lineno, funcname,
fn_calls[pid, filename, funcname, lineno]);
}
delete fn_calls;
}

File diff suppressed because it is too large Load Diff

View File

@ -1,30 +0,0 @@
diff -up Python-3.5.0/configure.ac.than Python-3.5.0/configure.ac
--- Python-3.5.0/configure.ac.than 2015-11-13 11:51:32.039560172 -0500
+++ Python-3.5.0/configure.ac 2015-11-13 11:52:11.670168157 -0500
@@ -804,9 +804,9 @@ cat >> conftest.c <<EOF
powerpc-linux-gnuspe
# elif defined(__powerpc64__)
# if defined(__LITTLE_ENDIAN__)
- powerpc64le-linux-gnu
+ ppc64le-linux-gnu
# else
- powerpc64-linux-gnu
+ ppc64-linux-gnu
# endif
# elif defined(__powerpc__)
powerpc-linux-gnu
diff -up Python-3.5.0/configure.than Python-3.5.0/configure
--- Python-3.5.0/configure.than 2015-11-13 12:13:19.039658399 -0500
+++ Python-3.5.0/configure 2015-11-13 12:13:35.199906857 -0500
@@ -5153,9 +5153,9 @@ cat >> conftest.c <<EOF
powerpc-linux-gnuspe
# elif defined(__powerpc64__)
# if defined(__LITTLE_ENDIAN__)
- powerpc64le-linux-gnu
+ ppc64le-linux-gnu
# else
- powerpc64-linux-gnu
+ ppc64-linux-gnu
# endif
# elif defined(__powerpc__)
powerpc-linux-gnu

84
python3.rpmlintrc Normal file
View File

@ -0,0 +1,84 @@
# KNOWN BUGS:
# https://bugzilla.redhat.com/show_bug.cgi?id=1489816
addFilter(r'crypto-policy-non-compliance-openssl')
# TESTS:
addFilter(r'(zero-length|pem-certificate|uncompressed-zip) /usr/lib(64)?/python3\.\d+/test')
# OTHER DELIBERATES:
# chroot function
addFilter(r'missing-call-to-chdir-with-chroot')
# intentionally unversioned and selfobsoleted
addFilter(r'unversioned-explicit-obsoletes python')
addFilter(r'self-obsoletion python3\d+ obsoletes python3\d+')
# intentionally hardcoded
addFilter(r'hardcoded-library-path in %{_prefix}/lib/(debug/%{_libdir}|python%{pybasever})')
# intentional for our pythonXY package
addFilter(r'python3\d+\.[^:]+: (E|W): devel-file-in-non-devel-package')
# we have non binary stuff, python files
addFilter(r'only-non-binary-in-usr-lib')
# some devel files that are deliberately needed
addFilter(r'devel-file-in-non-devel-package /usr/include/python3\.\d+m?/pyconfig-(32|64)\.h')
addFilter(r'devel-file-in-non-devel-package /usr/lib(64)?/python3\.\d+/distutils/tests/xxmodule\.c')
# SORRY, NOT SORRY:
# manual pages
addFilter(r'no-manual-page-for-binary (idle|pydoc|pyvenv|2to3|python3?-debug|pathfix|msgfmt|pygettext)')
addFilter(r'no-manual-page-for-binary python3?.*-config$')
addFilter(r'no-manual-page-for-binary python3\.\d+d?m?$')
# missing documentation from subpackages
addFilter(r'^python3\d*-(debug|tkinter|test|idle)\.[^:]+: (E|W): no-documentation')
# platform python is obsoleted, but not provided
addFilter(r'obsolete-not-provided platform-python')
# RPMLINT IMPERFECTIONS
# https://github.com/rpm-software-management/rpmlint/issues/123
addFilter(r'python-bytecode-wrong-magic-value .* expected 33\d\d \(3\.7\), found 3393')
# https://github.com/rpm-software-management/rpmlint/pull/133
addFilter(r'python-bytecode-wrong-magic-value .* expected 33\d\d \(3\.7\), found 3394')
# https://bugzilla.redhat.com/show_bug.cgi?id=1550562
# https://github.com/rpm-software-management/rpmlint/issues/128
addFilter(r'python-bytecode-inconsistent-mtime .* 1970')
# debugsource
addFilter(r'^python3\d*-debugsource\.[^:]+: (E|W): no-documentation')
# debuginfo
addFilter(r'^python3\d*-debuginfo\.[^:]+: (E|W): useless-provides debuginfo\(build-id\)')
# this is OK for F28+
addFilter(r'library-without-ldconfig-post')
# debug package contains devel and non-devel files
addFilter(r'python3\d*-debug\.[^:]+: (E|W): (non-)?devel-file-in-(non-)?devel-package')
# this goes to other subpackage, hence not actually dangling, the read error is bogus
addFilter(r'dangling-relative-symlink /usr/lib(64)?/pkgconfig/python-3\.\d+dm?(-embed)?\.pc python-3\.\d+(-embed)?\.pc')
addFilter(r'read-error /usr/lib(64)?/pkgconfig/python-3\.\d+dm?(-embed)?\.pc \[Errno 2\]')
# the python-unversioned-command package contains dangling symlinks by design
addFilter(r'^python-unversioned-command\.[^:]+: (E|W): dangling-relative-symlink '
r'(/usr/bin/python \./python3|/usr/share/man/man1/python\.1\S* ./python3\.1\S*)$')
# we need this macro to evaluate, even if the line starts with #
addFilter(r'macro-in-comment %\{_pyconfig(32|64)_h\}')
# Python modules don't need to be linked against libc
# Since 3.8 they are no longer linked against libpython3.8.so.1.0
addFilter(r'E: library-not-linked-against-libc /usr/lib(64)?/python3\.\d+/lib-dynload/')
addFilter(r'E: shared-lib-without-dependency-information /usr/lib(64)?/python3\.\d+/lib-dynload/')
# SPELLING ERRORS
addFilter(r'spelling-error .* en_US (bytecode|pyc|filename|tkinter|namespaces|pytest) ')

File diff suppressed because it is too large Load Diff

View File

@ -1 +1,2 @@
SHA512 (Python-3.6.1.tar.xz) = 8605fb7019386fec227d4b06d06f00ae500a8a89df289bfe6141bb56196c75483a60cc0ee553930742b31cefce68add5ccf226e0f27b7b915f5026d597e1ac29
SHA512 (Python-3.8.2.tar.xz) = ca37ad0e7c5845f5f228566aa8ff654a8f428c7d4a5aaabff29baebb0ca3219b31ba8bb2607f89e37cf3fc564f023b8407e53a4f2c47bd99122c1cc222613e37
SHA512 (Python-3.8.2.tar.xz.asc) = 765796ab5539576bbf1578e05cdb041dbc9a9ca0d6d2040a473a00a293b49f90be11ea6e33b47889da33b25f8e360fad4adeec292f0d43e5fae233d1f03bafd2

View File

@ -1,19 +0,0 @@
/*
Example usage of the Python systemtap tapset to show a nested view of all
Python function calls (and returns) across the whole system.
Run this using
stap systemtap-example.stp
to instrument all Python processes on the system, or (for example) using
stap systemtap-example.stp -c COMMAND
to instrument a specific program (implemented in Python)
*/
probe python.function.entry
{
printf("%s => %s in %s:%d\n", thread_indent(1), funcname, filename, lineno);
}
probe python.function.return
{
printf("%s <= %s in %s:%d\n", thread_indent(-1), funcname, filename, lineno);
}

View File

@ -1,25 +0,0 @@
diff -up Python-3.4.0/Lib/test/test_asyncio/test_events.py.orig Python-3.4.0/Lib/test/test_asyncio/test_events.py
--- Python-3.4.0/Lib/test/test_asyncio/test_events.py.orig 2014-04-15 13:18:49.696215288 +0200
+++ Python-3.4.0/Lib/test/test_asyncio/test_events.py 2014-04-15 13:18:56.104258453 +0200
@@ -1528,7 +1528,7 @@ class SubprocessTestsMixin:
self.check_terminated(proto.returncode)
transp.close()
- @unittest.skipIf(sys.platform == 'win32', "Don't have SIGHUP")
+ @unittest.skipIf(True, "Temporarily skipped (rhbz#1088233)")
def test_subprocess_send_signal(self):
prog = os.path.join(os.path.dirname(__file__), 'echo.py')
diff -up Python-3.4.0/Lib/test/test_asyncio/test_subprocess.py.orig Python-3.4.0/Lib/test/test_asyncio/test_subprocess.py
--- Python-3.4.0/Lib/test/test_asyncio/test_subprocess.py.orig 2014-04-17 12:03:32.777827520 +0200
+++ Python-3.4.0/Lib/test/test_asyncio/test_subprocess.py 2014-04-17 12:04:37.614210481 +0200
@@ -108,7 +108,7 @@ class SubprocessMixin:
else:
self.assertEqual(-signal.SIGTERM, returncode)
- @unittest.skipIf(sys.platform == 'win32', "Don't have SIGHUP")
+ @unittest.skipIf(True, "Temporarily skipped (rhbz#1088233)")
def test_send_signal(self):
code = 'import time; print("sleeping", flush=True); time.sleep(3600)'
args = [sys.executable, '-c', code]

1
tests/.fmf/version Normal file
View File

@ -0,0 +1 @@
1

4
tests/provision.fmf Normal file
View File

@ -0,0 +1,4 @@
---
standard-inventory-qcow2:
qemu:
m: 3G # Amount of VM memory

34
tests/tests.yml Normal file
View File

@ -0,0 +1,34 @@
---
- hosts: localhost
roles:
- role: standard-test-basic
tags:
- classic
repositories:
- repo: "https://src.fedoraproject.org/tests/python.git"
dest: "python"
tests:
- smoke:
dir: python/smoke
run: VERSION=3.8 ./venv.sh
- debugsmoke:
dir: python/smoke
run: PYTHON=python3-debug TOX=false VERSION=3.8 ./venv.sh
- selftest:
dir: python/selftest
run: VERSION=3.8 X="-x test_wsgiref" ./parallel.sh
- debugtest:
dir: python/selftest
run: VERSION=3.8 PYTHON=python3-debug X="-x test_wsgiref" ./parallel.sh
- debugflags:
dir: python/flags
run: python3-debug ./assertflags.py -Og
required_packages:
- gcc # for extension building in venv and selftest
- gdb # for test_gdb
- python3-debug # for leak testing
- python3-devel # for extension building in venv and selftest
- python3-tkinter # for selftest
- python3-test # for selftest
- python3-tox # for venv tests
- glibc-all-langpacks # for locale tests