Commit Graph

103 Commits

Author SHA1 Message Date
Fedora Release Engineering
7b3e3b30de Rebuilt for https://fedoraproject.org/wiki/Fedora_37_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2022-07-22 22:07:02 +00:00
Miro Hrončok
cf65060b7e https://fedoraproject.org/wiki/Changes/PythonDistPEP503ProvidesOnly 2022-06-02 12:43:00 +02:00
Miro Hrončok
0bd051d514 Don't include all requirements with True-evaluating markers in extras subpackages
The idea is that the extra subpackage only has requirements specific to that extra.
The logic however only excluded requirements without markers,
but requirements with *a* marker that was correct leaked to all extras subpackages.

E.g. with the following requirements:

    Requires-Dist: base-dependency
    Requires-Dist: base-dependency-with-matching-marker ; python_version < "3.15"
    Requires-Dist: base-dependency-with-unmatching-marker ; python_version < "3.8"
    Provides-Extra: an-extra
    Requires-Dist: extra-only-dependency-with-matching-marker ; extra == 'an-extra' and python_version < "3.15"
    Requires-Dist: extra-only-dependency-with-unmatching-marker ; extra == 'an-extra' and python_version < "3.8"

On Python 3.10, the base package generated the following requirements:

    python3.10dist(base-dependency)
    python3.10dist(base-dependency-with-matching-marker)

And for the [an-extra] extra:

    python3.10dist(base-dependency-with-matching-marker)  <--- REDUNDANT, WRONG
    python3.10dist(extra-only-dependency-with-matching-marker)

Now we no longer just check if the marker evaluates to True,
but we also check that the same marker evaluates to False when the extra is not given.

A real package with this issue is build[virtualenv] 0.8.0, which we use for tests. The package has:

    Requires-Dist: tomli (>=1.0.0) ; python_version < "3.11"

And on Python 3.10, it generated the following dependency for python3-build+virtualenv-0.8.0-2.fc37.noarch.rpm:

    python3.10dist(tomli) >= 1

Now it no longer does. This is asserted in tests.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2090186

Upstream PR: https://github.com/rpm-software-management/python-rpm-packaging/pull/16
2022-06-02 12:05:11 +02:00
Sandro Mani
76e71def2c Add namespace option to pythodistdeps.py
Co-authored-by: Miro Hrončok <miro@hroncok.cz>
2022-02-10 11:54:50 +01:00
Charalampos Stratakis
ecd2f8b3f8 Add rpminspect file 2022-02-02 15:24:19 +00:00
Tomas Orsava
e18b8c952c Add tests for automatically not generating Obsoletes tags on Fedora 2022-02-02 13:58:09 +01:00
Tomas Orsava
b1fa63bf02 From python3-foo packages automatically generate python3.X-foo Obsoletes tags on CentOS/RHEL 2022-01-26 17:07:05 +01:00
Tomas Orsava
fbd2f87265 Fix typo in lua comment 2022-01-26 17:06:51 +01:00
Fedora Release Engineering
2ff265d8fd - Rebuilt for https://fedoraproject.org/wiki/Fedora_36_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2022-01-21 15:19:07 +00:00
Gordon Messmer
2c2f8bd984 Handle legacy version specifiers that would previously raise exceptions. 2021-12-19 14:08:05 -08:00
Gordon Messmer
a3ad67b505 Additional fix for dev releases. 2021-10-29 20:00:41 -07:00
Gordon Messmer
27f9733f0b Sync dependency conversion with upstream pyreq2rpm.
Improve handling of > operator, preventing post-release from satisfying most rpm requirements.
Improve handling of < operator, preventing pre-release from satisfying rpm requirement.
Improve handling of != operator with prefix matching, preventing pre-release from satisfying rpm requirements.
2021-10-28 21:50:58 -07:00
Fedora Release Engineering
98fa009fc8 - Rebuilt for https://fedoraproject.org/wiki/Fedora_35_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2021-07-23 09:18:27 +00:00
Miro Hrončok
9bd2a43a74 Support multiple vendor files in pythonbundles.py
Not bumping the release, will happily wait until it bubbles trough.
2021-06-22 18:28:43 +00:00
Miro Hrončok
04ae9b96f2 CI: Adapt pythondist.spec for Python 3.10 being the main Python version
Preserves comaptbility with Python 3.9.
2021-06-18 19:20:16 +02:00
Tomas Orsava
cc489bde7a pythondistdeps.py: Catch all exceptions and terminate build if one is raised 2021-05-25 18:51:44 +02:00
Tomas Orsava
27d363833e pythondistdeps.py: Detect and error when metadata is corrupted 2021-05-25 18:11:00 +02:00
Miro Hrončok
20f8b2c775 Fix python(abi) generator (the one written in Python)
There were three problems:

 - sys.version was not imported
 - sys.version[:3] is not reliable on Python 3.10+
 - distutils is deprecated on Python 3.10+

We were not hit by the missing import in Fedora because we only run the script
on .dist-info/.egg-info/.egg and not on .py files, so this if-branch never runs.

But when the script was fed with a .py path, it errored:

    Traceback (most recent call last):
      File "/usr/lib/rpm/pythondistdeps.py", line 344, in <module>
        purelib = get_python_lib(standard_lib=0, plat_specific=0).split(version[:3])[0]
    NameError: name 'version' is not defined

The sys.version[:3] thing kinda works for Python 3.10+ because *in this
particular case* splitting on '3.1' and taking the prefix yields the same
results as splitting on '3.10', but I consider that mere coincidence.

Finally, since the distutils import happened at module-level,
we got the Deprecation warning in all Fedora's Python packages:

    /usr/lib/rpm/pythondistdeps.py:16: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12

Backported from https://github.com/rpm-software-management/python-rpm-packaging/commit/d12e039037
2021-04-19 22:59:10 +02:00
Miro Hrončok
0a12aa5a2f Do not generate setuptools requirement for console_scripts on Python 3.10+
See https://fedoraproject.org/wiki/Changes/Reduce_dependencies_on_python3-setuptools
2021-03-31 11:56:16 +02:00
Tomas Orsava
a295a58559 Add __pycache__ into .gitignore 2021-03-11 13:41:54 +01:00
Tomas Orsava
3a4efade98 pythondistdeps.py: Always output extras names in lowercase 2021-03-11 13:41:54 +01:00
Tomas Orsava
b44c808358 pythondistdeps.py: Compare extras as lowercase
- New test sources tarball with added test data
2021-03-11 13:41:25 +01:00
Tomas Orsava
103464475f pythondistdeps.py: Changing order in test-data 2021-03-11 12:46:23 +01:00
Tomas Orsava
48510eebae scripts/pythondistdeps: Fix for Python 3.10
self.name in PathDistribution is a property in Python 3.10+ and thus we
can't redefine it as an instance variable. Instead we explicitly define
it as a property, which works on all supported Python versions.
2021-02-24 14:07:24 +01:00
Tomas Orsava
438d8d3b70 scripts/pythondistdeps: Backport switch to importlib.metadata from upstream
Upstream change to importlib.metadata: https://github.com/rpm-software-management/rpm/pull/1317

Due to extras packages being hadled slightly differently by importlib,
one test case for this was added.  And due to changes in handling
requires.txt files, comments were removed from the pyreq2rpm.tests
testing package.

Also because of the switch, we removed the dependency on setuptools and
added a dependency on packaging.

Note: Some packages with egg-info files might provide a different name
due to this change if there is a conflict between the filename and the
name in the metadata. Previously, the filename was sometimes used to
parse the name, now it is always the content of that file, which is what
packaging does, and thus also pip and other Python tooling. Currently,
this is known to affect only 1 package in Fedora (ntpsec).

The resulting script is different from upstream because of not yet upstreamed changes in Fedora:
- scripts/pythondistdeps: Rework error messages
- scripts/pythondistdeps: Add parameter --package-name
- scripts/pythondistdeps: Implement provides/requires for extras packages
- pythondistdeps.py: When parsing extras name, take the rightmost +

These changes are proposed in this upstream PR: https://github.com/rpm-software-management/rpm/pull/1546
2021-02-18 16:08:27 +01:00
Miro Hrončok
2d631762c5 Remove unused 2.7 from --majorver-provides-versions
Fixup for 8c2a1c0ac9.
This makes no real difference, just a cleanup, hence not bumping.
2021-02-08 10:53:40 +01:00
Miro Hrončok
8c2a1c0ac9 Disable the dist generators for Python 2
https://fedoraproject.org/wiki/Changes/Disable_Python_2_Dist_RPM_Generators_and_Freeze_Python_2_Macros

The regex previously matched any Python version in a form of <single digit>.<at least one digit>.

Now it matches anything from 3.0 above: <single digit (3 or higher)>.<at least one digit>

It still does not match <multiple digits>.<at least one digit>, e.g. 11.0.

This is a breaking change, hence the version bump.
2021-02-03 14:09:45 +01:00
Fedora Release Engineering
b65cf8549a - Rebuilt for https://fedoraproject.org/wiki/Fedora_34_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2021-01-27 13:13:21 +00:00
Tomas Orsava
f328c9dd18 Add executable bit to pythonbundles.py
pythondistdeps.py is executable already
2020-10-19 12:56:51 +02:00
Tomas Orsava
d77d134c10 Run scripts in an isolated environment (#1889080) 2020-10-19 12:56:43 +02:00
Miro Hrončok
bfb7f70b99 Add a test for a requires with multiple underscores 2020-09-25 14:22:03 +02:00
Tomas Orsava
cb3aaf6d26 Add a test for a requires with an underscore
We already have PyQt5_sip as a test of a provides with an underscore
2020-09-23 11:39:53 +02:00
Fedora Release Engineering
df7ed92279 - Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
Signed-off-by: Fedora Release Engineering <releng@fedoraproject.org>
2020-07-29 03:42:44 +00:00
Tomas Orsava
64e5d7567a Sync tests for python dependency conversion with pyreq2rpm 2020-07-22 18:00:02 +02:00
Gordon Messmer
fbe1c77166 Sync python dependency conversion with pyreq2rpm. 2020-07-22 18:00:02 +02:00
Miro Hrončok
7398b71fbc pythondistdeps.py: When parsing extras name, take the rightmost + 2020-07-22 00:29:19 +02:00
Miro Hrončok
d1a02fdda7 pythondistdeps.py: Adapt Python version marker workaround for setuptools 42+
See https://bugzilla.redhat.com/show_bug.cgi?id=1853597#c11

pkg_resources from setuptools 42+ no longer only use platform.python_version(),
but also platform.python_version_tuple() -- this was updated in packaging 19.1+.

This fix makes it work again with both new and old setuptools,
hopefully for some while.

bf069fe9dd
86a443f318
2020-07-10 16:30:20 +02:00
Tomas Orsava
32a1b47f5b scripts/pythondistdeps: Tests: small tweaks 2020-07-10 15:46:19 +02:00
Tomas Orsava
c2e0f33565 scripts/pythondistdeps: Add tests for: Rework error messages 2020-07-10 15:37:11 +02:00
Tomas Orsava
098c48d46d scripts/pythondistdeps: Rework error messages 2020-07-10 13:45:59 +02:00
Tomas Orsava
b6e0638f7c Enable --require-extras-subpackages and bump release 2020-07-10 13:43:04 +02:00
Tomas Orsava
9df3e5bcb5 scripts/pythondistdeps: Add tests for: Implement provides/requires for extras packages 2020-07-10 13:42:12 +02:00
Tomas Orsava
0c9665427c scripts/pythondistdeps: Implement provides/requires for extras packages 2020-07-10 13:42:12 +02:00
Tomas Orsava
3b1100ba1f scripts/pythondistdeps: Add parameter --package-name 2020-07-10 13:42:12 +02:00
Miro Hrončok
48c0de39d9 Add a script to generate Python bundled provides
See https://src.fedoraproject.org/rpms/python-setuptools/pull-request/40

Strictly speaking, this is not an RPM generator, but:

 - it generates provides
 - it is tighly coupled with pythondistdeps.py

Usage:

 1. Run `$ /usr/lib/rpm/pythonbundles.py .../vendored.txt`

 2. Copy the output into the spec as a macro definition:

    %global bundled %{expand:
    Provides: bundled(python3dist(appdirs)) = 1.4.3
    Provides: bundled(python3dist(packaging)) = 16.8
    Provides: bundled(python3dist(pyparsing)) = 2.2.1
    Provides: bundled(python3dist(six)) = 1.15
    }

 3. Use the macro to expand the provides
 4. Verify the macro contents in %check:

    %check
    ...
    %{_rpmconfigdir}/pythonbundles.py src/_vendor/vendored.txt --compare-with '%{bundled}'
2020-07-07 16:01:37 +02:00
Miro Hrončok
e78c420523 Fix python(abi) requires generator, it picked files from almost good directories
The %__python_magic filter suddenly got actually working with file 5.39:

Before:

    file.cpython-38.opt-1.pyc: data

After:

    file.cpython-38.opt-1.pyc: python 3.8 byte-compiled

Hence, the filter started to pick all Python files regardless of their location.
Later, in the actual generator, paths like this were considered:

    /opt/usr/lib/python3.X/...

And generated requirements on python(abi).

We don't actually need to filter the files by file magic,
so we drop it to get the previously accidentally working behavior.

We could choose if the path and magic filters are applied as OR or AND.
However, we don't want either.

We actually want to mach any files in Python directories regardless of their magic.
We *could* filter by file type (and executable bit) for provides,
but that would require us to split the attr files into two.
2020-06-18 13:32:24 +02:00
Miro Hrončok
3a396fbf96 Use PEP 503 names for requires 2020-05-21 17:43:19 +02:00
Miro Hrončok
39315a6aa4 Adapt tests for the pythonXY -> pythonX.Y renaming
See https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/VIUS7WMQMDX6H2WEIH7TVTMBB6SUHY7E/
2020-05-07 17:59:18 +02:00
Miro Hrončok
33358b9a65 Deduplicate automatically provided names trough Python RPM Lua macros 2020-05-05 14:02:52 +02:00
Miro Hrončok
c3f90ed2e8 Fix reversed grep exit codes in integration tests
grep -v only fails if there are no unmatched lines, but that's not what we want to test.
2020-05-04 13:45:57 +02:00