Avoid using the full path and instead rely on PATH being correctly set
up to find the executable.
This fixes byte compilation for python2.7 when doing flatpak module
builds where python2.7 can be in either /usr/bin or /app/bin, depending
on how it's compiled.
A new script brp-fix-pyc-reproducibility creates an opt-in way of how to fix
problems with the reproducibility of byte-compiled Python files. The script
uses marshalparser [0] which currently doesn't provide solutions for all issues
but can fix at least problems with reference flags. For more info see
this Bugzilla [1].
If you want to use this new feature, you need to define
`%py_reproducible_pyc_path` to specify a path you want to fix `.pyc`
files in (recursively) and build-require /usr/bin/marshalparser.
if you forget to build-require the parser. The error message is:
```
+ /usr/lib/rpm/redhat/brp-python-bytecompile '' 1 0
Bytecompiling .py files below /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64/usr/lib/python3.9 using /usr/bin/python3.9
+ /usr/lib/rpm/redhat/brp-fix-pyc-reproducibility /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64
ERROR: If %py_reproducible_pyc_path is defined, you have to also BuildRequire: /usr/bin/marshalparser !
error: Bad exit status from /var/tmp/rpm-tmp.UUJr4v (%install)
```
A build fails if the parser is not able to parse any of the `.pyc` files.
And finally, if a build is properly configured it produces fixed `.pyc` files.
Currently, `.pyc` files in the tldr package contain a lot of unused reference flags:
```
$ dnf install -y tldr
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
… long output …
190 - Flag_ref(byte=9610, type='TYPE_SHORT_ASCII_INTERNED', content=b'init', usages=0)
191 - Flag_ref(byte=9633, type='TYPE_SHORT_ASCII_INTERNED', content=b'source', usages=0)
192 - Flag_ref(byte=9651, type='TYPE_SHORT_ASCII_INTERNED', content=b'argv', usages=0)
193 - Flag_ref(byte=9657, type='TYPE_SHORT_ASCII_INTERNED', content=b'print_help', usages=0)
194 - Flag_ref(byte=9669, type='TYPE_SHORT_ASCII_INTERNED', content=b'stderr', usages=0)
195 - Flag_ref(byte=9682, type='TYPE_SHORT_ASCII_INTERNED', content=b'parse_args', usages=0)
196 - Flag_ref(byte=9737, type='TYPE_SHORT_ASCII_INTERNED', content=b'encode', usages=0)
197 - Flag_ref(byte=9782, type='TYPE_SHORT_ASCII_INTERNED', content=b'parser', usages=0)
198 - Flag_ref(byte=9790, type='TYPE_SHORT_ASCII_INTERNED', content=b'options', usages=0)
199 - Flag_ref(byte=9799, type='TYPE_SHORT_ASCII_INTERNED', content=b'rest', usages=0)
200 - Flag_ref(byte=9821, type='TYPE_SHORT_ASCII_INTERNED', content=b'result', usages=0)
202 - Flag_ref(byte=10022, type='TYPE_SHORT_ASCII_INTERNED', content=b'__main__', usages=0)
203 - Flag_ref(byte=10102, type='TYPE_SHORT_ASCII_INTERNED', content=b'argparse', usages=0)
204 - Flag_ref(byte=10433, type='TYPE_SHORT_ASCII_INTERNED', content=b'__name__', usages=0)
205 - Flag_ref(byte=10463, type='TYPE_SHORT_ASCII_INTERNED', content=b'<module>', usages=0)
```
This new feature fixes them:
```
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
<empty output>
```
[0] https://github.com/fedora-python/marshalparser
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1686078
The Python compileall2 module in /usr/lib/rpm/redhat/
can be executed by various different Python interpreters.
We don't want to write several different `*.pyc` files
to this location - in most cases, that's not possible,
but somebody might run this as root.
Resolves: rhbz#1595265
The problem this change is intended to solve is with how `real_libdir`
is calculated. Let's assume we want to recursively byte-compile all
`*.py` files in
`/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8`.
Then, `real_libdir` is this path without `$RPM_BUILD_ROOT` with
the filename at the end which displays in the error message like this:
```
Bytecompiling .py files below /builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8 using /usr/bin/python3.8
*** Error compiling '/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8/site-packages/greplin/bar.py'...
File "/usr/lib/python3.8/bar.py", line 1
import sin from math
^
SyntaxError: invalid syntax
```
`/usr/lib/python3.8/bar.py` is obviously wrong.
One of the new features of the `compileall2` module (which will
be available in stdlib in Python 3.9) is that the path byte-compiled to
`*.pyc` files is calculated for each file. This means that by using
`-s` and `-p` we can strip `$RPM_BUILD_ROOT` and prepend `/` for each
file individually which will fix the problem.
```
Bytecompiling .py files below /builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8 using /usr/bin/python3.8
*** Error compiling '/builddir/build/BUILDROOT/python-scales-1.0.9-250.fc32.x86_64/usr/lib/python3.8/site-packages/greplin/bar.py'...
File "/usr/lib/python3.8/site-packages/greplin/bar.py", line 1
import sin from math
^
SyntaxError: invalid syntax
```
This change has an effect only for Python >= 3.4.
Kalev Lember
Miro Hrončok
Lumir Balhar
Tomas Orsava