A new script brp-fix-pyc-reproducibility creates an opt-in way of how to fix
problems with the reproducibility of byte-compiled Python files. The script
uses marshalparser [0] which currently doesn't provide solutions for all issues
but can fix at least problems with reference flags. For more info see
this Bugzilla [1].
If you want to use this new feature, you need to define
`%py_reproducible_pyc_path` to specify a path you want to fix `.pyc`
files in (recursively) and build-require /usr/bin/marshalparser.
if you forget to build-require the parser. The error message is:
```
+ /usr/lib/rpm/redhat/brp-python-bytecompile '' 1 0
Bytecompiling .py files below /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64/usr/lib/python3.9 using /usr/bin/python3.9
+ /usr/lib/rpm/redhat/brp-fix-pyc-reproducibility /builddir/build/BUILDROOT/tldr-0.5-2.fc33.x86_64
ERROR: If %py_reproducible_pyc_path is defined, you have to also BuildRequire: /usr/bin/marshalparser !
error: Bad exit status from /var/tmp/rpm-tmp.UUJr4v (%install)
```
A build fails if the parser is not able to parse any of the `.pyc` files.
And finally, if a build is properly configured it produces fixed `.pyc` files.
Currently, `.pyc` files in the tldr package contain a lot of unused reference flags:
```
$ dnf install -y tldr
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
… long output …
190 - Flag_ref(byte=9610, type='TYPE_SHORT_ASCII_INTERNED', content=b'init', usages=0)
191 - Flag_ref(byte=9633, type='TYPE_SHORT_ASCII_INTERNED', content=b'source', usages=0)
192 - Flag_ref(byte=9651, type='TYPE_SHORT_ASCII_INTERNED', content=b'argv', usages=0)
193 - Flag_ref(byte=9657, type='TYPE_SHORT_ASCII_INTERNED', content=b'print_help', usages=0)
194 - Flag_ref(byte=9669, type='TYPE_SHORT_ASCII_INTERNED', content=b'stderr', usages=0)
195 - Flag_ref(byte=9682, type='TYPE_SHORT_ASCII_INTERNED', content=b'parse_args', usages=0)
196 - Flag_ref(byte=9737, type='TYPE_SHORT_ASCII_INTERNED', content=b'encode', usages=0)
197 - Flag_ref(byte=9782, type='TYPE_SHORT_ASCII_INTERNED', content=b'parser', usages=0)
198 - Flag_ref(byte=9790, type='TYPE_SHORT_ASCII_INTERNED', content=b'options', usages=0)
199 - Flag_ref(byte=9799, type='TYPE_SHORT_ASCII_INTERNED', content=b'rest', usages=0)
200 - Flag_ref(byte=9821, type='TYPE_SHORT_ASCII_INTERNED', content=b'result', usages=0)
202 - Flag_ref(byte=10022, type='TYPE_SHORT_ASCII_INTERNED', content=b'__main__', usages=0)
203 - Flag_ref(byte=10102, type='TYPE_SHORT_ASCII_INTERNED', content=b'argparse', usages=0)
204 - Flag_ref(byte=10433, type='TYPE_SHORT_ASCII_INTERNED', content=b'__name__', usages=0)
205 - Flag_ref(byte=10463, type='TYPE_SHORT_ASCII_INTERNED', content=b'<module>', usages=0)
```
This new feature fixes them:
```
$ marshalparser --unused /usr/lib/python3.9/site-packages/__pycache__/tldr.cpython-39.pyc
<empty output>
```
[0] https://github.com/fedora-python/marshalparser
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1686078
This macro calls `rpm` on background, which is not good idea. Luckily,
it seems to be used just by samba package, so it should not cause any
substantial issues.
More details at \[[1]\] where the guideline to ban `rpm` call during
build is discussed.
[1]: https://pagure.io/packaging-committee/pull-request/954
This config is to let libtool recognize that our 64bit variant of
%_libdir is actually on the standard/default library path, so libtool
doesn't think it has to be hard-wired as RPATH. This is proper solution
for libtool RPATH issues described in:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_removing_rpath
The libtool script/macros (new enough, v2.4.6+) honor this variable when
it isn't possible to detect the system-wide default library path. It is
e.g. able to parse /etc/ld.so.* configuration, but there's no info about
/usr/lib64 on Fedora.
So to not force everybody to do:
%configure LT_SYS_LIBRARY_PATH=...
... rather set this system-wide. This is low-risk change since
older libtool scripts don't use this variable, and really no other
tools should.
If %source_date_epoch_from_changelog is true, RPM can set the SOURCE_DATE_EPOCH
environment variable to the timestamp of the topmost changelog entry. The
SOURCE_DATE_EPOCH can be in turn used by various projects to override otherwise
dynamically generated timestamps.
E.g. this might help to have stable timestamps in generated
documentation etc.
Rpm 4.15 removes various language-specific macros. Python side is
already covered by the versioned python macros but this is not the
case with Perl, macros. Add them here temporarily to avoid breaking
the world, but these really belong to perl-macros or such.
Once upon a time these did differ (for various bad reasons) but the
version in rpm has been identical to ours for many years, lets shed
some old baggage.
It says more where to look and what it is supposed to do.
Acked-by: Florian Festi <ffesti@redhat.com>
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
This patch adds two additional rpm macros, __brp_mangle_shebangs_exclude_file
and __brp_mangle_shebangs_exclude_from_file, to specify files from which
to read the extended regexps used for excluding shebangs and target
files.
Additionally, this adds documentation in the macros file and
--help/--usage/-?/-h to brp-mangle-shebangs, so that it's possible to
actually discover what the intended behavior is without reading the
script itself.
Signed-off-by: Peter Jones <pjones@redhat.com>
Introduces __brp_mangle_shebangs_exclude_from and __brp_mangle_shebangs_exclude
* the first allows to explude specific paths from the mangling
* the second allows to exlude specific shebangs
Both are used with `grep -E`. Similar escaping rules as in [1] apply.
[1] https://fedoraproject.org/wiki/Packaging:AutoProvidesAndRequiresFiltering
If there are unpackaged symlinks, build will fail with unpackaged files.
People can %undefine __brp_ldconfig if they need to and they should make
sure that they call ldconfig themselves.
Right now, script doesn't guide packagers what to do, but it's not
prerequisite.
Signed-off-by: Igor Gnatenko <ignatenkobrain@fedoraproject.org>
Add a separate macro for each brp we have, using standard naming
convention and conditionalize the usage in %__os_install_post.
Voilà, we have a standard way to disable (and also override) any brp
scripts from specs that need it and a common scheme for new brps
to follow.
Note that this is not supposed to change the existing behavior and
default build root policy invocations at all, any change in those
would be a thinko/typo/copy-paste error in this commit.
Per fweimer: "Sorry, we need to revert the -Werror=implicit-*
bits. There is no chance we can get this working in any
reasonable time frame, there is simply too much breakage."
Introduce new language specific __global_fooflags for C, C++ and Fortran
This looks mildly strange and suspicious since because the global flags
get pulled in indirectly via %optflags (that doesn't change here but becomes
more obvious).
Depends on previous commit (8fe5b07871)
which was done separately to make the actual change (or lack of thereof)
stand out here.
This is not supposed to change any actual values for current usages,
so if it does it's a bug.
However there's a minor bonus involved for Fortran users who can now get
the correct FFLAGS/FCFLAGS for non-autoconf projects too by using
__global_fflags/fcflags
Preparing for language specific compiler flag macros, this is
simply:
perl -pi -e "s:__global_cflags:__global_compiler_flags:g" macros rpmrc
Since this looks like a much bigger change than it actually is, doing
it in a separate step without an associated build.
If java people say brp-java-repack-jars is not needed then it
probably isn't (#1235770). brp-implant-ident-static hasn't been enabled
in 13+ years, I THINK it's safe to say its not critically needed.
Leaving the actual scripts in the repo for now (amusement for
archeologists of future generations, eh?)
One possible incompatibility, hopefully non-issue: our brp-strip*
allowed setting strip and objdump to use via args and STRIP and
OBJDUMP env vars whereas the rpm ones allow it through args only
(i.e. %{__strip} and %{__objdump} as far as specfiles are concerned).
Specifically, the following are gone from here now: %_prefix,
%_sysconfdir, %_infodir, %_mandir, %_defaultdocdir, %_configure,
%makeinstall, %debug_package, %_use_internal_dependency_generator,
%_missing_doc_files_terminate_build, %_unpackaged_files_terminate_build
- Stop overriding rpm external dependency generator settings by default
- No normal package should ever end up using the old unmaintained
dependency generator scripts from here, but the kmp system depends
for now on the way this was previously set up here so letting
that old cruft live in the non-default package for now.
This reverts commit 8174ec3d10.
remove patch that forces --disable-silent-rules to configure it breaks anything set to not ignore unknown configure options
Various projects have been adding AM_SILENT_RULES from Automake to
their Makefiles for "developer convenience"; the goal being that they
see warnings more easily.
Now really the right way to do this is to have a make wrapper (or an IDE)
that knows how to filter out warnings, but let's leave that aside for now.
But for debugging builds, we really need the full log data. Being
able to see exactly how e.g. libtool is being run helps a lot for
debugging link problems as an example.
- brp-strip-comment-note is not only unnecessary here but is also
now messing up things by resetting EI_OSABI to zero (#568921)
- patch from Roland McGrath
- with %_python_bytecompile_errors_terminate_build set to non-zero,
byte-compilation errors will abort the build, this helps catch out
silly "improt foo" syntax errors early on
- not all .py files are valid python (they can be templates, inteded for
jython consumption etc), and what's valid can depend on the python
version (notably 2.x vs 3.x) so allow overriding from spec
- rpm < 4.6 used to try and fetch and import any missing keys from
keyserver automatically on rpmdb iteration if hkp_keyserver was set, which
caused hideous slowdowns and huge load on pgp keyservers AND was a
security hazard as rpm thinks imported == trusted key. This is safe
enable now as rpm will only ever import keys when explicitly told to do
so with --import
- this makes pgp import directly from PGP servers work, ie
'rpm --import 0x<keyid>'
- autotools dependency tracking isn't generally useful in rpm builds;
disabling it results in cleaner build logs and possibly slight build speedups
- patch from Ville Skyttä
- --target is only ever useful for handful of compiler toolchain packages
and cross-compiler packages are better off setting it themselves if
necessary, rpm messing here only gets in the way
- patch originally from Stepan Kasal
- syncs up with rpm upstream setup
- FFLAGS has a Fedora-specific override forcing us to carry this %configure
copy, need to fix rpm to permit more fine-grained overrides...
- as per https://fedoraproject.org/wiki/Features/XZRpmPayloads
- lowish compression preset level to keep deltarpm rebuild time tolerable
- source rpms dont really benefit from XZ compression as the contents are
typically tarballs which are already compressed
- patch from Bill Nottingham
- remove any existing buildroot contents and safely create a new one
- patch originally from OpenSUSE / Michael Schroeder, adopted to Fedora
by Tom "spot" Callaway
