Bash 5.2 has a new globskipdots option, which is enabled by default. The
check_dotfiles lint fails with globskipdots due to the assumption that
at least the "." and ".." paths will match. Disabling globskipdots would
be the usual solution, but that fails on bash<5.2. Instead, enable
nullglob for this check.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit a6b06a5b17)
In some cases packages are built outside of a directory which contains
pkgname-pkgver, this results in source listing in debug packages having
a conflicting path like `/usr/src/debug/build/` which is not ideal.
This patch ensures we always include the pkgbase to ensure the paths are
unique.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 776b7c1e75)
We use a multi step process during stripping to ensure permissions do
not get changed. However, if the initial objcopy fails, the subsequent cat
results in a blank file. Abandon early if objcopy fails.
Fixes FS#74486
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 18152a211a)
bsdtar uses the "pax" TAR archive format by default, which has support for
storing sparse file information in the archive. Unfortunately this is a source
of unreproducibility because the sparse encoding is taken from the file system
and different file systems handle sparse files differently: some file systems
have no support for sparsely encoded files at all, and even file systems with
sparse file support can report different file information for identical files
due to differing implementations.
As a real world example where this happens, consider the Arch Linux package
"brotli-testdata 1.0.9-7", which contains a sparsely encoded all-zeros file
"usr/share/brotli/testdata/zeros". Building this package on a btrfs file system
yields a different package than building it on tmpfs or ext4 solely due to
different sparse file information that gets recorded in the package tarball.
To improve the reproducibility of archives containing sparsely encoded files,
libarchive version 3.6.0 introduces a new --no-read-sparse option. This skips
reading sparse file information from disk entirely and therefore stores files
"expanded" in the archive, which is the only way to make them reliably
reproducible across file systems.
makepkg will use this option if libarchive is recent enough to support it,
which is detected at build time.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit f5f9e0f4ea)
We want to use -flto=auto in Arch Linux to speed up building, but we
can't hardcode it in buildenv/lto.sh because other downstreams might
have clang < 13.0.0 which did not recognize -flto=auto as equivalent
to -flto=full.
Introducing an LTOFLAGS variable to makepkg.conf seems the way to go.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit e1ce2351f5)
This moves us from the fairly ugly AWK parsing line to debugedit which
originally comes out of the rpm project.
The original code has issues parsing anything that was not straight
C/C++ and languages like Rust or Go would return invalid source code
files. debugedit handles all these cases better.
Fixes FS#66755
Fixes FS#66888
Fixes FS#65677
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit ae2f506ddf)
This means options=('!buildflags') will disable the addition of CFLAG
etc for LTO and debug building.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 42fe4864a0)
When trying to identify debug packages among other packages we
discovered that it's pkgname used in pkgdesc. Since pkgname can
sometimes be an array when building debug packages for a split package,
this could potentially include a pkgname that might not make sense
depending on the order of the array.
This patch simply uses pkgbase as it seems more correct.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 2fe1ba5d81)
>From gcc(1):
-ffile-prefix-map=old=new
[...] Specifying this option is equivalent to specifying all the
individual -f*-prefix-map options. This can be used to make reproducible
builds that are location independent.
Specifically, this additionally enables -fmacro-prefix-map=, which causes
prefix mapping to be applied to expansions of __FILE__ and similar macros.
Without this option, if source files are compiled by passing the
absolute file path to the compiler (as done by e.g. cmake), any
expansions of __FILE__ (e.g. from uses of assert()) will contain
$srcdir.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit e37f7d8b6b)
GCC automatically detects when it is linking LTO objects, but clang does
not. Add -flto to LDFLAGS to make this work for clang too.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 26ee6ff6ad)
Every time we modify gpg's state by signing or revoking a key, gpg
marks the trustdb as stale and rechecks it the next time key_is_lsigned()
or key_is_revoked() is called.
Currently, we alternate calls signing of keys and calling key_is_lsigned()
(idem for revoking) which means that for each key we sign (or revoke), gpg
will check the trustdb once.
To avoid checking the trustb so many times, we can simply do all the
key_is_lsigned() and key_is_revoked() checks upfront. Inbetween read
operations the trustdb is not marked stale and inbetween write operations
the trustdb is also not marked stale. This reduces the amount of trustdb
checks from 50 to 1.
Signed-off-by: Allan McRae <allan@archlinux.org>
Currently, when running pacman-key --populate, gpg prints the
trustdb check output once for each locally signed and revoked key.
When bootstrapping a new container image, about 50 keys get signed
and revoked which leads to a huge amount of output when running
pacman-key which is repeated 50x.
To avoid overloading the user with gpg output, we add --quiet to the gpg
calls generating the trustdb checking output to silence those calls which
gets rid of the trustdb check output on the terminal.
Signed-off-by: Daan De Meyer <daan.j.demeyer@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Arch Linux has been setting PYTHONHASHSEED=0 to create deterministic
.pyc files. After a thorough review by the Arch Security Team, setting
this variable was determined not to generated vulnerable .pyc files, as
when the loader loads the .pyc file and unmarshalls it, the internal
runtime will just populate the unordered data structures and use a new
runtime hash for them.
Signed-off-by: Allan McRae <allan@archlinux.org>
We usually set this up to default to the build time configured install
location, but a couple of files crept in without this.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Binutils commit 93df3340fd5ad32f784214fc125de71811da72ff enabled readelf
to report "Position-Independent Executable" files. Fix stripping to
account for this change.
Signed-off-by: Allan McRae <allan@archlinux.org>
readelf --debug-dump sometimes reports inscrutable warnings which don't
actually affect our extraction of source filenames. For example:
readelf: Warning: There is a hole [0xd3d - 0xd89] in .debug_loc section.
Now gcc 11 seems to have dramatically increased the number of warnings:
readelf: Warning: Corrupt offset (0x0000008e) in range entry 9
[...]
readelf: Warning: Corrupt offset (0x000010f0) in range entry 250
The resulting debuginfo created by the very same toolchain works fine,
as does the list of source filenames. But the warnings are quite
noisy... send them to /dev/null since they are not actionable in the
context of getting source files
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Previously TotalDownload would switch the % download from per package to
overall. Meaning you had a choice of which information to dispplay.
Now with parallel downloads TotalDownload adds an extra progress bar.
There's no reason to have this an off by default feature. Let's just
make it always on.
Signed-off-by: Allan McRae <allan@archlinux.org>
If a makepkg consumer uses a build wrapper to override compiler
flags this may lead to unreproducible packages as there is no way to
know which exact files were used for tooling that tries to reproduce
said package.
Instead of vendoring the whole used makepkg.conf file into buildinfo,
this patch adds two new properties to the .BUILDINFO file named
BUILDTOOL and BUILDTOOLVER which by default are simply makepkg's own
values. Downstream consumers may override those values: For example in
Arch Linux the devtools package can set those values and allow
reproducible builds tooling to fetch the appropriate makepkg.conf.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If specified, this will be used no matter what. If not, then we check if
sudo exists and use that, or else fall back on su.
Implements FS#32621
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Broken via refactoring in commit aa6fe1160b
but for obvious reasons only one person in the last 9 years has ever
actually tried to do this. Still, it's technically correct to allow it.
Fixes FS#70254
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
The current gcc build from git master give different output from
readelf:
gcc-10.2.0
$ readelf "hello" --debug-dump | grep hello
<11> DW_AT_name : (indirect string, offset: 0xbfc): hello.cpp
gcc-git
$ readelf "hello" --debug-dump | grep hello
<12> DW_AT_name : (indirect line string, offset: 0x0): hello.cpp
This causes the awk statement extracting the file name to fail as it
relied on the information being in the 8th field. Instead, extract
the information from the final field.
Fixes FS#70168
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit 0f75ab3224 some unbalanced
quotes were added by the committer while editing an error message.
Fixes FS#69865
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
GCC's LTO implementation emits bytecodes into .o files it generates.
These bytecodes are _not_ considered stable from one release of GCC
to the next. There we need to strip the LTO bytecode out of any .o
(and .a) file that gets installed into the package.
Signed-off-by: Allan McRae <allan@archlinux.org>
Add the 'lto' option to enable building with link time optimization
by adding '-flto' to both CFLAGS and CXXFLAGS. The 'lto' option can
be specificed both in the PKGBUILD or by setting the default in
makepkg.conf.
Signed-off-by: Allan McRae <allan@archlinux.org>
With the recent outages of the keyservers there is a possibility of
`--refresh-keys` failing to fetch new keys. A lot of current key
distribution is done over WKD these days, and `pacman-key` has the
ability to use it for `--recv-key`.
There was a hope `gpg` would end up supporting WKD for the refresh
functionality, but this seems to be limited to expired keys fetched
through WKD. Since this functionality isn't yet available it makes sense
to stuff it into `pacman-key`.
The current implementation looks over all available keyids in the
keyring, attempts to fetch over WKD and then fall backs to keyservers if
no email has a valid WKD available. The downside of this approach is
that it takes a bit longer to refresh the keys, but it should be more
robust as the distribution should be providing their own WKDs.
Co-authored-by: Jonas Witschel <diabonas@archlinux.org>
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
This enables us to extract files in the source array and ensures that we
can decompress files if the uncompressed signature is served.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
buildenv is set once for build() and a second time for package(). When
using both distcc and ccache, this lead to CCACHE_PREFIX="distcc distcc"
in package(), which breaks PKGBUILDs that execute the compiler in
package() because distcc complains:
distcc[383041] (main) CRITICAL! distcc seems to have invoked itself
recursively!
Avoid causing this error by only adding "distcc" to CCACHE_PREFIX if
it's not yet there.
Signed-off-by: Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
Signed-off-by: Allan McRae <allan@archlinux.org>
It updates the stripped/objcopied file by creating a temp file,
chown/chmodding it, and replacing the original file. But upstream
binutils has CVE-worthy issues with this if running strip as root, and
some recent versions of strip don't play nicely with fakeroot.
Also, this has always destroyed xattrs. :/
Sidestep the issue by telling strip/objcopy to write to a temporary
file, and manually dump the contents of that back into the original
binary. Since the original binary is intact, albeit with different
contents, it retains its correct attributes in fakeroot.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In 19980a61e9 there was a msg added which
didn't get the string closed.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
Generating the pacman master key can take some time on systems
without enough entropy. Warn the user that the generation may
take some time.
Fixes FS#30286.
Signed-off-by: Allan McRae <allan@archlinux.org>
With commit 74aacf4495 creating uncompressed .tar
packages fails.
-> Compressing package...
/usr/share/makepkg/util/compress.sh: line 70: COMPRESS.TAR[@]: invalid variable name
bsdtar: Write error
Empty the '$ext' variable for the '.tar' extension in get_compress_command() to
fix this. We would fallback to cat for 'tar' anyways.
Signed-off-by: Michael Straube <michael.straubej@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Fix typo in a comment in tidy_emptydirs().
Signed-off-by: Michael Straube <michael.straubej@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
We leaked fullver and pkgarch all over the place, and only conditionally
unset the other variables. Marking them local is a more proactive
solution.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit c6b04c0465 the signing stage was
moved out of fakeroot, and thus into the main control flow instead of
create_{,src}package
While the function for signing binary packages has logic to build
and gpg-sign multiple filenames, the source package never got this
special treatment. This would be fine, except it uses the standard
variables to set define the filename... like ${fullver}, which is
usually set beforehand, but in this case is not. We don't define fullver
globally as it's an internal implementation detail, except by sheer
coincidence if PKGVERFUNC is false due to improperly guarded code.
Result: source packages didn't end up signed. Instead, we raised a logic
error:
==> WARNING: Failed to sign package file somepackage-.src.tar.gz.
==> ERROR: An unknown error has occurred. Exiting...
Instead, let's just build the version inline, since we only use it once.
Reported-by: GaKu999 <g4ku999@gmail.com>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If multiple files match the pattern libfoo.so*, we want to check each of
them and see if they are shared libraries, and if so, if they have
versions attached.
But some packages can have both shared libraries and random files which
match the filename pattern. This is true at least for files in
/usr/share/gdb/auto-load/, which must match the filename they are paired
with, followed by "-gdb.py" (or some other gdb scripting ext), but
definitely don't contain a shared library. In this case, we don't want
to double-report the library in the generated provides.
It's also possible (probably) for a package to provide a versioned as
well as an unversioned shared library, but in such cases a single
provides entry is sufficient to cover both cases (and the libdepends
for the depending package would contain an unversioned dependency).
Solve this by keeping track of whether we have added a versioned soname
provides already, and then only adding a maximum of one unversioned
provides *iff* there isn't a versioned one yet.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Currently the list of supported formats for an archive, is maintained in
two places. And repo-add does not actually get updated. :(
In the process, remove some of the logical duplication when calling
bsdtar/compress_as.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
get_compression_command() can now be used to do upfront checks for
whether a given extension is known to do something successfully. This is
useful when writing tools in which an unknown compression type is a
fatal error.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each
section is concluded with an empty line. This means that at the end of
the file, an empty line remains. This is considered a trailing
whitespace error. In fact, `git diff --check` will warn about this,
saying "new blank line at EOF."
Instead of closing each section off with an empty line, use the empty
line to separate sections, omitting the empty line at the end of the
file.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
