Bash 5.2 has a new globskipdots option, which is enabled by default. The
check_dotfiles lint fails with globskipdots due to the assumption that
at least the "." and ".." paths will match. Disabling globskipdots would
be the usual solution, but that fails on bash<5.2. Instead, enable
nullglob for this check.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit a6b06a5b17)
Currently our gitlab CI is failing due to valgrind breakage. With
Arch stripping glibc, valgrind now requires debuginfod to be active.
However the gitlab CI system combined without our testsuite does not
retrieve these symbols, even when the appropriate environmental
variable is set.
Work around this by installing the glibc-debug package directly
using a slight kludge... All blame for this approach is assigned
to foutrelis!
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit d1e95c4e54)
alpm_pkg_get_builddate() and alpm_pkg_get_installdate() both return -1 on
error. Correctly handle the error condition in pacman.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 7bcc2d9b23)
In some cases packages are built outside of a directory which contains
pkgname-pkgver, this results in source listing in debug packages having
a conflicting path like `/usr/src/debug/build/` which is not ideal.
This patch ensures we always include the pkgbase to ensure the paths are
unique.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 776b7c1e75)
We use a multi step process during stripping to ensure permissions do
not get changed. However, if the initial objcopy fails, the subsequent cat
results in a blank file. Abandon early if objcopy fails.
Fixes FS#74486
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 18152a211a)
bsdtar uses the "pax" TAR archive format by default, which has support for
storing sparse file information in the archive. Unfortunately this is a source
of unreproducibility because the sparse encoding is taken from the file system
and different file systems handle sparse files differently: some file systems
have no support for sparsely encoded files at all, and even file systems with
sparse file support can report different file information for identical files
due to differing implementations.
As a real world example where this happens, consider the Arch Linux package
"brotli-testdata 1.0.9-7", which contains a sparsely encoded all-zeros file
"usr/share/brotli/testdata/zeros". Building this package on a btrfs file system
yields a different package than building it on tmpfs or ext4 solely due to
different sparse file information that gets recorded in the package tarball.
To improve the reproducibility of archives containing sparsely encoded files,
libarchive version 3.6.0 introduces a new --no-read-sparse option. This skips
reading sparse file information from disk entirely and therefore stores files
"expanded" in the archive, which is the only way to make them reliably
reproducible across file systems.
makepkg will use this option if libarchive is recent enough to support it,
which is detected at build time.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit f5f9e0f4ea)
Parsing of Content-Disposition relies on well formed headers.
A malformed header such as:
Content-Disposition="";
will result in a strnduppayload->content_disp_name, -1, ptr),
which will copy memory until it hits a \0.
Prevent this by only copying the value if it exists.
Fixes FS#73704.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 40583ebe89)
In order to use WKD in pacman -U/--upgrade operations, we need to
get the packager information from the .PKGINFO within the package.
That has obvious security implications. e.g. something like this
could convince a user to download a different key to what they
expect:
packager = foo bar <>^[[2K^[[0G:: Import PGP key DEADBEEF, "foo <bar>
While downloading an untrusted key has little impact due to the
web-of-trust model used by pacman, this could be bad in combination
with an exploit that allowed trust of keys in the keyring to be
altered.
To be safe, do not use WKD when installing using -U.
Fixes FS#73703.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 632eb9739d)
Avoid a segfault when a search of the keyserver returns that the
key is found but returns no primary IDs. We are then likely going
to fail the import, but attempt anyway because no-one know what
a keyserver will do!
Fixes FS#73534.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 6b37d99431)
Looking up a key using WKD just ensures you have a key with the
same email address, it does not ensure that a key with the correct
fingerprint has been downloaded.
Check a key with the relevant fingerprint is available after a
WKD import.
(cherry picked from commit e1246baddd)
We want to use -flto=auto in Arch Linux to speed up building, but we
can't hardcode it in buildenv/lto.sh because other downstreams might
have clang < 13.0.0 which did not recognize -flto=auto as equivalent
to -flto=full.
Introducing an LTOFLAGS variable to makepkg.conf seems the way to go.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit e1ce2351f5)
This moves us from the fairly ugly AWK parsing line to debugedit which
originally comes out of the rpm project.
The original code has issues parsing anything that was not straight
C/C++ and languages like Rust or Go would return invalid source code
files. debugedit handles all these cases better.
Fixes FS#66755
Fixes FS#66888
Fixes FS#65677
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit ae2f506ddf)
This means options=('!buildflags') will disable the addition of CFLAG
etc for LTO and debug building.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 42fe4864a0)
When trying to identify debug packages among other packages we
discovered that it's pkgname used in pkgdesc. Since pkgname can
sometimes be an array when building debug packages for a split package,
this could potentially include a pkgname that might not make sense
depending on the order of the array.
This patch simply uses pkgbase as it seems more correct.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 2fe1ba5d81)
>From gcc(1):
-ffile-prefix-map=old=new
[...] Specifying this option is equivalent to specifying all the
individual -f*-prefix-map options. This can be used to make reproducible
builds that are location independent.
Specifically, this additionally enables -fmacro-prefix-map=, which causes
prefix mapping to be applied to expansions of __FILE__ and similar macros.
Without this option, if source files are compiled by passing the
absolute file path to the compiler (as done by e.g. cmake), any
expansions of __FILE__ (e.g. from uses of assert()) will contain
$srcdir.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit e37f7d8b6b)
This is the error value generally used and the calling function
explicitly checks for -1, later causing the error to be missed
and the transaction to continue.
> pacman -S xterm
warning: xterm-369-1 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...
Package (1) Old Version New Version Net Change Download Size
extra/xterm 369-1 369-1 0.00 MiB 0.42 MiB
Total Download Size: 0.42 MiB
Total Installed Size: 1.05 MiB
Net Upgrade Size: 0.00 MiB
:: Proceed with installation? [Y/n]
error: no servers configured for repository: extra
(1/1) checking keys in keyring [--------------------------------------------------------] 100%
(1/1) checking package integrity [--------------------------------------------------------] 100%
error: failed to commit transaction (wrong or NULL argument passed)
Errors occurred, no packages were upgraded.
(cherry picked from commit 58c81fa213)
GCC automatically detects when it is linking LTO objects, but clang does
not. Add -flto to LDFLAGS to make this work for clang too.
Signed-off-by: Allan McRae <allan@archlinux.org>
(cherry picked from commit 26ee6ff6ad)
When removing files we check _alpm_access() to see if we can write
(delete) the file. If not, we check if the file exists because if the
file does not exist then we don't actually need to remove it so there's
no issue.
However the second call uses acess() instead of _alpm_access() which
does not the rootdir into account.
(cherry picked from commit 5287cc7251)
Previously, when printing a package changelog to stdout, we would write
chunks of data that were not necessarily nul-terminated to stdout using
a function (fputs) which requires the input string to be nul-terminated.
On my system, this would result in occasional garbage characters showing
up in the "pacman -Qc" output.
Fix this by never nul-terminating the chunk, and using the fwrite()
function which takes an explicit input size and does not require a
nul-terminated string.
Signed-off-by: Carlo Teubner <carlo@cteubner.net>
(cherry picked from commit 806ccd90ed)
Added, seemingly by accident, with meson in
51db84750e
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
(cherry picked from commit 529e208f39)
Every time we modify gpg's state by signing or revoking a key, gpg
marks the trustdb as stale and rechecks it the next time key_is_lsigned()
or key_is_revoked() is called.
Currently, we alternate calls signing of keys and calling key_is_lsigned()
(idem for revoking) which means that for each key we sign (or revoke), gpg
will check the trustdb once.
To avoid checking the trustb so many times, we can simply do all the
key_is_lsigned() and key_is_revoked() checks upfront. Inbetween read
operations the trustdb is not marked stale and inbetween write operations
the trustdb is also not marked stale. This reduces the amount of trustdb
checks from 50 to 1.
Signed-off-by: Allan McRae <allan@archlinux.org>
Currently, when running pacman-key --populate, gpg prints the
trustdb check output once for each locally signed and revoked key.
When bootstrapping a new container image, about 50 keys get signed
and revoked which leads to a huge amount of output when running
pacman-key which is repeated 50x.
To avoid overloading the user with gpg output, we add --quiet to the gpg
calls generating the trustdb checking output to silence those calls which
gets rid of the trustdb check output on the terminal.
Signed-off-by: Daan De Meyer <daan.j.demeyer@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
All of these links are broken since the recent move to
gitlab.archlinux.org.
A few projects are, apparently, only available on GitHub, so I've linked
to that source (hopefully that's only temporary).
For git-clone URLs, I've opted for the https URLs since those can be
used by anyone -- whereas the ssh URLs require the user to be registered
on the gitlab instance which is not open to the public yet.
Signed-off-by: Hugo Osvaldo Barrera <hugo@barrera.io>
Signed-off-by: Allan McRae <allan@archlinux.org>
When downloading in parallel, sort by package size so that the larger
packages are queued first to fully leverage parallelism.
Addresses FS#70172
Signed-off-by: Charlie Sale <softwaresale01@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Test for downloads that redirect to some sort of cdn where the
redirected url does not relate to the original filename.
Signed-off-by: Allan McRae <allan@archlinux.org>
Github and other sites redirect their downloads to a cdn. So the
download http://foo.org/myrepo.db may redirect to something like
https://cdn.foo.org/83749327439.
This then causes pacman to try and download the sig as
https://cdn.foo.org/83749327439.sig which is incorrect. In this case
pacman should append .sig to the original url.
However urls like https://archlinux.org/packages/community/x86_64/0ad/download/
Redirect to the mirror, so .sig has to appended after the redirects and
not before.
So we decide if we should append .sig on the original or effective url
based on if the effective url (minus the query part) has .db or .pkg in it.
Fixes FS#71148
---
v2: move variable decleration to start of block
v3: use dbext instead of db
archweb's download links all ended in /download. This cause all the temp
files to be named download.part. With parallel downloads this results in
multiple downloads to go to the same temp file and breaks the transaction.
Assign random temporary filenames to downloads from URLs that are either
missing a filename, or if the filename does not contain at least three
hyphens (as a well formed package filename does).
While this approach to determining when to use a temporary filename is
not 100% foolproof, it does keep nice looking download progress bar names
when a proper package filename is given. The only downside of not using
temporary files when provided with a filename with three or more hyphens
is URLs created specifically to bypass temporary filename usage can not
be downloaded in parallel. We probably do not want to download packages
from such URLs anyway.
Fixes FS#71464
Modified-by: Allan McRae (do not use temporary files for realish URLs)
Signed-off-by: Allan McRae <allan@archlinux.org>
Arch Linux has been setting PYTHONHASHSEED=0 to create deterministic
.pyc files. After a thorough review by the Arch Security Team, setting
this variable was determined not to generated vulnerable .pyc files, as
when the loader loads the .pyc file and unmarshalls it, the internal
runtime will just populate the unordered data structures and use a new
runtime hash for them.
Signed-off-by: Allan McRae <allan@archlinux.org>
We usually set this up to default to the build time configured install
location, but a couple of files crept in without this.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Binutils commit 93df3340fd5ad32f784214fc125de71811da72ff enabled readelf
to report "Position-Independent Executable" files. Fix stripping to
account for this change.
Signed-off-by: Allan McRae <allan@archlinux.org>
pacman_date is set to the current date during build without respecting
SOURCE_DATE_EPOCH. As a result, a build cannot be fully reproduced on a later
date because the date embedded into the man pages does not match.
In contrast, the built-in asciidoc attribute "localdate" respects
SOURCE_DATE_EPOCH and has the desired ISO 8601 format, so simply use that
instead of the custom "pacman_date" attribute.
Fixes: FS#71154
Signed-off-by: Jonas Witschel <diabonas at archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If the original download redirects to to a different url then alpm would
try to name the sig file after the url instead of <original_file>.sig.
Instead force this naming scheme regardless of url.
Fixes FS#71274
Signed-off-by: Allan McRae <allan@archlinux.org>
Since commit 08f4ae70, makepkg supports downloading from fossil.
However, the PKGBUILD man page was only partially updated to reflect
this change.
Signed-off-by: Allan McRae <allan@archlinux.org>
Comit 5151de30 tried to fix leaking memory when importing a key. However
key_search_keyserver() writes to the key passed in, making the original
uid and fingerprint unreachable, causing the new uid and fingerprint to
double free.
Fixes FS#71107
Signed-off-by: Allan McRae <allan@archlinux.org>
Around the same time retry events were added, there was a patch to pass
sig download events to the frontend. The retry code was not updated to
account for this.
Signed-off-by: morganamilo <morganamilo@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Some servers respond with error pages (e.g. 404.html) when a package is
not present. These were getting written to packages before moving onto
the next server. Reset the download progress on 400+ error conditions
to avoid this.
Signed-off-by: Allan McRae <allan@archlinux.org>
