Commit graph

7029 commits

Author SHA1 Message Date
Jelle van der Waa
c294b7cb4c Update urls to not use www. for archlinux.org
www.archlinux.org now redirects to archlinux.org.

Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-08 10:14:33 +10:00
Jelle van der Waa
f09047f0b0 test: update mailing list url
The Arch Linux mailing lists are these days served from the lists
subdomain.

Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-08 10:14:30 +10:00
Eli Schwartz
2535611d6c makepkg: add PACMAN_AUTH configurable setting for sudo elevation
If specified, this will be used no matter what. If not, then we check if
sudo exists and use that, or else fall back on su.

Implements FS#32621

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-07 22:41:54 +10:00
Eli Schwartz
d6f8659443 ensure tests are parsed as TAP since they are, in fact, TAP
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-07 22:38:52 +10:00
Eli Schwartz
8aa2a48fc3 makepkg: fix the use of spaces in the localname:: component of sources
Broken via refactoring in commit aa6fe1160b
but for obvious reasons only one person in the last 9 years has ever
actually tried to do this. Still, it's technically correct to allow it.

Fixes FS#70254

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-07 22:38:34 +10:00
Andrew Gregory
9060058393 include retries and signatures in download count
Fixes: FS#69881

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-07 22:34:29 +10:00
Andrew Gregory
4bf7aa119d skip servers with too many errors
Keep track of errors from servers so that bad ones can be skipped once
a threshold is reached.  Key the error tracking off the hostname because
hosts may serve multiple repos under different url's and errors are
likely to be host-wide.

Implements: FS#29293.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-07 22:33:52 +10:00
Allan McRae
bdf6aa3fb7 libmakepkg: fix detection of source file names for debug packages
The current gcc build from git master give different output from
readelf:

gcc-10.2.0
$ readelf "hello" --debug-dump | grep hello
<11> DW_AT_name : (indirect string, offset: 0xbfc): hello.cpp

gcc-git
$ readelf "hello" --debug-dump | grep hello
<12> DW_AT_name : (indirect line string, offset: 0x0): hello.cpp

This causes the awk statement extracting the file name to fail as it
relied on the information being in the 8th field. Instead, extract
the information from the final field.

Fixes FS#70168

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-27 09:22:15 +10:00
Anatol Pomozov
1e60a5f006 Remove "total download" callback in favor of generic event callback
Total download callback called right before packages start downloaded.
But we already have an event for such event (ALPM_EVENT_PKG_RETRIEVE_START)
and it is naturally to use the event to pass information about expected
download size.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-25 11:39:03 +10:00
Eli Schwartz
9bf3d6a760 remove sed command lookup and hardcoding in edit-script.sh
We should not need to hardcode the path to sed as we simply don't care.
We don't check what kind of sed we found, and we're using the same one
we initially found on the PATH, which is surely still on the PATH.

At one point we did care to find the system copy of sed and hardcode it
in makepkg, because we also passed non-portable -i options to it and
makepkg needed to continue working on macOS even if some incompatible
GNU sed got installed afterward, elsewhere on the PATH. But this was
never relevant to the in-tree buildsystem script running sed.

In commit 3a814ee6bc we removed even that,
so we don't need to look it up at all.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-25 11:38:58 +10:00
Eli Schwartz
ede73f4d89 fix syntax error when running pacman-key --help
In commit 0f75ab3224 some unbalanced
quotes were added by the committer while editing an error message.

Fixes FS#69865

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-06 12:01:26 +10:00
Allan McRae
c118a61f62 Strip LTO symbols from distributed .a/.o files
GCC's LTO implementation emits bytecodes into .o files it generates.
These bytecodes are _not_ considered stable from one release of GCC
to the next. There we need to strip the LTO bytecode out of any .o
(and .a) file that gets installed into the package.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-03 14:35:52 +10:00
Allan McRae
4a0891f49d Add link time optimization support to makepkg
Add the 'lto' option to enable building with link time optimization
by adding '-flto' to both CFLAGS and CXXFLAGS.  The 'lto' option can
be specificed both in the PKGBUILD or by setting the default in
makepkg.conf.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-03 14:35:45 +10:00
Allan McRae
17f9911ffc Update copyright year
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-01 12:22:20 +10:00
Morten Linderud
0f75ab3224 pacman-key: --refresh-keys queries WKD before keyserver
With the recent outages of the keyservers there is a possibility of
`--refresh-keys` failing to fetch new keys. A lot of current key
distribution is done over WKD these days, and `pacman-key` has the
ability to use it for `--recv-key`.

There was a hope `gpg` would end up supporting WKD for the refresh
functionality, but this seems to be limited to expired keys fetched
through WKD. Since this functionality isn't yet available it makes sense
to stuff it into `pacman-key`.

The current implementation looks over all available keyids in the
keyring, attempts to fetch over WKD and then fall backs to keyservers if
no email has a valid WKD available. The downside of this approach is
that it takes a bit longer to refresh the keys, but it should be more
robust as the distribution should be providing their own WKDs.

Co-authored-by: Jonas Witschel <diabonas@archlinux.org>
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-24 23:05:48 +10:00
Morten Linderud
7587153a44 libmakepkg: Support zstd decompression for sources
This enables us to extract files in the source array and ensures that we
can decompress files if the uncompressed signature is served.

Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-24 22:51:14 +10:00
Emil Velikov
17d3da4777 Revert "ci: cache packages"
This reverts commit e348ba3881.

With the above commit we started caching the downloaded packages. Based
on some testing and, it saves ~30s in the "step_script" stage while
adding 18s for "Restoring/Saving cache". A net saving of ~10s.

With earlier commit, we no longer use an ancient image which also pulls
base-devel - thus the packages we have to download is minimal.

Now comparing the uncached "step_script", vs the cached one - it is
slowed by 2-3 seconds (1:01 -> 1:03), while we eliminate the 18s (and
growing) caching.

Tl:Dr: With up-to date image, package caching in not worth it - be that
time, disk or network wise.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:28:34 +10:00
Emil Velikov
cc1d23d333 ci: use official image
The archlinux/base have been deprecated. Since we depend on base-devel
simply use archlinux:base-devel

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:25:53 +10:00
Matti Niemenmaa
9e425cd1ea makepkg: Don't double-layer distcc on ccache
buildenv is set once for build() and a second time for package(). When
using both distcc and ccache, this lead to CCACHE_PREFIX="distcc distcc"
in package(), which breaks PKGBUILDs that execute the compiler in
package() because distcc complains:

    distcc[383041] (main) CRITICAL! distcc seems to have invoked itself
    recursively!

Avoid causing this error by only adding "distcc" to CCACHE_PREFIX if
it's not yet there.

Signed-off-by: Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:24:49 +10:00
Eli Schwartz
2b8d00b799 doxygen: fix missing parameter name
This was raising a warning during the build.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:24:18 +10:00
Eli Schwartz
3a23abb2ec support xattr when extracting packages
This permits storing the result of setcap during package() and applying
the resulting capabilities to the installed program. Formerly, it was
necessary to edit the binary after the fact (and thus dirty the file
according to -Qkk) by using an install scriptlet.

One problem that needs to be solved before this is useful, is preventing
the strip routine from destroying xattrs. This is taken care of in the
previous patch.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:23:45 +10:00
Eli Schwartz
88d054093c makepkg: don't let the strip routine mess up file attributes
It updates the stripped/objcopied file by creating a temp file,
chown/chmodding it, and replacing the original file. But upstream
binutils has CVE-worthy issues with this if running strip as root, and
some recent versions of strip don't play nicely with fakeroot.

Also, this has always destroyed xattrs. :/

Sidestep the issue by telling strip/objcopy to write to a temporary
file, and manually dump the contents of that back into the original
binary. Since the original binary is intact, albeit with different
contents, it retains its correct attributes in fakeroot.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:22:55 +10:00
morganamilo
ab549c8467 libalpm: fix total download reporting wrong total
When a package does not need to be downloaded but a signature does,
total download didn't count that towards the total.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:20:48 +10:00
Andrew Gregory
b40c5553b4 add --upgrade download tests
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:26 +10:00
Andrew Gregory
b82a975e76 pmtest: add ability to spawn simple http servers
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:16 +10:00
Andrew Gregory
ad84a572aa pmdb: add option to skip populating file:// server
Populating a file:// Server prevents any manually registered HTTP
servers from ever being used.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:01 +10:00
Andrew Gregory
743e041dbc add CACHE_FEXISTS and CACHE_FCONTENTS test rules
The existing CACHE_EXISTS rule takes a package, which is not suitable
for -U tests that need to be able to check for specific files.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:37 +10:00
Andrew Gregory
1fdf8c0076 gitlab-ci: print output and logs for failed tests
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:24 +10:00
Andrew Gregory
f9bc6c2b09 pactest.py: read options from PACTEST_OPTS
Makes it easier to pass options when not running pactest directly.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:14 +10:00
Andrew Gregory
84f9cb021e pacman: add -w to -U
Mostly for testing.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:04 +10:00
Pascal Ernster
05aefb8f82 pacman: correct length of ".files.sig" string
Running "pacman -Sc" deletes /var/lib/pacman/sync/*.files.sig due to a
wrong string length being used when checking filename suffixes in that
directory. In turn, these missing signature files cause both the
corresponding "*.files" files and their signatures being forcibly
re-downloaded again when "pacman -Sy" is executed.

Since official Arch Linux repos don't use signed database files yet, this
only affects people who use custom repos with signed database files, for
which they have set the "SigLevel" directive to "Required" or
"DatabaseRequired" in /etc/pacman.conf.

Fixes FS#66472

Signed-off-by: Pascal Ernster <pacman-dev@hardfalcon.net>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:17:37 +10:00
Andrew Gregory
1c4e079899 do not try to download an empty db list
Starting the download process, even if there is nothing to actually
download, causes an error when pacman is built without curl and has no
XferCommand defined (like our test suite).

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:39:37 +10:00
Andrew Gregory
79227dabe0 make alpm download config accessible without curl
Download-related config options are currently limited to builds with
curl.  This causes compilation errors when those options are used
without an appropriate guard which often goes unnoticed because we all
use curl.  Front-ends providing their own download callback may also
want to use these settings.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:39:10 +10:00
Andrew Gregory
8ed98023b4 sighandler: initialize sigaction fields
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:57 +10:00
Andrew Gregory
a493170420 fully free error list in check_keyring
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:34 +10:00
Andrew Gregory
5151de3011 plug memory leaks in _alpm_key_import
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:02 +10:00
Andrew Gregory
5d21b2d44c pacman-conf: free rootdir before replacing
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:37:24 +10:00
Andrew Gregory
4311cc3648 pacman: indicate --overwrite takes a glob
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:03:08 +10:00
Morten Linderud
8566cd9be9 pacman-key: Close msg string in generate_master_key
In 19980a61e9 there was a msg added which
didn't get the string closed.

Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Allan McRae
57a75dedb1 mailmap addition
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
morganamilo
e9cc95f372 Read targets from stdin before chrooting
Operations involving --sysroot and reading targets from stdin were
failing due to attempting to read targets after chrooting.  Move the
chroot to happen after targets are read.

Fixes FS#68630

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Emil Velikov
8f7415c41b pacman: add file checksum validation against mtree
With libarchive v3.5.0 we have API to fetch the digest from the mtree.
Use that to validate if the installed files are modified or not.

As always, a modified backup file will trigger a warning but will not
result in an actual failure.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Allan McRae
3f5125e47c doc: move missed sections from alpm_ to libalpm_
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:10:55 +10:00
Allan McRae
4cdbbcfc34 doc: reference all libalpm sections within libalpm man page
Also, fix minor typo.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 09:19:50 +10:00
Allan McRae
c1d23c0ff2 doc: rename libalpm-list man page for consistency
All other libalpm man pages are in the form libalpm_<name>.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 09:19:03 +10:00
morganamilo
8fda300e06 doc: change group names to libalpm_*
Also change the group's title to point to the group's man page.

This makes generated man pages be named libalpm_* which is more
consistent with what library man pages are usually called.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:18 +10:00
Emil Velikov
45a648961b libalpm: const annotate struct db_operations
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:14 +10:00
Emil Velikov
1d2dd9a128 libalpm: const annotate struct pkg_operations
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:04 +10:00
Allan McRae
f8614d8c5b get_file_pkg_ops: update comment 2021-01-09 00:14:58 +10:00
Emil Velikov
027d76b9f5 libalpm: introduce get_sync_pkg_ops() helper
Currently default_pkg_ops is accessed in two different ways.

There is get_file_pkg_ops (in be_package.c) creating a local once-off
'tweaked' copy. As well as load_pkg_for_entry (be_sync.c) which modifies
in-place and uses default_pkg_ops.

This seems rather misleading and fragile approach. Introduce a helper
for the second use-case so that default_pkg_ops is handled consistently
and essentially remains unchanged throughout.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:56 +10:00