Commit graph

7017 commits

Author SHA1 Message Date
Allan McRae
4a0891f49d Add link time optimization support to makepkg
Add the 'lto' option to enable building with link time optimization
by adding '-flto' to both CFLAGS and CXXFLAGS.  The 'lto' option can
be specificed both in the PKGBUILD or by setting the default in
makepkg.conf.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-03 14:35:45 +10:00
Allan McRae
17f9911ffc Update copyright year
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-01 12:22:20 +10:00
Morten Linderud
0f75ab3224 pacman-key: --refresh-keys queries WKD before keyserver
With the recent outages of the keyservers there is a possibility of
`--refresh-keys` failing to fetch new keys. A lot of current key
distribution is done over WKD these days, and `pacman-key` has the
ability to use it for `--recv-key`.

There was a hope `gpg` would end up supporting WKD for the refresh
functionality, but this seems to be limited to expired keys fetched
through WKD. Since this functionality isn't yet available it makes sense
to stuff it into `pacman-key`.

The current implementation looks over all available keyids in the
keyring, attempts to fetch over WKD and then fall backs to keyservers if
no email has a valid WKD available. The downside of this approach is
that it takes a bit longer to refresh the keys, but it should be more
robust as the distribution should be providing their own WKDs.

Co-authored-by: Jonas Witschel <diabonas@archlinux.org>
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-24 23:05:48 +10:00
Morten Linderud
7587153a44 libmakepkg: Support zstd decompression for sources
This enables us to extract files in the source array and ensures that we
can decompress files if the uncompressed signature is served.

Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-24 22:51:14 +10:00
Emil Velikov
17d3da4777 Revert "ci: cache packages"
This reverts commit e348ba3881.

With the above commit we started caching the downloaded packages. Based
on some testing and, it saves ~30s in the "step_script" stage while
adding 18s for "Restoring/Saving cache". A net saving of ~10s.

With earlier commit, we no longer use an ancient image which also pulls
base-devel - thus the packages we have to download is minimal.

Now comparing the uncached "step_script", vs the cached one - it is
slowed by 2-3 seconds (1:01 -> 1:03), while we eliminate the 18s (and
growing) caching.

Tl:Dr: With up-to date image, package caching in not worth it - be that
time, disk or network wise.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:28:34 +10:00
Emil Velikov
cc1d23d333 ci: use official image
The archlinux/base have been deprecated. Since we depend on base-devel
simply use archlinux:base-devel

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:25:53 +10:00
Matti Niemenmaa
9e425cd1ea makepkg: Don't double-layer distcc on ccache
buildenv is set once for build() and a second time for package(). When
using both distcc and ccache, this lead to CCACHE_PREFIX="distcc distcc"
in package(), which breaks PKGBUILDs that execute the compiler in
package() because distcc complains:

    distcc[383041] (main) CRITICAL! distcc seems to have invoked itself
    recursively!

Avoid causing this error by only adding "distcc" to CCACHE_PREFIX if
it's not yet there.

Signed-off-by: Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:24:49 +10:00
Eli Schwartz
2b8d00b799 doxygen: fix missing parameter name
This was raising a warning during the build.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:24:18 +10:00
Eli Schwartz
3a23abb2ec support xattr when extracting packages
This permits storing the result of setcap during package() and applying
the resulting capabilities to the installed program. Formerly, it was
necessary to edit the binary after the fact (and thus dirty the file
according to -Qkk) by using an install scriptlet.

One problem that needs to be solved before this is useful, is preventing
the strip routine from destroying xattrs. This is taken care of in the
previous patch.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:23:45 +10:00
Eli Schwartz
88d054093c makepkg: don't let the strip routine mess up file attributes
It updates the stripped/objcopied file by creating a temp file,
chown/chmodding it, and replacing the original file. But upstream
binutils has CVE-worthy issues with this if running strip as root, and
some recent versions of strip don't play nicely with fakeroot.

Also, this has always destroyed xattrs. :/

Sidestep the issue by telling strip/objcopy to write to a temporary
file, and manually dump the contents of that back into the original
binary. Since the original binary is intact, albeit with different
contents, it retains its correct attributes in fakeroot.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-02-08 14:22:55 +10:00
morganamilo
ab549c8467 libalpm: fix total download reporting wrong total
When a package does not need to be downloaded but a signature does,
total download didn't count that towards the total.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:20:48 +10:00
Andrew Gregory
b40c5553b4 add --upgrade download tests
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:26 +10:00
Andrew Gregory
b82a975e76 pmtest: add ability to spawn simple http servers
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:16 +10:00
Andrew Gregory
ad84a572aa pmdb: add option to skip populating file:// server
Populating a file:// Server prevents any manually registered HTTP
servers from ever being used.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:19:01 +10:00
Andrew Gregory
743e041dbc add CACHE_FEXISTS and CACHE_FCONTENTS test rules
The existing CACHE_EXISTS rule takes a package, which is not suitable
for -U tests that need to be able to check for specific files.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:37 +10:00
Andrew Gregory
1fdf8c0076 gitlab-ci: print output and logs for failed tests
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:24 +10:00
Andrew Gregory
f9bc6c2b09 pactest.py: read options from PACTEST_OPTS
Makes it easier to pass options when not running pactest directly.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:14 +10:00
Andrew Gregory
84f9cb021e pacman: add -w to -U
Mostly for testing.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:18:04 +10:00
Pascal Ernster
05aefb8f82 pacman: correct length of ".files.sig" string
Running "pacman -Sc" deletes /var/lib/pacman/sync/*.files.sig due to a
wrong string length being used when checking filename suffixes in that
directory. In turn, these missing signature files cause both the
corresponding "*.files" files and their signatures being forcibly
re-downloaded again when "pacman -Sy" is executed.

Since official Arch Linux repos don't use signed database files yet, this
only affects people who use custom repos with signed database files, for
which they have set the "SigLevel" directive to "Required" or
"DatabaseRequired" in /etc/pacman.conf.

Fixes FS#66472

Signed-off-by: Pascal Ernster <pacman-dev@hardfalcon.net>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-21 17:17:37 +10:00
Andrew Gregory
1c4e079899 do not try to download an empty db list
Starting the download process, even if there is nothing to actually
download, causes an error when pacman is built without curl and has no
XferCommand defined (like our test suite).

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:39:37 +10:00
Andrew Gregory
79227dabe0 make alpm download config accessible without curl
Download-related config options are currently limited to builds with
curl.  This causes compilation errors when those options are used
without an appropriate guard which often goes unnoticed because we all
use curl.  Front-ends providing their own download callback may also
want to use these settings.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:39:10 +10:00
Andrew Gregory
8ed98023b4 sighandler: initialize sigaction fields
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:57 +10:00
Andrew Gregory
a493170420 fully free error list in check_keyring
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:34 +10:00
Andrew Gregory
5151de3011 plug memory leaks in _alpm_key_import
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:38:02 +10:00
Andrew Gregory
5d21b2d44c pacman-conf: free rootdir before replacing
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:37:24 +10:00
Andrew Gregory
4311cc3648 pacman: indicate --overwrite takes a glob
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-19 12:03:08 +10:00
Morten Linderud
8566cd9be9 pacman-key: Close msg string in generate_master_key
In 19980a61e9 there was a msg added which
didn't get the string closed.

Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Allan McRae
57a75dedb1 mailmap addition
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
morganamilo
e9cc95f372 Read targets from stdin before chrooting
Operations involving --sysroot and reading targets from stdin were
failing due to attempting to read targets after chrooting.  Move the
chroot to happen after targets are read.

Fixes FS#68630

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Emil Velikov
8f7415c41b pacman: add file checksum validation against mtree
With libarchive v3.5.0 we have API to fetch the digest from the mtree.
Use that to validate if the installed files are modified or not.

As always, a modified backup file will trigger a warning but will not
result in an actual failure.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:11:06 +10:00
Allan McRae
3f5125e47c doc: move missed sections from alpm_ to libalpm_
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:10:55 +10:00
Allan McRae
4cdbbcfc34 doc: reference all libalpm sections within libalpm man page
Also, fix minor typo.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 09:19:50 +10:00
Allan McRae
c1d23c0ff2 doc: rename libalpm-list man page for consistency
All other libalpm man pages are in the form libalpm_<name>.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 09:19:03 +10:00
morganamilo
8fda300e06 doc: change group names to libalpm_*
Also change the group's title to point to the group's man page.

This makes generated man pages be named libalpm_* which is more
consistent with what library man pages are usually called.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:18 +10:00
Emil Velikov
45a648961b libalpm: const annotate struct db_operations
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:14 +10:00
Emil Velikov
1d2dd9a128 libalpm: const annotate struct pkg_operations
Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:04 +10:00
Allan McRae
f8614d8c5b get_file_pkg_ops: update comment 2021-01-09 00:14:58 +10:00
Emil Velikov
027d76b9f5 libalpm: introduce get_sync_pkg_ops() helper
Currently default_pkg_ops is accessed in two different ways.

There is get_file_pkg_ops (in be_package.c) creating a local once-off
'tweaked' copy. As well as load_pkg_for_entry (be_sync.c) which modifies
in-place and uses default_pkg_ops.

This seems rather misleading and fragile approach. Introduce a helper
for the second use-case so that default_pkg_ops is handled consistently
and essentially remains unchanged throughout.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:56 +10:00
morganamilo
4472ce55d7 pacman: total download show how many packages have been download
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:52 +10:00
Emil Velikov
b9ab7790d5 Remove unused SYMHIDDEN macros
The macro hasn't been used since 2007 with commit
7f7da2b5fc. Although it was still copied
over into alpm_list.c an year or so later with commit ca1a1871 ("More
cleanup to alpm_list")

Just remove all instances of it.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:50 +10:00
Emil Velikov
67a42b9549 meson: use hidden symbol visiblity by default
All the required public API is annotated with SYMEXPORT, so we can just
add the meson notation, to hide all the symbols by default.

Thus we no longer spill all the internal API into the global namespace.

This is effectively a regression from the autotools build, which used
hidden and internal for DARWIN and others respectively.

The use of hidden is considered sufficient, considering:
 - internal was introduced with commit 920b0d20 ("Update usage of gcc
   __attribute__ flags"), referencing the GCC manual and potential
   optimisations, although
 - the details about the optimisations or respective benefits are close
   to non-existent,
 - the code/data size of the binaries is identical across hidden and
   internal. While the latter produces slightly larger overall binaries.
 - Internal is not widely supported - missing on Darwin, the CMake build
   system lacks a wrapper (unlike for hidden)
 - Internal is not widely used in projects.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:42 +10:00
Emil Velikov
d53ba019f5 meson: pacman-conf add missing libcommon link
Currently, we are erroneously exporting all the symbols via the
libalpm.so. As such, the libcommon dependency is resolved.

The libalpm.so exports are about to be resolved shortly, yet that
exposed that pacman-conf is missing a link against libcommon.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:14:33 +10:00
morganamilo
793e2097a6 libalpm: pass the number of packages being downloaded in totaldlcb
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:12:32 +10:00
morganamilo
f5b373788f libalpm: don't use curl's deprecated functions
This bumps the minimun curl version from 7.32.0 to 7.55.0.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:12:19 +10:00
morganamilo
bc1591a0b8 pacman: clean filename for downlaods
The progress bar already did this. But the init event and up to date
message printed the full file name.  Unify these for consistency.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:12:08 +10:00
morganamilo
7cc8e0181f libalpm: remove useless if
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:11:44 +10:00
morganamilo
8e291e6f4b doc: fix typos
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-03 11:24:24 +10:00
Allan McRae
19980a61e9 pacman-key: warn about time taken for master key generation
Generating the pacman master key can take some time on systems
without enough entropy. Warn the user that the generation may
take some time.

Fixes FS#30286.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-03 11:19:21 +10:00
Eli Schwartz
a023565ed3 doc: make doxygen build from any directory
In the autotools build, it only built in-tree, from cwd = doc/ and
resolving doc/../lib/libalpm

In the meson build, this accidentally worked if cwd =
pacman/builddir/ and resolved to builddir/../lib/libalpm/

But... this should always have been configured with the actual path to
the inputs. So, we will now proceed to do so.

Fixes building man3 if your out of tree builddir doesn't happen to be a
direct subdirectory of the source root.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-29 13:15:40 +10:00
Emil Velikov
ccdd1e3fd9 Move hex_representation() to src/common
We'll reuse the function in pacman with a later commit.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-29 12:38:23 +10:00