Commit graph

6716 commits

Author SHA1 Message Date
Eli Schwartz
a0f4429e95 makepkg: use more schema.sh to clean the environment of special variables
Fixes "arch" and "checkdepends" never having been unset, fixes b2sums
(but not ${!b2sums_@}) being recently left out.
The "build" function used to be unset as well, explicitly unset it as a
function and do the same for other official functions as well.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 12:45:26 +10:00
Eli Schwartz
583f3122ce meson: fix build of executables with nonstandard libarchive path
The libarchive header is used in alpm.h, and several binaries include
this header. This is noticeably a problem when using e.g. the musl-gcc
compiler which does not include /usr/include by default, and thus the
build system reports:

...../lib/libalpm/alpm.h:35:10: fatal error: archive.h: No such file or directory

More commonly, this will result in compiling against potentially the
wrong headers, if the libarchive installation picked up by pkg-config is
different from the one with headers in /usr/include, and /usr/include is
in the -isystem path.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 10:15:52 +10:00
Eli Schwartz
6911904a03 makepkg: fix bash 5 compatibility when packaging symlinks to a directory
In commit b5191ea140 we moved to using
shell globbing to print package files for a couple of reasons including
reproducible packaging of .METADATA files.

Unfortunately, this only works reliably when the glob pattern does not
resolve to a symlinked directory due to a change in the bash 5.0
release. Note that the previous, desired behavior was rather to merely
refuse to recurse into symlinked directories, but due to an unrelated
issue, the symlink handling for globstar was reworked in a way that had
this side effect.
See https://lists.gnu.org/archive/html/bug-bash/2019-04/msg00015.html
for discussion; this may be fixed at some point, but bash 5.0 is broken
either way.

The appropriate way of handling this seems to be to use **/* to match
instead; this produces the same results on both bash 4 and bash 5, as
the ** matches any leading directory component (or none), and the *
matches any file, directory, or symlink to either one.

Fixes FS#62278

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 10:14:32 +10:00
Rikard Falkeborn
f2a7fb2b96 Fix clang 8 string-plus-int warnings
Clang 8 warns that adding a string to an integer does not append to
string. Indeed it doesn't, but that was not the intentetion. Use array
indexing as suggested by the compiler to silence the warning. There
should be no functional change.

Example of warning message:

    alpm.c:71:54: warning: adding 'int' to a string does not append to the string [-Wstring-plus-int]
            sprintf(hookdir, "%s%s", myhandle->root, SYSHOOKDIR + 1);
                                                     ~~~~~~~~~~~^~~
    alpm.c:71:54: note: use array indexing to silence this warning
            sprintf(hookdir, "%s%s", myhandle->root, SYSHOOKDIR + 1);
                                                                ^
                                                     &          [  ]
    1 warning generated.
2019-05-08 10:12:14 +10:00
Eli Schwartz
a8b3d1a62a makepkg: correctly handle hg sources with updates on a non-default branch
The "tip" ref actually signifies the most recently updated branch. hg
does not support a default branch named anything other than "default",
except by creating a "@" bookmark. The correct way to explicitly update
to the default clone ref, is therefore to use one of these, rather than
"tip".

Fixes FS#62092

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 10:10:57 +10:00
Eli Schwartz
915ac9ff9b libmakepkg: fix migration to schema.sh for integsums
One of the callers was changed to use known_hash_algos, one was not.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 10:05:23 +10:00
Eli Schwartz
d37e6d40a1 bash-completion: support file redirection completions
The current completions don't properly handle redirection operators, and
attempt to complete command completions rather than completing filenames
to redirect to.

bash-completion provides both _get_comp_words_by_ref and a higher-level
wrapper _init_completion, but the latter provides handling of redirection
operators, so switch to using that.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-25 11:15:26 +10:00
Allan McRae
7a598d0845 Use bug tracker to track bugs
The mailing list requires subscription.  So does the bug tracker,
but that is more obvious.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-20 09:58:16 +10:00
Eli Schwartz
9f5412d809 doc: generalize description of the bash nature of PKGBUILD functions
Currently this is scoped to the build() function, which is simply wrong
as it equally applies to any function. Simply moving the paragraphs up
to the main manpage section makes this clear.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:33:50 +10:00
Que Quotion
7b15918b65 Clarify that build_options only apply during build()
Signed-off-by: Que Quotion <quequotion@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:24:18 +10:00
Eli Schwartz
ee5f24e50c libmakepkg: fix reporting of invalid archive extensions in compress.sh
In commit 1825bd6716 this was split out
from makepkg, but the warning was not properly migrated; $ext did not
ever exist.

As a result, no matter what you did, the only possible warning was:

==> WARNING: '' is not a valid archive extension.

Fix to filter based on the presence of .tar in the argument, and
building the $ext variable for all checking and messaging purposes
within the function.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:12:44 +10:00
Eli Schwartz
35a0d5e744 makepkg: use "shared" git clones when checking out sources
In order to cache sources offline, makepkg creates *two* copies of every
git repo. This is a useful tradeoff for network time, but comes at the
cost of increased disk space.

Normally, git can smooth this over automagically. Whenever possible, git
objects are hardlinked to save space, but this does not work when
SRCDEST and BUILDDIR are on separate filesystems.

When the repo in question is both very large (linux.git for example is
2.2 GB) and crosses filesystem boundaries, this results in a lot of
extra disk space being used; the most likely scenario is where BUILDDIR
is a tmpfs for bonus ouch.

git(1) has a builtin feature which serves this case handily: the
--shared flag will create the info/alternates file instructing git to
not copy or hardlink or create objects/packs at all, but merely look for
them in an external location (that being the source of the clone).

The downside of using shared clones, is that if you modify and drop
commits from the original repo, or simply delete the whole repo
altogether, you break the copy. But we don't care about that here,
because

1) the BUILDDIR copy is meant to be a temporary copy strictly derived
   via PKGBUILD syntax from the SRCDEST, and must be able to be
   recreated at any time,
2) if the SRCDEST disappears, makepkg will redownload it, thus restoring
   the objects needed by the BUILDDIR clone,
3) if the user does non-default things like hacking on the BUILDDIR copy
   then deleting and re-cloning the SRCDEST may result in momentary
   breakage, but ultimately should be fine -- the unique objects they
   created will be stored in the BUILDDIR copy.

While it's theoretically possible that upstream will force-push to
overwrite the base tree from which makepkg is building (which they
should not do), *and* the user deleted their SRCDEST which they should
not do, *and* they saved work in makepkg's working directory which they
should not do either...
... this is an unlikely chain of events for which we should not care.

Using --shared is therefore helpful in immediately useful ways and IMHO
has no actual downsides; we should use it.

An alternative implementation would be to use worktrees. I've rejected
this since it is essentially the same as shared clones, except adding
additional restrictions on the branch namespace, and could potentially
break existing use cases such as manually handling the SRCDEST in order
to share repositories with normal working copies.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:09:00 +10:00
Eli Schwartz
0a72874734 build: check for gpgme with pkg-config before gpgme-config
gpgme in git master now supports pkg-config and with the next release we
can and should prefer its use. However, retain the legacy code that
enables building with older versions of gpgme, as a fallback.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:05:53 +10:00
Santiago Torres
226d2c1248 drop DU* config variables
Since DUFLAGS and DUPATH are not needed anymore remove them from the
source

Signed-off-by: Santiago Torres <santiago@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 12:07:15 +10:00
Eli Schwartz
bbfb75fd29 build: remove references to variable replacements from pacman-optimize
MODECMD and OWNERCMD are not used by pacman itself, so we don't need to
check for and replace them now that pacman-optimize is removed.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 12:04:51 +10:00
Santiago Torres
f26cb61cb6 Make makepkg compute sizes properly
Makepkg used to use du --apparent-size to compute the size of the
package. Unfortunately, this would result in different sizes depending
on the filesystem used (e.g., btrfs vs ext4), which would affect
reproducible builds. Use a wc-based approach to compute sizes

Signed-off-by: Santiago Torres <santiago@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 11:54:50 +10:00
Florian Wehner
c61cd050f9 Reformatting log timestamp to include time-zone
The time logged is currently given as localtime without any timezone
information. This is confusing in various scenarios.

Examples:
* If one is travelling across time-zones and the timestamps in the log
appear out of order.
* Comparing dates with `datediff` gives an offset by the time-zone

This patch would reformat the time-stamp to a full ISO-8601 version.
It includes the 'T' separating date and time including seconds.

Old: [2019-03-04 16:15]
New: [2019-03-04T16:15:45-05:00]

Signed-off-by: Florian Wehner <florian@whnr.de>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 11:53:30 +10:00
Allan McRae
c0e9be7973 Remove support for deltas from libalpm
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Allan McRae
e7bb0f8824 Make pacman forget deltas exist
Dummy callbacks are still present to prevent compiler warnings until
libalpm is delta free.

Also remove Delta parsing from pacman.conf.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Allan McRae
40391c564a Remove cleanupdelta
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Allan McRae
2883b869c7 Remove pkgdelta
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Allan McRae
9adb0d5b37 Remove delta support from repo-add
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Jelle van der Waa
377d47142f doc: add man page for pacman-conf
Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Jelle van der Waa
298bcf7986 doc: set rootdir correctly in man pages
If an alternative rootdir is specified in either meson or configure it's
not respected in the generated man pages.

Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Eli Schwartz
e03752e6ad makepkg: add new checksum algorithm via coreutils b2sum
coreutils 8.26 in December 2016 added this new hashing method which is
compatible with the existing md5sum and sha*sum tool usage, while using
the blake2 hash algorithm.

makepkg uses coreutils to provide source file integrity checks via
${integ}sum binaries and it makes sense to offer this as an additional
option.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
morganamilo
0113214db9 pacman: fix segfault when Usage is specified without a value
And extract all the common code to a macro.

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
Andrew Gregory
d197d8ab82 Sanitize file name received from Content-Disposition header
When installing a remote package with "pacman -U <url>", pacman renames
the downloaded package file to match the name given in the
Content-Disposition header. However, pacman does not sanitize this name,
which may contain slashes, before calling rename(). A malicious server (or
a network MitM if downloading over HTTP) can send a content-disposition
header to make pacman place the file anywhere in the filesystem,
potentially leading to arbitrary root code execution. Notably, this
bypasses pacman's package signature checking.

For example, a malicious package-hosting server (or a network
man-in-the-middle, if downloading over HTTP) could serve the following
header:

Content-Disposition: filename=../../../../../../usr/share/libalpm/hooks/evil.hook

and pacman would move the downloaded file to
/usr/share/libalpm/hooks/evil.hook. This invocation of "pacman -U" would
later fail, unable to find the downloaded package in the cache directory,
but the hook file would remain in place. The commands in the malicious
hook would then be run (as root) the next time any package is installed.

Discovered-by: Adam Suhl <asuhl@mit.edu>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-01 11:23:20 +10:00
Sever Oraz
adb961a88e Use standard, consistent units in the download progress.
Rather than use M/s which can be either MB or MiB, specify that it uses
MiB (consistent with the displayed total size).

Fixes FS#59201

Signed-off-by: Sever Oraz <severoraz@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21 14:55:49 +10:00
Eli Schwartz
ba7a41031c meson: fix static/shared split again
shared_library does not generate a sane pkg-config file because it
assumes we don't want dependencies.

Additionally, since we key off of buildstatic, when *not* using
buildstatic but attempting to build libalpm on its own as static using
-Ddefault_library=static, we are building and linking to a shared
libalpm anyway.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21 14:34:02 +10:00
Eli Schwartz
3c774252ef libmakepkg: migrate to schema.sh for integsums
We cannot use most of the arrays defined in schema.sh as srcinfo is dependent
on the order, but migrate the hashes for now.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21 14:26:59 +10:00
Eli Schwartz
656e851f0a makepkg: simplify run_pacman logic
We don't need to check the options twice, since it is the same check
both times. Instead, merge the conditionals.

As far as I can tell, the only reason the checks for:

- PACMAN_OPTS and
- whether to use sudo

were ever separated is due to the historic existence of --asroot, since
the second check included a check for (( ! ASROOT )) until it was
cleaned up in commit 61ba5c961e.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21 14:18:25 +10:00
Eli Schwartz
4b83bcfcee makepkg: implement locking for pacman commands
When pacman is run as root to do -S, -U, or -R, it would immediately
abort if pacman is not ready for use. Instead, poll the lockfile and
wait until it becomes available.

Implements FS#28840

Original-patch-by: Georges Dubus <georges.dubus@compiletoi.net>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21 14:16:16 +10:00
Eli Schwartz
477a66cd0e build: link vercmp with a static copy of libalpm
This has historically been the case in autotools since we want vercmp to
not break mid-transaction in an install script.

For convenience, we create libalpm.a and use this to optionally generate
libalpm.so (when not configured with -Dbuildstatic=true) as well as to
link any binary which explicitly wishes to be built statically "with
libalpm", but does not care where a function is defined. meson then
treats this correctly: it builds the object file only once for both
libraries, and the compiler strips out unused functionality from the
final static binary.

Currently the only binary which requires this is vercmp.

Fixes FS#61719

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-12 21:58:19 +10:00
Allan McRae
86004227d1 Add implicit fall through warning
Requires modification to our comment about fall through to match compilers
expectations.  Works for GCC and Clang.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-12 11:29:13 +10:00
Mark Ulrich
db102c67ef libalpm: prevent 301 redirect loop from hanging the process
If a mirror responds with a 301 redirect to itself, it will create an
infinite redirect loop. This will cause pacman to hang, unresponsive to
even a SIGINT. The result is pacman being unable to sync or
download any package from a particular repo if its current mirror
is stuck in a redirect loop. Setting libcurl's MAXREDIRS option
effectively prevents a redirect loop from hanging the process.

Signed-off-by: Mark Ulrich <mark.ulrich.86@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-07 09:51:01 +10:00
Allan McRae
714a1b4e4e I blame everyone but me for this
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-07 09:50:49 +10:00
morganamilo
04e77591d6 fix various typos
Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-04 10:17:30 +10:00
morganamilo
11bc315cdb libmakepkg: lint disallowed architecture specific variables
Variables such as 'pkgdesc_x86_64' are invalid, instead of ignoring them
raise an error.

This also disallows using 'any' as an architecture specific variable

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31 15:03:28 +10:00
morganamilo
14755df06f libmakepkg: lint disallowed variables in package()
makepkg will now error if disallowed variables are set inside of
the package function.

Disallowed variables are variables that do exist, like 'makedepends'
and 'pkgver' but can not be set inside of a package function.

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31 15:03:25 +10:00
morganamilo
75aae126c4 libmakepkg: centralise random arrays of pkgbuild variables
Refactor many of the different arrays of pkgbuild variables
into scripts/libmakepkg/util/schema.sh.in.

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31 10:02:37 +10:00
Allan McRae
6cf0548128 makepkg: use --unneeded for pacman call in remove_deps()
This patch was inspired by FS#32723 which asks makepkg to install makedepends
before depends.  The use case is to build a package depending on a virtual
package that is only provided by other packages (e.g. java-runtime in Arch
Linux), but wanting to build against a specific version.  Installing makedepends
first (but not at the same time as depends) would allow specifying the version
to build against, instead of pacman resolving to the default version when
installing depends.

It turns out, we can already achieve installing makedepends first by specifying
dependencies only in the package function (and making sure makedepends includes
everything needed). The only issue is that if we use makepkg to install the
built package with the --install flag and along with the --rmdeps flag, we will
try to remove any installed dependencies that are specified in the depends
array in the package function.  To counter this, we need to use the --unneeded
flag for the pacman call.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-30 13:39:35 +10:00
Chloe Kudryavtsev
776fbe1cde libmakepkg: Add lzip ext support
lzip is a lossless data compressor designed to replace gzip and bzip2 as
the standard general-purpose compressed format.
- add .lz (lzip) support to libmakepkg/util/compress.sh:compress_as
- add COMPRESSLZ to makepkg.conf.in
- document COMPRESSLZ
- document PKGEXT with `.tar.lz`

Signed-off-by: Chloe Kudryavtsev <toast@toastin.space>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-30 10:06:55 +10:00
morganamilo
5bd54fa0c6 libmakepkg: move checkdepends to the correct array
Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 11:06:11 +10:00
Eli Schwartz
1e040153bc libmakepkg: Implement extendable signature verification
Lookup the existence of matching functions for each protocol, and
fallback on the generic file handler. New verification protocols can
then be added via thirdparty libmakepkg drop-ins without requiring
modifications to verify_signature.sh

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 09:42:38 +10:00
Eli Schwartz
9c817b6549 libmakepkg: implement extendable source protocols
Lookup the existence of matching functions for each protocol, and
fallback on the generic file handler. New source protocols can then be
added via thirdparty libmakepkg drop-ins without requiring modifications
to source.sh

Fixes FS#49076

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 09:38:31 +10:00
Eli Schwartz
ac0e21a6df libmakepkg: optimize get_protocol to always return proto, not proto+uri
e.g. git+https:// is commonly used for git repositories cloned over
HTTPS, but we assume a proto with a plus in it is actually a protocol
followed by some URI handler. So we might as well simplify the return
value and not have to always add glob matching everywhere when checking
the proto in use.

This is required in order to use the proto directly in function calls,
which will be used in a followup patch.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 09:38:21 +10:00
Eli Schwartz
1b9e358f1d libmakepkg: make gettext replacements more templated
Making the undescore be translated is probably not something we need
translators to think about.

Additionally, a number of places which use the same text differ only by
the variable being referenced, so simplifying the string means we can
drop a redundant translation.

Bonus: we save a few bytes here and there. \o/
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 07:32:41 +10:00
morganamilo
4d1d857ef0 libalpm: fix minor typo in _alpm_db_usage_
Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-21 13:47:23 +10:00
Allan McRae
ff8c96fa7d Fix typo in pacman man page
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-21 13:20:47 +10:00
morganamilo
c15f7ae606 libmakepkg: disallow using 'any' with other arches
Error if the arch array contains any and any other values. This also
fixes a bug where the check for `$arch == 'any'` which only evaluated
the first value in the array, meaning the rest of the values would not
be linted.

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-16 11:23:17 +10:00