This implements pkgtype into .PKGINFO. This is useful to ensure tools
parsing packages do not miss important context on the creation of the
package.
For instance discovering if a given .pkg.tar is a debug package, one
would have to do heuristics on the pkgdesc and "${pkgbase}-debug".
However both of these values are controlled by the packager.
Similarly, the heuristic for discovering split packages is if pkgbase
and pkgname differ, which can happen in any package as both values are
packager controlled.
This should ensure we don't need to rely on heuristics and instead
include the context of how the package was created.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
When trying to identify debug packages among other packages we
discovered that it's pkgname used in pkgdesc. Since pkgname can
sometimes be an array when building debug packages for a split package,
this could potentially include a pkgname that might not make sense
depending on the order of the array.
This patch simply uses pkgbase as it seems more correct.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
If a makepkg consumer uses a build wrapper to override compiler
flags this may lead to unreproducible packages as there is no way to
know which exact files were used for tooling that tries to reproduce
said package.
Instead of vendoring the whole used makepkg.conf file into buildinfo,
this patch adds two new properties to the .BUILDINFO file named
BUILDTOOL and BUILDTOOLVER which by default are simply makepkg's own
values. Downstream consumers may override those values: For example in
Arch Linux the devtools package can set those values and allow
reproducible builds tooling to fetch the appropriate makepkg.conf.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If specified, this will be used no matter what. If not, then we check if
sudo exists and use that, or else fall back on su.
Implements FS#32621
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We leaked fullver and pkgarch all over the place, and only conditionally
unset the other variables. Marking them local is a more proactive
solution.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit c6b04c0465 the signing stage was
moved out of fakeroot, and thus into the main control flow instead of
create_{,src}package
While the function for signing binary packages has logic to build
and gpg-sign multiple filenames, the source package never got this
special treatment. This would be fine, except it uses the standard
variables to set define the filename... like ${fullver}, which is
usually set beforehand, but in this case is not. We don't define fullver
globally as it's an internal implementation detail, except by sheer
coincidence if PKGVERFUNC is false due to improperly guarded code.
Result: source packages didn't end up signed. Instead, we raised a logic
error:
==> WARNING: Failed to sign package file somepackage-.src.tar.gz.
==> ERROR: An unknown error has occurred. Exiting...
Instead, let's just build the version inline, since we only use it once.
Reported-by: GaKu999 <g4ku999@gmail.com>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If multiple files match the pattern libfoo.so*, we want to check each of
them and see if they are shared libraries, and if so, if they have
versions attached.
But some packages can have both shared libraries and random files which
match the filename pattern. This is true at least for files in
/usr/share/gdb/auto-load/, which must match the filename they are paired
with, followed by "-gdb.py" (or some other gdb scripting ext), but
definitely don't contain a shared library. In this case, we don't want
to double-report the library in the generated provides.
It's also possible (probably) for a package to provide a versioned as
well as an unversioned shared library, but in such cases a single
provides entry is sufficient to cover both cases (and the libdepends
for the depending package would contain an unversioned dependency).
Solve this by keeping track of whether we have added a versioned soname
provides already, and then only adding a maximum of one unversioned
provides *iff* there isn't a versioned one yet.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If it's not listed by --list-secret-key we don't care if it has been
imported into your keyring, it's unusable. And you might not have a
private key at all in the no-keyid-specified case.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We pass this to gpg -u and this gpg option can accept a number of
different formats, not just the historical hexadecimal fingerprint we
assumed. We should not barf hard if a format is used which happens to
contain spaces.
This also fixes a validation bug. When we initially check if the desired
key is available, we don't quote spaces, so gpg goes ahead and treats
each space-separated string as a *different key* to search for,
returning partial matches, and returning success if at least one key is
found. But gpg --detach-sign -u will certainly not accept multiple keys!
Fixes FS#66949
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit 882e707e40 we changed message
output to go to stdout by default, unless it was an error. The plain()
function doesn't *look* like an error function, but in practice it was
-- it's used to continue multiline messages, and all in-tree uses were
for warning/error.
This is a problem both because we're sending output to the wrong place,
and because in some cases, we were performing error logging from a
function which would otherwise return a value to be captured in a
variable using command substution.
Fix this and straighten out the API by providing two functions: one for
continuing msg output, and one which wraps this by sending output to
stderr, for continuing error output.
Change all callers to use the second function.
This removed all information on dependency failures if the --syncdeps
flag was not used. A better approach is needed.
This reverts commit 4246a4cc4f.
Signed-off-by: Allan McRae <allan@archlinux.org>
While iterating over the provides array, the find call for locating a
shared library may result in listing multiple entries which by itself
does not produce a stable deterministic order and may vary depending on
the underlying filesystem.
To provide a stable listing and a reproducible .PKGINFO file the result
of find is piped to sort with a static LC_ALL=C localisation.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When pacman fails to satisfy deps, we might see output like the
following:
==> Making package: spiderfoot 3.0-1 (Thu 06 Feb 2020 12:45:10 PM CET)
==> Checking runtime dependencies...
==> Installing missing dependencies...
error: target not found: python-pygexf
==> ERROR: 'pacman' failed to install missing dependencies.
==> Missing dependencies:
-> python-dnspython
-> python-exifread
-> python-cherrypy
-> python-beautifulsoup4
-> python-netaddr
-> python-pysocks
-> python-ipwhois
-> python-ipaddress
-> python-phonenumbers
-> python-pypdf2
-> python-stem
-> python-whois
-> python-future
-> python-pyopenssl
-> python-docx
-> python-pptx
-> python-networkx
-> python-cryptography
-> python-secure
-> python-pygexf
-> python-adblockparser
==> Checking buildtime dependencies...
==> ERROR: Could not resolve all dependencies.
This is misleading -- the only truly missing package is python-pygexf,
but we fail to remove sync-able deps from our deplist and report
everything as if it were missing. Simply drop this extra reporting
because pacman already tells us exactly what couldn't be resolved.
Signed-off-by: Allan McRae <allan@archlinux.org>
Checksums arrays should be filled with values provided by upstream. We
currently have md5 set as an unsecure default, and are constantly asked to
change it to sha2. However, just changing the default to a stronger checksum
gives the user the impression that "makepkg -g" checksums are perfect.
Instead, change the default checksum to a CRC, to make it clear that any
checksum generated purely by "makepkg -g" is not ideal.
Signed-off-by: Allan McRae <allan@archlinux.org>
Extracting function variables containing arbitrarily scoped variables of
arbitrary nature is a disaster, but let's at least cover the common case
of using the actual '$pkgname' in an install/changelog file. It's the
odd case of actually being basically justified use of disambiguating
between the same variable used in multiple different split packages...
and also, --printsrcinfo already uses and overwrites the variable
'pkgname' in pkgbuild_extract_to_srcinfo, so this "works" in .SRCINFO
but doesn't work in .src.tar.gz
It doesn't work in lint_pkgbuild either, but in that case the problem is
being too permissive, not too restrictive -- we might end up checking
the same file twice, and printing that it is missing twice.
Fixes FS#64932
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
This value is needed for reproducible builds. The reason is because
$BUILDDIR changes its behavior depending on whether it is the same as
$startdir, and the result is that we cannot know whether $srcdir (the
path that is potentially embedded into the final package) is actually
"$BUILDDIR/src" or "$BUILDDIR/$pkgbase/src".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Reads PKGBUILD into an array and replaces the pkgver and pkgrel with
bash parameter substitution, then uses shell redirection to write to to
the file. Because shell redirection follows symlinks, this accomplishes
the same thing as the previous default of using the GNU-specific
--follow-symlinks sed flag.
Removes SEDPATH and SEDINPLACEFLAGS from the build systems as they are
not used elsewhere.
Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Fixes issue where users were allowed to run cleanup while running
--geninteg or --printsrcinfo or --packagelist, thus mixing invalid
responses into stdout.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Exclude files with hardlinks when cat'ing all the files, and do a second
run to look at each file with hardlinks, keep track of the ones we've
already operated on, and only cat each inode once. Then use "wc -c" to get
the size of all (deduplicated) files the same way we were already doing.
Original-patch-by: Ronan Pigott <rpigott@berkeley.edu>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
zipman:
read -r protects against those evil manpages whose filenames contain
backslash escapes, (muahahaha?)
IFS= read protects against filenames with:
- leading whitespace (but no one is actually stupid enough to configure
their MAN_DIRS=() in makepkg.conf with such silly directories, *right*?)
- trailing whitespace (but likewise, no one should be stupid enough to
write an uncompressed manpage for section '1 ' or something)
Also fix several other cases where we read filenames without protecting
against surrounding whitespace, or without using null-delimited
filenames when we could trivially do so.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Saving fflages breaks reproducible builds due to encoding information
specific to the filesystem that was used to build the package. This
information is not needed for packaging purposes anyway.
Including fflags also means that attempting to extract a package file as
root (or fakeroot) might result in angry warnings being printed to the
console by bsdtar, followed by a non-zero exit code, unless the user
remembers to use --no-fflags during extraction. This is unpleasant UI, even
if pacman itself won't care about these.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
remove_deps is called once, at the end of clean_up() before makepkg
exit. If remove_deps returns >0 (e.g. when pressing "n" in the resulting
prompt), the error is caught by the ERR signal handler. This in turns
sends SIGUSR1 to the process group, with resulting exit code 138.
In case remove_deps fails, this patch exits makepkg with E_REMOVE_DEPS
if there was no previous error (that is, EXIT_CODE equals E_OK).
Otherwise, makepkg exits with EXIT_CODE.
Signed-off-by: Allan McRae <allan@archlinux.org>
When running `makepkg -i` it may be necessary to first remove make- and
checkdepends before installing the built package - for example if they
conflict each other. This is the case for wireguard-arch which
makedepends and conflicts wireguard-dkms.
Signed-off-by: Erich Eckner <git@eckner.net>
Signed-off-by: Allan McRae <allan@archlinux.org>
Without the -f option to wait, we might move on and try to delete the
logpipe before the process is completed.
Signed-off-by: Allan McRae <allan@archlinux.org>
The logpipe fifo can remain when exiting on a non-error condition such
as recieving signals INT and USR1. This can be seen by doing either a
manual CTRL-C to interrupt the build or by sending a signal such as:
$ makepkg & sleep 5 ; kill -USR1 $!
Remove the fifo in all cases on script exit if it still exists.
Signed-off-by: Austin Lund <austin.lund@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
When scripting/automating around makepkg, it is sometimes desirable to
know how makepkg will be configured to operate. One example is the
archlinux devtools, which must forward select makepkg.conf variables
into a build chroot (for example PACKAGER) or use those variables itself
(for example {SRC,PKG,LOG}DEST).
The configuration file can be in up to 3 places, and should be capable
of being overridden via environment variables. It is sufficiently
complex to represent distinct functionality, and sufficiently useful to
merit easy accessibility in other scripts, therefore, let us move it
into a publicly exposed utility library.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Fixes "arch" and "checkdepends" never having been unset, fixes b2sums
(but not ${!b2sums_@}) being recently left out.
The "build" function used to be unset as well, explicitly unset it as a
function and do the same for other official functions as well.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit b5191ea140 we moved to using
shell globbing to print package files for a couple of reasons including
reproducible packaging of .METADATA files.
Unfortunately, this only works reliably when the glob pattern does not
resolve to a symlinked directory due to a change in the bash 5.0
release. Note that the previous, desired behavior was rather to merely
refuse to recurse into symlinked directories, but due to an unrelated
issue, the symlink handling for globstar was reworked in a way that had
this side effect.
See https://lists.gnu.org/archive/html/bug-bash/2019-04/msg00015.html
for discussion; this may be fixed at some point, but bash 5.0 is broken
either way.
The appropriate way of handling this seems to be to use **/* to match
instead; this produces the same results on both bash 4 and bash 5, as
the ** matches any leading directory component (or none), and the *
matches any file, directory, or symlink to either one.
Fixes FS#62278
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Makepkg used to use du --apparent-size to compute the size of the
package. Unfortunately, this would result in different sizes depending
on the filesystem used (e.g., btrfs vs ext4), which would affect
reproducible builds. Use a wc-based approach to compute sizes
Signed-off-by: Santiago Torres <santiago@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
coreutils 8.26 in December 2016 added this new hashing method which is
compatible with the existing md5sum and sha*sum tool usage, while using
the blake2 hash algorithm.
makepkg uses coreutils to provide source file integrity checks via
${integ}sum binaries and it makes sense to offer this as an additional
option.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We don't need to check the options twice, since it is the same check
both times. Instead, merge the conditionals.
As far as I can tell, the only reason the checks for:
- PACMAN_OPTS and
- whether to use sudo
were ever separated is due to the historic existence of --asroot, since
the second check included a check for (( ! ASROOT )) until it was
cleaned up in commit 61ba5c961e.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When pacman is run as root to do -S, -U, or -R, it would immediately
abort if pacman is not ready for use. Instead, poll the lockfile and
wait until it becomes available.
Implements FS#28840
Original-patch-by: Georges Dubus <georges.dubus@compiletoi.net>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Refactor many of the different arrays of pkgbuild variables
into scripts/libmakepkg/util/schema.sh.in.
Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
This patch was inspired by FS#32723 which asks makepkg to install makedepends
before depends. The use case is to build a package depending on a virtual
package that is only provided by other packages (e.g. java-runtime in Arch
Linux), but wanting to build against a specific version. Installing makedepends
first (but not at the same time as depends) would allow specifying the version
to build against, instead of pacman resolving to the default version when
installing depends.
It turns out, we can already achieve installing makedepends first by specifying
dependencies only in the package function (and making sure makedepends includes
everything needed). The only issue is that if we use makepkg to install the
built package with the --install flag and along with the --rmdeps flag, we will
try to remove any installed dependencies that are specified in the depends
array in the package function. To counter this, we need to use the --unneeded
flag for the pacman call.
Signed-off-by: Allan McRae <allan@archlinux.org>
This is not really an error with a "user function".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
paths can contain printf-unsafe chars, and printf -v is not somehow immune to this
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
There are state variables for everything else, and we use them to do
conditional checks on things, but it's currently a bit difficult to test
whether a package is being built, as it's the default action if *no*
options are specified.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Merge the similar code handling unsplit PKGBUILDs and individual
packages in a split PKGBUILD and make it a new function.
Signed-off-by: Allan McRae <allan@archlinux.org>
