If a makepkg consumer uses a build wrapper to override compiler
flags this may lead to unreproducible packages as there is no way to
know which exact files were used for tooling that tries to reproduce
said package.
Instead of vendoring the whole used makepkg.conf file into buildinfo,
this patch adds two new properties to the .BUILDINFO file named
BUILDTOOL and BUILDTOOLVER which by default are simply makepkg's own
values. Downstream consumers may override those values: For example in
Arch Linux the devtools package can set those values and allow
reproducible builds tooling to fetch the appropriate makepkg.conf.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If specified, this will be used no matter what. If not, then we check if
sudo exists and use that, or else fall back on su.
Implements FS#32621
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Broken via refactoring in commit aa6fe1160b
but for obvious reasons only one person in the last 9 years has ever
actually tried to do this. Still, it's technically correct to allow it.
Fixes FS#70254
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
The current gcc build from git master give different output from
readelf:
gcc-10.2.0
$ readelf "hello" --debug-dump | grep hello
<11> DW_AT_name : (indirect string, offset: 0xbfc): hello.cpp
gcc-git
$ readelf "hello" --debug-dump | grep hello
<12> DW_AT_name : (indirect line string, offset: 0x0): hello.cpp
This causes the awk statement extracting the file name to fail as it
relied on the information being in the 8th field. Instead, extract
the information from the final field.
Fixes FS#70168
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit 0f75ab3224 some unbalanced
quotes were added by the committer while editing an error message.
Fixes FS#69865
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
GCC's LTO implementation emits bytecodes into .o files it generates.
These bytecodes are _not_ considered stable from one release of GCC
to the next. There we need to strip the LTO bytecode out of any .o
(and .a) file that gets installed into the package.
Signed-off-by: Allan McRae <allan@archlinux.org>
Add the 'lto' option to enable building with link time optimization
by adding '-flto' to both CFLAGS and CXXFLAGS. The 'lto' option can
be specificed both in the PKGBUILD or by setting the default in
makepkg.conf.
Signed-off-by: Allan McRae <allan@archlinux.org>
With the recent outages of the keyservers there is a possibility of
`--refresh-keys` failing to fetch new keys. A lot of current key
distribution is done over WKD these days, and `pacman-key` has the
ability to use it for `--recv-key`.
There was a hope `gpg` would end up supporting WKD for the refresh
functionality, but this seems to be limited to expired keys fetched
through WKD. Since this functionality isn't yet available it makes sense
to stuff it into `pacman-key`.
The current implementation looks over all available keyids in the
keyring, attempts to fetch over WKD and then fall backs to keyservers if
no email has a valid WKD available. The downside of this approach is
that it takes a bit longer to refresh the keys, but it should be more
robust as the distribution should be providing their own WKDs.
Co-authored-by: Jonas Witschel <diabonas@archlinux.org>
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
This enables us to extract files in the source array and ensures that we
can decompress files if the uncompressed signature is served.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
buildenv is set once for build() and a second time for package(). When
using both distcc and ccache, this lead to CCACHE_PREFIX="distcc distcc"
in package(), which breaks PKGBUILDs that execute the compiler in
package() because distcc complains:
distcc[383041] (main) CRITICAL! distcc seems to have invoked itself
recursively!
Avoid causing this error by only adding "distcc" to CCACHE_PREFIX if
it's not yet there.
Signed-off-by: Matti Niemenmaa <matti.niemenmaa+git@iki.fi>
Signed-off-by: Allan McRae <allan@archlinux.org>
It updates the stripped/objcopied file by creating a temp file,
chown/chmodding it, and replacing the original file. But upstream
binutils has CVE-worthy issues with this if running strip as root, and
some recent versions of strip don't play nicely with fakeroot.
Also, this has always destroyed xattrs. :/
Sidestep the issue by telling strip/objcopy to write to a temporary
file, and manually dump the contents of that back into the original
binary. Since the original binary is intact, albeit with different
contents, it retains its correct attributes in fakeroot.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In 19980a61e9 there was a msg added which
didn't get the string closed.
Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Allan McRae <allan@archlinux.org>
Generating the pacman master key can take some time on systems
without enough entropy. Warn the user that the generation may
take some time.
Fixes FS#30286.
Signed-off-by: Allan McRae <allan@archlinux.org>
With commit 74aacf4495 creating uncompressed .tar
packages fails.
-> Compressing package...
/usr/share/makepkg/util/compress.sh: line 70: COMPRESS.TAR[@]: invalid variable name
bsdtar: Write error
Empty the '$ext' variable for the '.tar' extension in get_compress_command() to
fix this. We would fallback to cat for 'tar' anyways.
Signed-off-by: Michael Straube <michael.straubej@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Fix typo in a comment in tidy_emptydirs().
Signed-off-by: Michael Straube <michael.straubej@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
We leaked fullver and pkgarch all over the place, and only conditionally
unset the other variables. Marking them local is a more proactive
solution.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit c6b04c0465 the signing stage was
moved out of fakeroot, and thus into the main control flow instead of
create_{,src}package
While the function for signing binary packages has logic to build
and gpg-sign multiple filenames, the source package never got this
special treatment. This would be fine, except it uses the standard
variables to set define the filename... like ${fullver}, which is
usually set beforehand, but in this case is not. We don't define fullver
globally as it's an internal implementation detail, except by sheer
coincidence if PKGVERFUNC is false due to improperly guarded code.
Result: source packages didn't end up signed. Instead, we raised a logic
error:
==> WARNING: Failed to sign package file somepackage-.src.tar.gz.
==> ERROR: An unknown error has occurred. Exiting...
Instead, let's just build the version inline, since we only use it once.
Reported-by: GaKu999 <g4ku999@gmail.com>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If multiple files match the pattern libfoo.so*, we want to check each of
them and see if they are shared libraries, and if so, if they have
versions attached.
But some packages can have both shared libraries and random files which
match the filename pattern. This is true at least for files in
/usr/share/gdb/auto-load/, which must match the filename they are paired
with, followed by "-gdb.py" (or some other gdb scripting ext), but
definitely don't contain a shared library. In this case, we don't want
to double-report the library in the generated provides.
It's also possible (probably) for a package to provide a versioned as
well as an unversioned shared library, but in such cases a single
provides entry is sufficient to cover both cases (and the libdepends
for the depending package would contain an unversioned dependency).
Solve this by keeping track of whether we have added a versioned soname
provides already, and then only adding a maximum of one unversioned
provides *iff* there isn't a versioned one yet.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Currently the list of supported formats for an archive, is maintained in
two places. And repo-add does not actually get updated. :(
In the process, remove some of the logical duplication when calling
bsdtar/compress_as.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
get_compression_command() can now be used to do upfront checks for
whether a given extension is known to do something successfully. This is
useful when writing tools in which an unknown compression type is a
fatal error.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each
section is concluded with an empty line. This means that at the end of
the file, an empty line remains. This is considered a trailing
whitespace error. In fact, `git diff --check` will warn about this,
saying "new blank line at EOF."
Instead of closing each section off with an empty line, use the empty
line to separate sections, omitting the empty line at the end of the
file.
Signed-off-by: Denton Liu <liu.denton@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
If it's not listed by --list-secret-key we don't care if it has been
imported into your keyring, it's unusable. And you might not have a
private key at all in the no-keyid-specified case.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We pass this to gpg -u and this gpg option can accept a number of
different formats, not just the historical hexadecimal fingerprint we
assumed. We should not barf hard if a format is used which happens to
contain spaces.
This also fixes a validation bug. When we initially check if the desired
key is available, we don't quote spaces, so gpg goes ahead and treats
each space-separated string as a *different key* to search for,
returning partial matches, and returning success if at least one key is
found. But gpg --detach-sign -u will certainly not accept multiple keys!
Fixes FS#66949
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit 882e707e40 we changed message
output to go to stdout by default, unless it was an error. The plain()
function doesn't *look* like an error function, but in practice it was
-- it's used to continue multiline messages, and all in-tree uses were
for warning/error.
This is a problem both because we're sending output to the wrong place,
and because in some cases, we were performing error logging from a
function which would otherwise return a value to be captured in a
variable using command substution.
Fix this and straighten out the API by providing two functions: one for
continuing msg output, and one which wraps this by sending output to
stderr, for continuing error output.
Change all callers to use the second function.
This was broken in commit 882e707e40,
which changed 'plain()' messages to go to stdout, which was then
captured as the download client in question: cmdline=("Aborting...").
The result was a very confusing error message e.g.
/usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found
or with makepkg --nocolor:
/usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found
The problem here is that we checked to see if an asynchronous subshell,
in our case <(...), failed, by checking if its captured stdout is
non-empty. Which is terrible, and also a limitation of old bash. But
bash 4.4 can use wait $! to retrieve the return value of an asynchronous
subshell. Now we target that as our minimum, we can sanely handle errors
in such functions.
Losing error messages on stdout by capturing them in a variable instead
of printing them, continues to be a problem, but this will be fixed
systematically in a later commit.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
If something like source=(..."#commit=") is used, e.g. due to failed
variable expansion, we try to check out an empty refspec as nothing at
all, and end up just running "git checkout". This happens because we
fail at variable expansion too -- so let's quote our variables properly
and make sure git sees this as an empty refspec, so it can error out.
Also make sure it is interpreted as a ref instead of a path.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
This removed all information on dependency failures if the --syncdeps
flag was not used. A better approach is needed.
This reverts commit 4246a4cc4f.
Signed-off-by: Allan McRae <allan@archlinux.org>
It's either a waste of work, or triggers edge cases in some packages
(like coreutils-8.31) where the source file is readonly and cp gets a
permission denied error trying to overwrite it with an identical copy of
itself.
Also while we are at it, make the variable names be something readable,
because I could barely tell what this was doing while editing it.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
While iterating over the provides array, the find call for locating a
shared library may result in listing multiple entries which by itself
does not produce a stable deterministic order and may vary depending on
the underlying filesystem.
To provide a stable listing and a reproducible .PKGINFO file the result
of find is piped to sort with a static LC_ALL=C localisation.
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When pacman fails to satisfy deps, we might see output like the
following:
==> Making package: spiderfoot 3.0-1 (Thu 06 Feb 2020 12:45:10 PM CET)
==> Checking runtime dependencies...
==> Installing missing dependencies...
error: target not found: python-pygexf
==> ERROR: 'pacman' failed to install missing dependencies.
==> Missing dependencies:
-> python-dnspython
-> python-exifread
-> python-cherrypy
-> python-beautifulsoup4
-> python-netaddr
-> python-pysocks
-> python-ipwhois
-> python-ipaddress
-> python-phonenumbers
-> python-pypdf2
-> python-stem
-> python-whois
-> python-future
-> python-pyopenssl
-> python-docx
-> python-pptx
-> python-networkx
-> python-cryptography
-> python-secure
-> python-pygexf
-> python-adblockparser
==> Checking buildtime dependencies...
==> ERROR: Could not resolve all dependencies.
This is misleading -- the only truly missing package is python-pygexf,
but we fail to remove sync-able deps from our deplist and report
everything as if it were missing. Simply drop this extra reporting
because pacman already tells us exactly what couldn't be resolved.
Signed-off-by: Allan McRae <allan@archlinux.org>
Checksums arrays should be filled with values provided by upstream. We
currently have md5 set as an unsecure default, and are constantly asked to
change it to sha2. However, just changing the default to a stronger checksum
gives the user the impression that "makepkg -g" checksums are perfect.
Instead, change the default checksum to a CRC, to make it clear that any
checksum generated purely by "makepkg -g" is not ideal.
Signed-off-by: Allan McRae <allan@archlinux.org>
Extracting function variables containing arbitrarily scoped variables of
arbitrary nature is a disaster, but let's at least cover the common case
of using the actual '$pkgname' in an install/changelog file. It's the
odd case of actually being basically justified use of disambiguating
between the same variable used in multiple different split packages...
and also, --printsrcinfo already uses and overwrites the variable
'pkgname' in pkgbuild_extract_to_srcinfo, so this "works" in .SRCINFO
but doesn't work in .src.tar.gz
It doesn't work in lint_pkgbuild either, but in that case the problem is
being too permissive, not too restrictive -- we might end up checking
the same file twice, and printing that it is missing twice.
Fixes FS#64932
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
This value is needed for reproducible builds. The reason is because
$BUILDDIR changes its behavior depending on whether it is the same as
$startdir, and the result is that we cannot know whether $srcdir (the
path that is potentially embedded into the final package) is actually
"$BUILDDIR/src" or "$BUILDDIR/$pkgbase/src".
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
We now generate the scripts using their real name, install them using
meson's builtin facility instead of an install_script, and generate the
wrapper scripts in the root of the build directory, instead of a
subdirectory.
This gets us closer to resolving FS#64394.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
In commit 9c817b6549 we made these sources
extendable, and heuristically determined the correct extraction
functions to use. But our fallback for protos that didn't have an exact
extract_* function didn't take into account that 'extract_file' matches
an actual proto... so we passed the netfile in while the function
expected a file.
Solution: the function should expect a netfile too, thereby allowing us
to delay an attempted resolution of netfile -> file, to the one case
where it is actually used. This makes us slightly more efficient in the
non-file case, makes our functions a bit more consistent, and makes
file:// extraction work again.
Fixes FS#64648
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Redirect file to stdin so wc -c doesn't print a file name that needs to
be stripped.
Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Now that library/ is fully gone, we don't need this anymore.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Reads PKGBUILD into an array and replaces the pkgver and pkgrel with
bash parameter substitution, then uses shell redirection to write to to
the file. Because shell redirection follows symlinks, this accomplishes
the same thing as the previous default of using the GNU-specific
--follow-symlinks sed flag.
Removes SEDPATH and SEDINPLACEFLAGS from the build systems as they are
not used elsewhere.
Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Previously parseopts checked if there was an argument by checking
that the string was non-empty, resulting in empty arguments being
incorrectly considered non-existent. This change makes parseopts check
if arguments exist at all, rather than checking that they are non-empty
Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Added two new functions, key_is_lsigned() and key_is_revoked()
that check whether a key has been locally signed or revoked
respectively during --populate. If the key is already signed
or revoked, it is quietly ignored.
Suggested-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Matthew Sexton <wsdmatty@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
