Commit graph

15 commits

Author SHA1 Message Date
Kevin Mihelich
99c5809bbf Support application/gzip MIME type in extraction
file 5.37 changed the gzip MIME type from application/x-gzip to
application/gzip, so support this when checking to extract source files.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28 10:48:42 +10:00
Eli Schwartz
b93dfa935f scripts: protect against unintended glob matching in [[ ]] RHS
The right-hand side of the [[ ... = ... ]] keyword is an exception to
the general rule that quoting is unnecessary with [[

This is usually not a problem, e.g. in libmakepkg, lint_one_pkgname will
already fail if pkgname has an asterisk, but it certainly doesn't hurt
to be "more proper" and go with the spec; it is more dangerous in
repo-add, which can get caught in an infinite loop instead of safely
asserting there is no package named 'foo*'.

Reported-by: Rafael Ascensão <rafa.almas@gmail.com>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 12:45:26 +10:00
Eli Schwartz
a8b3d1a62a makepkg: correctly handle hg sources with updates on a non-default branch
The "tip" ref actually signifies the most recently updated branch. hg
does not support a default branch named anything other than "default",
except by creating a "@" bookmark. The correct way to explicitly update
to the default clone ref, is therefore to use one of these, rather than
"tip".

Fixes FS#62092

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08 10:10:57 +10:00
Eli Schwartz
35a0d5e744 makepkg: use "shared" git clones when checking out sources
In order to cache sources offline, makepkg creates *two* copies of every
git repo. This is a useful tradeoff for network time, but comes at the
cost of increased disk space.

Normally, git can smooth this over automagically. Whenever possible, git
objects are hardlinked to save space, but this does not work when
SRCDEST and BUILDDIR are on separate filesystems.

When the repo in question is both very large (linux.git for example is
2.2 GB) and crosses filesystem boundaries, this results in a lot of
extra disk space being used; the most likely scenario is where BUILDDIR
is a tmpfs for bonus ouch.

git(1) has a builtin feature which serves this case handily: the
--shared flag will create the info/alternates file instructing git to
not copy or hardlink or create objects/packs at all, but merely look for
them in an external location (that being the source of the clone).

The downside of using shared clones, is that if you modify and drop
commits from the original repo, or simply delete the whole repo
altogether, you break the copy. But we don't care about that here,
because

1) the BUILDDIR copy is meant to be a temporary copy strictly derived
   via PKGBUILD syntax from the SRCDEST, and must be able to be
   recreated at any time,
2) if the SRCDEST disappears, makepkg will redownload it, thus restoring
   the objects needed by the BUILDDIR clone,
3) if the user does non-default things like hacking on the BUILDDIR copy
   then deleting and re-cloning the SRCDEST may result in momentary
   breakage, but ultimately should be fine -- the unique objects they
   created will be stored in the BUILDDIR copy.

While it's theoretically possible that upstream will force-push to
overwrite the base tree from which makepkg is building (which they
should not do), *and* the user deleted their SRCDEST which they should
not do, *and* they saved work in makepkg's working directory which they
should not do either...
... this is an unlikely chain of events for which we should not care.

Using --shared is therefore helpful in immediately useful ways and IMHO
has no actual downsides; we should use it.

An alternative implementation would be to use worktrees. I've rejected
this since it is essentially the same as shared clones, except adding
additional restrictions on the branch namespace, and could potentially
break existing use cases such as manually handling the SRCDEST in order
to share repositories with normal working copies.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:09:00 +10:00
Eli Schwartz
9c817b6549 libmakepkg: implement extendable source protocols
Lookup the existence of matching functions for each protocol, and
fallback on the generic file handler. New source protocols can then be
added via thirdparty libmakepkg drop-ins without requiring modifications
to source.sh

Fixes FS#49076

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22 09:38:31 +10:00
Dave Reisner
51db84750e Add meson.build files to build with meson
Provide both build systems in parallel for now, to ensure that we work
out all the differences between the two. Some time from now, we'll give
up on autotools.

Meson tends to be faster and probably easier to read/maintain. On my
machine, the full meson configure+build+install takes a little under
half as long as a similar autotools-based invocation.

Building with meson is a two step process. First, configure the build:

  meson build

Then, compile the project:

  ninja -C build

There's some mild differences in functionality between meson and
autotools.  specifically:

1) No singular update-po target. meson only generates individual
update-po targets for each textdomain (of which we have 3).  To make
this easier, there's a build-aux/update-po script which finds all
update-po targets and runs them.

2) No 'make dist' equivalent. Just run 'git archive' to generate a
suitable tarball for distribution.
2018-11-02 03:16:34 -04:00
Allan McRae
b6bb8cb7dc Update coyrights for 2018
make update-copyright OLD=2017 NEW=201

Signed-off-by: Allan McRae <allan@archlinux.org>
2018-03-14 13:31:31 +10:00
Eli Schwartz
39319c1860 libmakepkg: check for invalid tags in git
As per https://lists.archlinux.org/pipermail/arch-general/2017-July/043876.html
git doesn't check that the tag name matches what an annotated tag object
*thinks* it should be called. This is a bit of a theoretical attack and
some would argue that we should always use commits since upstream can
legitimately change a tag, but nevertheless this can result in a
downgrade attack if the git download transport was manipulated or the
upstream repository hacked.

So, check the tag blob to make sure the tag actually matches the name we
used for `git checkout`.

This really should be fixed in git itself, rather than forcing all
downstream users of git verify-tag to implement their own checks, but
the git developers disagree, see the discussion surrounding
https://public-inbox.org/git/xmqqk2hzldx8.fsf@gitster.mtv.corp.google.com/

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2017-09-13 14:20:35 +10:00
Eli Schwartz
9c604af0a0 libmakepkg: fix unsanitized source filenames
There were a couple places where filenames beginning with "-" were not
properly guarded against by passing them after "--". Some PKGBUILD
authors are crazy, but we still take those into account.

Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2017-07-06 12:55:25 +10:00
Eli Schwartz
eaa82b4d07 makepkg: Verify git signatures
A git repository is marked as signed if it contains the query "signed"
as defined by https://tools.ietf.org/html/rfc3986

Adds two utility functions in util/source.sh.in to extract fragments and
queries, and modifies source/git.sh.in to use them.

Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04 13:59:15 +10:00
Allan McRae
1a2d5bee3b Update copyright years
Signed-off-by: Allan McRae <allan@archlinux.org>
2017-01-04 13:59:14 +10:00
Allan McRae
4742f5929d Update copyright years for 2016
make update-copyright OLD=2015 NEW=2016

Signed-off-by: Allan McRae <allan@archlinux.org>
2016-01-04 13:27:08 +10:00
Florian Pritz
a671fa497c makepkg: Output full URL in case of download failure
Most entries in $sources contain variables so finding out why a URL
fails to download is hard because one has to manually replace the
variables when looking at the PKGBUILD. Simply output the full URL here
so that it can be easily seen what is wrong.

Old:
==> ERROR: Failure while downloading example-1.2.4.tar.gz

New:
==> ERROR: Failure while downloading http://example.org/releases/1.1/example-1.2.4.tar.gz

With the new format it is much more obvious that the directory name is
the culprint (1.1 vs 1.2) while the old one would not display that
information.

Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Allan McRae <allan@archlinux.org>
2015-12-15 20:34:30 +10:00
Allan McRae
5301d3fe8f Remove space before ellipses
Makes all use of ellipses consistent...

Signed-off-by: Allan McRae <allan@archlinux.org>
2015-10-19 13:48:56 +10:00
Allan McRae
3d4529335c libmakepkg: extract functions for source download and extraction
Signed-off-by: Allan McRae <allan@archlinux.org>
2015-05-19 23:43:00 +10:00