pacman/scripts/libmakepkg
Eli Schwartz 35a0d5e744 makepkg: use "shared" git clones when checking out sources
In order to cache sources offline, makepkg creates *two* copies of every
git repo. This is a useful tradeoff for network time, but comes at the
cost of increased disk space.

Normally, git can smooth this over automagically. Whenever possible, git
objects are hardlinked to save space, but this does not work when
SRCDEST and BUILDDIR are on separate filesystems.

When the repo in question is both very large (linux.git for example is
2.2 GB) and crosses filesystem boundaries, this results in a lot of
extra disk space being used; the most likely scenario is where BUILDDIR
is a tmpfs for bonus ouch.

git(1) has a builtin feature which serves this case handily: the
--shared flag will create the info/alternates file instructing git to
not copy or hardlink or create objects/packs at all, but merely look for
them in an external location (that being the source of the clone).

The downside of using shared clones, is that if you modify and drop
commits from the original repo, or simply delete the whole repo
altogether, you break the copy. But we don't care about that here,
because

1) the BUILDDIR copy is meant to be a temporary copy strictly derived
   via PKGBUILD syntax from the SRCDEST, and must be able to be
   recreated at any time,
2) if the SRCDEST disappears, makepkg will redownload it, thus restoring
   the objects needed by the BUILDDIR clone,
3) if the user does non-default things like hacking on the BUILDDIR copy
   then deleting and re-cloning the SRCDEST may result in momentary
   breakage, but ultimately should be fine -- the unique objects they
   created will be stored in the BUILDDIR copy.

While it's theoretically possible that upstream will force-push to
overwrite the base tree from which makepkg is building (which they
should not do), *and* the user deleted their SRCDEST which they should
not do, *and* they saved work in makepkg's working directory which they
should not do either...
... this is an unlikely chain of events for which we should not care.

Using --shared is therefore helpful in immediately useful ways and IMHO
has no actual downsides; we should use it.

An alternative implementation would be to use worktrees. I've rejected
this since it is essentially the same as shared clones, except adding
additional restrictions on the branch namespace, and could potentially
break existing use cases such as manually handling the SRCDEST in order
to share repositories with normal working copies.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19 14:09:00 +10:00
..
buildenv Split prepare_buildenv() to libmakepkg 2018-11-27 22:48:43 +10:00
executable fix various typos 2019-02-04 10:17:30 +10:00
integrity libmakepkg: centralise random arrays of pkgbuild variables 2019-01-31 10:02:37 +10:00
lint_config libmakepkg: add routine for linting $SOURCE_DATE_EPOCH 2019-01-10 17:01:30 +10:00
lint_package Add meson.build files to build with meson 2018-11-02 03:16:34 -04:00
lint_pkgbuild libmakepkg: lint disallowed architecture specific variables 2019-01-31 15:03:28 +10:00
source makepkg: use "shared" git clones when checking out sources 2019-03-19 14:09:00 +10:00
tidy Add meson.build files to build with meson 2018-11-02 03:16:34 -04:00
util makepkg: add new checksum algorithm via coreutils b2sum 2019-03-07 11:12:12 +10:00
.gitignore libmakepkg: fix .gitignore to simply ignore all generated .sh files 2017-12-07 15:37:20 +10:00
buildenv.sh.in Split prepare_buildenv() to libmakepkg 2018-11-27 22:48:43 +10:00
executable.sh.in libmakepkg/executable: don't rely on scoped value of $ret to flag outcomes 2018-12-04 17:22:27 +10:00
integrity.sh.in Update coyrights for 2018 2018-03-14 13:31:31 +10:00
lint_config.sh.in libmakepkg: Add basic rules to lint makepkg.conf variables 2018-03-15 14:34:13 +10:00
lint_package.sh.in Update coyrights for 2018 2018-03-14 13:31:31 +10:00
lint_pkgbuild.sh.in Update coyrights for 2018 2018-03-14 13:31:31 +10:00
meson.build meson: remove useless mkdir -p 2018-12-12 10:12:41 +10:00
source.sh.in libmakepkg: implement extendable source protocols 2019-01-22 09:38:31 +10:00
srcinfo.sh.in libmakepkg: migrate to schema.sh for integsums 2019-02-21 14:26:59 +10:00
tidy.sh.in Update coyrights for 2018 2018-03-14 13:31:31 +10:00
util.sh.in Update coyrights for 2018 2018-03-14 13:31:31 +10:00