Allow finding which mirror was used to fetch a file.
This makes it a bit easier to debug situations in which mirrors serve
bad files with HTTP 200.
Signed-off-by: Vladimir Panteleev <archlinux@cy.md>
When constructing an import question we never really used a proper gpg
key. We just zero initialize the key, set the uid and fingerprint, and
sent that to the front end.
Instead lets just give the import question a uid and fingerprint field.
Signed-off-by: Allan McRae <allan@archlinux.org>
When downloading in parallel, sort by package size so that the larger
packages are queued first to fully leverage parallelism.
Addresses FS#70172
Signed-off-by: Charlie Sale <softwaresale01@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Github and other sites redirect their downloads to a cdn. So the
download http://foo.org/myrepo.db may redirect to something like
https://cdn.foo.org/83749327439.
This then causes pacman to try and download the sig as
https://cdn.foo.org/83749327439.sig which is incorrect. In this case
pacman should append .sig to the original url.
However urls like https://archlinux.org/packages/community/x86_64/0ad/download/
Redirect to the mirror, so .sig has to appended after the redirects and
not before.
So we decide if we should append .sig on the original or effective url
based on if the effective url (minus the query part) has .db or .pkg in it.
Fixes FS#71148
---
v2: move variable decleration to start of block
v3: use dbext instead of db
archweb's download links all ended in /download. This cause all the temp
files to be named download.part. With parallel downloads this results in
multiple downloads to go to the same temp file and breaks the transaction.
Assign random temporary filenames to downloads from URLs that are either
missing a filename, or if the filename does not contain at least three
hyphens (as a well formed package filename does).
While this approach to determining when to use a temporary filename is
not 100% foolproof, it does keep nice looking download progress bar names
when a proper package filename is given. The only downside of not using
temporary files when provided with a filename with three or more hyphens
is URLs created specifically to bypass temporary filename usage can not
be downloaded in parallel. We probably do not want to download packages
from such URLs anyway.
Fixes FS#71464
Modified-by: Allan McRae (do not use temporary files for realish URLs)
Signed-off-by: Allan McRae <allan@archlinux.org>
If the original download redirects to to a different url then alpm would
try to name the sig file after the url instead of <original_file>.sig.
Instead force this naming scheme regardless of url.
Fixes FS#71274
Signed-off-by: Allan McRae <allan@archlinux.org>
Comit 5151de30 tried to fix leaking memory when importing a key. However
key_search_keyserver() writes to the key passed in, making the original
uid and fingerprint unreachable, causing the new uid and fingerprint to
double free.
Fixes FS#71107
Signed-off-by: Allan McRae <allan@archlinux.org>
Around the same time retry events were added, there was a patch to pass
sig download events to the frontend. The retry code was not updated to
account for this.
Signed-off-by: morganamilo <morganamilo@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
Some servers respond with error pages (e.g. 404.html) when a package is
not present. These were getting written to packages before moving onto
the next server. Reset the download progress on 400+ error conditions
to avoid this.
Signed-off-by: Allan McRae <allan@archlinux.org>
This partially fixes FS#67850
It fixes the case for -S'ing packages but not -U'ing urls.
pacman -S a/a b/b
resolving dependencies...
error: packages a and b have the same filename: a-1-1-any.pkg.tar.zst
error: failed to prepare transaction (duplicate filename)
Signed-off-by: Allan McRae <allan@archlinux.org>
When a download fails on one mirror a new download is started on the
next mirror. This causes the ammount downloaded to reset, confusing the
rate math and making it display a negative rate.
This is further complicated by the fact that a download may be resumed
from where it is or started over.
To account for this we alert the frontend that the download was
restarted. Pacman then starts the progress bar over.
Signed-off-by: Allan McRae <allan@archlinux.org>
Every alpm_option_set function clones the input so lets be more
consistent. Also this fixes servers not being sanatized.
Signed-off-by: Allan McRae <allan@archlinux.org>
With a repo using "SigLevel = Optional" and a package already downloaded
into the cache, download_files() returns 1 (via _alpm_download) to indicate
no files were downloaded. This causes installation of the package to
fail.
Explicitly check that download_files() returns -1 (error) rather than
non-zero.
Signed-off-by: Allan McRae <allan@archlinux.org>
Restore the prior indicator whether or not databases were up to date.
0 is used to indicate if *any* db was actually updated as callers are
more likely to care about that than if *all* dbs were updated.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
An extra break causes _alpm_download to break out of the payload loop as
soon as it sees a successful url download with XferCommand.
Fixes: FS#70608 - -U fails to download all files with XferCommand
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Our callbacks require front-ends to maintain state in order to provide
reasonable output. The new download callback in particular requires
much more complex state information to be saved. Without the ability to
provide context, state must be saved globally, which may not be possible
for all front-ends. Scripting language bindings in particular have no
way to register per-handle callbacks without some form of context.
Implements: FS#12721
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
This allows architecture to be multivalued. On x86-64 machines, this
could be something like:
Architecture = x86-64-v3 x86-64
We use the first specified Architecture value in mirrorlist $arch
variable replacement, as this is backwards-compatible and sane.
Original-patch-by: Dan McGee <dan@archlinux.org>
Patch-updated-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
On Linux, SIGPOLL is a valid signal, but on systems like FreeBSD, it is
not. This patch does a preprocessor check to see if SIGPOLL is available
or not.
Signed-off-by: Mark Weiman <mark.weiman@markzz.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
On Linux, signal.h is not required to have access to the signal
constants. On FreeBSD, this is not the case and requires signal.h to be
explicitly included.
This patch adds an include for signal.h in any source file that uses it.
Signed-off-by: Mark Weiman <mark.weiman@markzz.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Keep track of errors from servers so that bad ones can be skipped once
a threshold is reached. Key the error tracking off the hostname because
hosts may serve multiple repos under different url's and errors are
likely to be host-wide.
Implements: FS#29293.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Total download callback called right before packages start downloaded.
But we already have an event for such event (ALPM_EVENT_PKG_RETRIEVE_START)
and it is naturally to use the event to pass information about expected
download size.
Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
This permits storing the result of setcap during package() and applying
the resulting capabilities to the installed program. Formerly, it was
necessary to edit the binary after the fact (and thus dirty the file
according to -Qkk) by using an install scriptlet.
One problem that needs to be solved before this is useful, is preventing
the strip routine from destroying xattrs. This is taken care of in the
previous patch.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
When a package does not need to be downloaded but a signature does,
total download didn't count that towards the total.
Signed-off-by: Allan McRae <allan@archlinux.org>
Starting the download process, even if there is nothing to actually
download, causes an error when pacman is built without curl and has no
XferCommand defined (like our test suite).
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Download-related config options are currently limited to builds with
curl. This causes compilation errors when those options are used
without an appropriate guard which often goes unnoticed because we all
use curl. Front-ends providing their own download callback may also
want to use these settings.
Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
Also change the group's title to point to the group's man page.
This makes generated man pages be named libalpm_* which is more
consistent with what library man pages are usually called.
Signed-off-by: Allan McRae <allan@archlinux.org>
