Commit graph

476 commits

Author SHA1 Message Date
morganamilo
618fa4f675 libalpm: check for duplicate filenames
This partially fixes FS#67850

It fixes the case for -S'ing packages but not -U'ing urls.

pacman -S a/a b/b
resolving dependencies...
error: packages a and b have the same filename: a-1-1-any.pkg.tar.zst
error: failed to prepare transaction (duplicate filename)

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-05-10 07:58:44 +10:00
morganamilo
8bf17b29a2 libalpm: fix download rates becoming negative
When a download fails on one mirror a new download is started on the
next mirror. This causes the ammount downloaded to reset, confusing the
rate math and making it display a negative rate.

This is further complicated by the fact that a download may be resumed
from where it is or started over.

To account for this we alert the frontend that the download was
restarted. Pacman then starts the progress bar over.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-05-09 23:28:04 +10:00
morganamilo
4fead44e3c libalpm: clone data on alpm_db_set_servers
Every alpm_option_set function clones the input so lets be more
consistent. Also this fixes servers not being sanatized.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-05-09 22:54:20 +10:00
Andrew Gregory
eb1a63a516 alpm_db_update: indicate if dbs were up to date
Restore the prior indicator whether or not databases were up to date.
0 is used to indicate if *any* db was actually updated as callers are
more likely to care about that than if *all* dbs were updated.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
2021-05-01 12:08:14 +10:00
Andrew Gregory
e7fa35baa2 add front-end provided context to callbacks
Our callbacks require front-ends to maintain state in order to provide
reasonable output.  The new download callback in particular requires
much more complex state information to be saved.  Without the ability to
provide context, state must be saved globally, which may not be possible
for all front-ends.  Scripting language bindings in particular have no
way to register per-handle callbacks without some form of context.

Implements: FS#12721

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
2021-05-01 12:08:14 +10:00
Dan McGee
3179db108a Add support for multiple 'Architecture' values
This allows architecture to be multivalued. On x86-64 machines, this
could be something like:
    Architecture = x86-64-v3 x86-64

We use the first specified Architecture value in mirrorlist $arch
variable replacement, as this is backwards-compatible and sane.

Original-patch-by: Dan McGee <dan@archlinux.org>
Patch-updated-by: Allan McRae <allan@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-04-22 00:15:21 +10:00
Anatol Pomozov
1e60a5f006 Remove "total download" callback in favor of generic event callback
Total download callback called right before packages start downloaded.
But we already have an event for such event (ALPM_EVENT_PKG_RETRIEVE_START)
and it is naturally to use the event to pass information about expected
download size.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-25 11:39:03 +10:00
Allan McRae
17f9911ffc Update copyright year
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-03-01 12:22:20 +10:00
Allan McRae
3f5125e47c doc: move missed sections from alpm_ to libalpm_
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-11 11:10:55 +10:00
Allan McRae
4cdbbcfc34 doc: reference all libalpm sections within libalpm man page
Also, fix minor typo.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 09:19:50 +10:00
morganamilo
8fda300e06 doc: change group names to libalpm_*
Also change the group's title to point to the group's man page.

This makes generated man pages be named libalpm_* which is more
consistent with what library man pages are usually called.

Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:15:18 +10:00
morganamilo
793e2097a6 libalpm: pass the number of packages being downloaded in totaldlcb
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-09 00:12:32 +10:00
morganamilo
8e291e6f4b doc: fix typos
Signed-off-by: Allan McRae <allan@archlinux.org>
2021-01-03 11:24:24 +10:00
morganamilo
e67c306f91 doc: add extra documentation to the man page
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 12:01:58 +10:00
morganamilo
ea975c328d doc: rename alpm_api group to alpm
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 12:01:58 +10:00
morganamilo
1e8816f13d doc: move top level items into groups
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 12:01:58 +10:00
morganamilo
2a16434bfd doc: move alpm_api group to top of file
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 12:01:58 +10:00
morganamilo
4613527738 doc: document options
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 12:01:58 +10:00
morganamilo
50d2171ef0 doc: document misc
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:46 +10:00
morganamilo
a321e1f8a5 doc: document transactions
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:46 +10:00
morganamilo
3d60a829e1 doc: document files and groups
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:45 +10:00
morganamilo
5901ac9cb2 doc: document packages
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:45 +10:00
morganamilo
af7a1e834f doc: document logging
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:45 +10:00
morganamilo
55efa43f23 doc: document databases
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:45 +10:00
morganamilo
e9ac13776e doc: document callbacks
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:45 +10:00
morganamilo
bf26b6bbf7 doc: document depends
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:55:44 +10:00
morganamilo
5e9635317b doc: document signatures
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:51:07 +10:00
morganamilo
ab431f6810 doc: document handle
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:51:07 +10:00
morganamilo
49e33aad7c doc: document error
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:51:06 +10:00
morganamilo
48c8869339 doc: add doc header to alpm.h
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-12-21 11:49:33 +10:00
morganamilo
73e0d7dedc libalpm: add alpm_option_get_parallel_downloads
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-11-26 21:10:16 +10:00
morganamilo
75cfce9468 libalpm: clarify alpm_download_event_completed_t status
The comment makes it seem that the result itself is an error code. But
all it does is simply return -1 to indicate an error occured;

Signed-off-by: Allan McRae <allan@archlinux.org>
2020-11-26 21:07:48 +10:00
Anatol Pomozov
b01bcc7d3d Fallback to detached signatures during keyring check
Pacman has a 'key in keyring' verification step that makes sure the signatures
have a valid keyid. Currently pacman parses embedded package signatures only.

Add a fallback to detached signatures. If embedded signature is missing then it
tries to read corresponding *.sig file and get keyid from there.

Verification:
  debug: found cached pkg: /var/cache/pacman/pkg/glib-networking-2.64.3-1-x86_64.pkg.tar.zst
  debug: found detached signature /var/cache/pacman/pkg/glib-networking-2.64.3-1-x86_64.pkg.tar.zst.sig with size 310
  debug: found signature key: A5E9288C4FA415FA
  debug: looking up key A5E9288C4FA415FA locally
  debug: key lookup success, key exists

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-07-07 21:38:13 +10:00
Anatol Pomozov
16d98d6577 Convert '-U pkg1 pkg2' codepath to parallel download
Installing remote packages using its URL is an interesting case for ALPM
API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with
server mirror list. Thus _alpm_multi_download() should be able to
handle file download for payloads that either have 'fileurl' field
or pair of fields ('servers' and 'filepath') set.

Signature for alpm_fetch_pkgurl() has changed and it accepts an
output list that is populated with filepaths to fetched packages.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
2020-06-26 15:59:08 +10:00
Anatol Pomozov
22a58f5420 Swap alpm_db_update() implementation to multiplexed version
Now when all callers of the old alpm_db_update() function are gone we can
remove this implementation. And then rename alpm_dbs_update() function to
alpm_db_update().

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:39 +10:00
Anatol Pomozov
0346e0eef2 Convert download packages logic to multiplexed API
Create a list of dload_payloads and pass it to the new _alpm_multi_*
interface.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:39 +10:00
Anatol Pomozov
c78eb48d91 Extend download callback interface with start/complete events
With the previous download interface the callback uses the first progress
event as 'download has started' signal. Unfortunately it does not work with
up-to-date files that never receive 'download progress' events.
Up-to-date database messages are currently handled in sync_syncdbs()
after the sequential download is completed and a result from ALPM is
received. But this is not going to work with multiplexed download
interface that returns the result only after all files are completed.

Another problem with 'first progress event is the beginning of the
download' is that such events time are unpredictable. Thus the UI progress
bar order might differ from what has been passed by client to
alpm_dbs_update() function. We actually want to keep the dbs progress bars
in a strict order.

To help to solve the given problems extend the download callback to
allow 2 more events - download started and completed. 'Download started'
events appear in the same order as in the list given by a client.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:21 +10:00
Anatol Pomozov
64c4669f57 Introduce event types for start/end database list download
Multiplexed database/files downloads will use multiple progress bars.
The UI logic is quite complicated and printing error messages while
handling multiple progress bars is going to be challenging.

Instead we are going to save all ALPM error messages to a list and flush
it at the end of the download process. Use on_progress variable that
blocks error messages printing.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:21 +10:00
Anatol Pomozov
a8a1a1bb3e Introduce alpm_dbs_update() function for parallel db updates
This is an equivalent of alpm_db_update but for multiplexed (parallel)
download. The difference is that this function accepts list of
databases to update. And then ALPM internals download it in parallel if
possible.

Add a stub for _alpm_multi_download the function that will do parallel
payloads downloads in the future.

Introduce dload_payload->filepath field that contains url path to the
file we download. It is like fileurl field but does not contain
protocol/server part. The rationale for having this field is that with
the curl multidownload the server retry logic is going to move to a curl
callback. And the callback needs to be able to reconstruct the 'next'
fileurl. One will be able to do it by getting the next server url from
'servers' list and then concat with filepath. Once the 'parallel download'
refactoring is over 'fileurl' field will go away.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:21 +10:00
Anatol Pomozov
fe8e13341b Add config option to specify amount of parallel download streams
It includes pacman.conf new 'ParallelDownloads' option that
specifies how many concurrent downloads cURL starts in parallel.

Add alpm_option_set_parallel_downloads() ALPM function that
allows to set this config option programmatically.

Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-05-09 11:58:21 +10:00
Allan McRae
e76ec94083 build-aux/update-copyright 2019 2020
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-10 10:46:03 +10:00
morganamilo
d0c487d4dc Docs docs docs
libalpm: move docs from .c files into alpm.h And fix/expand some
along the way.

Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-28 10:46:27 +10:00
Allan McRae
1e23b45851 Fix documentation of alpm_mtree_next and remove libarchive exposure
The documentation of the return types of alpm_mtree_next was incorrect.
This extended into the relevant function in be_local.c.

Also, return explicit integer values, rather than the ARCHIVE_xxx values,
to avoid unnecessarily exposing frontends to libarchive internals (even
though it makes no functional difference).

Original-work-by: morganamilo <morganamilo@archlinux.org>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-26 11:37:32 +10:00
Allan McRae
55233b69c7 Improved documentation for alpm_db_search() parameters
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-22 09:35:31 +10:00
morganamilo
27f354a787 pacman+libalpm: handle search errors
Previously, pacman treated no matches and an error during search the
same.

To fix this, alpm_db_search now returns its status as an int and
instead takes the to be returned list as a param. Allowing front ends to
easily differentiate between errors and no matches.

Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-08 16:32:55 +10:00
Allan McRae
f37a3752b3 Update copyright years
make update-copyright OLD=2018 NEW=2019

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-23 22:06:54 +10:00
Allan McRae
c0e9be7973 Remove support for deltas from libalpm
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07 11:12:12 +10:00
morganamilo
4d1d857ef0 libalpm: fix minor typo in _alpm_db_usage_
Signed-off-by: morganamilo <morganamilo@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-21 13:47:23 +10:00
Allan McRae
e9d91a688d libalpm/sync.c: change alpm_sync_newversion() to alpm_sync_get_new_version()
The behaviour of "pacman -Qu" was very strange...  It would only consider
packages from repos with Usage = Search (or All), and ignore those with
Usage = Sync, Install or Upgrade.

This is because the function alpm_sync_newversion() used ALPM_DB_USAGE_SEARCH
for its filtering. Given this function is documented (at least in the source)
to "Check for new version of pkg in sync repos", I would expect that to look at
all repos. However, just changing this parameter, would result in a fairly
silent change in behaviour of this function. Instead, rename the function
and remove this filtering altogether.  Users of this function can filter
the dbs passed to this function to achieve their desired output.

Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-10 14:14:04 +10:00
Andrew Gregory
3726693612 add specific error for missing gpg support
"wrong or NULL argument passed" is a useless error for end users.

Fixes FS#60880.

Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com>
Signed-off-by: Allan McRae <allan@archlinux.org>
2018-11-27 22:48:43 +10:00