bryson/pacman
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pacman/lib/libalpm/dload.h

62 lines
1.8 KiB
C
Raw Normal View History

Move download code out of server.c This is the first in what will be a series of patches to clean up the current download code in libalpm. Start by moving download code out of server.c and into download.c. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-22 21:36:11 -06:00
/*
* dload.h
*
build-aux/update-copyright 2019 2020 Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-10 10:46:03 +10:00
* Copyright (c) 2006-2020 Pacman Development Team <pacman-dev@archlinux.org>
Update copyright headers and messages Signed-off-by: Dan McGee <dan@archlinux.org>
2009-07-01 02:08:33 -05:00
* Copyright (c) 2002-2006 by Judd Vinet <jvinet@zeroflux.org>
Move download code out of server.c This is the first in what will be a series of patches to clean up the current download code in libalpm. Start by moving download code out of server.c and into download.c. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-22 21:36:11 -06:00
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
Do not #define _RESERVED_IDENTIFIERS Signed-off-by: Ivy Foster <ivy.foster@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-09-03 22:14:49 -05:00
#ifndef ALPM_DLOAD_H
#define ALPM_DLOAD_H
Move download code out of server.c This is the first in what will be a series of patches to clean up the current download code in libalpm. Start by moving download code out of server.c and into download.c. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-22 21:36:11 -06:00
#include "alpm_list.h"
#include "alpm.h"
absorb fileinfo struct into dload_payload This transitional struct becomes delicious noms for dload_payload. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-30 22:00:07 -04:00
struct dload_payload {
Rename pmhandle_t to alpm_handle_t Signed-off-by: Allan McRae <allan@archlinux.org>
2011-06-28 14:04:00 +10:00
alpm_handle_t *handle;
dload: add open_mode to payload struct This is a precursor to a following patch which will move the setting of options to a separate function. With the open mode as part of the struct, we can avoid modifying stack allocated variables. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19 13:42:53 -04:00
const char *tempfile_openmode;
Fix memory leak in download payload->remote_name In the sync code, we explicitly allocated a string for this field, while in the dload code itself it was filled in with a pointer to another string. This led to a memory leak in the sync download case. Make remote_name non-const and always explicitly allocate it. This patch ensures this as well as uses malloc + snprintf (rather than calloc) in several codepaths, and eliminates the only use of PATH_MAX in the download code. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28 02:56:36 -05:00
char *remote_name;
dload: move tempfile and destfile to payload struct These are private to the download operation already, so glob them onto the struct. This is an ugly rename patch, with the only logical change being that destfile and tempfile are now freed by the payload_free function. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19 11:56:37 -04:00
char *tempfile_name;
char *destfile_name;
dload: rename cd_filename to content_disp_name This is more in line with the menagerie of file name members that we now have on the payload struct. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-19 11:58:45 -04:00
char *content_disp_name;
lib/dload: prevent large file attacks This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-24 00:18:01 -04:00
char *fileurl;
Introduce alpm_dbs_update() function for parallel db updates This is an equivalent of alpm_db_update but for multiplexed (parallel) download. The difference is that this function accepts list of databases to update. And then ALPM internals download it in parallel if possible. Add a stub for _alpm_multi_download the function that will do parallel payloads downloads in the future. Introduce dload_payload->filepath field that contains url path to the file we download. It is like fileurl field but does not contain protocol/server part. The rationale for having this field is that with the curl multidownload the server retry logic is going to move to a curl callback. And the callback needs to be able to reconstruct the 'next' fileurl. One will be able to do it by getting the next server url from 'servers' list and then concat with filepath. Once the 'parallel download' refactoring is over 'fileurl' field will go away. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-26 13:19:59 -07:00
char *filepath; /* download URL path */
Minor struct member reordering for packing concerns Noticed using clang and `-Wpadded`. Signed-off-by: Dan McGee <dan@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2014-01-02 12:37:08 -06:00
alpm_list_t *servers;
long respcode;
Use off_t rather than double where possible Beautiful of libcurl to use floating point types for what are never fractional values. We can do better, and we usually want these values in their integer form anyway. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-24 19:48:03 -05:00
off_t initial_size;
off_t max_size;
move prevprogress onto payload handle This is a poor place for it, and it will likely move again in the future, but it's better to have it here than as a static variable. Initialization of this variable is now no longer necessary as its zeroed on creation of the payload struct. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-29 12:17:16 -04:00
off_t prevprogress;
absorb some _alpm_download params into payload struct Restore some sanity to the number of arguments passed to _alpm_download and curl_download_internal. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-30 21:19:25 -04:00
int force;
int allow_resume;
int errors_ok;
dload: add 'unlink_on_fail' to payload struct Let callers of _alpm_download state whether we should delete on fail, rather than inferring it from context. We still override this decision and always unlink when a temp file is used. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-18 14:17:30 -04:00
int unlink_on_fail;
dload: avoid renaming files downloaded via sync operations If the server redirects from ${repo}.db to ${repo}.db.tar.gz pacman gets this wrong: It saves to new filename and fails when accessing ${repo}.db. We need the remote filename only when downloading remote files with pacman's -U operation. This introduces a new field 'trust_remote_name' to payload. If set pacman downloads to the filename given by the server. The field trust_remote_name is set in alpm_fetch_pkgurl(). Fixes FS#36791 ([pacman] downloads to wrong filename with redirect). [dave: remove redundant assignment leading to memory leak] Signed-off-by: Allan McRae <allan@archlinux.org>
2013-09-05 16:08:23 +02:00
int trust_remote_name;
dload: ensure callback is always initialized once Frontends rely on an initialization call for setup between downloads. Checking for intialization after checking for a completed download can skip initialization in cases where files are small enough to be downloaded all at once (FS#56408). Relying on previous download size can result in multiple initializations if there are multiple non-transfer events prior to the download starting (fS#56468). Introduce a new cb_initialized variable to the payload struct and use it to ensure that the callback is initialized exactly once prior to any actual events. Fixes FS#56408, FS#56468 Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2017-12-16 12:41:11 -05:00
int cb_initialized;
Add missing #ifdef around cURL error code in download struct Thanks to Eduardo Tongson on the mailing list. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-14 07:38:58 -05:00
#ifdef HAVE_LIBCURL
Initialize cURL library on first use Rather than always initializing it on any handle creation. There are several frontend operations (search, info, etc.) that never need the download code, so spending time initializing this every single time is a bit silly. This makes it a bit more like the GPGME code init path. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28 12:06:36 -05:00
CURLcode curlerr; /* last error produced by curl */
Add missing #ifdef around cURL error code in download struct Thanks to Eduardo Tongson on the mailing list. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-10-14 07:38:58 -05:00
#endif
lib/dload: prevent large file attacks This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-24 00:18:01 -04:00
};
Refactor download payload reset and free This was done to squash a memory leak in the sync database download code. When we downloaded a database and then reused the payload struct, we could find ourselves calling get_fullpath() for the signatures and overwriting non-freed values we had left over from the database download. Refactor the payload_free function into a payload_reset function that we can call that does NOT free the payload itself, so we can reuse payload structs. This also allows us to move the payload to the stack in some call paths, relieving us of the need to alloc space. Signed-off-by: Dan McGee <dan@archlinux.org>
2011-09-28 12:58:43 -05:00
void _alpm_dload_payload_reset(struct dload_payload *payload);
Parametrise the different ways in which the payload is reset In FS#43434, Downloads which fail and are restarted on a different server will resume and may display a negative download speed. The payload's progress in libalpm was not properly reset which ultimately caused terminal noise because the line width calculation assumes positive download speeds. This patch fixes the incomplete reset of the payload by mimicing what be_sync.c:alpm_db_update() does over in sync.c:download_single_file(). The new dload.c:_alpm_dload_payload_reset_for_retry() extends beyond the current behavior by updating initial_size and prevprogress for this case. This makes pacman reset the progress properly in the next invocation of the callback and display positive download speeds. Fixes FS#43434. Signed-off-by: Martin Kühne <mysatyre@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-10-17 16:40:37 +02:00
void _alpm_dload_payload_reset_for_retry(struct dload_payload *payload);
lib/dload: prevent large file attacks This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-24 00:18:01 -04:00
absorb some _alpm_download params into payload struct Restore some sanity to the number of arguments passed to _alpm_download and curl_download_internal. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
2011-06-30 21:19:25 -04:00
int _alpm_download(struct dload_payload *payload, const char *localpath,
dload: mark final_url as const Signed-off-by: Allan McRae <allan@archlinux.org>
2014-10-16 15:36:55 +02:00
char **final_file, const char **final_url);
Move download code out of server.c This is the first in what will be a series of patches to clean up the current download code in libalpm. Start by moving download code out of server.c and into download.c. Signed-off-by: Dan McGee <dan@archlinux.org>
2008-01-22 21:36:11 -06:00
Introduce alpm_dbs_update() function for parallel db updates This is an equivalent of alpm_db_update but for multiplexed (parallel) download. The difference is that this function accepts list of databases to update. And then ALPM internals download it in parallel if possible. Add a stub for _alpm_multi_download the function that will do parallel payloads downloads in the future. Introduce dload_payload->filepath field that contains url path to the file we download. It is like fileurl field but does not contain protocol/server part. The rationale for having this field is that with the curl multidownload the server retry logic is going to move to a curl callback. And the callback needs to be able to reconstruct the 'next' fileurl. One will be able to do it by getting the next server url from 'servers' list and then concat with filepath. Once the 'parallel download' refactoring is over 'fileurl' field will go away. Signed-off-by: Anatol Pomozov <anatol.pomozov@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-03-26 13:19:59 -07:00
int _alpm_multi_download(alpm_handle_t *handle,
alpm_list_t *payloads /* struct dload_payload */,
const char *localpath);
Do not #define _RESERVED_IDENTIFIERS Signed-off-by: Ivy Foster <ivy.foster@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2016-09-03 22:14:49 -05:00
#endif /* ALPM_DLOAD_H */
Reference in a new issue Copy permalink