Commit Graph

36 Commits

Author SHA1 Message Date
ed
db33d68d42 zip-download: eagerly 64bit data-descriptors; closes #155
this avoids a false-positive in the info-zip unzip zipbomb detector.

unfortunately,

* now impossible to extract large (4 GiB) zipfiles using old software
   (WinXP, macos 10.12)

* now less viable to stream download-as-zip into a zipfile unpacker
   (please use download-as-tar for that purpose)

context:

the zipfile specification (APPNOTE.TXT) is slightly ambiguous as to when
data-descriptor (0x504b0708) filesize-fields change from 32bit to 64bit;
both copyparty and libarchive independently made the same interpretation
that this is only when the local header is zip64, AND the size-fields
are both 0xFFFFFFFF. This makes sense because the data descriptor is
only necessary when that particular file-to-be-added exceeds 4 GiB,
and/or when the crc32 is not known ahead of time.

another interpretation, seen in an early version of the patchset
to fix CVE-2019-13232 (zip-bombs) in the info-zip unzip command,
believes the only requirement is that the local header is zip64.

in many linux distributions, the unzip command would thus fail on
zipfiles created by copyparty, since they (by default) satisfy
the three requirements to hit the zipbomb false-positive:

* total filesize exceeds 4 GiB, and...
* a mix of regular (32bit) and zip64 entries, and...
* streaming-mode zipfile (not made with ?zip=crc)

this issue no longer exists in a more recent version of that patchset,
https://github.com/madler/unzip/commit/af0d07f95809653b
but this fix has not yet made it into most linux distros
2025-04-17 18:52:47 +00:00
ed
57a56073d8 use zlib-ng when available
download-as-tar-gz becomes 2.4x faster in docker

segfaults on windows, so don't use it there

does not affect fedora or gentoo,
since zlib-ng is already system-default on those

also adds a global-option to write list of successful
binds to a textfile, for automation / smoketest purposes
2025-03-23 20:15:21 +00:00
ed
ce3cab0295 v1.16.0 2024-11-10 19:32:37 +00:00
ed
19a5985f29 allow uploading logues; closes #100 2024-10-02 23:16:59 +00:00
ed
d5c9c8ebbd make it 5% faster 2024-07-31 17:51:53 +00:00
ed
5919607ad0 sanitize fs-paths in archive error summary
also gets rid of a dumb debug print i forgot
2024-05-30 23:55:37 +00:00
ed
2b24c50eb7 add option --iobuf (file r/w buffersize):
the default (256 KiB) appears optimal in the most popular scenario
(linux host with storage on local physical disk, usually NVMe)

was previously a mix of 64 and 512 KiB;
now the same value is enforced everywhere

download-as-tar is now 20% faster with the default value
2024-03-23 16:17:40 +00:00
ed
6420c4bd03 up to 2.6x faster download-as-zip
when there's lots of files, and especially small ones
and also reduces cpu load by at least 15%
2023-09-05 22:57:03 +00:00
ed
767696185b add ?tar=gz, ?tar=bz2, ?tar=xz with optional level;
defaults are ?tar=gz:3, ?tar=bz2:9, ?tar=xz:1
2023-09-01 23:44:10 +00:00
ed
d8dfc4ccb2 support davfs2 LOCK (uploads) + misc windows support + logue filtering 2023-01-31 18:53:38 +00:00
ed
db194ab519 support location-based rproxy 2022-12-10 23:43:31 +00:00
ed
2fbdc0a85e misc fixes / cleanup 2022-12-02 23:42:46 +00:00
ed
edad3246e0 make pylance happier 2022-10-29 20:40:25 +00:00
ed
2fb1e6dab8 mute exception on zip abort 2022-07-25 01:20:38 +02:00
ed
4a76663fb2 ensure free disk space 2022-07-17 22:33:08 +02:00
ed
438384425a add types, isort, errorhandling 2022-06-16 01:07:15 +02:00
ed
fe73f2d579 cleanup 2022-06-07 23:08:43 +02:00
ed
c20b7dac3d ah whatever, still 16 years left 2022-05-15 17:23:52 +02:00
ed
1f779d5094 zip: add ntfs and unix extensions for utc time 2022-05-15 16:13:49 +02:00
ed
715401ca8e fix timezone in search, zipfiles, fuse 2022-05-15 13:51:44 +02:00
ed
fa12c81a03 zip-download files older than 1980-01-01 2022-03-24 01:31:50 +01:00
ed
d020527c6f centralize mojibake support stuff 2021-07-24 21:56:55 +02:00
ed
8837c8f822 print zip/tar errors to log 2021-07-12 00:47:22 +02:00
ed
bac301ed66 get rid of iffy default-args 2021-07-12 00:15:13 +02:00
ed
e041a2b197 fix centos7 support 2021-07-08 23:35:28 +02:00
ed
79c7639aaf haha memes 2021-06-08 21:10:25 +02:00
ed
26e18ae800 disallow uploading logues 2021-05-12 23:22:43 +02:00
ed
e83c63d239 fix unix permissions in zip files 2021-03-27 17:28:25 +01:00
ed
b38533b0cc recover from file access errors when zipping 2021-03-27 17:16:59 +01:00
ed
b12131e91c v0.10.1 2021-03-27 02:44:40 +01:00
ed
d09ec6feaa tehe 2021-03-27 01:49:58 +01:00
ed
21405c3fda be nice to windows 2021-03-27 01:43:02 +01:00
ed
13e5c96cab finish adding zip-crc (semi-streaming) 2021-03-27 01:27:12 +01:00
ed
871dde79a9 download as tar + utf8 zip + optimize walk 2021-03-26 20:43:25 +01:00
ed
514d046d1f download folders as zip 2021-03-26 01:51:38 +01:00
ed
73ebd917d1 i know too much about zip now 2021-03-25 02:31:25 +01:00