copyparty/copyparty
ed 8785d2f9fe add volflag sparse to force use of sparse files;
this improves performance on s3-backed volumes

noktuas reported on discord that the upload performance was
unexpectedly poor when writing to an s3 bucket through a JuiceFS
fuse-mount, only getting 1.5 MiB/s with copyparty, meanwhile a
regular filecopy averaged 30 MiB/s plus

the issue was that s3 does not support sparse files, so copyparty
would fall back to sequential uploading, and also disable fpool,
causing JuiceFS to repeatedly commit the same 5 MiB range to
the storage provider as each chunk arrived from the client

by forcing use of sparse files, s3 adapters such as JuiceFS and
geesefs will "only" write the entire file to s3 *twice*, initially
it writes the full filesize of zerobytes (depending on adapter,
hopefully using gzip compression to reduce the bandwidth necessary)
and then the actual file data in an adapter-specific chunksize

with this volflag, copyparty appears to reach the full expected speed
2024-03-08 18:20:29 +00:00
..
bos ensure nested symlinks are not broken during deletes; 2023-12-08 01:11:03 +00:00
res replace setup.py with pyproject.toml + misc cleanup 2023-05-07 14:37:57 +00:00
stolen cleanup 2023-12-14 10:47:52 +00:00
web improve smoketests, warnings and error-messages: 2024-03-07 19:47:38 +00:00
__init__.py make linters happier 2023-11-30 17:33:07 +00:00
__main__.py improve smoketests, warnings and error-messages: 2024-03-07 19:47:38 +00:00
__version__.py v1.10.2 2024-02-21 21:32:11 +00:00
authsrv.py improve smoketests, warnings and error-messages: 2024-03-07 19:47:38 +00:00
broker_mp.py make linters happier 2023-11-30 17:33:07 +00:00
broker_mpw.py suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
broker_thr.py add landing page with mounting instructions 2022-11-26 19:47:27 +00:00
broker_util.py make pylance happier 2022-10-29 20:40:25 +00:00
cert.py add cfssl to packaging + improve certgen expiration check 2023-11-11 15:30:03 +00:00
cfg.py add volflag sparse to force use of sparse files; 2024-03-08 18:20:29 +00:00
dxml.py make pylance happier 2022-10-29 20:40:25 +00:00
fsutil.py make pylance happier 2022-10-29 20:40:25 +00:00
ftpd.py fix a handful of tftp crashes: 2024-02-21 00:06:47 +00:00
httpcli.py improve smoketests, warnings and error-messages: 2024-03-07 19:47:38 +00:00
httpconn.py up2k: fix a mostly-harmless race 2024-02-13 19:24:06 +00:00
httpsrv.py get rid of brotli due to poor support; closes #73 2024-02-24 22:24:44 +00:00
ico.py "upload only" icon on write-only folders 2024-02-21 20:57:18 +00:00
mdns.py actually impl --mc-hop (and improve --zm-spam) 2023-08-20 21:27:28 +00:00
metrics.py add more prometheus metrics; breaking changes: 2023-11-04 20:32:34 +00:00
mtag.py suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
multicast.py v1.9.0 2023-08-20 22:02:40 +00:00
pwhash.py cosmetics: 2023-11-03 16:52:43 +00:00
smbd.py support environment variables mostly everywhere, 2023-11-30 10:22:52 +00:00
ssdp.py smb: upgrade to impacket 0.11, full user account support, 2023-09-09 12:46:37 +00:00
star.py suboptimizations and some future safeguards 2024-01-10 23:20:42 +01:00
sutil.py utcfromtimestamp was deprecated and nobody told me, 2023-10-20 23:41:58 +00:00
svchub.py improve smoketests, warnings and error-messages: 2024-03-07 19:47:38 +00:00
szip.py up to 2.6x faster download-as-zip 2023-09-05 22:57:03 +00:00
tcpsrv.py add tftp server 2024-02-10 18:37:21 +00:00
tftpd.py fix a handful of tftp crashes: 2024-02-21 00:06:47 +00:00
th_cli.py enforce thumbnail config serverside 2024-02-18 15:36:59 +00:00
th_srv.py add hi-res thumbs (togglebtn/servercfg) 2024-02-18 13:04:22 +00:00
u2idx.py list dotfiles only for specific volumes or users (#66): 2023-12-16 15:38:48 +00:00
up2k.py add volflag sparse to force use of sparse files; 2024-03-08 18:20:29 +00:00
util.py fix other possible division-by-zeros; 2024-02-17 23:19:11 +00:00