Compare commits

...

241 Commits

Author SHA1 Message Date
ed
4accef00fb v1.16.16 2025-02-28 18:46:32 +00:00
ed
d779525500 move -volflag warning to avoid false positives 2025-02-28 18:13:23 +00:00
ed
65a7706f77 add helptext for volflags dk, dks, dky 2025-02-28 17:56:51 +00:00
ed
5e12abbb9b ignore impossible lastmod on upload; closes #142
android-chrome bug https://issues.chromium.org/issues/393149335
sends last-modified time `-11644473600` for all uploads

this has been fixed in chromium, but there might be similar
bugs in other browsers, so add server-side and client-side
detection for unreasonable lastmod times

previously, if the js detected a similar situation, it would
substitute the lastmod-time with the client's wallclock, but
now the server's wallclock is always preferrred as fallback
2025-02-28 17:48:14 +00:00
daimond113
e0fe2b97be nix: add mainProgram
Silences warnings like "getExe: Package
"copyparty-1.16.15" does not have the
meta.mainProgram attribute. We'll assume that the
main program has the same name for now, but this
behavior is deprecated, because it leads to
surprising errors when the assumption does not
hold. If the package has a main program, please
set `meta.mainProgram` in its definition to make
this warning go away. Otherwise, if the package
does not have a main program, or if you don't
control its definition, use getExe' to specify
the name to the program, such as lib.getExe' foo
"bar"."
2025-02-26 23:07:19 +01:00
ed
bd33863f9f update pkgs to 1.16.15 2025-02-25 01:25:15 +00:00
ed
a011139894 v1.16.15 2025-02-25 00:17:58 +00:00
ed
36866f1d36 dangit.wav 2025-02-25 00:11:57 +00:00
ed
407531bcb1 fix markdown / text-editor jank
* only indicate file-history for markdown files since
   other files won't load into the editor which makes
   that entirely pointless; do file extension instead

* text-editor: in files containing one single line,
   ^C followed by ^V ^Z would accidentally a letter

and fix unhydrated extensions
2025-02-25 00:03:22 +00:00
ed
3adbb2ff41 https://youtu.be/WyXebd3I3Vo 2025-02-24 23:32:03 +00:00
ed
499ae1c7a1 other minor html-escaping fixes
mostly related to error-handling for uploads, network-loss etc,
nothing worse than the dom-xss just now
2025-02-24 22:42:05 +00:00
ed
438ea6ccb0 fix GHSA-m2jw-cj8v-937r ;
this fixes a DOM-Based XSS when preparing files for upload;
empty files would have their filenames rendered as HTML in
a messagebox, making it possible to trick users into running
arbitrary javascript by giving them maliciously-named files

note that, being a general-purpose webserver, it is still
intentionally possible to upload and execute arbitrary
javascript, just not in this unexpected manner
2025-02-24 21:23:13 +00:00
ed
598a29a733 mention sony psp support (thx dwarf) 2025-02-23 21:37:21 +00:00
ed
6d102fc826 mention risc-v support 2025-02-20 04:51:04 +00:00
ed
fca07fbb62 update pkgs to 1.16.14 2025-02-19 23:35:05 +00:00
ed
cdedcc24b8 v1.16.14 2025-02-19 23:09:14 +00:00
ed
60d5f27140 new example: randpic.py 2025-02-19 22:41:30 +00:00
ed
cb413bae49 webdav: a healthy dash of paranoia
there's probably at least one client sending `Overwrite: False`
instead of the spec-correct `Overwrite: F`
2025-02-19 22:07:26 +00:00
ed
e9f78ea70c up2k: tristate option for overwriting files; closes #139
adds a third possible value for the `replace` property in handshakes:

* absent or False: never overwrite an existing file on the server,
   and instead generate a new filename to avoid collision

* True: always overwrite existing files on the server

* "mt": only overwrite if client's last-modified is more recent
   (this is the new option)

the new UI button toggles between all three options,
defaulting to never-overwrite
2025-02-19 21:58:56 +00:00
ed
6858cb066f spinner: themes + improve positioning
loading-spinner is either `#dlt_t` or `#dlt_f`
(tree or files), appearing top-left or top-right,
regardless of page/tree scroll (position:fixed)
2025-02-19 18:55:33 +00:00
ed
4be0d426f4 option to forget uploader-IP from db after some time
does this mean copyparty is GDPR-compliant now? idklol
2025-02-17 23:47:59 +00:00
ed
7d7d5d6c3c fix custom spinner css on initial page load 2025-02-17 23:26:21 +00:00
ed
0422387e90 readme: changing the loading spinner (#138) 2025-02-16 19:28:57 +00:00
ed
2ed5fd9ac4 readme: diagnosing broken thumbnails (#137) 2025-02-16 19:22:17 +00:00
ed
2beb2acc24 readme: permanent cloudflare tunnel (#137) 2025-02-16 18:59:18 +00:00
ed
56ce591908 synology dsm: add updating 2025-02-16 18:12:35 +00:00
ed
b190e676b4 fix cosmetic volflag stuff:
* `xz` would show the "unrecognized volflag" warning,
   but it still applied correctly

* removing volflags with `-foo` would also show the warning
   but it would still get removed correctly

* hide `ext_th_d` in the startup volume-listing
2025-02-14 20:54:13 +00:00
ed
19520b2ec9 remove patch for musl cve (no longer necessary) 2025-02-14 09:15:52 +00:00
ed
eeb96ae8b5 update pkgs to 1.16.13 2025-02-13 21:43:32 +00:00
ed
cddedd37d5 v1.16.13 2025-02-13 20:57:04 +00:00
ed
4d6626b099 workaround musl 1.2.5 cve 2025-02-13 20:53:47 +00:00
ed
7a55833bb2 silence linter 2025-02-13 18:34:41 +00:00
ed
7e4702cf09 file-extension icons
global-option / volflag `ext_th` specifies
custom thumbnail for a given file extension
2025-02-13 18:32:01 +00:00
ed
685f08697a alternative loader spinners 2025-02-13 17:07:48 +00:00
ed
a255db706d make volflags less confusing
1. warn about unrecognized volflags

previously, when specifying an unknown volflag, it would
be silently ignored, giving the impression that it applied

2. also allow uppercase, kebab-case
    (previously, only snake_case was accepted)

3. mention every volflag in --help-flags
    (some volflags were missing)
2025-02-13 00:34:46 +00:00
ed
9d76902710 WebDAV: adjust 401-mask for GETs (#136)
some clients, including KDE Dolphin (kioworker/6.10) keeps
sending requests without the basic-auth header, expecting
the server to respond with a 401 before it does

most clients only do this for the initial request, which is
usually a PROPFIND, which makes this nice and simple -- but
turns out we need to consider this for GET as well...

this is tricky because a graphical webbrowser must never
receive a 401 lest it becomes near-impossible to deauth,
and that's exactly what Dolphin pretends to be in its UA

man ( ´_ゝ`)

note: `KIO/` hits konqueror so don't
2025-02-11 23:32:44 +00:00
ed
62ee7f6980 WebDAV: support COPY, KDE-Dolphin (#136):
* add support for the COPY verb

* COPY/MOVE: add overwrite support;
   default is True according to rfc
   (only applies to single files for now)

* COPY/MOVE/MKCOL: return 401 as necessary
   for clients which rechallenge frequently
   such as KDE Dolphin (KIO/6.10)

* MOVE: support webdav:// Destination prefix
   as used by KDE Dolphin (KIO/6.10)

* MOVE: vproxy support
2025-02-11 21:34:24 +00:00
ed
2f6707825a improve usb-eject hook:
* don't crash internet explorer
* support running as root
* support old linuxen
2025-02-10 04:52:57 +00:00
ed
7dda77dcb4 update pkgs to 1.16.12 2025-02-10 00:04:04 +00:00
ed
ddec22d04c v1.16.12 2025-02-09 23:31:26 +00:00
ed
32e90859f4 readme: add config file examples 2025-02-09 23:03:46 +00:00
ed
8b8970c787 add note about dead docker experiments 2025-02-09 21:17:25 +00:00
ed
03d35ba799 rename bubblewrap.sh to bubbleparty.sh (2/2)
reduces confusion if it ends up in $PATH, if there
are multiple such wrappers with different semantics
2025-02-09 20:40:16 +00:00
ed
c035d7d88a rename bubblewrap.sh to bubbleparty.sh (1/2)
reduces confusion if it ends up in $PATH, if there
are multiple such wrappers with different semantics
2025-02-09 20:39:32 +00:00
ed
46f9e9efff add plugin: quickmove
defines hotkey W to move selected files into a subfolder
2025-02-09 19:40:36 +00:00
ed
4fa8d7ed79 hotkey S toggles selection of playing song
does not apply if image gallery is open

also ensure the hotkey handler is only attached once;
makes it easier to hook/modify it from plugins
2025-02-09 19:22:29 +00:00
ed
cd71b505a9 safeguard against accidental config loss
when running copyparty without any config, it defaults to sharing
the current folder read-write for everyone. This makes sense for
quick one-off instances, but not in more permanent deployments

especially for docker, where the config can get lost by accident
in too many ways (compose typos, failed upgrade, selinux, ...)
the default should be to reject all access

add a safeguard which disables read-access if one or more
config-files were specified, but no volumes are defined

should prevent issues such as filebrowser/filebrowser#3719
2025-02-08 20:37:30 +00:00
ed
c7db08ed3e remember file selection per-folder
avoids losing file selection when
accidentally changing to another folder
2025-02-08 15:18:07 +00:00
Leon van Kammen
3582a1004c added bubblewrap docs + script 2025-02-07 21:12:39 +01:00
ed
22cbd2dbb5 handlers: add http-redirect example 2025-02-07 19:03:13 +00:00
ed
c87af9e85c option to restrict download-as-zip/tar
new global-option / volflag `zip_who` specifies
who gets to use the download-as-zip/tar function;

* 0: nobody, same as --no-zip
* 1: admins
* 2: authorized users with read-access
* 3: anyone with read-access
2025-02-05 20:45:50 +00:00
ed
6c202effa4 add plugin: graft-thumbs.js; #133
"sidecar thumbnails"; if a folder contains both foo.mp3 and foo.png
then this plugin takes the png thumbnail and applies to the mp3
while in the grid-view
2025-02-05 19:48:59 +00:00
ed
632f52af22 warn that RTL support is currently not planned
the current approach to html generation is
probably too jank to handle RTL correctly
2025-02-03 18:41:02 +00:00
ed
46e59529a4 ensure intended order in language selector 2025-02-03 18:32:58 +00:00
ed
bdf060236a improve(?) bugreport template 2025-02-03 05:27:19 +00:00
ed
d9d2a09282 mention fuse/rclone hijinks from #132 2025-02-02 23:22:06 +00:00
ed
b020fd4ad2 make some 403s less ambiguous in logs 2025-02-02 23:02:54 +00:00
ed
4ef3526354 bbox: try to detect media load errors
listen for errors from <img> and <video> in the media gallery and
show an error-toast to indicate that the file isn't going to appear

unfortunately, when iOS-Safari fails to decode an unsupported video,
Safari itself appears to believe that everything is fine, and doesn't
issue the expected error-event, meaning we cannot detect this...

for example, trying to play non-yuv420p vp9 webm will silently fail,
with the only symptom being the play() promise throwing as the
<video> is destroyed during cleanup (bbox-close or media unload)
2025-01-31 21:13:35 +00:00
ed
20ddeb6e1b include last rtt in next req 2025-01-31 20:09:45 +00:00
ed
d27f110498 http rtt in serverinfo panel 2025-01-31 20:00:33 +00:00
ed
910797ccb6 ping.html: add mdev, limit 2025-01-31 19:16:44 +00:00
ed
7de9d15aef add ping.html (from old php project) 2025-01-31 18:56:12 +00:00
ed
6a9ffe7e06 traefik-example: fix disconnect during big uploads
if an upload takes longer than 60 seconds,
by default, traefik closes the connection

thx to @JuvenoiaAgent@lemmy.ca for catching this
2025-01-29 21:03:18 +00:00
ed
12dcea4f70 improve iPad detection;
recent iPads do not indicate being an iPad in the user-agent,
so the audio-player would fall back on transcoding to mp3,
assuming the device cannot play opus-caf

improve this with pessimistic feature-detection for caf
hopefully still avoiding false-positives
2025-01-27 21:06:47 +00:00
ed
b3b39bd8f1 update pkgs to 1.16.11 2025-01-27 02:01:25 +00:00
ed
c7caecf77c v1.16.11 2025-01-27 01:40:23 +00:00
ed
1fe30363c7 u2c: option to print download links 2025-01-27 01:35:36 +00:00
ed
54a7256c8d fix js-panic if audio transcoding disabled 2025-01-27 00:37:03 +00:00
ed
8e8e4ff132 update pkgs to 1.16.10 2025-01-25 18:45:50 +00:00
ed
1dace72092 v1.16.10 2025-01-25 18:09:52 +00:00
ed
3a5c1d9faf allow zeromq to veto uploads 2025-01-25 17:49:03 +00:00
ed
f38c754301 add hook: usb-eject 2025-01-25 17:02:41 +00:00
ed
fff38f484d prefer opus-caf due to iOS bugs 2025-01-25 14:27:34 +00:00
ed
95390b655f ensure opus to owa is remux, not transcode
also add transcoding from opus to mp3 if client wants mp3,
overriding the feature detection for opus support
2025-01-25 13:44:19 +00:00
ed
5967c421ca zmq tweaks 2025-01-25 11:29:38 +00:00
ed
b8b5214f44 option to restrict recent-uploads visibility 2025-01-25 11:13:12 +00:00
ed
cdd3b67a5c msg-to-log includes xm / ZeroMQ response 2025-01-25 10:59:15 +00:00
ed
28c9de3f6a add opus-weba transcoding (for iOS 18 and newer)
support for "owa", audio-only webm, was introduced in iOS 17.5

owa is a more compliant alternative to opus-caf from iOS 11,
which was technically limited to CBR opus, a limitation which
we ignored since it worked mostly fine for regular opus too

being the new officially-recommended way to do things,
we'll default to owa for iOS 18 and later, even though
iOS still has some bugs affecting our use specifically:

if a weba file is preloaded into a 2nd audio object,
safari will throw a spurious exception as playback is
initiated, even as the file is playing just fine

the `.ld` stuff is an attempt at catching and ignoring this
spurious error without eating any actual network exceptions
2025-01-25 10:15:44 +00:00
ed
f3b9bfc114 option to disable caf transcoding + misc cleanup 2025-01-24 22:40:52 +00:00
ed
c9eba39edd fix audio-transcoding for iOS v10 and older
opus-in-caf was added in iOS v11;
use caf in iOS v12 and newer (iPhone 5s, iPad Air/mini2, iPod 6),
use mp3 in iOS v11 and older (iPhone 5c, iPad 4)
2025-01-24 19:57:23 +00:00
ed
40a1c7116e golf useragent to ua 2025-01-24 19:56:56 +00:00
ed
c03af9cfcc update pkgs to 1.16.9 2025-01-22 23:54:04 +00:00
ed
c4cbc32cc5 v1.16.9 2025-01-22 23:26:17 +00:00
ed
1231ce199e create utf8 zipfiles by default
previously, the `?zip` url-suffix would create a cp437 zipfile,
and `?zip=utf` would use utf-8, which is now generally expected

now, both `?zip=utf` and `?zip` will produce a utf8 zipfile,
and `?zip=dos` provides the old behavior
2025-01-22 22:50:03 +00:00
ed
e0cac6fd99 clamp filekeys to max 72 chars
fixes a bug reported on discord:

a sha512 checksum does not cleanly encode to base64, and the
padding runs afoul of the safety-check added in 988a7223f4

as there is not a single reason to use a filekey that long,
fix it by setting an upper limit (which is still ridiculous)
2025-01-22 22:17:57 +00:00
ed
d9db1534b1 hooks: send zeromq/zmq/0mq messages
adds an optional dependency on pyzmq
2025-01-22 21:18:42 +00:00
ed
6a0aaaf069 md/logue sandbox: custom allow prop
add global-option and volflag to specify the
value of the iframe's allow-property
2025-01-21 22:51:00 +00:00
ed
4c04798aa5 u2c: fix hash-calculator mode
it produced the correct chunk-hashes with --chs
but the total file-hash was wrong regardless
2025-01-21 22:04:20 +00:00
ed
3f84b0a015 failsafe against unsafe reverse-proxy misconfiguration:
if an untrusted x-forwarded-for is received, then disable
some features which assume the client-ip to be correct:

* listing dotfiles recently uploaded from own ip
* listing ongoing uploads from own ip
* unpost recently uploaded files

this is in addition to the existing vivid warning in
the serverlogs, which empirically is possible to miss
2025-01-20 18:52:39 +00:00
ed
917380ddbb add nosparse volflag + update s3 readme:
may improve upload performance in some particular uncommon scenarios,
for example if hdd-writes are uncached, and/or the hdd is drastically
slower than the network throughput

one particular usecase where nosparse *might* improve performance
is when the upload destination is cloud-storage provided by FUSE
(for example an s3 bucket) but this is educated guesswork
2025-01-19 16:28:40 +00:00
ed
d9ae067e52 stop recommending webworkers on android
as of v1.16.8 this is counter-productive on android;
see ec50788987
2025-01-19 16:09:54 +00:00
ed
b2e8bf6e89 selftest dxml on startup:
try to decode some malicious xml on startup; if this succeeds,
then force-disable all xml-based features (primarily WebDAV)

this is paranoid future-proofing against unanticipated changes
in future versions of python, specifically if the importlib or
xml.etree.ET behavior changes in a way that somehow reenables
entity expansion, which (still hypothetically) would probably
be caused by failing to unload the `_elementtree` c-module

no past or present python versions are affected by this change
2025-01-17 06:06:36 +00:00
ed
170cbe98c5 refactor github urls 2025-01-17 05:48:49 +00:00
exci
c94f662095 fix optional package name in PKGBUILD (#130) 2025-01-16 08:01:17 +01:00
ed
0987dcfb1c versus: seafile fix, filebrowser fork
* versus: seafile DOES support write-only folders

* versus: mention https://github.com/gtsteffaniak/filebrowser

* connect-page: link the correct v12.x version of sharex
2025-01-13 22:30:46 +00:00
ed
6920c01d4a update pkgs to 1.16.8 2025-01-11 16:47:33 +00:00
ed
cc0cc8cdf0 v1.16.8 2025-01-11 16:11:15 +00:00
ed
fb13969798 connect-page: add flameshot too 2025-01-11 16:08:12 +00:00
ed
278258ee9f connect-page:
* add sharex, ishare

* change placeholder password from `pw` to `hunter2`

* add a button to use a real password instead of a placeholder
2025-01-11 15:23:47 +00:00
ed
9e542cf86b these can also trigger reloads; dd6e9ea7 2025-01-11 12:52:11 +00:00
ed
244e952f79 copyparty.exe: update pillow 2025-01-11 12:49:07 +00:00
ed
aa2a8fa223 up2k-snap: remove deprecated properties
v1.15.7 is the oldest version which still
has any chance of reading the up2k.snap
2025-01-11 12:16:45 +00:00
ed
467acb47bf up2k-snap-load: assert .PARTIAL for unfinished
when loading up2k snaps, entries are forgotten if
the relevant file has been deleted since last run

when the entry is an unfinished upload, the file that should
be asserted is the .PARTIAL, and not the placeholder / final
filename (which, unintentionally, was the case until now)

if .PARTIAL is missing but the placeholder still exists,
the only safe alternative is to forget/disown the file,
since its state is obviously wrong and unknown
2025-01-11 11:49:53 +00:00
ed
0c0d6b2bfc add ishare config example (macos screenshot uploader)
also includes a slight tweak to the json upload info:

when exactly one file is uploaded, the json-response has a
new top-level property, `fileurl` -- this is just a copy of
`files[0].url` as a workaround for castdrian/ishare#107
("only toplevel json properties can be referenced")
2025-01-10 21:13:20 +00:00
ed
ce0e5be406 bup: alias ?j to request-header Accept: json
and teach PUT to answer in json too
2025-01-10 20:32:12 +00:00
ed
65ce4c90fa link the idp-webdav docs from the main readme too 2025-01-10 18:54:45 +00:00
ed
9897a08d09 hotlink from the connect-page to the idp client-auth docs added in #129 2025-01-10 18:47:12 +00:00
ed
f5753ba720 add chunksize cheat-sheet 2025-01-10 18:24:40 +00:00
Wuast94
fcf32a935b add idp client section to docs 2025-01-10 18:17:57 +01:00
ed
ec50788987 up2k.js: 10x faster hashing on android-chrome
when hashing files on android-chrome, read a contiguous range of
several chunks at a time, ensuring each read is at least 48 MiB
and then slice that cache into the correct chunksizes for hashing

especially on GrapheneOS Vanadium (where webworkers are forbidden),
improves worst-case speed (filesize <= 256 MiB) from 13 to 139 MiB/s

48M was chosen wrt RAM usage (48*4 MiB); a target read-size of
16M would have given 76 MiB/s, 32M = 117 MiB/s, and 64M = 154 MiB/s

additionally, on all platforms (not just android and/or chrome),
allow async hashing of <= 3 chunks in parallel on main-thread
when chunksize <= 48 MiB, and <= 2 at <= 96 MiB; this gives
decent speeds approaching that of webworkers (around 50%)

this is a new take on c06d928bb5
which was removed in 184af0c603
when a chrome-beta temporarily fixed the poor file-read performance
(afaict the fix was reverted and never made it to chrome stable)

as for why any of this is necessary,

the security features in android have the unfortunate side-effect
of making file-reads from web-browsers extremely expensive;
this is especially noticeable in android-chrome, where
file-hashing is painfully slow, around 9 MiB/s worst-case

this is due to a fixed-time overhead for each read operation;
reading 1 MiB takes 60 msec, while reading 16 MiB takes 112 msec
2025-01-10 05:29:55 +00:00
ed
ac0a2da3b5 add/improve reverse-proxy examples
* add haproxy, lighttpd, traefik, caddy

* adjust nginx buffer sizes for way faster downloads

* move unix-socket to /dev/shm/ because
   fedora sets PrivateTmp=true for nginx (orz)
2025-01-07 05:49:40 +00:00
ed
9f84dc42fe recommend kamelåså instead of very-bad-idea; closes #75 2025-01-01 20:26:09 +00:00
ed
21f9304235 add synology howto 2024-12-27 02:16:20 +00:00
ed
5cedd22bbd update pkgs to 1.16.7 2024-12-23 18:24:35 +00:00
ed
c0dacbc4dd v1.16.7 2024-12-23 00:05:49 +00:00
ed
dd6e9ea70c when idp is enabled, always daemon(up2k-rescan)
fixes a bug reported on discord;

1. run with `--idp-h-usr=iu -v=srv::A`
2. upload a file with up2k; this succeeds
3. announce an idp user: `curl -Hiu:a 127.1:3923`
4. upload another file; fails with "fs-reload"

the idp announce would `up2k.reload` which raises the
`reload_flag` and `rescan_cond`, but there is nothing
listening on `rescan_cond` because `have_e2d` was false

must assume e2d if idp is enabled, because `have_e2d` will
only be true if there are non-idp volumes with e2d enabled
2024-12-23 17:16:56 +00:00
ed
87598dcd7f recent-uploads: move rendering to js
* loads 50% faster, reducing server-load by 30%

* inhibits search engines from indexing it

* eyecandy (filter applies automatically on edit)
2024-12-20 23:52:03 +00:00
ed
3bb7b677f8 jinja optimizations 2024-12-20 16:34:17 +00:00
ed
988a7223f4 remove some footguns
in case someone writes a plugin which
expects certain params to be sanitized

note that because mojibake filenames are supported,
URLs and filepaths can still be absolutely bonkers

this fixes one known issue:
invalid rss-feed xml if ?pw contains special chars

...and somehow things now run 2% faster, idgi
2024-12-20 14:03:40 +00:00
ed
7f044372fa 18:17:14 +Mai | ed: volume bar is bad design
18:17:26  &ed | what's wrong with it
18:17:38 +Mai | that you don't know it's the volume bar before you try it
18:17:46  &ed | oh
18:17:48  &ed | yeah i guess
18:17:54 +Mai | especially when it's at 100
18:18:00  &ed | how do i fix it tho
18:19:50 +Mai | you could add an icon that's also a mute button (to not make it a useless icon)
18:22:38  &ed | i'll make the volume text always visible and include a speaker icon before it
18:23:53 +Mai | that is better at least
2024-12-19 18:49:51 +00:00
ed
552897abbc fix log colors on loss of ext.ip 2024-12-19 18:48:03 +00:00
ed
946a8c5baa u2c: fix windowtitle 2024-12-19 18:02:29 +00:00
ed
888b31aa92 update pkgs to 1.16.6 2024-12-19 01:08:34 +00:00
ed
e2dec2510f v1.16.6 2024-12-19 00:37:24 +00:00
ed
da5ad2ab9f warn on ambiguous comments in config files 2024-12-19 00:25:10 +00:00
ed
eaa4b04a22 list recent uploads
also makes the unpost lister 5x faster
2024-12-18 22:17:30 +01:00
ed
3051b13108 try to avoid printing mojibake in logs
unprintable and side-effect-inducing paths and names are hex-escaped,
preserving greppability and making log-parsing slightly more okay
2024-12-18 01:45:54 +01:00
ed
4c4e48bab7 improve dotfile handling; closes #126
when deleting a folder, any dotfiles/folders within would only
be deleted if the user had the dot-permission to see dotfiles;
this gave the confusing behavior of not removing the "empty"
folders after deleting them

fix this to only require the delete-permission, and always
delete the entire folder, including any dotfiles within

similar behavior would also apply to moves, renames, and copies;

fix moves and renames to only require the move-permission in
the source volume; dotfiles will now always be included,
regardless of whether the user does (or does not) have the
dot-permission in either the source and/or destination volumes

copying folders now also behaves more intuitively: if the user has
the dot-permission in the target volume, then dotfiles will only be
included from source folders where the user also has the dot-perm,
to prevent the user from seeing intentionally hidden files/folders
2024-12-17 22:47:34 +01:00
ed
01a3eb29cb ui: improve some eta/idle fields
cpanel db-idle-time indicator would glitch on 0.0s

upload windowtitle was %.2f seconds, but the value is int
2024-12-17 22:01:36 +01:00
ed
73f7249c5f decode and log request URLs; closes #125
as processing of a HTTP request begins (GET, HEAD, PUT, POST, ...),
the original query line is printed in its encoded form. This makes
debugging easier, since there is no ambiguity in how the client
phrased its request.

however, this results in very opaque logs for non-ascii languages;
basically a wall of percent-encoded characters. Avoid this issue
by printing an additional log-message if the URL contains `%`,
immediately below the original url-encoded entry.

also fix tests on macos, and an unrelated bad logmsg in up2k
2024-12-16 00:53:22 +01:00
ed
18c6559199 update pkgs to 1.16.5 2024-12-11 22:59:44 +00:00
ed
e66ece993f v1.16.5 2024-12-11 22:36:19 +00:00
ed
0686860624 connectpage nitpick + update dompurify 2024-12-11 22:24:31 +00:00
ed
24ce46b380 avoid chrome webworker OOM bug; closes #124
chrome (and chromium-based browsers) can OOM when:

* the OS is Windows, MacOS, or Android (but not Linux?)
* the website is hosted on a remote IP (not localhost)
* webworkers are used to read files

unfortunately this also applies to Android, which heavily relies
on webworkers to make read-speeds anywhere close to acceptable

as for android, there are diminishing returns with more than 4
webworkers (1=1x, 2=2.3x, 3=3.8x, 4=4.2x, 6=4.5x, 8=5.3x), and
limiting the number of workers to ensure at least one idle core
appears to sufficiently reduce the OOM probability

on desktop, webworkers are only necessary for hashwasm, so
limit the number of workers to 2 if crypto.subtle is available
and otherwise use the nproc-1 rule for hashwasm in workers

bug report: https://issues.chromium.org/issues/383568268
2024-12-11 22:11:54 +00:00
ed
a49bf81ff2 mdns: improve nic-ip changelog
if a NIC is brought up with several IPs,
it would only mention one of the new IPs in the logs

or if a PCIe bus crashes and all NICs drop dead,
it would only mention one of the IPs that disappeared

as both scenarios are oddly common, be more verbose
2024-12-10 00:36:58 +00:00
ed
64501fd7f1 hybrid IdP (check regular users too); closes #122
previously, when IdP was enabled, the password-based login would be
entirely disabled. This was a semi-conscious decision, based on the
assumption that you would always want to use IdP after enabling it.

it makes more sense to keep password-based login working as usual,
conditionally disengaging it for requests which contains a valid
IdP username header. This makes it possible to define fallback
users, or API-only users, and all similar escape hatches.
2024-12-08 17:18:20 +00:00
ed
db3c0b0907 nice 2024-12-07 22:24:13 +00:00
ed
edda117a7a update pkgs to 1.16.4 2024-12-07 01:10:50 +00:00
ed
cdface0dd5 v1.16.4 2024-12-07 00:24:37 +00:00
ed
be6afe2d3a improve ux for relocating partial uploads
if someone accidentally starts uploading a file in the wrong folder,
it was not obvious that you can forget that upload in the unpost tab

this '(explain)' button in the upload-error hopefully explains that,
and upload immediately commences when the initial attempt is aborted

on the backend, cleanup the dupesched when an upload is
aborted, and save some cpu by adding unique entries only
2024-12-06 23:34:47 +00:00
ed
9163780000 u2c: misc windows fixes
* support globbing/wildcards on windows

* add `osc 9;4` to show upload progress in the taskbar
   (currently windows-only; linux is picking it up)

* workaround msys2-terminal not normalizing
   absolute paths which contain whitespace

* show a helpful "now hashing..." while the
   first file is being hashed, since it kinda
   looks like a deadlock on windows otherwise
2024-12-06 18:44:05 +00:00
ed
d7aa7dfe64 translations: new strings 2024-12-04 09:46:04 +00:00
ed
f1decb531d update pkgs to 2024-12-04 00:41:34 +00:00
ed
99399c698b v1.16.3 2024-12-04 00:03:55 +00:00
ed
1f5f42f216 fix #121 (GET toplevel files with h) 2024-12-03 23:53:21 +00:00
ed
9082c4702f accesslog: exclude thumbnails by default 2024-12-03 22:17:49 +00:00
ed
6cedcfbf77 update deps:
* copyparty.exe: python 3.12.7 => 3.12.8
* webdeps:
  * hashwasm 4.10.0 => 4.12.0
  * dompurify 3.1.7 => 3.2.2
  * codemirror 5.65.16 => 5.65.18
2024-12-03 22:11:07 +00:00
ed
8a631f045e ui: fix final time-elapsed and speed for fsearch 2024-12-03 21:43:38 +00:00
ed
a6a2ee5b6b sort on filename should forget other sorts
filenames are unique per folder; remembering other keys is pointless
2024-12-03 21:06:00 +00:00
ed
016708276c add sorting granularity options for media URLs 2024-12-03 20:01:19 +00:00
ed
4cfdc4c513 preserve active sort-prefs if more specific than url-hash 2024-12-03 02:13:12 +00:00
ed
0f257c9308 embed sort-order in gallery/media URLs
so anyone who clicks the link sees the files in the same order
2024-12-03 01:46:56 +00:00
ed
c8104b6e78 js: 7x faster deepclone 2024-12-03 01:44:55 +00:00
ed
1a1d731043 misc cleanup:
* u2c: remove superfluous pathsep (harmless)
* tl: new strings
2024-12-02 22:52:39 +00:00
ed
c5a000d2ae url-option for upload checksum type
url-param / header `ck` specifies hashing algo;
md5 sha1 sha256 sha512 b2 blake2 b2s blake2s

value 'no' or blank disables checksumming,
for when copyparty is running on ancient gear
and you don't really care about file integrity
2024-12-02 13:51:39 +00:00
ed
94d1924fa9 improve avahi-379 workaround 2024-12-01 21:24:41 +00:00
ed
6c1cf68bca mdns: add workaround for https://github.com/avahi/avahi/issues/379
Avahi's mDNS-reflection feature does not understand NSEC, so
it corrupts mDNS packets by not rewriting compressed labels
2024-12-01 19:01:40 +00:00
ed
395af051bd mdns: option to ignore invalid packets on the lan 2024-12-01 15:38:24 +00:00
ed
42fd66675e tests: improve specificity 2024-12-01 15:36:35 +00:00
ed
21a3f3699b webdav: add tests + fix minor edgecases
* allow depth:0 at top of unmapped root

* cannot use the Referer header to identify
   graphical browsers since rclone sends it
2024-12-01 14:44:41 +00:00
ed
d168b2acac forget all shadowed files (uploads too); closes #120
shadowing is the act of intentinoally blocking off access to
files in a volume by placing another volume atop of a file/folder.

say you have volume '/' with a file '/a/b/c/d.txt'; if you create a
volume at '/a/b', then all files/folders inside the original folder
becomes inaccessible, and replaced with the contents of the new vol

the initial code for forgetting shadowed files from the parent vol
database would only forget files which were discovered during a
filesystem scan; any uploaded files would be intentionally preseved
in the parent volume's database, probably to avoid losing uploader
info in the event of a brief mistaken config change, where a volume
is shadowed by accident.

this precaution was a mistake, currently causing far more
issues than it solves (#61 and #120), so away it goes.

huge thanks to @Gremious for doing all the legwork on this!
2024-11-28 22:01:18 +00:00
ed
2ce8233921 webdav: auth-challenge clients correctly:
* return 403 instead of 404 in the following sitations:
  * viewing an RSS feed without necessary auth
  * accessing a file with the wrong filekey
  * accessing a file/folder without necessary auth
     (would previously 404 for intentional ambiguity)

* only allow PROPFIND if user has either read or write;
   previously a blank response was returned if user has
   get-access, but this could confuse webdav clients into
   skipping authentication (for example AuthPass)

* return 401 basic-challenge instead of 403 if the client
   appears to be non-graphical, because many webdav clients
   do not provide the credentials until they're challenged.
   There is a heavy bias towards assuming the client is a
   browser, because browsers must NEVER EVER get a 401
   (tricky state that is near-impossible to deal with)

* return 401 basic-challenge instead of 403 if a PUT
   is attempted without any credentials included; this
   should be safe, as graphical browsers never do that

this fixes the interoperability issues mentioned in
https://github.com/authpass/authpass/issues/379
where AuthPass would GET files without providing the
password because it expected a 401 instead of a 403;
AuthPass is behaving correctly, this is not a bug
2024-11-27 22:07:53 +00:00
ed
697a4fa8a4 exclude search results by regex (#120)
a better alternative to using `--no-idx` for this purpose since
this also excludes recent uploads, not just during fs-indexing,
and it doesn't prevent deduplication

also speeds up searches by a tiny amount due to building the
sanchecks into the exclude-filter while parsing the config,
instead of during each search query
2024-11-26 23:57:01 +00:00
ed
2f83c6c7d1 drop caches if certain volflags change (#120)
dhash would prevent a new noidx value from taking effect
2024-11-26 19:25:47 +00:00
ed
127f414e9c improve phrasing in indexer messages (#120) 2024-11-26 18:52:23 +00:00
ed
33c4ccffab vendor foss licenses
license downloader (for generating COPYING.txt) broke after
opensource.org changed their html, so just vendor all of it
2024-11-26 00:39:38 +00:00
ed
bafe7f5a09 improve helptext exporters 2024-11-24 21:28:21 +00:00
ed
baf41112d1 update pkgs to 1.16.2 2024-11-23 23:55:09 +00:00
ed
a90dde94e1 v1.16.2 2024-11-23 23:36:15 +00:00
ed
7dfbfc7227 fix v1.16.0 webdav upload regression; closes #119 2024-11-23 23:32:56 +00:00
ed
b10843d051 cosmetic eta improvements:
* u2c: strip hh:mm:ss past 30 days
* u2js: fix "infini.ty" in elapsed-times
2024-11-23 19:58:25 +00:00
ed
520ac8f4dc fix opening md files from gridview 2024-11-23 17:55:05 +00:00
ed
537a6e50e9 javascript... 2024-11-22 23:10:23 +00:00
ed
2d0cbdf1a8 video-player: support mov files 2024-11-22 22:47:42 +00:00
ed
5afb562aa3 avoid layout-shift for qr-codes 2024-11-22 22:44:44 +00:00
ed
db069c3d4a fix shares qr-code on chrome 2024-11-22 22:28:00 +00:00
ed
fae40c7e2f black 2024-11-22 22:26:34 +00:00
ed
0c43b592dc pave the way for more ux volflags
makes directory listings a tiny bit faster, about 7% or so
2024-11-22 22:24:56 +00:00
ed
2ab8924e2d tests/debug: plug some resource leaks 2024-11-22 22:21:43 +00:00
akp
0e31cfa784 Allow multiple CIDR ranges when using lan shorthands
Signed-off-by: akp <abi@tdpain.net>
2024-11-22 19:51:56 +00:00
ed
8f7ffcf350 add nsort option/volflag 2024-11-19 18:39:40 +00:00
ed
9c8507a0fd fix downloads-eta layout jank 2024-11-17 19:39:44 +00:00
ed
e9b2cab088 update pkgs to 1.16.1 2024-11-15 22:40:41 +00:00
ed
d3ccacccb1 v1.16.1 2024-11-15 22:18:11 +00:00
ed
df386c8fbc ux: fix paste msg + cleanup css 2024-11-15 22:11:51 +00:00
ed
4d15dd6e17 cbz thumbnails 2024-11-15 21:33:37 +00:00
ed
56a0499636 fix gallery links when msel enabled 2024-11-15 20:04:13 +00:00
ed
10fc4768e8 fix dl from jumpvols with -j0 2024-11-15 19:29:44 +00:00
ed
2b63d7d10d detect invalid config (prevent db loss) 2024-11-15 08:04:58 +00:00
ed
1f177528c1 fix advanced options for password-hashing
and allow raising scrypt ram usage past OpenSSL's default 32 MiB
2024-11-15 00:42:08 +00:00
ed
fc3bbb70a3 update pkgs to 1.16.0 2024-11-10 20:00:38 +00:00
ed
ce3cab0295 v1.16.0 2024-11-10 19:32:37 +00:00
ed
c784e5285e u2c: adaptive connection:keepalive expiration 2024-11-10 17:43:40 +00:00
ed
2bf9055cae detect free RAM on startup for sane defaults
* if free ram on startup is less than 2 GiB,
   use smaller chunks for parallel file hashing

* if --th-max-ram is lower than 0.25 (256 MiB),
   print a warning that thumbnails will not work

* make thumbnail cleaner immediately do a sweep on startup,
   forgetting any failed conversions so they can be retried
   in case the memory limit was increased since last run
2024-11-10 15:43:19 +00:00
ed
8aba5aed4f list active downloads in controlpanel 2024-11-10 02:12:18 +00:00
ed
0ce7cf5e10 update comparison / versus.md 2024-11-09 14:44:03 +00:00
ed
96edcbccd7 https://ocv.me/stuff/goed-gedaan.jpg 2024-11-08 22:11:33 +00:00
ed
4603afb6de don't consume ctrl-shift-c (devtools inspector) 2024-11-08 21:51:54 +00:00
ed
56317b00af filecopy: ui for resolving name conflicts 2024-11-08 02:12:28 +00:00
ed
cacec9c1f3 support copying files/folders; closes #115
behaves according to the target volume's deduplication config;
will create symlinks / hardlinks instead if dedup is enabled
2024-11-07 21:41:53 +00:00
ed
44ee07f0b2 IdP: async reload; closes #114
whenever a new idp user is registered, up2k will continuously
reload in the background until all users have been processed

just like before, this blocks up2k uploads from each user
until said user makes it into a reload, but as of now,
reloads will batch and execute without interrupting read-access

needs further testing before next release,
probably some rough edges to sand down
2024-11-04 22:31:48 +00:00
ed
6a8d5e1731 ui: batch-rename: remember last regex + format 2024-11-02 18:06:39 +00:00
ed
d9962f65b3 ui: folder loading indicator stole focus
show a spinning halfcircle around the +/- instead of
moving the focus to the selected folder in the sidebar,
since that could mess with keyboard scrolling
2024-11-02 17:58:30 +00:00
ed
119e88d87b bubble OS-filesystem errors to client
send a 500 or 404 if a folder is inaccessible or does not exist

previously it would return an empty directory listing instead
2024-11-02 17:38:17 +00:00
ed
71d9e010d9 ui: make hotkey-help less eager to show itself
would appear when typing `?` into textboxes
2024-10-30 19:40:48 +00:00
ed
5718caa957 ui: url-options to set grid/thumbs on/off 2024-10-30 19:24:00 +00:00
ed
efd8a32ed6 ui: show switch-to-https on 403s too 2024-10-28 03:38:15 +00:00
ed
b22d700e16 update pkgs to 1.15.10 2024-10-27 09:27:38 +00:00
ed
ccdacea0c4 v1.15.10 2024-10-27 07:51:11 +00:00
ed
4bdcbc1cb5 shares: allow upload, unpost
* files can be uploaded into writeable shares

* add "write-only" button to the create-share ui

* unpost is possible while viewing the relevant share
2024-10-26 21:36:07 +00:00
ed
833c6cf2ec partyfuse: bump dircache size
dircache size should exceed max dir depth, because the OS
may periodically listdir all parents of current folder
2024-10-26 18:25:21 +00:00
ed
dd6dbdd90a http 304: client-option to force-disable cache
an extremely brutish workaround for issues such as #110 where
browsers receive an HTTP 304 and misinterpret as HTTP 200

option `--no304=1` adds the button `no304` to the controlpanel
which can be enabled to force-disable caching in that browser

the button is default-disabled; by specifying `--no304=2`
instead of `--no304=1` the button becomes default-enabled

can also always be enabled by accessing `/?setck=no304=y`
2024-10-26 17:56:54 +00:00
ed
63013cc565 http 304: k304 obsoleted for ie11 by Vary
the Vary header killed caching in all versions of internet explorer
so there's no point conditionally enabling k304 for trident anymore
2024-10-25 22:32:58 +00:00
ed
912402364a http 304: strip Content-Length and Content-Type
these response headers are usually not included in 304 replies,
and their presence are suspected to confuse some clients (#110)

also strip `out_headerlist` (primarily cookie assignments)
2024-10-25 22:24:40 +00:00
ed
159f51b12b http 304: if-range, backdating
add support for the `If-Range` header which is generally used to
prevent resuming a partial download after the source file on the
server has been modified, by returning HTTP 200 instead of a 206

also simplifies `If-Modified-Since` and `If-Range` handling;
previously this was a spec-compliant lexical comparison,
now it's a basic string-comparison instead. The server will now
reply 200 also when the server mtime is older than the client's.
This is technically not according to spec, but should be safer,
as it allows backdating timestamps without purging client cache
2024-10-25 22:05:59 +00:00
ed
7678a91b0e debug: --ohead (log response headers) 2024-10-25 20:00:19 +00:00
ed
b13899c63d make --u2sz more intuitive
previously, it only accepted the 3-tuple `min,default,max`

if given a single integer (or any other unexpected value),
the up2k js would enter an infinite loop, eat all the ram
and crash the browser (nice)

fix this by accepting a single integer (for example 96)
and translating it to `1,96,96`
2024-10-22 21:37:51 +00:00
ed
3a0d882c5e fix NetMap -j0 compat
would crash on startup if `-j0` was
combined with `--ipa` or `--ipu`
2024-10-22 20:53:19 +00:00
ed
cb81f0ad6d readme: add nintendo 3ds to supported browsers 2024-10-21 00:06:13 +00:00
ed
518bacf628 add pingvin-share to comparison 2024-10-20 01:18:07 +00:00
ed
ca63b03e55 update pkgs to 1.15.9 2024-10-18 23:54:46 +00:00
ed
cecef88d6b v1.15.9 2024-10-18 23:42:20 +00:00
ed
7ffd805a03 add RSS feed output; closes #109 2024-10-18 23:24:12 +00:00
ed
a7e2a0c981 up2k: fix chinese-specific js crash; closes #108
the client-side ETA, included as metadata in POSTs,
would crash the js with the initial "Starting..." text
2024-10-18 19:04:22 +00:00
ed
2a570bb4ca fix --df for webdav; closes #107
PUT uploads, as used by webdav, would stat the absolute
path of the file to be created, which would throw ENOENT

strip components until the path is an existing directory

and also try to enforce disk space / volume size limits
even when the incoming file is of unknown size
2024-10-18 18:14:35 +00:00
ed
5ca8f0706d up2k.js: detect broken webworkers;
the first time a file is to be hashed after a website refresh,
a set of webworkers are launched for efficient parallelization

in the unlikely event of a network outage exactly at this point,
the workers will fail to start, and the hashing would never begin

add a ping/pong sequence to smoketest the workers, and
fallback to hashing on the main-thread when necessary
2024-10-18 16:50:15 +00:00
ed
a9b4436cdc up2k: improve upload retry/timeout
* `js:` make handshake retries more aggressive
* `u2c:` reduce chunks timeout + ^
* `main:` reduce tcp timeout to 128sec (js is 42s)
* `httpcli:` less confusing log messages
2024-10-18 16:24:31 +00:00
ed
5f91999512 update pkgs to 1.15.8 2024-10-16 22:22:29 +00:00
ed
9f000beeaf v1.15.8 2024-10-16 21:53:23 +00:00
ed
ff0a71f212 gallery: play m4v videos 2024-10-16 21:36:11 +00:00
ed
22dfc6ec24 ui-toast: hide countdown if infinite 2024-10-16 21:32:47 +00:00
ed
48147c079e subchunks: fix eta, cfg-ui 2024-10-16 21:17:00 +00:00
ed
d715479ef6 add chickenbit to force hashwasm 2024-10-16 20:23:02 +00:00
ed
fc8298c468 up2k: avoid cloudflare upload size-limit
previously, the biggest file that could be uploaded through
cloudflare was 383 GiB, due to max num chunks being 4096

`--u2sz`, which takes three ints (min-size, target, max-size)
can now be used to enforce a max chunksize; chunks larger
than max-size gets split into smaller subchunks / chunklets

subchunks cannot be stitched/joined, and subchunks of the
same chunk must be uploaded sequentially, one at a time

if a subchunk fails due to bitflips or connection-loss,
then the entire chunk must (and will) be reuploaded
2024-10-16 19:29:08 +00:00
ed
e94ca5dc91 up2k: improve logging 2024-10-16 15:41:19 +00:00
ed
114b71b751 up2k: fix filesystem toctou
previously and currently, as an upload completes, its "done" flag
is not set until all the data has been flushed to disk

however, the list of missing chunks becomes empty before the flush,
and that list was incorrectly used to determine completion state
in some dedup-related logic

as a result, duplicate uploads could initially fail, and would
succeed after the client automatically retried a handful of times
2024-10-16 15:32:58 +00:00
ed
b2770a2087 u2c: support more crazy filenames
newlines, invalid utf8, and worst of all... %20 (whitespace)

due to up2k protocol limitations,
filenames are normalized when they hit the server,
but folders get to keep their intended jank
2024-10-15 23:01:07 +00:00
ed
cba1878bb2 u2c: don't get stuck at fifos and such 2024-10-15 22:53:55 +00:00
ed
a2e037d6af u2c: fix chunksize calculation
files which were exactly 128 GiB large would fail
(you can't make this shit up)
2024-10-15 22:39:48 +00:00
ed
65a2b6a223 u2c: fix excessive FDs
it would open separate FDs for all chunks to be uploaded...

open and close files as they are needed during upload instead
2024-10-15 22:30:15 +00:00
ed
9ed799e803 update pkgs to 1.15.7 2024-10-13 23:07:31 +00:00
132 changed files with 7779 additions and 1456 deletions

View File

@@ -11,30 +11,38 @@ NOTE:
all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md
**Describe the bug**
### Describe the bug
a description of what the bug is
**To Reproduce**
### To Reproduce
List of steps to reproduce the issue, or, if it's hard to reproduce, then at least a detailed explanation of what you did to run into it
**Expected behavior**
### Expected behavior
a description of what you expected to happen
**Screenshots**
### Screenshots
if applicable, add screenshots to help explain your problem, such as the kickass crashpage :^)
**Server details**
if the issue is possibly on the server-side, then mention some of the following:
* server OS / version:
* python version:
* copyparty arguments:
* filesystem (`lsblk -f` on linux):
### Server details (if you are using docker/podman)
remove the ones that are not relevant:
* **server OS / version:**
* **how you're running copyparty:** (docker/podman/something-else)
* **docker image:** (variant, version, and arch if you know)
* **copyparty arguments and/or config-file:**
**Client details**
### Server details (if you're NOT using docker/podman)
remove the ones that are not relevant:
* **server OS / version:**
* **what copyparty did you grab:** (sfx/exe/pip/aur/...)
* **how you're running it:** (in a terminal, as a systemd-service, ...)
* run copyparty with `--version` and grab the last 3 lines (they start with `copyparty`, `CPython`, `sqlite`) and paste them below this line:
* **copyparty arguments and/or config-file:**
### Client details
if the issue is possibly on the client-side, then mention some of the following:
* the device type and model:
* OS version:
* browser version:
**Additional context**
### Additional context
any other context about the problem here

View File

@@ -28,6 +28,8 @@ aside from documentation and ideas, some other things that would be cool to have
* **translations** -- the copyparty web-UI has translations for english and norwegian at the top of [browser.js](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.js); if you'd like to add a translation for another language then that'd be welcome! and if that language has a grammar that doesn't fit into the way the strings are assembled, then we'll fix that as we go :>
* but please note that support for [RTL (Right-to-Left) languages](https://en.wikipedia.org/wiki/Right-to-left_script) is currently not planned, since the javascript is a bit too jank for that
* **UI ideas** -- at some point I was thinking of rewriting the UI in react/preact/something-not-vanilla-javascript, but I'll admit the comfiness of not having any build stage combined with raw performance has kinda convinced me otherwise :p but I'd be very open to ideas on how the UI could be improved, or be more intuitive.
* **docker improvements** -- I don't really know what I'm doing when it comes to containers, so I'm sure there's a *huge* room for improvement here, mainly regarding how you're supposed to use the container with kubernetes / docker-compose / any of the other popular ways to do things. At some point I swear I'll start learning about docker so I can pick up clach04's [docker-compose draft](https://github.com/9001/copyparty/issues/38) and learn how that stuff ticks, unless someone beats me to it!

473
README.md
View File

@@ -47,6 +47,8 @@ turn almost any device into a file server with resumable uploads/downloads using
* [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
* [shares](#shares) - share a file or folder by creating a temporary link
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [rss feeds](#rss-feeds) - monitor a folder with your RSS reader
* [recent uploads](#recent-uploads) - list all recent uploads
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
@@ -78,6 +80,7 @@ turn almost any device into a file server with resumable uploads/downloads using
* [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
* [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
* [event hooks](#event-hooks) - trigger a program on uploads, renames etc ([examples](./bin/hooks/))
* [zeromq](#zeromq) - event-hooks can send zeromq messages
* [upload events](#upload-events) - the older, more powerful approach ([examples](./bin/mtag/))
* [handlers](#handlers) - redefine behavior with plugins ([examples](./bin/handlers/))
* [ip auth](#ip-auth) - autologin based on IP range (CIDR)
@@ -90,9 +93,12 @@ turn almost any device into a file server with resumable uploads/downloads using
* [listen on port 80 and 443](#listen-on-port-80-and-443) - become a *real* webserver
* [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
* [real-ip](#real-ip) - teaching copyparty how to see client IPs
* [reverse-proxy performance](#reverse-proxy-performance)
* [permanent cloudflare tunnel](#permanent-cloudflare-tunnel) - if you have a domain and want to get your copyparty online real quick
* [prometheus](#prometheus) - metrics/stats can be enabled
* [other extremely specific features](#other-extremely-specific-features) - you'll never find a use for these
* [custom mimetypes](#custom-mimetypes) - change the association of a file extension
* [GDPR compliance](#GDPR-compliance) - imagine using copyparty professionally...
* [feature chickenbits](#feature-chickenbits) - buggy feature? rip it out
* [packages](#packages) - the party might be closer than you think
* [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
@@ -138,7 +144,11 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
* or install [on arch](#arch-package) [on NixOS](#nixos-module) [through nix](#nix-package)
* or if you are on android, [install copyparty in termux](#install-on-android)
* or maybe you have a [synology nas / dsm](./docs/synology-dsm.md)
* or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
* or if you don't trust copyparty yet and want to isolate it a little, then...
* ...maybe [prisonparty](./bin/prisonparty.sh) to create a tiny [chroot](https://wiki.archlinux.org/title/Chroot) (very portable),
* ...or [bubbleparty](./bin/bubbleparty.sh) to wrap it in [bubblewrap](https://github.com/containers/bubblewrap) (much better)
* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
* docker has all deps built-in, so skip this step:
@@ -151,8 +161,8 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
* **MacOS:** `port install py-Pillow ffmpeg`
* **MacOS** (alternative): `brew install pillow ffmpeg`
* **Windows:** `python -m pip install --user -U Pillow`
* install python and ffmpeg manually; do not use `winget` or `Microsoft Store` (it breaks $PATH)
* copyparty.exe comes with `Pillow` and only needs `ffmpeg`
* install [python](https://www.python.org/downloads/windows/) and [ffmpeg](#optional-dependencies) manually; do not use `winget` or `Microsoft Store` (it breaks $PATH)
* copyparty.exe comes with `Pillow` and only needs [ffmpeg](#optional-dependencies) for mediatags/videothumbs
* see [optional dependencies](#optional-dependencies) to enable even more features
running copyparty without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; you may want [accounts and volumes](#accounts-and-volumes)
@@ -175,6 +185,8 @@ first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/co
as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you
but if you have a domain, then you probably want to skip the random autogenerated URL and instead make a [permanent cloudflare tunnel](#permanent-cloudflare-tunnel)
since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
@@ -216,10 +228,11 @@ also see [comparison to similar software](./docs/versus.md)
* ☑ [upnp / zeroconf / mdns / ssdp](#zeroconf)
* ☑ [event hooks](#event-hooks) / script runner
* ☑ [reverse-proxy support](https://github.com/9001/copyparty#reverse-proxy)
* ☑ cross-platform (Windows, Linux, Macos, Android, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64)
* upload
* ☑ basic: plain multipart, ie6 support
* ☑ [up2k](#uploading): js, resumable, multithreaded
* **no filesize limit!** ...unless you use Cloudflare, then it's 383.9 GiB
* **no filesize limit!** even on Cloudflare
* ☑ stash: simple PUT filedropper
* ☑ filename randomizer
* ☑ write-only folders
@@ -250,7 +263,7 @@ also see [comparison to similar software](./docs/versus.md)
* ☑ search by name/path/date/size
* ☑ [search by ID3-tags etc.](#searching)
* client support
* ☑ [folder sync](#folder-sync)
* ☑ [folder sync](#folder-sync) (one-way only; full sync will never be supported)
* ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
* ☑ [opengraph](#opengraph) (discord embeds)
* markdown
@@ -338,6 +351,9 @@ same order here too
* [Chrome issue 1352210](https://bugs.chromium.org/p/chromium/issues/detail?id=1352210) -- plaintext http may be faster at filehashing than https (but also extremely CPU-intensive)
* [Chrome issue 383568268](https://issues.chromium.org/issues/383568268) -- filereaders in webworkers can OOM / crash the browser-tab
* copyparty has a workaround which seems to work well enough
* [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files
* Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)
@@ -345,10 +361,19 @@ same order here too
* iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
* `AudioContext` will probably never be a viable workaround as apple introduces new issues faster than they fix current ones
* iPhones: music volume goes on a rollercoaster during song changes
* nothing I can do about it because `AudioContext` is still broken in safari
* iPhones: the preload feature (in the media-player-options tab) can cause a tiny audio glitch 20sec before the end of each song, but disabling it may cause worse iOS bugs to appear instead
* just a hunch, but disabling preloading may cause playback to stop entirely, or possibly mess with bluetooth speakers
* tried to add a tooltip regarding this but looks like apple broke my tooltips
* iPhones: preloaded awo files make safari log MEDIA_ERR_NETWORK errors as playback starts, but the song plays just fine so eh whatever
* awo, opus-weba, is apple's new take on opus support, replacing opus-caf which was technically limited to cbr opus
* iPhones: preloading another awo file may cause playback to stop
* can be somewhat mitigated with `mp.au.play()` in `mp.onpreload` but that can hit a race condition in safari that starts playing the same audio object twice in parallel...
* Windows: folders cannot be accessed if the name ends with `.`
* python or windows bug
@@ -381,6 +406,9 @@ upgrade notes
"frequently" asked questions
* can I change the 🌲 spinning pine-tree loading animation?
* [yeah...](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner) :-(
* is it possible to block read-access to folders unless you know the exact URL for a particular file inside?
* yes, using the [`g` permission](#accounts-and-volumes), see the examples there
* you can also do this with linux filesystem permissions; `chmod 111 music` will make it possible to access files and folders inside the `music` folder but not list the immediate contents -- also works with other software, not just copyparty
@@ -403,6 +431,14 @@ upgrade notes
* copyparty seems to think I am using http, even though the URL is https
* your reverse-proxy is not sending the `X-Forwarded-Proto: https` header; this could be because your reverse-proxy itself is confused. Ensure that none of the intermediates (such as cloudflare) are terminating https before the traffic hits your entrypoint
* thumbnails are broken (you get a colorful square which says the filetype instead)
* you need to install `FFmpeg` or `Pillow`; see [thumbnails](#thumbnails)
* thumbnails are broken (some images appear, but other files just get a blank box, and/or the broken-image placeholder)
* probably due to a reverse-proxy messing with the request URLs and stripping the query parameters (`?th=w`), so check your URL rewrite rules
* could also be due to incorrect caching settings in reverse-proxies and/or CDNs, so make sure that nothing is set to ignore the query string
* could also be due to misbehaving privacy-related browser extensions, so try to disable those
* i want to learn python and/or programming and am considering looking at the copyparty source code in that occasion
* ```bash
_| _ __ _ _|_
@@ -427,7 +463,7 @@ configuring accounts/volumes with arguments:
permissions:
* `r` (read): browse folder contents, download files, download as zip/tar, see filekeys/dirkeys
* `w` (write): upload files, move files *into* this folder
* `w` (write): upload files, move/copy files *into* this folder
* `m` (move): move files/folders *from* this folder
* `d` (delete): delete files/folders
* `.` (dots): user can ask to show dotfiles in directory listings
@@ -455,6 +491,40 @@ examples:
anyone trying to bruteforce a password gets banned according to `--ban-pw`; default is 24h ban for 9 failed attempts in 1 hour
and if you want to use config files instead of commandline args (good!) then here's the same examples as a configfile; save it as `foobar.conf` and use it like this: `python copyparty-sfx.py -c foobar.conf`
```yaml
[accounts]
u1: p1 # create account "u1" with password "p1"
u2: p2 # (note that comments must have
u3: p3 # two spaces before the # sign)
[/] # this URL will be mapped to...
/srv # ...this folder on the server filesystem
accs:
r: * # read-only for everyone, no account necessary
[/music] # create another volume at this URL,
/mnt/music # which is mapped to this folder
accs:
r: u1, u2 # only these accounts can read,
rw: u3 # and only u3 can read-write
[/inc]
/mnt/incoming
accs:
w: u1 # u1 can upload but not see/download any files,
rm: u2 # u2 can browse + move files out of this volume
[/i]
/mnt/ss
accs:
rw: u1 # u1 can read-write,
g: * # everyone can access files if they know the URL
flags:
fk: 4 # each file URL will have a 4-character password
```
## shadowing
@@ -462,6 +532,8 @@ hiding specific subfolders by mounting another volume on top of them
for example `-v /mnt::r -v /var/empty:web/certs:r` mounts the server folder `/mnt` as the webroot, but another volume is mounted at `/web/certs` -- so visitors can only see the contents of `/mnt` and `/mnt/web` (at URLs `/` and `/web`), but not `/mnt/web/certs` because URL `/web/certs` is mapped to `/var/empty`
the example config file right above this section may explain this better; the first volume `/` is mapped to `/srv` which means http://127.0.0.1:3923/music would try to read `/srv/music` on the server filesystem, but since there's another volume at `/music` mapped to `/mnt/music` then it'll go to `/mnt/music` instead
## dotfiles
@@ -473,6 +545,19 @@ a client can request to see dotfiles in directory listings if global option `-ed
dotfiles do not appear in search results unless one of the above is true, **and** the global option / volflag `dotsrch` is set
config file example, where the same permission to see dotfiles is given in two different ways just for reference:
```yaml
[/foo]
/srv/foo
accs:
r.: ed # user "ed" has read-access + dot-access in this volume;
# dotfiles are visible in listings, but not in searches
flags:
dotsrch # dotfiles will now appear in search results too
dots # another way to let everyone see dotfiles in this vol
```
# the browser
@@ -507,7 +592,8 @@ the browser has the following hotkeys (always qwerty)
* `ESC` close various things
* `ctrl-K` delete selected files/folders
* `ctrl-X` cut selected files/folders
* `ctrl-V` paste
* `ctrl-C` copy selected files/folders to clipboard
* `ctrl-V` paste (move/copy)
* `Y` download selected files
* `F2` [rename](#batch-rename) selected file/folder
* when a file/folder is selected (in not-grid-view):
@@ -576,12 +662,14 @@ click the `🌲` or pressing the `B` hotkey to toggle between breadcrumbs path (
press `g` or `` to toggle grid-view instead of the file listing and `t` toggles icons / thumbnails
* can be made default globally with `--grid` or per-volume with volflag `grid`
* enable by adding `?imgs` to a link, or disable with `?imgs=0`
![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
* pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
* disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
* for installing FFmpeg on windows, see [optional dependencies](#optional-dependencies)
audio files are converted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
@@ -593,6 +681,26 @@ enabling `multiselect` lets you click files to select them, and then shift-click
* `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
* the `sel` option can be made default globally with `--gsel` or per-volume with volflag `gsel`
to show `/icons/exe.png` as the thumbnail for all .exe files, `--ext-th=exe=/icons/exe.png` (optionally as a volflag)
config file example:
```yaml
[global]
no-thumb # disable ALL thumbnails and audio transcoding
no-vthumb # only disable video thumbnails
[/music]
/mnt/nas/music
accs:
r: * # everyone can read
flags:
dthumb # disable ALL thumbnails and audio transcoding
dvthumb # only disable video thumbnails
ext-th: exe=/ico/exe.png # /ico/exe.png is the thumbnail of *.exe
th-covers: folder.png,folder.jpg,cover.png,cover.jpg # the default
```
## zip downloads
@@ -606,8 +714,8 @@ select which type of archive you want in the `[⚙️] config` tab:
| `pax` | `?tar=pax` | pax-format tar, futureproof, not as fast |
| `tgz` | `?tar=gz` | gzip compressed gnu-tar (slow), for `curl \| tar -xvz` |
| `txz` | `?tar=xz` | gnu-tar with xz / lzma compression (v.slow) |
| `zip` | `?zip=utf8` | works everywhere, glitchy filenames on win7 and older |
| `zip_dos` | `?zip` | traditional cp437 (no unicode) to fix glitchy filenames |
| `zip` | `?zip` | works everywhere, glitchy filenames on win7 and older |
| `zip_dos` | `?zip=dos` | traditional cp437 (no unicode) to fix glitchy filenames |
| `zip_crc` | `?zip=crc` | cp437 with crc32 computed early for truly ancient software |
* gzip default level is `3` (0=fast, 9=best), change with `?tar=gz:9`
@@ -615,7 +723,7 @@ select which type of archive you want in the `[⚙️] config` tab:
* bz2 default level is `2` (1=fast, 9=best), change with `?tar=bz2:9`
* hidden files ([dotfiles](#dotfiles)) are excluded unless account is allowed to list them
* `up2k.db` and `dir.txt` is always excluded
* bsdtar supports streaming unzipping: `curl foo?zip=utf8 | bsdtar -xv`
* bsdtar supports streaming unzipping: `curl foo?zip | bsdtar -xv`
* good, because copyparty's zip is faster than tar on small files
* `zip_crc` will take longer to download since the server has to read each file twice
* this is only to support MS-DOS PKZIP v2.04g (october 1993) and older
@@ -639,7 +747,7 @@ dragdrop is the recommended way, but you may also:
* select some files (not folders) in your file explorer and press CTRL-V inside the browser window
* use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy)
* upload using [curl or sharex](#client-examples)
* upload using [curl, sharex, ishare, ...](#client-examples)
when uploading files through dragdrop or CTRL-V, this initiates an upload using `up2k`; there are two browser-based uploaders available:
* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
@@ -654,7 +762,7 @@ up2k has several advantages:
* uploads resume if you reboot your browser or pc, just upload the same files again
* server detects any corruption; the client reuploads affected chunks
* the client doesn't upload anything that already exists on the server
* no filesize limit unless imposed by a proxy, for example Cloudflare, which blocks uploads over 383.9 GiB
* no filesize limit, even when a proxy limits the request size (for example Cloudflare)
* much higher speeds than ftp/scp/tarpipe on some internet connections (mainly american ones) thanks to parallel connections
* the last-modified timestamp of the file is preserved
@@ -673,8 +781,11 @@ the up2k UI is the epitome of polished intuitive experiences:
* "parallel uploads" specifies how many chunks to upload at the same time
* `[🏃]` analysis of other files should continue while one is uploading
* `[🥔]` shows a simpler UI for faster uploads from slow devices
* `[🛡️]` decides when to overwrite existing files on the server
* `🛡️` = never (generate a new filename instead)
* `🕒` = overwrite if the server-file is older
* `♻️` = always overwrite if the files are different
* `[🎲]` generate random filenames during upload
* `[📅]` preserve last-modified timestamps; server times will match yours
* `[🔎]` switch between upload and [file-search](#file-search) mode
* ignore `[🔎]` if you add files by dragging them into the browser
@@ -690,6 +801,8 @@ note that since up2k has to read each file twice, `[🎈] bup` can *theoreticall
if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option `--u2sz` (the default is 96 MiB to support Cloudflare)
### file-search
@@ -709,12 +822,20 @@ files go into `[ok]` if they exist (and you get a link to where it is), otherwis
### unpost
undo/delete accidental uploads
undo/delete accidental uploads using the `[🧯]` tab in the UI
![copyparty-unpost-fs8](https://user-images.githubusercontent.com/241032/129635368-3afa6634-c20f-418c-90dc-ec411f3b3897.png)
you can unpost even if you don't have regular move/delete access, however only for files uploaded within the past `--unpost` seconds (default 12 hours) and the server must be running with `-e2d`
config file example:
```yaml
[global]
e2d # enable up2k database (remember uploads)
unpost: 43200 # 12 hours (default)
```
### self-destruct
@@ -753,10 +874,11 @@ file selection: click somewhere on the line (not the link itself), then:
* shift-click another line for range-select
* cut: select some files and `ctrl-x`
* copy: select some files and `ctrl-c`
* paste: `ctrl-v` in another folder
* rename: `F2`
you can move files across browser tabs (cut in one tab, paste in another)
you can copy/move files across browser tabs (cut/copy in one tab, paste in another)
## shares
@@ -843,6 +965,52 @@ or a mix of both:
the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
## rss feeds
monitor a folder with your RSS reader , optionally recursive
must be enabled per-volume with volflag `rss` or globally with `--rss`
the feed includes itunes metadata for use with podcast readers such as [AntennaPod](https://antennapod.org/)
a feed example: https://cd.ocv.me/a/d2/d22/?rss&fext=mp3
url parameters:
* `pw=hunter2` for password auth
* `recursive` to also include subfolders
* `title=foo` changes the feed title (default: folder name)
* `fext=mp3,opus` only include mp3 and opus files (default: all)
* `nf=30` only show the first 30 results (default: 250)
* `sort=m` sort by mtime (file last-modified), newest first (default)
* `u` = upload-time; NOTE: non-uploaded files have upload-time `0`
* `n` = filename
* `a` = filesize
* uppercase = reverse-sort; `M` = oldest file first
## recent uploads
list all recent uploads by clicking "show recent uploads" in the controlpanel
will show uploader IP and upload-time if the visitor has the admin permission
* global-option `--ups-when` makes upload-time visible to all users, and not just admins
* global-option `--ups-who` (volflag `ups_who`) specifies who gets access (0=nobody, 1=admins, 2=everyone), default=2
note that the [🧯 unpost](#unpost) feature is better suited for viewing *your own* recent uploads, as it includes the option to undo/delete them
config file example:
```yaml
[global]
ups-when # everyone can see upload times
ups-who: 1 # but only admins can see the list,
# so ups-when doesn't take effect
```
## media player
plays almost every audio format there is (if the server has FFmpeg installed for on-demand transcoding)
@@ -879,6 +1047,11 @@ open the `[🎺]` media-player-settings tab to configure it,
* `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
* `[oth]` converts all other known formats into opus (if supported by browser) or mp3
* `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
* "transcode to":
* `[opus]` produces an `opus` whenever transcoding is necessary (the best choice on Android and PCs)
* `[awo]` is `opus` in a `weba` file, good for iPhones (iOS 17.5 and newer) but Apple is still fixing some state-confusion bugs as of iOS 18.2.1
* `[caf]` is `opus` in a `caf` file, good for iPhones (iOS 11 through 17), technically unsupported by Apple but works for the mos tpart
* `[mp3]` -- the myth, the legend, the undying master of mediocre sound quality that definitely works everywhere
* "tint" reduces the contrast of the playback bar
@@ -981,7 +1154,16 @@ using arguments or config files, or a mix of both:
announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png)) -- `-z` enables both [mdns](#mdns) and [ssdp](#ssdp)
* `--z-on` / `--z-off`' limits the feature to certain networks
* `--z-on` / `--z-off` limits the feature to certain networks
config file example:
```yaml
[global]
z # enable all zeroconf features (mdns, ssdp)
zm # only enables mdns (does nothing since we already have z)
z-on: 192.168.0.0/16, 10.1.2.0/24 # restrict to certain subnets
```
### mdns
@@ -1059,6 +1241,8 @@ on macos, connect from finder:
in order to grant full write-access to webdav clients, the volflag `daw` must be set and the account must also have delete-access (otherwise the client won't be allowed to replace the contents of existing files, which is how webdav works)
> note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)
### connecting to webdav from windows
@@ -1067,11 +1251,12 @@ using the GUI (winXP or later):
* on winXP only, click the `Sign up for online storage` hyperlink instead and put the URL there
* providing your password as the username is recommended; the password field can be anything or empty
known client bugs:
the webdav client that's built into windows has the following list of bugs; you can avoid all of these by connecting with rclone instead:
* win7+ doesn't actually send the password to the server when reauthenticating after a reboot unless you first try to login with an incorrect password and then switch to the correct password
* or just type your password into the username field instead to get around it entirely
* connecting to a folder which allows anonymous read will make writing impossible, as windows has decided it doesn't need to login
* workaround: connect twice; first to a folder which requires auth, then to the folder you actually want, and leave both of those mounted
* or set the server-option `--dav-auth` to force password-auth for all webdav clients
* win7+ may open a new tcp connection for every file and sometimes forgets to close them, eventually needing a reboot
* maybe NIC-related (??), happens with win10-ltsc on e1000e but not virtio
* windows cannot access folders which contain filenames with invalid unicode or forbidden characters (`<>:"/\|?*`), or names ending with `.`
@@ -1119,7 +1304,7 @@ dependencies: `python3 -m pip install --user -U impacket==0.11.0`
some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
* not entirely confident that read-only is read-only
* the smb backend is not fully integrated with vfs, meaning there could be security issues (path traversal). Please use `--smb-port` (see below) and [prisonparty](./bin/prisonparty.sh)
* the smb backend is not fully integrated with vfs, meaning there could be security issues (path traversal). Please use `--smb-port` (see below) and [prisonparty](./bin/prisonparty.sh) or [bubbleparty](./bin/bubbleparty.sh)
* account passwords work per-volume as expected, and so does account permissions (read/write/move/delete), but `--smbw` must be given to allow write-access from smb
* [shadowing](#shadowing) probably works as expected but no guarantees
@@ -1205,6 +1390,18 @@ advantages of using symlinks (default):
global-option `--xlink` / volflag `xlink` additionally enables deduplication across volumes, but this is probably buggy and not recommended
config file example:
```yaml
[global]
e2dsa # scan and index filesystem on startup
dedup # symlink-based deduplication for all volumes
[/media]
/mnt/nas/media
flags:
hardlinkonly # this vol does hardlinks instead of symlinks
```
## file indexing
@@ -1236,19 +1433,41 @@ note:
* `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
* the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
config file example (these options are recommended btw):
```yaml
[global]
e2dsa # scan and index all files in all volumes on startup
e2ts # check newly-discovered or uploaded files for media tags
```
### exclude-patterns
to save some time, you can provide a regex pattern for filepaths to only index by filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash \.iso$` or the volflag `:c,nohash=\.iso$`, this has the following consequences:
to save some time, you can provide a regex pattern for filepaths to only index by filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash '\.iso$'` or the volflag `:c,nohash=\.iso$`, this has the following consequences:
* initial indexing is way faster, especially when the volume is on a network disk
* makes it impossible to [file-search](#file-search)
* if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
similarly, you can fully ignore files/folders using `--no-idx [...]` and `:c,noidx=\.iso$`
NOTE: `no-idx` and/or `no-hash` prevents deduplication of those files
* when running on macos, all the usual apple metadata files are excluded by default
if you set `--no-hash [...]` globally, you can enable hashing for specific volumes using flag `:c,nohash=`
to exclude certain filepaths from search-results, use `--srch-excl` or volflag `srch_excl` instead of `--no-idx`, for example `--srch-excl 'password|logs/[0-9]'`
config file example:
```yaml
[/games]
/mnt/nas/games
flags:
noidx: \.iso$ # skip indexing iso-files
srch_excl: password|logs/[0-9] # filter search results
```
### filesystem guards
avoid traversing into other filesystems using `--xdev` / volflag `:c,xdev`, skipping any symlinks or bind-mounts to another HDD for example
@@ -1269,6 +1488,20 @@ argument `--re-maxage 60` will rescan all volumes every 60 sec, same as volflag
uploads are disabled while a rescan is happening, so rescans will be delayed by `--db-act` (default 10 sec) when there is write-activity going on (uploads, renames, ...)
note: folder-thumbnails are selected during filesystem indexing, so periodic rescans can be used to keep them accurate as images are uploaded/deleted (or manually do a rescan with the `reload` button in the controlpanel)
config file example:
```yaml
[global]
re-maxage: 3600
[/pics]
/mnt/nas/pics
flags:
scan: 900
```
## upload rules
@@ -1294,6 +1527,26 @@ you can also set transaction limits which apply per-IP and per-volume, but these
notes:
* `vmaxb` and `vmaxn` requires either the `e2ds` volflag or `-e2dsa` global-option
config file example:
```yaml
[/inc]
/mnt/nas/uploads
accs:
w: * # anyone can upload here
rw: ed # only user "ed" can read-write
flags:
e2ds: # filesystem indexing is required for many of these:
sz: 1k-3m # accept upload only if filesize in this range
df: 4g # free disk space cannot go lower than this
vmaxb: 1g # volume can never exceed 1 GiB
vmaxn: 4k # ...or 4000 files, whichever comes first
nosub # must upload to toplevel folder
lifetime: 300 # uploads are deleted after 5min
maxn: 250,3600 # each IP can upload 250 files in 1 hour
maxb: 1g,300 # each IP can upload 1 GiB over 5 minutes
```
## compress uploads
@@ -1339,10 +1592,24 @@ this can instead be kept in a single place using the `--hist` argument, or the `
* `--hist ~/.cache/copyparty -v ~/music::r:c,hist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
note:
* putting the hist-folders on an SSD is strongly recommended for performance
* markdown edits are always stored in a local `.hist` subdirectory
* on windows the volflag path is cyglike, so `/c/temp` means `C:\temp` but use regular paths for `--hist`
* you can use cygpaths for volumes too, `-v C:\Users::r` and `-v /c/users::r` both work
config file example:
```yaml
[global]
hist: ~/.cache/copyparty # put db/thumbs/etc. here by default
[/pics]
/mnt/nas/pics
flags:
hist: - # restore the default (/mnt/nas/pics/.hist/)
hist: /mnt/nas/cache/pics/ # can be absolute path
```
## metadata from audio files
@@ -1394,6 +1661,18 @@ copyparty can invoke external programs to collect additional metadata for files
if something doesn't work, try `--mtag-v` for verbose error messages
config file example; note that `mtp` is an additive option so all of the mtp options will take effect:
```yaml
[/music]
/mnt/nas/music
flags:
mtp: .bpm=~/bin/audio-bpm.py # assign ".bpm" (numeric) with script
mtp: key=f,t5,~/bin/audio-key.py # force/overwrite, 5sec timeout
mtp: ext=an,~/bin/file-ext.py # will only run on non-audio files
mtp: arch,built,ver,orig=an,eexe,edll,~/bin/exe.py # only exe/dll
```
## event hooks
@@ -1406,12 +1685,51 @@ there's a bunch of flags and stuff, see `--help-hooks`
if you want to write your own hooks, see [devnotes](./docs/devnotes.md#event-hooks)
### zeromq
event-hooks can send zeromq messages instead of running programs
to send a 0mq message every time a file is uploaded,
* `--xau zmq:pub:tcp://*:5556` sends a PUB to any/all connected SUB clients
* `--xau t3,zmq:push:tcp://*:5557` sends a PUSH to exactly one connected PULL client
* `--xau t3,j,zmq:req:tcp://localhost:5555` sends a REQ to the connected REP client
the PUSH and REQ examples have `t3` (timeout after 3 seconds) because they block if there's no clients to talk to
* the REQ example does `t3,j` to send extended upload-info as json instead of just the filesystem-path
see [zmq-recv.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/zmq-recv.py) if you need something to receive the messages with
config file example; note that the hooks are additive options, so all of the xau options will take effect:
```yaml
[global]
xau: zmq:pub:tcp://*:5556` # send a PUB to any/all connected SUB clients
xau: t3,zmq:push:tcp://*:5557` # send PUSH to exactly one connected PULL cli
xau: t3,j,zmq:req:tcp://localhost:5555` # send REQ to the connected REP cli
```
### upload events
the older, more powerful approach ([examples](./bin/mtag/)):
```
-v /mnt/inc:inc:w:c,mte=+x1:c,mtp=x1=ad,kn,/usr/bin/notify-send
-v /mnt/inc:inc:w:c,e2d,e2t,mte=+x1:c,mtp=x1=ad,kn,/usr/bin/notify-send
```
that was the commandline example; here's the config file example:
```yaml
[/inc]
/mnt/inc
accs:
w: *
flags:
e2d, e2t # enable indexing of uploaded files and their tags
mte: +x1
mtp: x1=ad,kn,/usr/bin/notify-send
```
so filesystem location `/mnt/inc` shared at `/inc`, write-only for everyone, appending `x1` to the list of tags to index (`mte`), and using `/usr/bin/notify-send` to "provide" tag `x1` for any filetype (`ad`) with kill-on-timeout disabled (`kn`)
@@ -1425,6 +1743,8 @@ note that this is way more complicated than the new [event hooks](#event-hooks)
note that it will occupy the parsing threads, so fork anything expensive (or set `kn` to have copyparty fork it for you) -- otoh if you want to intentionally queue/singlethread you can combine it with `--mtag-mt 1`
for reference, if you were to do this using event hooks instead, it would be like this: `-e2d --xau notify-send,hello,--`
## handlers
@@ -1432,6 +1752,8 @@ redefine behavior with plugins ([examples](./bin/handlers/))
replace 404 and 403 errors with something completely different (that's it for now)
as for client-side stuff, there is [plugins for modifying UI/UX](./contrib/plugins/)
## ip auth
@@ -1455,7 +1777,9 @@ replace copyparty passwords with oauth and such
you can disable the built-in password-based login system, and instead replace it with a separate piece of software (an identity provider) which will then handle authenticating / authorizing of users; this makes it possible to login with passkeys / fido2 / webauthn / yubikey / ldap / active directory / oauth / many other single-sign-on contraptions
a popular choice is [Authelia](https://www.authelia.com/) (config-file based), another one is [authentik](https://goauthentik.io/) (GUI-based, more complex)
* the regular config-defined users will be used as a fallback for requests which don't include a valid (trusted) IdP username header
some popular identity providers are [Authelia](https://www.authelia.com/) (config-file based) and [authentik](https://goauthentik.io/) (GUI-based, more complex)
there is a [docker-compose example](./docs/examples/docker/idp-authelia-traefik) which is hopefully a good starting point (alternatively see [./docs/idp.md](./docs/idp.md) if you're the DIY type)
@@ -1491,12 +1815,18 @@ connecting to an aws s3 bucket and similar
there is no built-in support for this, but you can use FUSE-software such as [rclone](https://rclone.org/) / [geesefs](https://github.com/yandex-cloud/geesefs) / [JuiceFS](https://juicefs.com/en/) to first mount your cloud storage as a local disk, and then let copyparty use (a folder in) that disk as a volume
you may experience poor upload performance this way, but that can sometimes be fixed by specifying the volflag `sparse` to force the use of sparse files; this has improved the upload speeds from `1.5 MiB/s` to over `80 MiB/s` in one case, but note that you are also more likely to discover funny bugs in your FUSE software this way, so buckle up
if copyparty is unable to access the local folder that rclone/geesefs/JuiceFS provides (for example if it looks invisible) then you may need to run rclone with `--allow-other` and/or enable `user_allow_other` in `/etc/fuse.conf`
you will probably get decent speeds with the default config, however most likely restricted to using one TCP connection per file, so the upload-client won't be able to send multiple chunks in parallel
> before [v1.13.5](https://github.com/9001/copyparty/releases/tag/v1.13.5) it was recommended to use the volflag `sparse` to force-allow multiple chunks in parallel; this would improve the upload-speed from `1.5 MiB/s` to over `80 MiB/s` at the risk of provoking latent bugs in S3 or JuiceFS. But v1.13.5 added chunk-stitching, so this is now probably much less important. On the contrary, `nosparse` *may* now increase performance in some cases. Please try all three options (default, `sparse`, `nosparse`) as the optimal choice depends on your network conditions and software stack (both the FUSE-driver and cloud-server)
someone has also tested geesefs in combination with [gocryptfs](https://nuetzlich.net/gocryptfs/) with surprisingly good results, getting 60 MiB/s upload speeds on a gbit line, but JuiceFS won with 80 MiB/s using its built-in encryption
you may improve performance by specifying larger values for `--iobuf` / `--s-rd-sz` / `--s-wr-sz`
> if you've experimented with this and made interesting observations, please share your findings so we can add a section with specific recommendations :-)
## hiding from google
@@ -1619,10 +1949,16 @@ some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatical
for improved security (and a 10% performance boost) consider listening on a unix-socket with `-i unix:770:www:/tmp/party.sock` (permission `770` means only members of group `www` can access it)
example webserver configs:
example webserver / reverse-proxy configs:
* [nginx config](contrib/nginx/copyparty.conf) -- entire domain/subdomain
* [apache2 config](contrib/apache/copyparty.conf) -- location-based
* [apache config](contrib/apache/copyparty.conf)
* caddy uds: `caddy reverse-proxy --from :8080 --to unix///dev/shm/party.sock`
* caddy tcp: `caddy reverse-proxy --from :8081 --to http://127.0.0.1:3923`
* [haproxy config](contrib/haproxy/copyparty.conf)
* [lighttpd subdomain](contrib/lighttpd/subdomain.conf) -- entire domain/subdomain
* [lighttpd subpath](contrib/lighttpd/subpath.conf) -- location-based (not optimal, but in case you need it)
* [nginx config](contrib/nginx/copyparty.conf) -- recommended
* [traefik config](contrib/traefik/copyparty.yaml)
### real-ip
@@ -1634,6 +1970,58 @@ if you (and maybe everybody else) keep getting a message that says `thank you fo
for most common setups, there should be a helpful message in the server-log explaining what to do, but see [docs/xff.md](docs/xff.md) if you want to learn more, including a quick hack to **just make it work** (which is **not** recommended, but hey...)
### reverse-proxy performance
most reverse-proxies support connecting to copyparty either using uds/unix-sockets (`/dev/shm/party.sock`, faster/recommended) or using tcp (`127.0.0.1`)
with copyparty listening on a uds / unix-socket / unix-domain-socket and the reverse-proxy connecting to that:
| index.html | upload | download | software |
| ------------ | ----------- | ----------- | -------- |
| 28'900 req/s | 6'900 MiB/s | 7'400 MiB/s | no-proxy |
| 18'750 req/s | 3'500 MiB/s | 2'370 MiB/s | haproxy |
| 9'900 req/s | 3'750 MiB/s | 2'200 MiB/s | caddy |
| 18'700 req/s | 2'200 MiB/s | 1'570 MiB/s | nginx |
| 9'700 req/s | 1'750 MiB/s | 1'830 MiB/s | apache |
| 9'900 req/s | 1'300 MiB/s | 1'470 MiB/s | lighttpd |
when connecting the reverse-proxy to `127.0.0.1` instead (the basic and/or old-fasioned way), speeds are a bit worse:
| index.html | upload | download | software |
| ------------ | ----------- | ----------- | -------- |
| 21'200 req/s | 5'700 MiB/s | 6'700 MiB/s | no-proxy |
| 14'500 req/s | 1'700 MiB/s | 2'170 MiB/s | haproxy |
| 11'100 req/s | 2'750 MiB/s | 2'000 MiB/s | traefik |
| 8'400 req/s | 2'300 MiB/s | 1'950 MiB/s | caddy |
| 13'400 req/s | 1'100 MiB/s | 1'480 MiB/s | nginx |
| 8'400 req/s | 1'000 MiB/s | 1'000 MiB/s | apache |
| 6'500 req/s | 1'270 MiB/s | 1'500 MiB/s | lighttpd |
in summary, `haproxy > caddy > traefik > nginx > apache > lighttpd`, and use uds when possible (traefik does not support it yet)
* if these results are bullshit because my config exampels are bad, please submit corrections!
## permanent cloudflare tunnel
if you have a domain and want to get your copyparty online real quick, either from your home-PC behind a CGNAT or from a server without an existing [reverse-proxy](#reverse-proxy) setup, one approach is to create a [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/) (formerly "Argo Tunnel")
I'd recommend making a `Locally-managed tunnel` for more control, but if you prefer to make a `Remotely-managed tunnel` then this is currently how:
* `cloudflare dashboard` » `zero trust` » `networks` » `tunnels` » `create a tunnel` » `cloudflared` » choose a cool `subdomain` and leave the `path` blank, and use `service type` = `http` and `URL` = `127.0.0.1:3923`
* and if you want to just run the tunnel without installing it, skip the `cloudflared service install BASE64` step and instead do `cloudflared --no-autoupdate tunnel run --token BASE64`
NOTE: since people will be connecting through cloudflare, as mentioned in [real-ip](#real-ip) you should run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
config file example:
```yaml
[global]
xff-hdr: cf-connecting-ip
```
## prometheus
metrics/stats can be enabled at URL `/.cpr/metrics` for grafana / prometheus / etc (openmetrics 1.0.0)
@@ -1657,6 +2045,7 @@ scrape_configs:
currently the following metrics are available,
* `cpp_uptime_seconds` time since last copyparty restart
* `cpp_boot_unixtime_seconds` same but as an absolute timestamp
* `cpp_active_dl` number of active downloads
* `cpp_http_conns` number of open http(s) connections
* `cpp_http_reqs` number of http(s) requests handled
* `cpp_sus_reqs` number of 403/422/malicious requests
@@ -1708,7 +2097,7 @@ change the association of a file extension
using commandline args, you can do something like `--mime gif=image/jif` and `--mime ts=text/x.typescript` (can be specified multiple times)
in a config-file, this is the same as:
in a config file, this is the same as:
```yaml
[global]
@@ -1719,6 +2108,18 @@ in a config-file, this is the same as:
run copyparty with `--mimes` to list all the default mappings
### GDPR compliance
imagine using copyparty professionally... **TINLA/IANAL; EU laws are hella confusing**
* remember to disable logging, or configure logrotation to an acceptable timeframe with `-lo cpp-%Y-%m%d.txt.xz` or similar
* if running with the database enabled (recommended), then have it forget uploader-IPs after some time using `--forget-ip 43200`
* don't set it too low; [unposting](#unpost) a file is no longer possible after this takes effect
* if you actually *are* a lawyer then I'm open for feedback, would be fun
### feature chickenbits
buggy feature? rip it out by setting any of the following environment variables to disable its associated bell or whistle,
@@ -1906,6 +2307,10 @@ quick summary of more eccentric web-browsers trying to view a directory index:
| **ie4** and **netscape** 4.0 | can browse, upload with `?b=u`, auth with `&pw=wark` |
| **ncsa mosaic** 2.7 | does not get a pass, [pic1](https://user-images.githubusercontent.com/241032/174189227-ae816026-cf6f-4be5-a26e-1b3b072c1b2f.png) - [pic2](https://user-images.githubusercontent.com/241032/174189225-5651c059-5152-46e9-ac26-7e98e497901b.png) |
| **SerenityOS** (7e98457) | hits a page fault, works with `?b=u`, file upload not-impl |
| **sony psp** 5.50 | can browse, upload/mkdir/msg (thx dwarf) [screenshot](https://github.com/user-attachments/assets/9d21f020-1110-4652-abeb-6fc09c533d4f) |
| **nintendo 3ds** | can browse, upload, view thumbnails (thx bnjmn) |
<p align="center"><img src="https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853" /></p>
# client examples
@@ -1943,7 +2348,8 @@ interact with copyparty using non-browser clients
* can be downloaded from copyparty: controlpanel -> connect -> [partyfuse.py](http://127.0.0.1:3923/.cpr/a/partyfuse.py)
* [rclone](https://rclone.org/) as client can give ~5x performance, see [./docs/rclone.md](docs/rclone.md)
* sharex (screenshot utility): see [./contrib/sharex.sxcu](contrib/#sharexsxcu)
* sharex (screenshot utility): see [./contrib/sharex.sxcu](./contrib/#sharexsxcu)
* and for screenshots on macos, see [./contrib/ishare.iscu](./contrib/#ishareiscu)
* and for screenshots on linux, see [./contrib/flameshot.sh](./contrib/flameshot.sh)
* contextlet (web browser integration); see [contrib contextlet](contrib/#send-to-cppcontextletjson)
@@ -1964,6 +2370,8 @@ NOTE: curl will not send the original filename if you use `-T` combined with url
sync folders to/from copyparty
NOTE: full bidirectional sync, like what [nextcloud](https://docs.nextcloud.com/server/latest/user_manual/sv/files/desktop_mobile_sync.html) and [syncthing](https://syncthing.net/) does, will never be supported! Only single-direction sync (server-to-client, or client-to-server) is possible with copyparty
the commandline uploader [u2c.py](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy) with `--dr` is the best way to sync a folder to copyparty; verifies checksums and does files in parallel, and deletes unexpected files on the server after upload has finished which makes file-renames really cheap (it'll rename serverside and skip uploading)
alternatively there is [rclone](./docs/rclone.md) which allows for bidirectional sync and is *way* more flexible (stream files straight from sftp/s3/gcs to copyparty, ...), although there is no integrity check and it won't work with files over 100 MiB if copyparty is behind cloudflare
@@ -2207,13 +2615,13 @@ mandatory deps:
install these to enable bonus features
enable hashed passwords in config: `argon2-cffi`
enable [hashed passwords](#password-hashing) in config: `argon2-cffi`
enable ftp-server:
enable [ftp-server](#ftp-server):
* for just plaintext FTP, `pyftpdlib` (is built into the SFX)
* with TLS encryption, `pyftpdlib pyopenssl`
enable music tags:
enable [music tags](#metadata-from-audio-files):
* either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
* or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
@@ -2224,11 +2632,14 @@ enable [thumbnails](#thumbnails) of...
* **AVIF pictures:** `pyvips` or `ffmpeg` or `pillow-avif-plugin`
* **JPEG XL pictures:** `pyvips` or `ffmpeg`
enable [smb](#smb-server) support (**not** recommended):
* `impacket==0.12.0`
enable sending [zeromq messages](#zeromq) from event-hooks: `pyzmq`
enable [smb](#smb-server) support (**not** recommended): `impacket==0.12.0`
`pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
to install FFmpeg on Windows, grab [a recent build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) -- you need `ffmpeg.exe` and `ffprobe.exe` from inside the `bin` folder; copy them into `C:\Windows\System32` or any other folder that's in your `%PATH%`
### dependency chickenbits

View File

@@ -78,3 +78,6 @@ cd /mnt/nas/music/.hist
# [`prisonparty.sh`](prisonparty.sh)
* run copyparty in a chroot, preventing any accidental file access
* creates bindmounts for /bin, /lib, and so on, see `sysdirs=`
# [`bubbleparty.sh`](bubbleparty.sh)
* run copyparty in an isolated process, preventing any accidental file access and more

19
bin/bubbleparty.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/bin/sh
# usage: ./bubbleparty.sh ./copyparty-sfx.py ....
bwrap \
--unshare-all \
--ro-bind /usr /usr \
--ro-bind /bin /bin \
--ro-bind /lib /lib \
--ro-bind /etc/resolv.conf /etc/resolv.conf \
--dev-bind /dev /dev \
--dir /tmp \
--dir /var \
--bind $(pwd) $(pwd) \
--share-net \
--die-with-parent \
--file 11 /etc/passwd \
--file 12 /etc/group \
"$@" \
11< <(getent passwd $(id -u) 65534) \
12< <(getent group $(id -g) 65534)

View File

@@ -20,6 +20,8 @@ each plugin must define a `main()` which takes 3 arguments;
## on404
* [redirect.py](redirect.py) sends an HTTP 301 or 302, redirecting the client to another page/file
* [randpic.py](randpic.py) redirects `/foo/bar/randpic.jpg` to a random pic in `/foo/bar/`
* [sorry.py](answer.py) replies with a custom message instead of the usual 404
* [nooo.py](nooo.py) replies with an endless noooooooooooooo
* [never404.py](never404.py) 100% guarantee that 404 will never be a thing again as it automatically creates dummy files whenever necessary

35
bin/handlers/randpic.py Normal file
View File

@@ -0,0 +1,35 @@
import os
import random
from urllib.parse import quote
# assuming /foo/bar/ is a valid URL but /foo/bar/randpic.png does not exist,
# hijack the 404 with a redirect to a random pic in that folder
#
# thx to lia & kipu for the idea
def main(cli, vn, rem):
req_fn = rem.split("/")[-1]
if not cli.can_read or not req_fn.startswith("randpic"):
return
req_abspath = vn.canonical(rem)
req_ap_dir = os.path.dirname(req_abspath)
files_in_dir = os.listdir(req_ap_dir)
if "." in req_fn:
file_ext = "." + req_fn.split(".")[-1]
files_in_dir = [x for x in files_in_dir if x.lower().endswith(file_ext)]
if not files_in_dir:
return
selected_file = random.choice(files_in_dir)
req_url = "/".join([vn.vpath, rem]).strip("/")
req_dir = req_url.rsplit("/", 1)[0]
new_url = "/".join([req_dir, quote(selected_file)]).strip("/")
cli.reply(b"redirecting...", 302, headers={"Location": "/" + new_url})
return "true"

52
bin/handlers/redirect.py Normal file
View File

@@ -0,0 +1,52 @@
# if someone hits a 404, redirect them to another location
def send_http_302_temporary_redirect(cli, new_path):
"""
replies with an HTTP 302, which is a temporary redirect;
"new_path" can be any of the following:
- "http://a.com/" would redirect to another website,
- "/foo/bar" would redirect to /foo/bar on the same server;
note the leading '/' in the location which is important
"""
cli.reply(b"redirecting...", 302, headers={"Location": new_path})
def send_http_301_permanent_redirect(cli, new_path):
"""
replies with an HTTP 301, which is a permanent redirect;
otherwise identical to send_http_302_temporary_redirect
"""
cli.reply(b"redirecting...", 301, headers={"Location": new_path})
def send_errorpage_with_redirect_link(cli, new_path):
"""
replies with a website explaining that the page has moved;
"new_path" must be an absolute location on the same server
but without a leading '/', so for example "foo/bar"
would redirect to "/foo/bar"
"""
cli.redirect(new_path, click=False, msg="this page has moved")
def main(cli, vn, rem):
"""
this is the function that gets called by copyparty;
note that vn.vpath and cli.vpath does not have a leading '/'
so we're adding the slash in the debug messages below
"""
print(f"this client just hit a 404: {cli.ip}")
print(f"they were accessing this volume: /{vn.vpath}")
print(f"and the original request-path (straight from the URL) was /{cli.vpath}")
print(f"...which resolves to the following filesystem path: {vn.canonical(rem)}")
new_path = "/foo/bar/"
print(f"will now redirect the client to {new_path}")
# uncomment one of these:
send_http_302_temporary_redirect(cli, new_path)
#send_http_301_permanent_redirect(cli, new_path)
#send_errorpage_with_redirect_link(cli, new_path)
return "true"

View File

@@ -2,7 +2,7 @@ standalone programs which are executed by copyparty when an event happens (uploa
these programs either take zero arguments, or a filepath (the affected file), or a json message with filepath + additional info
run copyparty with `--help-hooks` for usage details / hook type explanations (xm/xbu/xau/xiu/xbr/xar/xbd/xad/xban)
run copyparty with `--help-hooks` for usage details / hook type explanations (xm/xbu/xau/xiu/xbc/xac/xbr/xar/xbd/xad/xban)
> **note:** in addition to event hooks (the stuff described here), copyparty has another api to run your programs/scripts while providing way more information such as audio tags / video codecs / etc and optionally daisychaining data between scripts in a processing pipeline; if that's what you want then see [mtp plugins](../mtag/) instead
@@ -30,4 +30,5 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
# on message
* [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
* [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
* [usb-eject.py](usb-eject.py) adds web-UI buttons to safe-remove usb flashdrives shared through copyparty
* [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder

57
bin/hooks/usb-eject.js Normal file
View File

@@ -0,0 +1,57 @@
// see usb-eject.py for usage
function usbclick() {
QS('#treeul a[href="/usb/"]').click();
}
function eject_cb() {
var t = this.responseText;
if (t.indexOf('can be safely unplugged') < 0 && t.indexOf('Device can be removed') < 0)
return toast.err(30, 'usb eject failed:\n\n' + t);
toast.ok(5, esc(t.replace(/ - /g, '\n\n')).trim());
usbclick(); setTimeout(usbclick, 10);
};
function add_eject_2(a) {
var aw = a.getAttribute('href').split(/\//g);
if (aw.length != 4 || aw[3])
return;
var v = aw[2],
k = 'umount_' + v,
o = ebi(k);
if (o)
o.parentNode.removeChild(o);
a.appendChild(mknod('span', k, '⏏'), a);
o = ebi(k);
o.style.cssText = 'position:absolute; right:1em; margin-top:-.2em; font-size:1.3em';
o.onclick = function (e) {
ev(e);
var xhr = new XHR();
xhr.open('POST', get_evpath(), true);
xhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded;charset=UTF-8');
xhr.send('msg=' + uricom_enc(':usb-eject:' + v + ':'));
xhr.onload = xhr.onerror = eject_cb;
toast.inf(10, "ejecting " + v + "...");
};
};
function add_eject() {
var o = QSA('#treeul a[href^="/usb/"]');
for (var a = o.length - 1; a > 0; a--)
add_eject_2(o[a]);
};
(function() {
var f0 = treectl.rendertree;
treectl.rendertree = function (res, ts, top0, dst, rst) {
var ret = f0(res, ts, top0, dst, rst);
add_eject();
return ret;
};
})();
setTimeout(add_eject, 50);

58
bin/hooks/usb-eject.py Normal file
View File

@@ -0,0 +1,58 @@
#!/usr/bin/env python3
import os
import stat
import subprocess as sp
import sys
"""
if you've found yourself using copyparty to serve flashdrives on a LAN
and your only wish is that the web-UI had a button to unmount / safely
remove those flashdrives, then boy howdy are you in the right place :D
put usb-eject.js in the webroot (or somewhere else http-accessible)
then run copyparty with these args:
-v /run/media/egon:/usb:A:c,hist=/tmp/junk
--xm=c1,bin/hooks/usb-eject.py
--js-browser=/usb-eject.js
which does the following respectively,
* share all of /run/media/egon as /usb with admin for everyone
and put the histpath somewhere it won't cause trouble
* run the usb-eject hook with stdout redirect to the web-ui
* add the complementary usb-eject.js to the browser
"""
def main():
try:
label = sys.argv[1].split(":usb-eject:")[1].split(":")[0]
mp = "/run/media/egon/" + label
# print("ejecting [%s]... " % (mp,), end="")
mp = os.path.abspath(os.path.realpath(mp.encode("utf-8")))
st = os.lstat(mp)
if not stat.S_ISDIR(st.st_mode):
raise Exception("not a regular directory")
# if you're running copyparty as root (thx for the faith)
# you'll need something like this to make dbus talkative
cmd = b"sudo -u egon DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus gio mount -e"
# but if copyparty and the ui-session is running
# as the same user (good) then this is plenty
cmd = b"gio mount -e"
cmd = cmd.split(b" ") + [mp]
ret = sp.check_output(cmd).decode("utf-8", "replace")
print(ret.strip() or (label + " can be safely unplugged"))
except Exception as ex:
print("unmount failed: %r" % (ex,))
if __name__ == "__main__":
main()

View File

@@ -31,6 +31,9 @@ plugins in this section should only be used with appropriate precautions:
* [very-bad-idea.py](./very-bad-idea.py) combined with [meadup.js](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/meadup.js) converts copyparty into a janky yet extremely flexible chromecast clone
* also adds a virtual keyboard by @steinuil to the basic-upload tab for comfy couch crowd control
* anything uploaded through the [android app](https://github.com/9001/party-up) (files or links) are executed on the server, meaning anyone can infect your PC with malware... so protect this with a password and keep it on a LAN!
* [kamelåså](https://github.com/steinuil/kameloso) is a much better (and MUCH safer) alternative to this plugin
* powered by [chicken-curry-banana-pineapple-peanut pizza](https://a.ocv.me/pub/g/i/2025/01/298437ce-8351-4c8c-861c-fa131d217999.jpg?cache) so you know it's good
* and, unlike this plugin, kamelåså even has windows support (nice)
# dependencies

View File

@@ -6,6 +6,11 @@ WARNING -- DANGEROUS PLUGIN --
running this plugin, they can execute malware on your machine
so please keep this on a LAN and protect it with a password
here is a MUCH BETTER ALTERNATIVE (which also works on Windows):
https://github.com/steinuil/kameloso
----------------------------------------------------------------------
use copyparty as a chromecast replacement:
* post a URL and it will open in the default browser
* upload a file and it will open in the default application

View File

@@ -393,7 +393,8 @@ class Gateway(object):
if r.status != 200:
self.closeconn()
info("http error %s reading dir %r", r.status, web_path)
raise FuseOSError(errno.ENOENT)
err = errno.ENOENT if r.status == 404 else errno.EIO
raise FuseOSError(err)
ctype = r.getheader("Content-Type", "")
if ctype == "application/json":
@@ -1128,7 +1129,7 @@ def main():
# dircache is always a boost,
# only want to disable it for tests etc,
cdn = 9 # max num dirs; 0=disable
cdn = 24 # max num dirs; keep larger than max dir depth; 0=disable
cds = 1 # numsec until an entry goes stale
where = "local directory"

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env python3
from __future__ import print_function, unicode_literals
S_VERSION = "2.2"
S_BUILD_DT = "2024-10-13"
S_VERSION = "2.10"
S_BUILD_DT = "2025-02-19"
"""
u2c.py: upload to copyparty
@@ -62,6 +62,9 @@ else:
unicode = str
WTF8 = "replace" if PY2 else "surrogateescape"
VT100 = platform.system() != "Windows"
@@ -151,6 +154,7 @@ class HCli(object):
self.tls = tls
self.verify = ar.te or not ar.td
self.conns = []
self.hconns = []
if tls:
import ssl
@@ -170,7 +174,7 @@ class HCli(object):
"User-Agent": "u2c/%s" % (S_VERSION,),
}
def _connect(self):
def _connect(self, timeout):
args = {}
if PY37:
args["blocksize"] = 1048576
@@ -182,9 +186,11 @@ class HCli(object):
if self.ctx:
args = {"context": self.ctx}
return C(self.addr, self.port, timeout=999, **args)
return C(self.addr, self.port, timeout=timeout, **args)
def req(self, meth, vpath, hdrs, body=None, ctype=None):
now = time.time()
hdrs.update(self.base_hdrs)
if self.ar.a:
hdrs["PW"] = self.ar.a
@@ -195,7 +201,11 @@ class HCli(object):
0 if not body else body.len if hasattr(body, "len") else len(body)
)
c = self.conns.pop() if self.conns else self._connect()
# large timeout for handshakes (safededup)
conns = self.hconns if ctype == MJ else self.conns
while conns and self.ar.cxp < now - conns[0][0]:
conns.pop(0)[1].close()
c = conns.pop()[1] if conns else self._connect(999 if ctype == MJ else 128)
try:
c.request(meth, vpath, body, hdrs)
if PY27:
@@ -204,8 +214,15 @@ class HCli(object):
rsp = c.getresponse()
data = rsp.read()
self.conns.append(c)
conns.append((time.time(), c))
return rsp.status, data.decode("utf-8")
except http_client.BadStatusLine:
if self.ar.cxp > 4:
t = "\nWARNING: --cxp probably too high; reducing from %d to 4"
print(t % (self.ar.cxp,))
self.ar.cxp = 4
c.close()
raise
except:
c.close()
raise
@@ -217,6 +234,10 @@ CLEN = "Content-Length"
web = None # type: HCli
links = [] # type: list[str]
linkmtx = threading.Lock()
linkfile = None
class File(object):
"""an up2k upload task; represents a single file"""
@@ -228,7 +249,7 @@ class File(object):
self.lmod = lmod # type: float
self.abs = os.path.join(top, rel) # type: bytes
self.name = self.rel.split(b"/")[-1].decode("utf-8", "replace") # type: str
self.name = self.rel.split(b"/")[-1].decode("utf-8", WTF8) # type: str
# set by get_hashlist
self.cids = [] # type: list[tuple[str, int, int]] # [ hash, ofs, sz ]
@@ -267,10 +288,41 @@ class FileSlice(object):
raise Exception(9)
tlen += clen
self.len = tlen
self.len = self.tlen = tlen
self.cdr = self.car + self.len
self.ofs = 0 # type: int
self.f = open(file.abs, "rb", 512 * 1024)
self.f = None
self.seek = self._seek0
self.read = self._read0
def subchunk(self, maxsz, nth):
if self.tlen <= maxsz:
return -1
if not nth:
self.car0 = self.car
self.cdr0 = self.cdr
self.car = self.car0 + maxsz * nth
if self.car >= self.cdr0:
return -2
self.cdr = self.car + min(self.cdr0 - self.car, maxsz)
self.len = self.cdr - self.car
self.seek(0)
return nth
def unsub(self):
self.car = self.car0
self.cdr = self.cdr0
self.len = self.tlen
def _open(self):
self.seek = self._seek
self.read = self._read
self.f = open(self.file.abs, "rb", 512 * 1024)
self.f.seek(self.car)
# https://stackoverflow.com/questions/4359495/what-is-exactly-a-file-like-object-in-python
@@ -282,10 +334,15 @@ class FileSlice(object):
except:
pass # py27 probably
def close(self, *a, **ka):
return # until _open
def tell(self):
return self.ofs
def seek(self, ofs, wh=0):
def _seek(self, ofs, wh=0):
assert self.f # !rm
if wh == 1:
ofs = self.ofs + ofs
elif wh == 2:
@@ -299,12 +356,22 @@ class FileSlice(object):
self.ofs = ofs
self.f.seek(self.car + ofs)
def read(self, sz):
def _read(self, sz):
assert self.f # !rm
sz = min(sz, self.len - self.ofs)
ret = self.f.read(sz)
self.ofs += len(ret)
return ret
def _seek0(self, ofs, wh=0):
self._open()
return self.seek(ofs, wh)
def _read0(self, sz):
self._open()
return self.read(sz)
class MTHash(object):
def __init__(self, cores):
@@ -557,13 +624,17 @@ def walkdir(err, top, excl, seen):
for ap, inf in sorted(statdir(err, top)):
if excl.match(ap):
continue
yield ap, inf
if stat.S_ISDIR(inf.st_mode):
yield ap, inf
try:
for x in walkdir(err, ap, excl, seen):
yield x
except Exception as ex:
err.append((ap, str(ex)))
elif stat.S_ISREG(inf.st_mode):
yield ap, inf
else:
err.append((ap, "irregular filetype 0%o" % (inf.st_mode,)))
def walkdirs(err, tops, excl):
@@ -609,11 +680,12 @@ def walkdirs(err, tops, excl):
# mostly from copyparty/util.py
def quotep(btxt):
# type: (bytes) -> bytes
quot1 = quote(btxt, safe=b"/")
if not PY2:
quot1 = quot1.encode("ascii")
return quot1.replace(b" ", b"+") # type: ignore
return quot1.replace(b" ", b"%20") # type: ignore
# from copyparty/util.py
@@ -641,7 +713,7 @@ def up2k_chunksize(filesize):
while True:
for mul in [1, 2]:
nchunks = math.ceil(filesize * 1.0 / chunksize)
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks < 4096):
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
return chunksize
chunksize += stepsize
@@ -693,6 +765,29 @@ def get_hashlist(file, pcb, mth):
file.kchunks[k] = [v1, v2]
def printlink(ar, purl, name, fk):
if not name:
url = purl # srch
else:
name = quotep(name.encode("utf-8", WTF8)).decode("utf-8")
if fk:
url = "%s%s?k=%s" % (purl, name, fk)
else:
url = "%s%s" % (purl, name)
url = "%s/%s" % (ar.burl, url.lstrip("/"))
with linkmtx:
if ar.u:
links.append(url)
if ar.ud:
print(url)
if linkfile:
zs = "%s\n" % (url,)
zb = zs.encode("utf-8", "replace")
linkfile.write(zb)
def handshake(ar, file, search):
# type: (argparse.Namespace, File, bool) -> tuple[list[str], bool]
"""
@@ -712,7 +807,9 @@ def handshake(ar, file, search):
else:
if ar.touch:
req["umod"] = True
if ar.ow:
if ar.owo:
req["replace"] = "mt"
elif ar.ow:
req["replace"] = True
file.recheck = False
@@ -720,7 +817,7 @@ def handshake(ar, file, search):
url = file.url
else:
if b"/" in file.rel:
url = quotep(file.rel.rsplit(b"/", 1)[0]).decode("utf-8", "replace")
url = quotep(file.rel.rsplit(b"/", 1)[0]).decode("utf-8")
else:
url = ""
url = ar.vtop + url
@@ -764,17 +861,22 @@ def handshake(ar, file, search):
raise Exception(txt)
if search:
if ar.uon and r["hits"]:
printlink(ar, r["hits"][0]["rp"], "", "")
return r["hits"], False
file.url = r["purl"]
file.url = quotep(r["purl"].encode("utf-8", WTF8)).decode("utf-8")
file.name = r["name"]
file.wark = r["wark"]
if ar.uon and not r["hash"]:
printlink(ar, file.url, r["name"], r.get("fk"))
return r["hash"], r["sprs"]
def upload(fsl, stats):
# type: (FileSlice, str) -> None
def upload(fsl, stats, maxsz):
# type: (FileSlice, str, int) -> None
"""upload a range of file data, defined by one or more `cid` (chunk-hash)"""
ctxt = fsl.cids[0]
@@ -792,21 +894,34 @@ def upload(fsl, stats):
if stats:
headers["X-Up2k-Stat"] = stats
nsub = 0
try:
sc, txt = web.req("POST", fsl.file.url, headers, fsl, MO)
while nsub != -1:
nsub = fsl.subchunk(maxsz, nsub)
if nsub == -2:
return
if nsub >= 0:
headers["X-Up2k-Subc"] = str(maxsz * nsub)
headers.pop(CLEN, None)
nsub += 1
if sc == 400:
if (
"already being written" in txt
or "already got that" in txt
or "only sibling chunks" in txt
):
fsl.file.nojoin = 1
sc, txt = web.req("POST", fsl.file.url, headers, fsl, MO)
if sc >= 400:
raise Exception("http %s: %s" % (sc, txt))
if sc == 400:
if (
"already being written" in txt
or "already got that" in txt
or "only sibling chunks" in txt
):
fsl.file.nojoin = 1
if sc >= 400:
raise Exception("http %s: %s" % (sc, txt))
finally:
fsl.f.close()
if fsl.f:
fsl.f.close()
if nsub != -1:
fsl.unsub()
class Ctl(object):
@@ -938,7 +1053,7 @@ class Ctl(object):
print(" %d up %s" % (ncs - nc, cid))
stats = "%d/0/0/%d" % (nf, self.nfiles - nf)
fslice = FileSlice(file, [cid])
upload(fslice, stats)
upload(fslice, stats, self.ar.szm)
print(" ok!")
if file.recheck:
@@ -952,8 +1067,8 @@ class Ctl(object):
handshake(self.ar, file, False)
def _fancy(self):
atexit.register(self.cleanup_vt100)
if VT100 and not self.ar.ns:
atexit.register(self.cleanup_vt100)
ss.scroll_region(3)
Daemon(self.hasher)
@@ -961,6 +1076,7 @@ class Ctl(object):
Daemon(self.handshaker)
Daemon(self.uploader)
last_sp = -1
while True:
with self.exit_cond:
self.exit_cond.wait(0.07)
@@ -999,6 +1115,12 @@ class Ctl(object):
else:
txt = " "
if not VT100: # OSC9;4 (taskbar-progress)
sp = int(self.up_b * 100 / self.nbytes) or 1
if last_sp != sp:
last_sp = sp
txt += "\033]9;4;1;%d\033\\" % (sp,)
if not self.up_br:
spd = self.hash_b / ((time.time() - self.t0) or 1)
eta = (self.nbytes - self.hash_b) / (spd or 1)
@@ -1009,11 +1131,15 @@ class Ctl(object):
spd = humansize(spd)
self.eta = str(datetime.timedelta(seconds=int(eta)))
if eta > 2591999:
self.eta = self.eta.split(",")[0] # truncate HH:MM:SS
sleft = humansize(self.nbytes - self.up_b)
nleft = self.nfiles - self.up_f
tail = "\033[K\033[u" if VT100 and not self.ar.ns else "\r"
t = "%s eta @ %s/s, %s, %d# left\033[K" % (self.eta, spd, sleft, nleft)
t = "%s eta @ %s/s, %s, %d# left" % (self.eta, spd, sleft, nleft)
if not self.hash_b:
t = " now hashing..."
eprint(txt + "\033]0;{0}\033\\\r{0}{1}".format(t, tail))
if self.ar.wlist:
@@ -1034,7 +1160,10 @@ class Ctl(object):
handshake(self.ar, file, False)
def cleanup_vt100(self):
ss.scroll_region(None)
if VT100:
ss.scroll_region(None)
else:
eprint("\033]9;4;0\033\\")
eprint("\033[J\033]0;\033\\")
def cb_hasher(self, file, ofs):
@@ -1049,7 +1178,9 @@ class Ctl(object):
isdir = stat.S_ISDIR(inf.st_mode)
if self.ar.z or self.ar.drd:
rd = rel if isdir else os.path.dirname(rel)
srd = rd.decode("utf-8", "replace").replace("\\", "/")
srd = rd.decode("utf-8", "replace").replace("\\", "/").rstrip("/")
if srd:
srd += "/"
if prd != rd:
prd = rd
ls = {}
@@ -1057,7 +1188,7 @@ class Ctl(object):
print(" ls ~{0}".format(srd))
zt = (
self.ar.vtop,
quotep(rd.replace(b"\\", b"/")).decode("utf-8", "replace"),
quotep(rd.replace(b"\\", b"/")).decode("utf-8"),
)
sc, txt = web.req("GET", "%s%s?ls&lt&dots" % zt, {})
if sc >= 400:
@@ -1066,13 +1197,16 @@ class Ctl(object):
j = json.loads(txt)
for f in j["dirs"] + j["files"]:
rfn = f["href"].split("?")[0].rstrip("/")
ls[unquote(rfn.encode("utf-8", "replace"))] = f
ls[unquote(rfn.encode("utf-8", WTF8))] = f
except Exception as ex:
print(" mkdir ~{0} ({1})".format(srd, ex))
if self.ar.drd:
dp = os.path.join(top, rd)
lnodes = set(os.listdir(dp))
try:
lnodes = set(os.listdir(dp))
except:
lnodes = list(ls) # fs eio; don't delete
if ptn:
zs = dp.replace(sep, b"/").rstrip(b"/") + b"/"
zls = [zs + x for x in lnodes]
@@ -1080,12 +1214,12 @@ class Ctl(object):
lnodes = [x.split(b"/")[-1] for x in zls]
bnames = [x for x in ls if x not in lnodes and x != b".hist"]
vpath = self.ar.url.split("://")[-1].split("/", 1)[-1]
names = [x.decode("utf-8", "replace") for x in bnames]
locs = [vpath + srd + "/" + x for x in names]
names = [x.decode("utf-8", WTF8) for x in bnames]
locs = [vpath + srd + x for x in names]
while locs:
req = locs
while req:
print("DELETING ~%s/#%s" % (srd, len(req)))
print("DELETING ~%s#%s" % (srd, len(req)))
body = json.dumps(req).encode("utf-8")
sc, txt = web.req(
"POST", self.ar.url + "?delete", {}, body, MJ
@@ -1149,7 +1283,7 @@ class Ctl(object):
for n, zsii in enumerate(file.cids)
]
print("chs: %s\n%s" % (vp, "\n".join(zsl)))
zsl = [self.ar.wsalt, str(file.size)] + [x[0] for x in file.kchunks]
zsl = [self.ar.wsalt, str(file.size)] + [x[0] for x in file.cids]
zb = hashlib.sha512("\n".join(zsl).encode("utf-8")).digest()[:33]
wark = ub64enc(zb).decode("utf-8")
if self.ar.jw:
@@ -1286,7 +1420,7 @@ class Ctl(object):
self._check_if_done()
continue
njoin = (self.ar.sz * 1024 * 1024) // chunksz
njoin = self.ar.sz // chunksz
cs = hs[:]
while cs:
fsl = FileSlice(file, cs[:1])
@@ -1338,7 +1472,7 @@ class Ctl(object):
)
try:
upload(fsl, stats)
upload(fsl, stats, self.ar.szm)
except Exception as ex:
t = "upload failed, retrying: %s #%s+%d (%s)\n"
eprint(t % (file.name, cids[0][:8], len(cids) - 1, ex))
@@ -1372,7 +1506,7 @@ class APF(argparse.ArgumentDefaultsHelpFormatter, argparse.RawDescriptionHelpFor
def main():
global web
global web, linkfile
time.strptime("19970815", "%Y%m%d") # python#7980
"".encode("idna") # python#29288
@@ -1406,9 +1540,15 @@ source file/folder selection uses rsync syntax, meaning that:
ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
ap.add_argument("--owo", action="store_true", help="overwrite existing files if server-file is older")
ap.add_argument("--spd", action="store_true", help="print speeds for each file")
ap.add_argument("--version", action="store_true", help="show version and exit")
ap = app.add_argument_group("print links")
ap.add_argument("-u", action="store_true", help="print list of download-links after all uploads finished")
ap.add_argument("-ud", action="store_true", help="print download-link after each upload finishes")
ap.add_argument("-uf", type=unicode, metavar="PATH", help="print list of download-links to file")
ap = app.add_argument_group("compatibility")
ap.add_argument("--cls", action="store_true", help="clear screen before start")
ap.add_argument("--rh", type=int, metavar="TRIES", default=0, help="resolve server hostname before upload (good for buggy networks, but TLS certs will break)")
@@ -1427,8 +1567,10 @@ source file/folder selection uses rsync syntax, meaning that:
ap.add_argument("-j", type=int, metavar="CONNS", default=2, help="parallel connections")
ap.add_argument("-J", type=int, metavar="CORES", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
ap.add_argument("--sz", type=int, metavar="MiB", default=64, help="try to make each POST this big")
ap.add_argument("--szm", type=int, metavar="MiB", default=96, help="max size of each POST (default is cloudflare max)")
ap.add_argument("-nh", action="store_true", help="disable hashing while uploading")
ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles and macos)")
ap.add_argument("--cxp", type=float, metavar="SEC", default=57, help="assume http connections expired after SEConds")
ap.add_argument("--cd", type=float, metavar="SEC", default=5, help="delay before reattempting a failed handshake/upload")
ap.add_argument("--safe", action="store_true", help="use simple fallback approach")
ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")
@@ -1448,15 +1590,54 @@ source file/folder selection uses rsync syntax, meaning that:
except:
pass
# msys2 doesn't uncygpath absolute paths with whitespace
if not VT100:
zsl = []
for fn in ar.files:
if re.search("^/[a-z]/", fn):
fn = r"%s:\%s" % (fn[1:2], fn[3:])
zsl.append(fn.replace("/", "\\"))
ar.files = zsl
fok = []
fng = []
for fn in ar.files:
if os.path.exists(fn):
fok.append(fn)
elif VT100:
fng.append(fn)
else:
# windows leaves glob-expansion to the invoked process... okayyy let's get to work
from glob import glob
fns = glob(fn)
if fns:
fok.extend(fns)
else:
fng.append(fn)
if fng:
t = "some files/folders were not found:\n %s"
raise Exception(t % ("\n ".join(fng),))
ar.files = fok
if ar.drd:
ar.dr = True
if ar.dr:
ar.ow = True
ar.sz *= 1024 * 1024
ar.szm *= 1024 * 1024
ar.x = "|".join(ar.x or [])
setattr(ar, "wlist", ar.url == "-")
setattr(ar, "uon", ar.u or ar.ud or ar.uf)
if ar.uf:
linkfile = open(ar.uf, "wb")
for k in "dl dr drd wlist".split():
errs = []
@@ -1519,6 +1700,12 @@ source file/folder selection uses rsync syntax, meaning that:
ar.z = True
ctl = Ctl(ar, ctl.stats)
if links:
print()
print("\n".join(links))
if linkfile:
linkfile.close()
if ctl.errs:
print("WARNING: %d errors" % (ctl.errs))

76
bin/zmq-recv.py Executable file
View File

@@ -0,0 +1,76 @@
#!/usr/bin/env python3
import sys
import zmq
"""
zmq-recv.py: demo zmq receiver
2025-01-22, v1.0, ed <irc.rizon.net>, MIT-Licensed
https://github.com/9001/copyparty/blob/hovudstraum/bin/zmq-recv.py
basic zmq-server to receive events from copyparty; try one of
the below and then "send a message to serverlog" in the web-ui:
1) dumb fire-and-forget to any and all listeners;
run this script with "sub" and run copyparty with this:
--xm zmq:pub:tcp://*:5556
2) one lucky listener gets the message, blocks if no listeners:
run this script with "pull" and run copyparty with this:
--xm t3,zmq:push:tcp://*:5557
3) blocking syn/ack mode, client must ack each message;
run this script with "rep" and run copyparty with this:
--xm t3,zmq:req:tcp://localhost:5555
note: to conditionally block uploads based on message contents,
use rep_server to answer with "return 1" and run copyparty with
--xau t3,c,zmq:req:tcp://localhost:5555
"""
ctx = zmq.Context()
def sub_server():
# PUB/SUB allows any number of servers/clients, and
# messages are fire-and-forget
sck = ctx.socket(zmq.SUB)
sck.connect("tcp://localhost:5556")
sck.setsockopt_string(zmq.SUBSCRIBE, "")
while True:
print("copyparty says %r" % (sck.recv_string(),))
def pull_server():
# PUSH/PULL allows any number of servers/clients, and
# each message is sent to a exactly one PULL client
sck = ctx.socket(zmq.PULL)
sck.connect("tcp://localhost:5557")
while True:
print("copyparty says %r" % (sck.recv_string(),))
def rep_server():
# REP/REQ is a server/client pair where each message must be
# acked by the other before another message can be sent, so
# copyparty will do a blocking-wait for the ack
sck = ctx.socket(zmq.REP)
sck.bind("tcp://*:5555")
while True:
print("copyparty says %r" % (sck.recv_string(),))
reply = b"thx"
# reply = b"return 1" # non-zero to block an upload
sck.send(reply)
mode = sys.argv[1].lower() if len(sys.argv) > 1 else ""
if mode == "sub":
sub_server()
elif mode == "pull":
pull_server()
elif mode == "rep":
rep_server()
else:
print("specify mode as first argument: SUB | PULL | REP")

View File

@@ -12,14 +12,19 @@
* assumes the webserver and copyparty is running on the same server/IP
* modify `10.13.1.1` as necessary if you wish to support browsers without javascript
### [`sharex.sxcu`](sharex.sxcu)
* sharex config file to upload screenshots and grab the URL
### [`sharex.sxcu`](sharex.sxcu) - Windows screenshot uploader
* [sharex](https://getsharex.com/) config file to upload screenshots and grab the URL
* `RequestURL`: full URL to the target folder
* `pw`: password (remove the `pw` line if anon-write)
* the `act:bput` thing is optional since copyparty v1.9.29
* using an older sharex version, maybe sharex v12.1.1 for example? dw fam i got your back 👉😎👉 [`sharex12.sxcu`](sharex12.sxcu)
### [`flameshot.sh`](flameshot.sh)
### [`ishare.iscu`](ishare.iscu) - MacOS screenshot uploader
* [ishare](https://isharemac.app/) config file to upload screenshots and grab the URL
* `RequestURL`: full URL to the target folder
* `pw`: password (remove the `pw` line if anon-write)
### [`flameshot.sh`](flameshot.sh) - Linux screenshot uploader
* takes a screenshot with [flameshot](https://flameshot.org/) on Linux, uploads it, and writes the URL to clipboard
### [`send-to-cpp.contextlet.json`](send-to-cpp.contextlet.json)
@@ -53,5 +58,10 @@ init-scripts to start copyparty as a service
* [`openrc/copyparty`](openrc/copyparty)
# Reverse-proxy
copyparty has basic support for running behind another webserver
* [`nginx/copyparty.conf`](nginx/copyparty.conf)
copyparty supports running behind another webserver
* [`apache/copyparty.conf`](apache/copyparty.conf)
* [`haproxy/copyparty.conf`](haproxy/copyparty.conf)
* [`lighttpd/subdomain.conf`](lighttpd/subdomain.conf)
* [`lighttpd/subpath.conf`](lighttpd/subpath.conf)
* [`nginx/copyparty.conf`](nginx/copyparty.conf) -- recommended
* [`traefik/copyparty.yaml`](traefik/copyparty.yaml)

View File

@@ -1,14 +1,29 @@
# when running copyparty behind a reverse proxy,
# the following arguments are recommended:
# if you would like to use unix-sockets (recommended),
# you must run copyparty with one of the following:
#
# -i 127.0.0.1 only accept connections from nginx
# -i unix:777:/dev/shm/party.sock
# -i unix:777:/dev/shm/party.sock,127.0.0.1
#
# if you are doing location-based proxying (such as `/stuff` below)
# you must run copyparty with --rp-loc=stuff
#
# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
LoadModule proxy_module modules/mod_proxy.so
ProxyPass "/stuff" "http://127.0.0.1:3923/stuff"
# do not specify ProxyPassReverse
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
# NOTE: do not specify ProxyPassReverse
##
## then, enable one of the below:
# use subdomain proxying to unix-socket (best)
ProxyPass "/" "unix:///dev/shm/party.sock|http://whatever/"
# use subdomain proxying to 127.0.0.1 (slower)
#ProxyPass "/" "http://127.0.0.1:3923/"
# use subpath proxying to 127.0.0.1 (slow and maybe buggy)
#ProxyPass "/stuff" "http://127.0.0.1:3923/stuff"

View File

@@ -0,0 +1,24 @@
# this config is essentially two separate examples;
#
# foo1 connects to copyparty using tcp, and
# foo2 uses unix-sockets for 27% higher performance
#
# to use foo2 you must run copyparty with one of the following:
#
# -i unix:777:/dev/shm/party.sock
# -i unix:777:/dev/shm/party.sock,127.0.0.1
defaults
mode http
option forwardfor
timeout connect 1s
timeout client 610s
timeout server 610s
listen foo1
bind *:8081
server srv1 127.0.0.1:3923 maxconn 512
listen foo2
bind *:8082
server srv1 /dev/shm/party.sock maxconn 512

10
contrib/ishare.iscu Normal file
View File

@@ -0,0 +1,10 @@
{
"Name": "copyparty",
"RequestURL": "http://127.0.0.1:3923/screenshots/",
"Headers": {
"pw": "PUT_YOUR_PASSWORD_HERE_MY_DUDE",
"accept": "json"
},
"FileFormName": "f",
"ResponseURL": "{{fileurl}}"
}

View File

@@ -0,0 +1,24 @@
# example usage for benchmarking:
#
# taskset -c 1 lighttpd -Df ~/dev/copyparty/contrib/lighttpd/subdomain.conf
#
# lighttpd can connect to copyparty using either tcp (127.0.0.1)
# or a unix-socket, but unix-sockets are 37% faster because
# lighttpd doesn't reuse tcp connections, so we're doing unix-sockets
#
# this means we must run copyparty with one of the following:
#
# -i unix:777:/dev/shm/party.sock
# -i unix:777:/dev/shm/party.sock,127.0.0.1
#
# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
server.port = 80
server.document-root = "/var/empty"
server.upload-dirs = ( "/dev/shm", "/tmp" )
server.modules = ( "mod_proxy" )
proxy.forwarded = ( "for" => 1, "proto" => 1 )
proxy.server = ( "" => ( ( "host" => "/dev/shm/party.sock" ) ) )
# if you really need to use tcp instead of unix-sockets, do this instead:
#proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "3923" ) ) )

View File

@@ -0,0 +1,31 @@
# example usage for benchmarking:
#
# taskset -c 1 lighttpd -Df ~/dev/copyparty/contrib/lighttpd/subpath.conf
#
# lighttpd can connect to copyparty using either tcp (127.0.0.1)
# or a unix-socket, but unix-sockets are 37% faster because
# lighttpd doesn't reuse tcp connections, so we're doing unix-sockets
#
# this means we must run copyparty with one of the following:
#
# -i unix:777:/dev/shm/party.sock
# -i unix:777:/dev/shm/party.sock,127.0.0.1
#
# also since this example proxies a subpath instead of the
# recommended subdomain-proxying, we must also specify this:
#
# --rp-loc files
#
# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
server.port = 80
server.document-root = "/var/empty"
server.upload-dirs = ( "/dev/shm", "/tmp" )
server.modules = ( "mod_proxy" )
$HTTP["url"] =~ "^/files" {
proxy.forwarded = ( "for" => 1, "proto" => 1 )
proxy.server = ( "" => ( ( "host" => "/dev/shm/party.sock" ) ) )
# if you really need to use tcp instead of unix-sockets, do this instead:
#proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => "3923" ) ) )
}

View File

@@ -36,9 +36,9 @@ upstream cpp_uds {
# but there must be at least one unix-group which both
# nginx and copyparty is a member of; if that group is
# "www" then run copyparty with the following args:
# -i unix:770:www:/tmp/party.sock
# -i unix:770:www:/dev/shm/party.sock
server unix:/tmp/party.sock fail_timeout=1s;
server unix:/dev/shm/party.sock fail_timeout=1s;
keepalive 1;
}
@@ -61,6 +61,10 @@ server {
client_max_body_size 0;
proxy_buffering off;
proxy_request_buffering off;
# improve download speed from 600 to 1500 MiB/s
proxy_buffers 32 8k;
proxy_buffer_size 16k;
proxy_busy_buffers_size 24k;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.15.6"
pkgver="1.16.15"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@@ -16,12 +16,13 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
"libkeyfinder-git: detection of musical keys"
"qm-vamp-plugins: BPM detection"
"python-pyopenssl: ftps functionality"
"python-argon2_cffi: hashed passwords in config"
"python-pyzmq: send zeromq messages from event-hooks"
"python-argon2-cffi: hashed passwords in config"
"python-impacket-git: smb support (bad idea)"
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" )
sha256sums=("abb5c1705cd80ea553d647d4a7b35b5e1dac5a517200551bcca79aa199f30875")
sha256sums=("080844a7458073c86714dde5fa659a64120909568005420ff81a27e353e91a72")
build() {
cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,4 +1,4 @@
{ lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, argon2-cffi, pillow, pyvips, ffmpeg, mutagen,
{ lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, argon2-cffi, pillow, pyvips, pyzmq, ffmpeg, mutagen,
# use argon2id-hashed passwords in config files (sha2 is always available)
withHashedPasswords ? true,
@@ -21,6 +21,9 @@ withMediaProcessing ? true,
# if MediaProcessing is not enabled, you probably want this instead (less accurate, but much safer and faster)
withBasicAudioMetadata ? false,
# send ZeroMQ messages from event-hooks
withZeroMQ ? true,
# enable FTPS support in the FTP server
withFTPS ? false,
@@ -43,6 +46,7 @@ let
++ lib.optional withMediaProcessing ffmpeg
++ lib.optional withBasicAudioMetadata mutagen
++ lib.optional withHashedPasswords argon2-cffi
++ lib.optional withZeroMQ pyzmq
);
in stdenv.mkDerivation {
pname = "copyparty";
@@ -60,4 +64,5 @@ in stdenv.mkDerivation {
--set PATH '${lib.makeBinPath ([ utillinux ] ++ lib.optional withMediaProcessing ffmpeg)}:$PATH' \
--add-flags "$out/share/copyparty-sfx.py"
'';
meta.mainProgram = "copyparty";
}

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.15.6/copyparty-sfx.py",
"version": "1.15.6",
"hash": "sha256-0ikt3jv9/XT/w/ew+R4rZxF6s7LwNhUvUYYIZtkQqbk="
"url": "https://github.com/9001/copyparty/releases/download/v1.16.15/copyparty-sfx.py",
"version": "1.16.15",
"hash": "sha256-vpI/N01pCeCiAv3haE1Zac1X9Oyp2vqPAC1WEcl5phA="
}

View File

@@ -15,6 +15,7 @@ save one of these as `.epilogue.html` inside a folder to customize it:
point `--js-browser` to one of these by URL:
* [`minimal-up2k.js`](minimal-up2k.js) is similar to the above `minimal-up2k.html` except it applies globally to all write-only folders
* [`quickmove.js`](quickmove.js) adds a hotkey to move selected files into a subfolder
* [`up2k-hooks.js`](up2k-hooks.js) lets you specify a ruleset for files to skip uploading
* [`up2k-hook-ytid.js`](up2k-hook-ytid.js) is a more specific example checking youtube-IDs against some API

View File

@@ -0,0 +1,117 @@
// USAGE:
// place this file somewhere in the webroot and then
// python3 -m copyparty --js-browser /.res/graft-thumbs.js
//
// DESCRIPTION:
// this is a gridview plugin which, for each file in a folder,
// looks for another file with the same filename (but with a
// different file extension)
//
// if one of those files is an image and the other is not,
// then this plugin assumes the image is a "sidecar thumbnail"
// for the other file, and it will graft the image thumbnail
// onto the non-image file (for example an mp3)
//
// optional feature 1, default-enabled:
// the image-file is then hidden from the directory listing
//
// optional feature 2, default-enabled:
// when clicking the audio file, the image will also open
(function() {
// `graft_thumbs` assumes the gridview has just been rendered;
// it looks for sidecars, and transplants those thumbnails onto
// the other file with the same basename (filename sans extension)
var graft_thumbs = function () {
if (!thegrid.en)
return; // not in grid mode
var files = msel.getall(),
pairs = {};
console.log(files);
for (var a = 0; a < files.length; a++) {
var file = files[a],
is_pic = /\.(jpe?g|png|gif|webp)$/i.exec(file.vp),
is_audio = re_au_all.exec(file.vp),
basename = file.vp.replace(/\.[^\.]+$/, ""),
entry = pairs[basename];
if (!entry)
// first time seeing this basename; create a new entry in pairs
entry = pairs[basename] = {};
if (is_pic)
entry.thumb = file;
else if (is_audio)
entry.audio = file;
}
var basenames = Object.keys(pairs);
for (var a = 0; a < basenames.length; a++)
(function(a) {
var pair = pairs[basenames[a]];
if (!pair.thumb || !pair.audio)
return; // not a matching pair of files
var img_thumb = QS('#ggrid a[ref="' + pair.thumb.id + '"] img[onload]'),
img_audio = QS('#ggrid a[ref="' + pair.audio.id + '"] img[onload]');
if (!img_thumb || !img_audio)
return; // something's wrong... let's bail
// alright, graft the thumb...
img_audio.src = img_thumb.src;
// ...and hide the sidecar
img_thumb.closest('a').style.display = 'none';
// ...and add another onclick-handler to the audio,
// so it also opens the pic while playing the song
img_audio.addEventListener('click', function() {
img_thumb.click();
return false; // let it bubble to the next listener
});
})(a);
};
// ...and then the trick! near the end of loadgrid,
// thegrid.bagit is called to initialize the baguettebox
// (image/video gallery); this is the perfect function to
// "hook" (hijack) so we can run our code :^)
// need to grab a backup of the original function first,
var orig_func = thegrid.bagit;
// and then replace it with our own:
thegrid.bagit = function (isrc) {
if (isrc !== '#ggrid')
// we only want to modify the grid, so
// let the original function handle this one
return orig_func(isrc);
graft_thumbs();
// when changing directories, the grid is
// rendered before msel returns the correct
// filenames, so schedule another run:
setTimeout(graft_thumbs, 1);
// and finally, call the original thegrid.bagit function
return orig_func(isrc);
};
if (ls0) {
// the server included an initial listing json (ls0),
// so the grid has already been rendered without our hook
graft_thumbs();
}
})();

View File

@@ -0,0 +1,140 @@
"use strict";
// USAGE:
// place this file somewhere in the webroot,
// for example in a folder named ".res" to hide it, and then
// python3 copyparty-sfx.py -v .::A --js-browser /.res/quickmove.js
//
// DESCRIPTION:
// the command above launches copyparty with one single volume;
// ".::A" = current folder as webroot, and everyone has Admin
//
// the plugin adds hotkey "W" which moves all selected files
// into a subfolder named "foobar" inside the current folder
(function() {
var action_to_perform = ask_for_confirmation_and_then_move;
// this decides what the new hotkey should do;
// ask_for_confirmation_and_then_move = show a yes/no box,
// move_selected_files = just move the files immediately
var move_destination = "foobar";
// this is the target folder to move files to;
// by default it is a subfolder of the current folder,
// but it can also be an absolute path like "/foo/bar"
// ===
// === END OF CONFIG
// ===
var main_hotkey_handler, // copyparty's original hotkey handler
plugin_enabler, // timer to engage this plugin when safe
files_to_move; // list of files to move
function ask_for_confirmation_and_then_move() {
var num_files = msel.getsel().length,
msg = "move the selected " + num_files + " files?";
if (!num_files)
return toast.warn(2, 'no files were selected to be moved');
modal.confirm(msg, move_selected_files, null);
}
function move_selected_files() {
var selection = msel.getsel();
if (!selection.length)
return toast.warn(2, 'no files were selected to be moved');
if (thegrid.bbox) {
// close image/video viewer
thegrid.bbox = null;
baguetteBox.destroy();
}
files_to_move = [];
for (var a = 0; a < selection.length; a++)
files_to_move.push(selection[a].vp);
move_next_file();
}
function move_next_file() {
var num_files = files_to_move.length,
filepath = files_to_move.pop(),
filename = vsplit(filepath)[1];
toast.inf(10, "moving " + num_files + " files...\n\n" + filename);
var dst = move_destination;
if (!dst.endsWith('/'))
// must have a trailing slash, so add it
dst += '/';
if (!dst.startsWith('/'))
// destination is a relative path, so prefix current folder path
dst = get_evpath() + dst;
// and finally append the filename
dst += '/' + filename;
// prepare the move-request to be sent
var xhr = new XHR();
xhr.onload = xhr.onerror = function() {
if (this.status !== 201)
return toast.err(30, 'move failed: ' + esc(this.responseText));
if (files_to_move.length)
return move_next_file(); // still more files to go
toast.ok(1, 'move OK');
treectl.goto(); // reload the folder contents
};
xhr.open('POST', filepath + '?move=' + dst);
xhr.send();
}
function our_hotkey_handler(e) {
// bail if either ALT, CTRL, or SHIFT is pressed
if (e.altKey || e.shiftKey || e.isComposing || ctrl(e))
return main_hotkey_handler(e); // let copyparty handle this keystroke
var key_name = (e.code || e.key) + '',
ae = document.activeElement,
aet = ae && ae != document.body ? ae.nodeName.toLowerCase() : '';
// check the current aet (active element type),
// only continue if one of the following currently has input focus:
// nothing | link | button | table-row | table-cell | div | text
if (aet && !/^(a|button|tr|td|div|pre)$/.test(aet))
return main_hotkey_handler(e); // let copyparty handle this keystroke
if (key_name == 'KeyW') {
// okay, this one's for us... do the thing
action_to_perform();
return ev(e);
}
return main_hotkey_handler(e); // let copyparty handle this keystroke
}
function enable_plugin() {
if (!window.hotkeys_attached)
return console.log('quickmove is waiting for the page to finish loading');
clearInterval(plugin_enabler);
main_hotkey_handler = document.onkeydown;
document.onkeydown = our_hotkey_handler;
console.log('quickmove is now enabled');
}
// copyparty doesn't enable its hotkeys until the page
// has finished loading, so we'll wait for that too
plugin_enabler = setInterval(enable_plugin, 100);
})();

View File

@@ -0,0 +1,25 @@
# ./traefik --configFile=copyparty.yaml
entryPoints:
web:
address: :8080
transport:
# don't disconnect during big uploads
respondingTimeouts:
readTimeout: "0s"
log:
level: DEBUG
providers:
file:
# WARNING: must be same filename as current file
filename: "copyparty.yaml"
http:
services:
service-cpp:
loadBalancer:
servers:
- url: "http://127.0.0.1:3923/"
routers:
my-router:
rule: "PathPrefix(`/`)"
service: service-cpp

View File

@@ -80,6 +80,7 @@ web/deps/prismd.css
web/deps/scp.woff2
web/deps/sha512.ac.js
web/deps/sha512.hw.js
web/iiam.gif
web/md.css
web/md.html
web/md.js
@@ -90,6 +91,9 @@ web/mde.html
web/mde.js
web/msg.css
web/msg.html
web/rups.css
web/rups.html
web/rups.js
web/shares.css
web/shares.html
web/shares.js

View File

@@ -50,8 +50,12 @@ from .util import (
PARTFTPY_VER,
PY_DESC,
PYFTPD_VER,
RAM_AVAIL,
RAM_TOTAL,
SQLITE_VER,
UNPLICATIONS,
URL_BUG,
URL_PRJ,
Daemon,
align_tab,
ansi_re,
@@ -330,17 +334,16 @@ def ensure_webdeps() -> None:
if has_resource(E, "web/deps/mini-fa.woff"):
return
warn(
"""could not find webdeps;
t = """could not find webdeps;
if you are running the sfx, or exe, or pypi package, or docker image,
then this is a bug! Please let me know so I can fix it, thanks :-)
https://github.com/9001/copyparty/issues/new?labels=bug&template=bug_report.md
%s
however, if you are a dev, or running copyparty from source, and you want
full client functionality, you will need to build or obtain the webdeps:
https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#building
%s/blob/hovudstraum/docs/devnotes.md#building
"""
)
warn(t % (URL_BUG, URL_PRJ))
def configure_ssl_ver(al: argparse.Namespace) -> None:
@@ -684,6 +687,8 @@ def get_sects():
\033[36mxbu\033[35m executes CMD before a file upload starts
\033[36mxau\033[35m executes CMD after a file upload finishes
\033[36mxiu\033[35m executes CMD after all uploads finish and volume is idle
\033[36mxbc\033[35m executes CMD before a file copy
\033[36mxac\033[35m executes CMD after a file copy
\033[36mxbr\033[35m executes CMD before a file rename/move
\033[36mxar\033[35m executes CMD after a file rename/move
\033[36mxbd\033[35m executes CMD before a file delete
@@ -735,6 +740,10 @@ def get_sects():
the \033[33m,,\033[35m stops copyparty from reading the rest as flags and
the \033[33m--\033[35m stops notify-send from reading the message as args
and the alert will be "hey" followed by the messagetext
\033[36m--xau zmq:pub:tcp://*:5556\033[35m announces uploads on zeromq;
\033[36m--xau t3,zmq:push:tcp://*:5557\033[35m also works, and you can
\033[36m--xau t3,j,zmq:req:tcp://localhost:5555\033[35m too for example
\033[0m
each hook is executed once for each event, except for \033[36mxiu\033[0m
which builds up a backlog of uploads, running the hook just once
@@ -766,11 +775,22 @@ def get_sects():
values for --urlform:
\033[36mstash\033[35m dumps the data to file and returns length + checksum
\033[36msave,get\033[35m dumps to file and returns the page like a GET
\033[36mprint,get\033[35m prints the data in the log and returns GET
(leave out the ",get" to return an error instead)\033[0m
\033[36mprint \033[35m prints the data to log and returns an error
\033[36mprint,xm \033[35m prints the data to log and returns --xm output
\033[36mprint,get\033[35m prints the data to log and returns GET\033[0m
note that the \033[35m--xm\033[0m hook will only run if \033[35m--urlform\033[0m
is either \033[36mprint\033[0m or the default \033[36mprint,get\033[0m
note that the \033[35m--xm\033[0m hook will only run if \033[35m--urlform\033[0m is
either \033[36mprint\033[0m or \033[36mprint,get\033[0m or the default \033[36mprint,xm\033[0m
if an \033[35m--xm\033[0m hook returns text, then
the response code will be HTTP 202;
http/get responses will be HTTP 200
if there are multiple \033[35m--xm\033[0m hooks defined, then
the first hook that produced output is returned
if there are no \033[35m--xm\033[0m hooks defined, then the default
\033[36mprint,xm\033[0m behaves like \033[36mprint,get\033[0m (returning html)
"""
),
],
@@ -874,8 +894,9 @@ def get_sects():
use argon2id with timecost 3, 256 MiB, 4 threads, version 19 (0x13/v1.3)
\033[36m--ah-alg scrypt\033[0m # which is the same as:
\033[36m--ah-alg scrypt,13,2,8,4\033[0m
use scrypt with cost 2**13, 2 iterations, blocksize 8, 4 threads
\033[36m--ah-alg scrypt,13,2,8,4,32\033[0m
use scrypt with cost 2**13, 2 iterations, blocksize 8, 4 threads,
and allow using up to 32 MiB RAM (ram=cost*blksz roughly)
\033[36m--ah-alg sha2\033[0m # which is the same as:
\033[36m--ah-alg sha2,424242\033[0m
@@ -950,7 +971,7 @@ def add_general(ap, nc, srvname):
ap2.add_argument("-v", metavar="VOL", type=u, action="append", help="add volume, \033[33mSRC\033[0m:\033[33mDST\033[0m:\033[33mFLAG\033[0m; examples [\033[32m.::r\033[0m], [\033[32m/mnt/nas/music:/music:r:aed\033[0m], see --help-accounts")
ap2.add_argument("--grp", metavar="G:N,N", type=u, action="append", help="add group, \033[33mNAME\033[0m:\033[33mUSER1\033[0m,\033[33mUSER2\033[0m,\033[33m...\033[0m; example [\033[32madmins:ed,foo,bar\033[0m]")
ap2.add_argument("-ed", action="store_true", help="enable the ?dots url parameter / client option which allows clients to see dotfiles / hidden files (volflag=dots)")
ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-form POSTs; see \033[33m--help-urlform\033[0m")
ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,xm", help="how to handle url-form POSTs; see \033[33m--help-urlform\033[0m")
ap2.add_argument("--wintitle", metavar="TXT", type=u, default="cpp @ $pub", help="server terminal title, for example [\033[32m$ip-10.1.2.\033[0m] or [\033[32m$ip-]")
ap2.add_argument("--name", metavar="TXT", type=u, default=srvname, help="server name (displayed topleft in browser and in mDNS)")
ap2.add_argument("--mime", metavar="EXT=MIME", type=u, action="append", help="map file \033[33mEXT\033[0mension to \033[33mMIME\033[0mtype, for example [\033[32mjpg=image/jpeg\033[0m]")
@@ -1017,7 +1038,8 @@ def add_upload(ap):
ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16 for cross-atlantic (max=64)")
ap2.add_argument("--u2sz", metavar="N,N,N", type=u, default="1,64,96", help="web-client: default upload chunksize (MiB); sets \033[33mmin,default,max\033[0m in the settings gui. Each HTTP POST will aim for this size. Cloudflare max is 96. Big values are good for cross-atlantic but may increase HDD fragmentation on some FS. Disable this optimization with [\033[32m1,1,1\033[0m]")
ap2.add_argument("--u2sz", metavar="N,N,N", type=u, default="1,64,96", help="web-client: default upload chunksize (MiB); sets \033[33mmin,default,max\033[0m in the settings gui. Each HTTP POST will aim for \033[33mdefault\033[0m, and never exceed \033[33mmax\033[0m. Cloudflare max is 96. Big values are good for cross-atlantic but may increase HDD fragmentation on some FS. Disable this optimization with [\033[32m1,1,1\033[0m]")
ap2.add_argument("--u2ow", metavar="NUM", type=int, default=0, help="web-client: default setting for when to overwrite existing files; [\033[32m0\033[0m]=never, [\033[32m1\033[0m]=if-client-newer, [\033[32m2\033[0m]=always (volflag=u2ow)")
ap2.add_argument("--u2sort", metavar="TXT", type=u, default="s", help="upload order; [\033[32ms\033[0m]=smallest-first, [\033[32mn\033[0m]=alphabetical, [\033[32mfs\033[0m]=force-s, [\033[32mfn\033[0m]=force-n -- alphabetical is a bit slower on fiber/LAN but makes it easier to eyeball if everything went fine")
ap2.add_argument("--write-uplog", action="store_true", help="write POST reports to textfiles in working-directory")
@@ -1037,7 +1059,7 @@ def add_network(ap):
else:
ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186.0, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=128.0, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
ap2.add_argument("--s-rd-sz", metavar="B", type=int, default=256*1024, help="socket read size in bytes (indirectly affects filesystem writes; recommendation: keep equal-to or lower-than \033[33m--iobuf\033[0m)")
ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0.0, help="debug: socket write delay in seconds")
@@ -1078,7 +1100,7 @@ def add_cert(ap, cert_path):
def add_auth(ap):
ses_db = os.path.join(E.cfg, "sessions.db")
ap2 = ap.add_argument_group('IdP / identity provider / user authentication options')
ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks and assume the request-header \033[33mHN\033[0m contains the username of the requesting user (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks if the request-header \033[33mHN\033[0m contains a username to associate the request with (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
@@ -1119,6 +1141,8 @@ def add_zc_mdns(ap):
ap2.add_argument("--zm6", action="store_true", help="IPv6 only")
ap2.add_argument("--zmv", action="store_true", help="verbose mdns")
ap2.add_argument("--zmvv", action="store_true", help="verboser mdns")
ap2.add_argument("--zm-no-pe", action="store_true", help="mute parser errors (invalid incoming MDNS packets)")
ap2.add_argument("--zm-nwa-1", action="store_true", help="disable workaround for avahi-bug #379 (corruption in Avahi's mDNS reflection feature)")
ap2.add_argument("--zms", metavar="dhf", type=u, default="", help="list of services to announce -- d=webdav h=http f=ftp s=smb -- lowercase=plaintext uppercase=TLS -- default: all enabled services except http/https (\033[32mDdfs\033[0m if \033[33m--ftp\033[0m and \033[33m--smb\033[0m is set, \033[32mDd\033[0m otherwise)")
ap2.add_argument("--zm-ld", metavar="PATH", type=u, default="", help="link a specific folder for webdav shares")
ap2.add_argument("--zm-lh", metavar="PATH", type=u, default="", help="link a specific folder for http shares")
@@ -1160,6 +1184,7 @@ def add_webdav(ap):
ap2.add_argument("--dav-mac", action="store_true", help="disable apple-garbage filter -- allow macos to create junk files (._* and .DS_Store, .Spotlight-*, .fseventsd, .Trashes, .AppleDouble, __MACOS)")
ap2.add_argument("--dav-rt", action="store_true", help="show symlink-destination's lastmodified instead of the link itself; always enabled for recursive listings (volflag=davrt)")
ap2.add_argument("--dav-auth", action="store_true", help="force auth for all folders (required by davfs2 when only some folders are world-readable) (volflag=davauth)")
ap2.add_argument("--dav-ua1", metavar="PTN", type=u, default=r" kioworker/", help="regex of tricky user-agents which expect 401 from GET requests; disable with [\033[32mno\033[0m] or blank")
def add_tftp(ap):
@@ -1201,6 +1226,8 @@ def add_hooks(ap):
ap2.add_argument("--xbu", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m before a file upload starts")
ap2.add_argument("--xau", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m after a file upload finishes")
ap2.add_argument("--xiu", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m after all uploads finish and volume is idle")
ap2.add_argument("--xbc", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m before a file copy")
ap2.add_argument("--xac", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m after a file copy")
ap2.add_argument("--xbr", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m before a file move/rename")
ap2.add_argument("--xar", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m after a file move/rename")
ap2.add_argument("--xbd", metavar="CMD", type=u, action="append", help="execute \033[33mCMD\033[0m before a file delete")
@@ -1233,16 +1260,17 @@ def add_optouts(ap):
ap2.add_argument("--no-dav", action="store_true", help="disable webdav support")
ap2.add_argument("--no-del", action="store_true", help="disable delete operations")
ap2.add_argument("--no-mv", action="store_true", help="disable move/rename operations")
ap2.add_argument("--no-cp", action="store_true", help="disable copy operations")
ap2.add_argument("-nth", action="store_true", help="no title hostname; don't show \033[33m--name\033[0m in <title>")
ap2.add_argument("-nih", action="store_true", help="no info hostname -- don't show in UI")
ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI")
ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
ap2.add_argument("--zip-who", metavar="LVL", type=int, default=3, help="who can download as zip/tar? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=authenticated-with-read-access, [\033[32m3\033[0m]=everyone-with-read-access (volflag=zip_who)\n\033[1;31mWARNING:\033[0m if a nested volume has a more restrictive value than a parent volume, then this will be \033[33mignored\033[0m if the download is initiated from the parent, more lenient volume")
ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar; same as \033[33m--zip-who=0\033[0m")
ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
ap2.add_argument("--no-up-list", action="store_true", help="don't show list of incoming files in controlpanel")
ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader-IP into the database; will also disable unpost, you may want \033[32m--forget-ip\033[0m instead (volflag=no_db_ip)")
def add_safety(ap):
@@ -1307,7 +1335,8 @@ def add_logging(ap):
ap2.add_argument("--log-conn", action="store_true", help="debug: print tcp-server msgs")
ap2.add_argument("--log-htp", action="store_true", help="debug: print http-server threadpool scaling")
ap2.add_argument("--ihead", metavar="HEADER", type=u, action='append', help="print request \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|\?th=[wj]$|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching regex \033[33mRE\033[0m")
ap2.add_argument("--ohead", metavar="HEADER", type=u, action='append', help="print response \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|[?&]th=[wjp]|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching regex \033[33mRE\033[0m")
def add_admin(ap):
@@ -1315,9 +1344,16 @@ def add_admin(ap):
ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
ap2.add_argument("--no-ups-page", action="store_true", help="disable ?ru (list of recent uploads)")
ap2.add_argument("--no-up-list", action="store_true", help="don't show list of incoming files in controlpanel")
ap2.add_argument("--dl-list", metavar="LVL", type=int, default=2, help="who can see active downloads in the controlpanel? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=everyone")
ap2.add_argument("--ups-who", metavar="LVL", type=int, default=2, help="who can see recent uploads on the ?ru page? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=everyone (volflag=ups_who)")
ap2.add_argument("--ups-when", action="store_true", help="let everyone see upload timestamps on the ?ru page, not just admins")
def add_thumbnail(ap):
th_ram = (RAM_AVAIL or RAM_TOTAL or 9) * 0.6
th_ram = int(max(min(th_ram, 6), 1) * 10) / 10
ap2 = ap.add_argument_group('thumbnail options')
ap2.add_argument("--no-thumb", action="store_true", help="disable all thumbnails (volflag=dthumb)")
ap2.add_argument("--no-vthumb", action="store_true", help="disable video thumbnails (volflag=dvthumb)")
@@ -1325,7 +1361,7 @@ def add_thumbnail(ap):
ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res (volflag=thsize)")
ap2.add_argument("--th-mt", metavar="CORES", type=int, default=CORES, help="num cpu cores to use for generating thumbnails")
ap2.add_argument("--th-convt", metavar="SEC", type=float, default=60.0, help="conversion timeout in seconds (volflag=convt)")
ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=6.0, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=th_ram, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
ap2.add_argument("--th-crop", metavar="TXT", type=u, default="y", help="crop thumbnails to 4:3 or keep dynamic height; client can override in UI unless force. [\033[32my\033[0m]=crop, [\033[32mn\033[0m]=nocrop, [\033[32mfy\033[0m]=force-y, [\033[32mfn\033[0m]=force-n (volflag=crop)")
ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32my\033[0m]=yes, [\033[32mn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
ap2.add_argument("--th-dec", metavar="LIBS", default="vips,pil,ff", help="image decoders, in order of preference")
@@ -1340,23 +1376,33 @@ def add_thumbnail(ap):
# https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
# https://github.com/libvips/libvips
# ffmpeg -hide_banner -demuxers | awk '/^ D /{print$2}' | while IFS= read -r x; do ffmpeg -hide_banner -h demuxer=$x; done | grep -E '^Demuxer |extensions:'
ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,dcx,dds,dib,emf,eps,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,cbz,dcx,dds,dib,emf,eps,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,pbm,pcx,pgm,png,pnm,ppm,psd,qoi,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,cbz,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,mdgz,mdxz,mdz,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz", help="audio formats to decompress before passing to ffmpeg")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz, cbz=jpg.cbz", help="audio/image formats to decompress before passing to ffmpeg")
def add_transcoding(ap):
ap2 = ap.add_argument_group('transcoding options')
ap2.add_argument("--q-opus", metavar="KBPS", type=int, default=128, help="target bitrate for transcoding to opus; set 0 to disable")
ap2.add_argument("--q-mp3", metavar="QUALITY", type=u, default="q2", help="target quality for transcoding to mp3, for example [\033[32m192k\033[0m] (CBR) or [\033[32mq0\033[0m] (CQ/CRF, q0=maxquality, q9=smallest); set 0 to disable")
ap2.add_argument("--no-caf", action="store_true", help="disable transcoding to caf-opus (affects iOS v12~v17), will use mp3 instead")
ap2.add_argument("--no-owa", action="store_true", help="disable transcoding to webm-opus (iOS v18 and later), will use mp3 instead")
ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
ap2.add_argument("--no-bacode", action="store_true", help="disable batch audio transcoding by folder download (zip/tar)")
ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete cached transcode output after \033[33mSEC\033[0m seconds")
def add_rss(ap):
ap2 = ap.add_argument_group('RSS options')
ap2.add_argument("--rss", action="store_true", help="enable RSS output (experimental) (volflag=rss)")
ap2.add_argument("--rss-nf", metavar="HITS", type=int, default=250, help="default number of files to return (url-param 'nf')")
ap2.add_argument("--rss-fext", metavar="E,E", type=u, default="", help="default list of file extensions to include (url-param 'fext'); blank=all")
ap2.add_argument("--rss-sort", metavar="ORD", type=u, default="m", help="default sort order (url-param 'sort'); [\033[32mm\033[0m]=last-modified [\033[32mu\033[0m]=upload-time [\033[32mn\033[0m]=filename [\033[32ms\033[0m]=filesize; Uppercase=oldest-first. Note that upload-time is 0 for non-uploaded files")
def add_db_general(ap, hcores):
noidx = APPLESAN_TXT if MACOS else ""
ap2 = ap.add_argument_group('general db options')
@@ -1374,6 +1420,7 @@ def add_db_general(ap, hcores):
ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice -- only useful for offloading uploads to a cloud service or something (volflag=noforget)")
ap2.add_argument("--forget-ip", metavar="MIN", type=int, default=0, help="remove uploader-IP from database (and make unpost impossible) \033[33mMIN\033[0m minutes after upload, for GDPR reasons. Default [\033[32m0\033[0m] is never-forget. [\033[32m1440\033[0m]=day, [\033[32m10080\033[0m]=week, [\033[32m43200\033[0m]=month. (volflag=forget_ip)")
ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see \033[33m--help-dbd\033[0m (volflag=dbd)")
ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (probably buggy, not recommended) (volflag=xlink)")
ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
@@ -1381,6 +1428,7 @@ def add_db_general(ap, hcores):
ap2.add_argument("--db-act", metavar="SEC", type=float, default=10.0, help="defer any scheduled volume reindexing until \033[33mSEC\033[0m seconds after last db write (uploads, renames, ...)")
ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than \033[33mSEC\033[0m seconds")
ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
ap2.add_argument("--srch-excl", metavar="PTN", type=u, default="", help="regex: exclude files from search results if the file-URL matches \033[33mPTN\033[0m (case-sensitive). Example: [\033[32mpassword|logs/[0-9]\033[0m] any URL containing 'password' or 'logs/DIGIT' (volflag=srch_excl)")
ap2.add_argument("--dotsrch", action="store_true", help="show dotfiles in search results (volflags: dotsrch | nodotsrch)")
@@ -1437,9 +1485,13 @@ def add_ui(ap, retry):
ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
ap2.add_argument("--au-vol", metavar="0-100", type=int, default=50, choices=range(0, 101), help="default audio/video volume percent")
ap2.add_argument("--sort", metavar="C,C,C", type=u, default="href", help="default sort order, comma-separated column IDs (see header tooltips), prefix with '-' for descending. Examples: \033[32mhref -href ext sz ts tags/Album tags/.tn\033[0m (volflag=sort)")
ap2.add_argument("--nsort", action="store_true", help="default-enable natural sort of filenames with leading numbers (volflag=nsort)")
ap2.add_argument("--hsortn", metavar="N", type=int, default=2, help="number of sorting rules to include in media URLs by default (volflag=hsortn)")
ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
ap2.add_argument("--ext-th", metavar="E=VP", type=u, action="append", help="use thumbnail-image \033[33mVP\033[0m for file-extension \033[33mE\033[0m, example: [\033[32mexe=/.res/exe.png\033[0m] (volflag=ext_th)")
ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
ap2.add_argument("--spinner", metavar="TXT", type=u, default="🌲", help="\033[33memoji\033[0m or \033[33memoji,css\033[0m Example: [\033[32m🥖,padding:0\033[0m]")
ap2.add_argument("--css-browser", metavar="L", type=u, default="", help="URL to additional CSS to include in the filebrowser html")
ap2.add_argument("--js-browser", metavar="L", type=u, default="", help="URL to additional JS to include in the filebrowser html")
ap2.add_argument("--js-other", metavar="L", type=u, default="", help="URL to additional JS to include in all other pages")
@@ -1449,11 +1501,14 @@ def add_ui(ap, retry):
ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty @ --name", help="title / service-name to show in html documents")
ap2.add_argument("--bname", metavar="TXT", type=u, default="--name", help="server name (displayed in filebrowser document title)")
ap2.add_argument("--pb-url", metavar="URL", type=u, default="https://github.com/9001/copyparty", help="powered-by link; disable with \033[33m-np\033[0m")
ap2.add_argument("--pb-url", metavar="URL", type=u, default=URL_PRJ, help="powered-by link; disable with \033[33m-np\033[0m")
ap2.add_argument("--ver", action="store_true", help="show version on the control panel (incompatible with \033[33m-nb\033[0m)")
ap2.add_argument("--k304", metavar="NUM", type=int, default=0, help="configure the option to enable/disable k304 on the controlpanel (workaround for buggy reverse-proxies); [\033[32m0\033[0m] = hidden and default-off, [\033[32m1\033[0m] = visible and default-off, [\033[32m2\033[0m] = visible and default-on")
ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox")
ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for prologue/epilogue docs (volflag=lg_sbf)")
ap2.add_argument("--no304", metavar="NUM", type=int, default=0, help="configure the option to enable/disable no304 on the controlpanel (workaround for buggy caching in browsers); [\033[32m0\033[0m] = hidden and default-off, [\033[32m1\033[0m] = visible and default-off, [\033[32m2\033[0m] = visible and default-on")
ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to allow in the iframe 'sandbox' attribute for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#sandbox")
ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to allow in the iframe 'sandbox' attribute for prologue/epilogue docs (volflag=lg_sbf)")
ap2.add_argument("--md-sba", metavar="TXT", type=u, default="", help="the value of the iframe 'allow' attribute for README.md docs, for example [\033[32mfullscreen\033[0m] (volflag=md_sba)")
ap2.add_argument("--lg-sba", metavar="TXT", type=u, default="", help="the value of the iframe 'allow' attribute for prologue/epilogue docs (volflag=lg_sba); see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy#iframes")
ap2.add_argument("--no-sb-md", action="store_true", help="don't sandbox README/PREADME.md documents (volflags: no_sb_md | sb_md)")
ap2.add_argument("--no-sb-lg", action="store_true", help="don't sandbox prologue/epilogue docs (volflags: no_sb_lg | sb_lg); enables non-js support")
@@ -1479,6 +1534,7 @@ def add_debug(ap):
ap2.add_argument("--bf-nc", metavar="NUM", type=int, default=200, help="bak-flips: stop if there's more than \033[33mNUM\033[0m files at \033[33m--kf-dir\033[0m already; default: 6.3 GiB max (200*32M)")
ap2.add_argument("--bf-dir", metavar="PATH", type=u, default="bf", help="bak-flips: store corrupted chunks at \033[33mPATH\033[0m; default: folder named 'bf' wherever copyparty was started")
ap2.add_argument("--bf-log", metavar="PATH", type=u, default="", help="bak-flips: log corruption info to a textfile at \033[33mPATH\033[0m")
ap2.add_argument("--no-cfg-cmt-warn", action="store_true", help=argparse.SUPPRESS)
# fmt: on
@@ -1526,6 +1582,7 @@ def run_argparse(
add_db_metadata(ap)
add_thumbnail(ap)
add_transcoding(ap)
add_rss(ap)
add_ftp(ap)
add_webdav(ap)
add_tftp(ap)
@@ -1710,7 +1767,7 @@ def main(argv: Optional[list[str]] = None) -> None:
except:
lprint("\nfailed to disable quick-edit-mode:\n" + min_ex() + "\n")
if al.ansi:
if not al.ansi:
al.wintitle = ""
# propagate implications
@@ -1748,6 +1805,9 @@ def main(argv: Optional[list[str]] = None) -> None:
if al.ihead:
al.ihead = [x.lower() for x in al.ihead]
if al.ohead:
al.ohead = [x.lower() for x in al.ohead]
if HAVE_SSL:
if al.ssl_ver:
configure_ssl_ver(al)

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 15, 7)
CODENAME = "fill the drives"
BUILD_DT = (2024, 10, 14)
VERSION = (1, 16, 16)
CODENAME = "COPYparty"
BUILD_DT = (2025, 2, 28)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -66,6 +66,7 @@ if PY2:
LEELOO_DALLAS = "leeloo_dallas"
SEE_LOG = "see log for details"
SEESLOG = " (see serverlog for details)"
SSEELOG = " ({})".format(SEE_LOG)
BAD_CFG = "invalid config; {}".format(SEE_LOG)
SBADCFG = " ({})".format(BAD_CFG)
@@ -164,8 +165,11 @@ class Lim(object):
self.chk_rem(rem)
if sz != -1:
self.chk_sz(sz)
self.chk_vsz(broker, ptop, sz, volgetter)
self.chk_df(abspath, sz) # side effects; keep last-ish
else:
sz = 0
self.chk_vsz(broker, ptop, sz, volgetter)
self.chk_df(abspath, sz) # side effects; keep last-ish
ap2, vp2 = self.rot(abspath)
if abspath == ap2:
@@ -205,7 +209,15 @@ class Lim(object):
if self.dft < time.time():
self.dft = int(time.time()) + 300
self.dfv = get_df(abspath)[0] or 0
df, du, err = get_df(abspath, True)
if err:
t = "failed to read disk space usage for %r: %s"
self.log(t % (abspath, err), 3)
self.dfv = 0xAAAAAAAAA # 42.6 GiB
else:
self.dfv = df or 0
for j in list(self.reg.values()) if self.reg else []:
self.dfv -= int(j["size"] / (len(j["hash"]) or 999) * len(j["need"]))
@@ -355,18 +367,21 @@ class VFS(object):
self.ahtml: dict[str, list[str]] = {}
self.aadmin: dict[str, list[str]] = {}
self.adot: dict[str, list[str]] = {}
self.all_vols: dict[str, VFS] = {}
self.js_ls = {}
self.js_htm = ""
if realpath:
rp = realpath + ("" if realpath.endswith(os.sep) else os.sep)
vp = vpath + ("/" if vpath else "")
self.histpath = os.path.join(realpath, ".hist") # db / thumbcache
self.all_vols = {vpath: self} # flattened recursive
self.all_nodes = {vpath: self} # also jumpvols
self.all_aps = [(rp, self)]
self.all_vps = [(vp, self)]
else:
self.histpath = ""
self.all_vols = {}
self.all_nodes = {}
self.all_aps = []
self.all_vps = []
@@ -384,9 +399,11 @@ class VFS(object):
def get_all_vols(
self,
vols: dict[str, "VFS"],
nodes: dict[str, "VFS"],
aps: list[tuple[str, "VFS"]],
vps: list[tuple[str, "VFS"]],
) -> None:
nodes[self.vpath] = self
if self.realpath:
vols[self.vpath] = self
rp = self.realpath
@@ -396,7 +413,7 @@ class VFS(object):
vps.append((vp, self))
for v in self.nodes.values():
v.get_all_vols(vols, aps, vps)
v.get_all_vols(vols, nodes, aps, vps)
def add(self, src: str, dst: str) -> "VFS":
"""get existing, or add new path to the vfs"""
@@ -509,7 +526,7 @@ class VFS(object):
"""returns [vfsnode,fs_remainder] if user has the requested permissions"""
if relchk(vpath):
if self.log:
self.log("vfs", "invalid relpath [{}]".format(vpath))
self.log("vfs", "invalid relpath %r @%s" % (vpath, uname))
raise Pebkac(422)
cvpath = undot(vpath)
@@ -526,11 +543,11 @@ class VFS(object):
if req and uname not in d and uname != LEELOO_DALLAS:
if vpath != cvpath and vpath != "." and self.log:
ap = vn.canonical(rem)
t = "{} has no {} in [{}] => [{}] => [{}]"
self.log("vfs", t.format(uname, msg, vpath, cvpath, ap), 6)
t = "%s has no %s in %r => %r => %r"
self.log("vfs", t % (uname, msg, vpath, cvpath, ap), 6)
t = 'you don\'t have %s-access in "/%s" or below "/%s"'
raise Pebkac(err, t % (msg, cvpath, vn.vpath))
t = "you don't have %s-access in %r or below %r"
raise Pebkac(err, t % (msg, "/" + cvpath, "/" + vn.vpath))
return vn, rem
@@ -540,15 +557,14 @@ class VFS(object):
return self._get_dbv(vrem)
shv, srem = src
return shv, vjoin(srem, vrem)
return shv._get_dbv(vjoin(srem, vrem))
def _get_dbv(self, vrem: str) -> tuple["VFS", str]:
dbv = self.dbv
if not dbv:
return self, vrem
tv = [self.vpath[len(dbv.vpath) :].lstrip("/"), vrem]
vrem = "/".join([x for x in tv if x])
vrem = vjoin(self.vpath[len(dbv.vpath) :].lstrip("/"), vrem)
return dbv, vrem
def canonical(self, rem: str, resolve: bool = True) -> str:
@@ -580,10 +596,11 @@ class VFS(object):
scandir: bool,
permsets: list[list[bool]],
lstat: bool = False,
throw: bool = False,
) -> tuple[str, list[tuple[str, os.stat_result]], dict[str, "VFS"]]:
"""replaces _ls for certain shares (single-file, or file selection)"""
vn, rem = self.shr_src # type: ignore
abspath, real, _ = vn.ls(rem, "\n", scandir, permsets, lstat)
abspath, real, _ = vn.ls(rem, "\n", scandir, permsets, lstat, throw)
real = [x for x in real if os.path.basename(x[0]) in self.shr_files]
return abspath, real, {}
@@ -594,11 +611,12 @@ class VFS(object):
scandir: bool,
permsets: list[list[bool]],
lstat: bool = False,
throw: bool = False,
) -> tuple[str, list[tuple[str, os.stat_result]], dict[str, "VFS"]]:
"""return user-readable [fsdir,real,virt] items at vpath"""
virt_vis = {} # nodes readable by user
abspath = self.canonical(rem)
real = list(statdir(self.log, scandir, lstat, abspath))
real = list(statdir(self.log, scandir, lstat, abspath, throw))
real.sort()
if not rem:
# no vfs nodes in the list of real inodes
@@ -640,7 +658,7 @@ class VFS(object):
seen: list[str],
uname: str,
permsets: list[list[bool]],
wantdots: bool,
wantdots: int,
scandir: bool,
lstat: bool,
subvols: bool = True,
@@ -660,6 +678,10 @@ class VFS(object):
"""
recursively yields from ./rem;
rel is a unix-style user-defined vpath (not vfs-related)
NOTE: don't invoke this function from a dbv; subvols are only
descended into if rem is blank due to the _ls `if not rem:`
which intention is to prevent unintended access to subvols
"""
fsroot, vfs_ls, vfs_virt = self.ls(rem, uname, scandir, permsets, lstat=lstat)
@@ -671,8 +693,8 @@ class VFS(object):
and fsroot in seen
):
if self.log:
t = "bailing from symlink loop,\n prev: {}\n curr: {}\n from: {}/{}"
self.log("vfs.walk", t.format(seen[-1], fsroot, self.vpath, rem), 3)
t = "bailing from symlink loop,\n prev: %r\n curr: %r\n from: %r / %r"
self.log("vfs.walk", t % (seen[-1], fsroot, self.vpath, rem), 3)
return
if "xdev" in self.flags or "xvol" in self.flags:
@@ -684,7 +706,7 @@ class VFS(object):
rm1.append(le)
_ = [vfs_ls.remove(x) for x in rm1] # type: ignore
dots_ok = wantdots and uname in dbv.axs.udot
dots_ok = wantdots and (wantdots == 2 or uname in dbv.axs.udot)
if not dots_ok:
vfs_ls = [x for x in vfs_ls if "/." not in "/" + x[0]]
@@ -738,7 +760,7 @@ class VFS(object):
# if single folder: the folder itself is the top-level item
folder = "" if flt or not wrap else (vpath.split("/")[-1].lstrip(".") or "top")
g = self.walk(folder, vrem, [], uname, [[True, False]], True, scandir, False)
g = self.walk(folder, vrem, [], uname, [[True, False]], 1, scandir, False)
for _, _, vpath, apath, files, rd, vd in g:
if flt:
files = [x for x in files if x[0] in flt]
@@ -796,8 +818,8 @@ class VFS(object):
if vdev != st.st_dev:
if self.log:
t = "xdev: {}[{}] => {}[{}]"
self.log("vfs", t.format(vdev, self.realpath, st.st_dev, ap), 3)
t = "xdev: %s[%r] => %s[%r]"
self.log("vfs", t % (vdev, self.realpath, st.st_dev, ap), 3)
return None
@@ -807,7 +829,7 @@ class VFS(object):
return vn
if self.log:
self.log("vfs", "xvol: [{}]".format(ap), 3)
self.log("vfs", "xvol: %r" % (ap,), 3)
return None
@@ -892,7 +914,7 @@ class AuthSrv(object):
self.idp_accs[uname] = gnames
t = "reinitializing due to new user from IdP: [%s:%s]"
t = "reinitializing due to new user from IdP: [%r:%r]"
self.log(t % (uname, gnames), 3)
if not broker:
@@ -900,7 +922,7 @@ class AuthSrv(object):
self._reload()
return True
broker.ask("_reload_blocking", False).get()
broker.ask("reload", False, True).get()
return True
def _map_volume_idp(
@@ -1267,10 +1289,10 @@ class AuthSrv(object):
# one or more bools before the final flag; eat them
n1, uname = uname.split(",", 1)
for _, vp, _, _ in vols:
self._read_volflag(flags[vp], n1, True, False)
self._read_volflag(vp, flags[vp], n1, True, False)
for _, vp, _, _ in vols:
self._read_volflag(flags[vp], uname, cval, False)
self._read_volflag(vp, flags[vp], uname, cval, False)
return
@@ -1357,20 +1379,42 @@ class AuthSrv(object):
def _read_volflag(
self,
vpath: str,
flags: dict[str, Any],
name: str,
value: Union[str, bool, list[str]],
is_list: bool,
) -> None:
if name not in flagdescs:
name = name.lower()
# volflags are snake_case, but a leading dash is the removal operator
stripped = name.lstrip("-")
zi = len(name) - len(stripped)
if zi > 1:
t = "WARNING: the config for volume [/%s] specified a volflag with multiple leading hyphens (%s); use one hyphen to remove, or zero hyphens to add a flag. Will now enable flag [%s]"
self.log(t % (vpath, name, stripped), 3)
name = stripped
zi = 0
if stripped not in flagdescs and "-" in stripped:
name = ("-" * zi) + stripped.replace("-", "_")
desc = flagdescs.get(name.lstrip("-"), "?").replace("\n", " ")
if not name:
self._e("└─unreadable-line")
t = "WARNING: the config for volume [/%s] indicated that a volflag was to be defined, but the volflag name was blank"
self.log(t % (vpath,), 3)
return
if re.match("^-[^-]+$", name):
t = "└─unset volflag [{}] ({})"
self._e(t.format(name[1:], desc))
flags[name] = True
return
zs = "mtp on403 on404 xbu xau xiu xbr xar xbd xad xm xban"
zs = "ext_th mtp on403 on404 xbu xau xiu xbc xac xbr xar xbd xad xm xban"
if name not in zs.split():
if value is True:
t = "└─add volflag [{}] = {} ({})"
@@ -1493,6 +1537,14 @@ class AuthSrv(object):
if not mount and not self.args.idp_h_usr:
# -h says our defaults are CWD at root and read/write for everyone
axs = AXS(["*"], ["*"], None, None)
if os.path.exists("/z/initcfg"):
t = "Read-access has been disabled due to failsafe: Docker detected, but the config does not define any volumes. This failsafe is to prevent unintended access if this is due to accidental loss of config. You can override this safeguard and allow read/write to all of /w/ by adding the following arguments to the docker container: -v .::rw"
self.log(t, 1)
axs = AXS()
elif self.args.c:
t = "Read-access has been disabled due to failsafe: No volumes were defined by the config-file. This failsafe is to prevent unintended access if this is due to accidental loss of config. You can override this safeguard and allow read/write to the working-directory by adding the following arguments: -v .::rw"
self.log(t, 1)
axs = AXS()
vfs = VFS(self.log_func, absreal("."), "", axs, {})
elif "" not in mount:
# there's volumes but no root; make root inaccessible
@@ -1518,14 +1570,27 @@ class AuthSrv(object):
assert vfs # type: ignore
vfs.all_vols = {}
vfs.all_nodes = {}
vfs.all_aps = []
vfs.all_vps = []
vfs.get_all_vols(vfs.all_vols, vfs.all_aps, vfs.all_vps)
for vol in vfs.all_vols.values():
vfs.get_all_vols(vfs.all_vols, vfs.all_nodes, vfs.all_aps, vfs.all_vps)
for vol in vfs.all_nodes.values():
vol.all_aps.sort(key=lambda x: len(x[0]), reverse=True)
vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
vol.root = vfs
zs = "neversymlink"
k_ign = set(zs.split())
for vol in vfs.all_vols.values():
unknown_flags = set()
for k, v in vol.flags.items():
ks = k.lstrip("-")
if ks not in flagdescs and ks not in k_ign:
unknown_flags.add(k)
if unknown_flags:
t = "WARNING: the config for volume [/%s] has unrecognized volflags; will ignore: '%s'"
self.log(t % (vol.vpath, "', '".join(unknown_flags)), 3)
enshare = self.args.shr
shr = enshare[1:-1]
shrs = enshare[1:]
@@ -1545,7 +1610,7 @@ class AuthSrv(object):
continue
if self.args.shr_v:
t = "loading %s share [%s] by [%s] => [%s]"
t = "loading %s share %r by %r => %r"
self.log(t % (s_pr, s_k, s_un, s_vp))
if s_pw:
@@ -1572,7 +1637,7 @@ class AuthSrv(object):
vfs.nodes[shr] = vfs.all_vols[shr] = shv
for vol in shv.nodes.values():
vfs.all_vols[vol.vpath] = vol
vfs.all_vols[vol.vpath] = vfs.all_nodes[vol.vpath] = vol
vol.get_dbv = vol._get_share_src
vol.ls = vol._ls_nope
@@ -1715,7 +1780,19 @@ class AuthSrv(object):
self.log("\n\n".join(ta) + "\n", c=3)
vfs.histtab = {zv.realpath: zv.histpath for zv in vfs.all_vols.values()}
rhisttab = {}
vfs.histtab = {}
for zv in vfs.all_vols.values():
histp = zv.histpath
is_shr = shr and zv.vpath.split("/")[0] == shr
if histp and not is_shr and histp in rhisttab:
zv2 = rhisttab[histp]
t = "invalid config; multiple volumes share the same histpath (database location):\n histpath: %s\n volume 1: /%s [%s]\n volume 2: %s [%s]"
t = t % (histp, zv2.vpath, zv2.realpath, zv.vpath, zv.realpath)
self.log(t, 1)
raise Exception(t)
rhisttab[histp] = zv
vfs.histtab[zv.realpath] = histp
for vol in vfs.all_vols.values():
lim = Lim(self.log_func)
@@ -1730,7 +1807,7 @@ class AuthSrv(object):
use = True
try:
_ = float(zs)
zs = "%sg" % (zs)
zs = "%sg" % (zs,)
except:
pass
lim.dfl = unhumanize(zs)
@@ -1774,12 +1851,12 @@ class AuthSrv(object):
vol.lim = lim
if self.args.no_robots:
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
# volflag "robots" overrides global "norobots", allowing indexing by search engines for this vol
if not vol.flags.get("robots"):
vol.flags["norobots"] = True
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
if self.args.no_vthumb:
vol.flags["dvthumb"] = True
if self.args.no_athumb:
@@ -1791,13 +1868,17 @@ class AuthSrv(object):
vol.flags["dithumb"] = True
have_fk = False
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
fk = vol.flags.get("fk")
fka = vol.flags.get("fka")
if fka and not fk:
fk = fka
if fk:
vol.flags["fk"] = int(fk) if fk is not True else 8
fk = 8 if fk is True else int(fk)
if fk > 72:
t = "max filekey-length is 72; volume /%s specified %d (anything higher than 16 is pointless btw)"
raise Exception(t % (vol.vpath, fk))
vol.flags["fk"] = fk
have_fk = True
dk = vol.flags.get("dk")
@@ -1823,7 +1904,7 @@ class AuthSrv(object):
zs = os.path.join(E.cfg, "fk-salt.txt")
self.log(t % (fk_len, 16, zs), 3)
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
if "pk" in vol.flags and "gz" not in vol.flags and "xz" not in vol.flags:
vol.flags["gz"] = False # def.pk
@@ -1834,7 +1915,7 @@ class AuthSrv(object):
all_mte = {}
errors = False
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
if (self.args.e2ds and vol.axs.uwrite) or self.args.e2dsa:
vol.flags["e2ds"] = True
@@ -1845,6 +1926,7 @@ class AuthSrv(object):
["no_hash", "nohash"],
["no_idx", "noidx"],
["og_ua", "og_ua"],
["srch_excl", "srch_excl"],
]:
if vf in vol.flags:
ptn = re.compile(vol.flags.pop(vf))
@@ -1870,11 +1952,8 @@ class AuthSrv(object):
if vf not in vol.flags:
vol.flags[vf] = getattr(self.args, ga)
for k in ("nrand",):
if k not in vol.flags:
vol.flags[k] = getattr(self.args, k)
for k in ("nrand", "u2abort"):
zs = "forget_ip nrand u2abort u2ow ups_who zip_who"
for k in zs.split():
if k in vol.flags:
vol.flags[k] = int(vol.flags[k])
@@ -1925,9 +2004,11 @@ class AuthSrv(object):
vol.flags[k] = odfusion(getattr(self.args, k), vol.flags[k])
# append additive args from argv to volflags
hooks = "xbu xau xiu xbr xar xbd xad xm xban".split()
for name in "mtp on404 on403".split() + hooks:
self._read_volflag(vol.flags, name, getattr(self.args, name), True)
hooks = "xbu xau xiu xbc xac xbr xar xbd xad xm xban".split()
for name in "ext_th mtp on404 on403".split() + hooks:
self._read_volflag(
vol.vpath, vol.flags, name, getattr(self.args, name), True
)
for hn in hooks:
cmds = vol.flags.get(hn)
@@ -1955,6 +2036,16 @@ class AuthSrv(object):
ncmds.append(ocmd)
vol.flags[hn] = ncmds
ext_th = vol.flags["ext_th_d"] = {}
etv = "(?)"
try:
for etv in vol.flags.get("ext_th") or []:
k, v = etv.split("=")
ext_th[k] = v
except:
t = "WARNING: volume [/%s]: invalid value specified for ext-th: %s"
self.log(t % (vol.vpath, etv), 3)
# d2d drops all database features for a volume
for grp, rm in [["d2d", "e2d"], ["d2t", "e2t"], ["d2d", "e2v"]]:
if not vol.flags.get(grp, False):
@@ -2051,8 +2142,24 @@ class AuthSrv(object):
self.log(t.format(mtp), 1)
errors = True
have_daw = False
for vol in vfs.all_vols.values():
re1: Optional[re.Pattern] = vol.flags.get("srch_excl")
excl = [re1.pattern] if re1 else []
vpaths = []
vtop = vol.vpath
for vp2 in vfs.all_vols.keys():
if vp2.startswith((vtop + "/").lstrip("/")) and vtop != vp2:
vpaths.append(re.escape(vp2[len(vtop) :].lstrip("/")))
if vpaths:
excl.append("^(%s)/" % ("|".join(vpaths),))
vol.flags["srch_re_dots"] = re.compile("|".join(excl or ["^$"]))
excl.extend([r"^\.", r"/\."])
vol.flags["srch_re_nodot"] = re.compile("|".join(excl))
have_daw = False
for vol in vfs.all_nodes.values():
daw = vol.flags.get("daw") or self.args.daw
if daw:
vol.flags["daw"] = True
@@ -2067,13 +2174,17 @@ class AuthSrv(object):
self.log("--smb can only be used when --ah-alg is none", 1)
errors = True
for vol in vfs.all_vols.values():
for vol in vfs.all_nodes.values():
for k in list(vol.flags.keys()):
if re.match("^-[^-]+$", k):
vol.flags.pop(k[1:], None)
vol.flags.pop(k)
zs = k[1:]
if zs in vol.flags:
vol.flags.pop(k[1:])
else:
t = "WARNING: the config for volume [/%s] tried to remove volflag [%s] by specifying [%s] but that volflag was not already set"
self.log(t % (vol.vpath, zs, k), 3)
for vol in vfs.all_vols.values():
if vol.flags.get("dots"):
for name in vol.axs.uread:
vol.axs.udot.add(name)
@@ -2130,11 +2241,11 @@ class AuthSrv(object):
if not self.args.no_voldump:
self.log(t)
if have_e2d:
if have_e2d or self.args.idp_h_usr:
t = self.chk_sqlite_threadsafe()
if t:
self.log("\n\033[{}\033[0m\n".format(t))
if have_e2d:
if not have_e2t:
t = "hint: enable multimedia indexing (artist/title/...) with argument -e2ts"
self.log(t, 6)
@@ -2215,6 +2326,11 @@ class AuthSrv(object):
for x, y in vfs.all_vols.items()
if x != shr and not x.startswith(shrs)
}
vfs.all_nodes = {
x: y
for x, y in vfs.all_nodes.items()
if x != shr and not x.startswith(shrs)
}
assert db and cur and cur2 and shv # type: ignore
for row in cur.execute("select * from sh"):
@@ -2267,6 +2383,77 @@ class AuthSrv(object):
cur.close()
db.close()
self.js_ls = {}
self.js_htm = {}
for vn in self.vfs.all_nodes.values():
vf = vn.flags
vn.js_ls = {
"idx": "e2d" in vf,
"itag": "e2t" in vf,
"dnsort": "nsort" in vf,
"dhsortn": vf["hsortn"],
"dsort": vf["sort"],
"dcrop": vf["crop"],
"dth3x": vf["th3x"],
"u2ts": vf["u2ts"],
"frand": bool(vf.get("rand")),
"lifetime": vf.get("lifetime") or 0,
"unlist": vf.get("unlist") or "",
"sb_lg": "" if "no_sb_lg" in vf else (vf.get("lg_sbf") or "y"),
}
js_htm = {
"SPINNER": self.args.spinner,
"s_name": self.args.bname,
"have_up2k_idx": "e2d" in vf,
"have_acode": not self.args.no_acode,
"have_shr": self.args.shr,
"have_zip": not self.args.no_zip,
"have_mv": not self.args.no_mv,
"have_del": not self.args.no_del,
"have_unpost": int(self.args.unpost),
"have_emp": self.args.emp,
"ext_th": vf.get("ext_th_d") or {},
"sb_md": "" if "no_sb_md" in vf else (vf.get("md_sbf") or "y"),
"sba_md": vf.get("md_sba") or "",
"sba_lg": vf.get("lg_sba") or "",
"txt_ext": self.args.textfiles.replace(",", " "),
"def_hcols": list(vf.get("mth") or []),
"unlist0": vf.get("unlist") or "",
"dgrid": "grid" in vf,
"dgsel": "gsel" in vf,
"dnsort": "nsort" in vf,
"dhsortn": vf["hsortn"],
"dsort": vf["sort"],
"dcrop": vf["crop"],
"dth3x": vf["th3x"],
"dvol": self.args.au_vol,
"idxh": int(self.args.ih),
"themes": self.args.themes,
"turbolvl": self.args.turbo,
"u2j": self.args.u2j,
"u2sz": self.args.u2sz,
"u2ts": vf["u2ts"],
"u2ow": vf["u2ow"],
"frand": bool(vf.get("rand")),
"lifetime": vn.js_ls["lifetime"],
"u2sort": self.args.u2sort,
}
vn.js_htm = json.dumps(js_htm)
vols = list(vfs.all_nodes.values())
if enshare:
assert shv # type: ignore # !rm
vols.append(shv)
vols.extend(list(shv.nodes.values()))
for vol in vols:
dbv = vol.get_dbv("")[0]
vol.js_ls = vol.js_ls or dbv.js_ls or {}
vol.js_htm = vol.js_htm or dbv.js_htm or "{}"
zs = str(vol.flags.get("tcolor") or self.args.tcolor)
vol.flags["tcolor"] = zs.lstrip("#")
def load_sessions(self, quiet=False) -> None:
# mutex me
if self.args.no_ses:
@@ -2376,7 +2563,7 @@ class AuthSrv(object):
self._reload()
return True, "new password OK"
broker.ask("_reload_blocking", False, False).get()
broker.ask("reload", False, False).get()
return True, "new password OK"
def setup_chpw(self, acct: dict[str, str]) -> None:
@@ -2417,7 +2604,7 @@ class AuthSrv(object):
return
elif self.args.chpw_v == 2:
t = "chpw: %d changed" % (len(uok))
t = "chpw: %d changed" % (len(uok),)
if urst:
t += ", \033[0munchanged:\033[35m %s" % (", ".join(list(urst)))
@@ -2575,7 +2762,7 @@ class AuthSrv(object):
[],
u,
[[True, False]],
True,
1,
not self.args.no_scandir,
False,
False,
@@ -2628,10 +2815,12 @@ class AuthSrv(object):
]
csv = set("i p th_covers zm_on zm_off zs_on zs_off".split())
zs = "c ihead mtm mtp on403 on404 xad xar xau xiu xban xbd xbr xbu xm"
zs = "c ihead ohead mtm mtp on403 on404 xac xad xar xau xiu xban xbc xbd xbr xbu xm"
lst = set(zs.split())
askip = set("a v c vc cgen exp_lg exp_md theme".split())
fskip = set("exp_lg exp_md mv_re_r mv_re_t rm_re_r rm_re_t".split())
t = "exp_lg exp_md ext_th_d mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot"
fskip = set(t.split())
# keymap from argv to vflag
amap = vf_bmap()
@@ -2896,6 +3085,19 @@ def expand_config_file(
ret.append("#\033[36m closed{}\033[0m".format(ipath))
zsl = []
for ln in ret:
zs = ln.split(" #")[0]
if " #" in zs and zs.split("#")[0].strip():
zsl.append(ln)
if zsl and "no-cfg-cmt-warn" not in "\n".join(ret):
t = "\033[33mWARNING: there is less than two spaces before the # in the following config lines, so instead of assuming that this is a comment, the whole line will become part of the config value:\n\n>>> %s\n\nif you are familiar with this and would like to mute this warning, specify the global-option no-cfg-cmt-warn\n\033[0m"
t = t % ("\n>>> ".join(zsl),)
if log:
log(t)
else:
print(t, file=sys.stderr)
def upgrade_cfg_fmt(
log: Optional["NamedLogger"], args: argparse.Namespace, orig: list[str], cfg_fp: str

View File

@@ -43,6 +43,9 @@ class BrokerMp(object):
self.procs = []
self.mutex = threading.Lock()
self.retpend: dict[int, Any] = {}
self.retpend_mutex = threading.Lock()
self.num_workers = self.args.j or CORES
self.log("broker", "booting {} subprocesses".format(self.num_workers))
for n in range(1, self.num_workers + 1):
@@ -54,6 +57,8 @@ class BrokerMp(object):
self.procs.append(proc)
proc.start()
Daemon(self.periodic, "mp-periodic")
def shutdown(self) -> None:
self.log("broker", "shutting down")
for n, proc in enumerate(self.procs):
@@ -90,8 +95,10 @@ class BrokerMp(object):
self.log(*args)
elif dest == "retq":
# response from previous ipc call
raise Exception("invalid broker_mp usage")
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
retq.put(args[0])
else:
# new ipc invoking managed service in hub
@@ -109,7 +116,6 @@ class BrokerMp(object):
proc.q_pend.put((retq_id, "retq", rv))
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
# new non-ipc invoking managed service in hub
obj = self.hub
for node in dest.split("."):
@@ -121,17 +127,30 @@ class BrokerMp(object):
retq.put(rv)
return retq
def wask(self, dest: str, *args: Any) -> list[Union[ExceptionalQueue, NotExQueue]]:
# call from hub to workers
ret = []
for p in self.procs:
retq = ExceptionalQueue(1)
retq_id = id(retq)
with self.retpend_mutex:
self.retpend[retq_id] = retq
p.q_pend.put((retq_id, dest, list(args)))
ret.append(retq)
return ret
def say(self, dest: str, *args: Any) -> None:
"""
send message to non-hub component in other process,
returns a Queue object which eventually contains the response if want_retval
(not-impl here since nothing uses it yet)
"""
if dest == "listen":
if dest == "httpsrv.listen":
for p in self.procs:
p.q_pend.put((0, dest, [args[0], len(self.procs)]))
elif dest == "set_netdevs":
elif dest == "httpsrv.set_netdevs":
for p in self.procs:
p.q_pend.put((0, dest, list(args)))
@@ -140,3 +159,19 @@ class BrokerMp(object):
else:
raise Exception("what is " + str(dest))
def periodic(self) -> None:
while True:
time.sleep(1)
tdli = {}
tdls = {}
qs = self.wask("httpsrv.read_dls")
for q in qs:
qr = q.get()
dli, dls = qr
tdli.update(dli)
tdls.update(dls)
tdl = (tdli, tdls)
for p in self.procs:
p.q_pend.put((0, "httpsrv.write_dls", tdl))

View File

@@ -82,37 +82,38 @@ class MpWorker(BrokerCli):
while True:
retq_id, dest, args = self.q_pend.get()
# self.logw("work: [{}]".format(d[0]))
if dest == "retq":
# response from previous ipc call
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
retq.put(args)
continue
if dest == "shutdown":
self.httpsrv.shutdown()
self.logw("ok bye")
sys.exit(0)
return
elif dest == "reload":
if dest == "reload":
self.logw("mpw.asrv reloading")
self.asrv.reload()
self.logw("mpw.asrv reloaded")
continue
elif dest == "reload_sessions":
if dest == "reload_sessions":
with self.asrv.mutex:
self.asrv.load_sessions()
continue
elif dest == "listen":
self.httpsrv.listen(args[0], args[1])
obj = self
for node in dest.split("."):
obj = getattr(obj, node)
elif dest == "set_netdevs":
self.httpsrv.set_netdevs(args[0])
elif dest == "retq":
# response from previous ipc call
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
retq.put(args)
else:
raise Exception("what is " + str(dest))
rv = obj(*args) # type: ignore
if retq_id:
self.say("retq", rv, retq_id=retq_id)
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
retq = ExceptionalQueue(1)
@@ -123,5 +124,5 @@ class MpWorker(BrokerCli):
self.q_yield.put((retq_id, dest, list(args)))
return retq
def say(self, dest: str, *args: Any) -> None:
self.q_yield.put((0, dest, list(args)))
def say(self, dest: str, *args: Any, retq_id=0) -> None:
self.q_yield.put((retq_id, dest, list(args)))

View File

@@ -53,11 +53,11 @@ class BrokerThr(BrokerCli):
return NotExQueue(obj(*args)) # type: ignore
def say(self, dest: str, *args: Any) -> None:
if dest == "listen":
if dest == "httpsrv.listen":
self.httpsrv.listen(args[0], 1)
return
if dest == "set_netdevs":
if dest == "httpsrv.set_netdevs":
self.httpsrv.set_netdevs(args[0])
return

View File

@@ -5,6 +5,9 @@ from __future__ import print_function, unicode_literals
zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
onedash = set(zs.split())
# verify that all volflags are documented here:
# grep volflag= __main__.py | sed -r 's/.*volflag=//;s/\).*//' | sort | uniq | while IFS= read -r x; do grep -E "\"$x(=[^ \"]+)?\": \"" cfg.py || printf '%s\n' "$x"; done
def vf_bmap() -> dict[str, str]:
"""argv-to-volflag: simple bools"""
@@ -40,12 +43,15 @@ def vf_bmap() -> dict[str, str]:
"gsel",
"hardlink",
"magic",
"no_db_ip",
"no_sb_md",
"no_sb_lg",
"nsort",
"og",
"og_no_head",
"og_s_title",
"rand",
"rss",
"xdev",
"xlink",
"xvol",
@@ -68,10 +74,15 @@ def vf_vmap() -> dict[str, str]:
}
for k in (
"dbd",
"forget_ip",
"hsortn",
"html_head",
"lg_sbf",
"md_sbf",
"lg_sba",
"md_sba",
"nrand",
"u2ow",
"og_desc",
"og_site",
"og_th",
@@ -88,6 +99,8 @@ def vf_vmap() -> dict[str, str]:
"unlist",
"u2abort",
"u2ts",
"ups_who",
"zip_who",
):
ret[k] = k
return ret
@@ -99,13 +112,16 @@ def vf_cmap() -> dict[str, str]:
for k in (
"exp_lg",
"exp_md",
"ext_th",
"mte",
"mth",
"mtp",
"xac",
"xad",
"xar",
"xau",
"xban",
"xbc",
"xbd",
"xbr",
"xbu",
@@ -139,10 +155,12 @@ flagcats = {
"noclone": "take dupe data from clients, even if available on HDD",
"nodupe": "rejects existing files (instead of linking/cloning them)",
"sparse": "force use of sparse files, mainly for s3-backed storage",
"nosparse": "deny use of sparse files, mainly for slow storage",
"daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
"nosub": "forces all uploads into the top folder of the vfs",
"magic": "enables filetype detection for nameless uploads",
"gz": "allows server-side gzip of uploads with ?gz (also c,xz)",
"gz": "allows server-side gzip compression of uploads with ?gz",
"xz": "allows server-side lzma compression of uploads with ?xz",
"pk": "forces server-side compression, optional arg: xz,9",
},
"upload rules": {
@@ -153,6 +171,7 @@ flagcats = {
"medialinks": "return medialinks for non-up2k uploads (not hotlinks)",
"rand": "force randomized filenames, 9 chars long by default",
"nrand=N": "randomized filenames are N chars long",
"u2ow=N": "overwrite existing files? 0=no 1=if-older 2=always",
"u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
"u2abort=1": "allow aborting unfinished uploads? 0=no 1=strict 2=ip-chk 3=acct-chk",
"sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
@@ -169,8 +188,11 @@ flagcats = {
"e2dsa": "scans all folders for new files on startup; also sets -e2d",
"e2t": "enable multimedia indexing; makes it possible to search for tags",
"e2ts": "scan existing files for tags on startup; also sets -e2t",
"e2tsa": "delete all metadata from DB (full rescan); also sets -e2ts",
"e2tsr": "delete all metadata from DB (full rescan); also sets -e2ts",
"d2ts": "disables metadata collection for existing files",
"e2v": "verify integrity on startup by hashing files and comparing to db",
"e2vu": "when e2v fails, update the db (assume on-disk files are good)",
"e2vp": "when e2v fails, panic and quit copyparty",
"d2ds": "disables onboot indexing, overrides -e2ds*",
"d2t": "disables metadata collection, overrides -e2t*",
"d2v": "disables file verification, overrides -e2v*",
@@ -180,6 +202,8 @@ flagcats = {
"nohash=\\.iso$": "skips hashing file contents if path matches *.iso",
"noidx=\\.iso$": "fully ignores the contents at paths matching *.iso",
"noforget": "don't forget files when deleted from disk",
"forget_ip=43200": "forget uploader-IP after 30 days (GDPR)",
"no_db_ip": "never store uploader-IP in the db; disables unpost",
"fat32": "avoid excessive reindexing on android sdcardfs",
"dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
"xlink": "cross-volume dupe detection / linking (dangerous)",
@@ -187,8 +211,11 @@ flagcats = {
"xvol": "do not follow symlinks leaving the volume root",
"dotsrch": "show dotfiles in search results",
"nodotsrch": "hide dotfiles in search results (default)",
"srch_excl": "exclude search results with URL matching this regex",
},
'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
"mte=artist,title": "media-tags to index/display",
"mth=fmt,res,ac": "media-tags to hide by default",
"mtp=.bpm=f,audio-bpm.py": 'uses the "audio-bpm.py" program to\ngenerate ".bpm" tags from uploads (f = overwrite tags)',
"mtp=ahash,vhash=media-hash.py": "collects two tags at once",
},
@@ -202,6 +229,7 @@ flagcats = {
"crop": "center-cropping (y/n/fy/fn)",
"th3x": "3x resolution (y/n/fy/fn)",
"convt": "conversion timeout in seconds",
"ext_th=s=/b.png": "use /b.png as thumbnail for file-extension s",
},
"handlers\n(better explained in --help-handlers)": {
"on404=PY": "handle 404s by executing PY file",
@@ -211,6 +239,8 @@ flagcats = {
"xbu=CMD": "execute CMD before a file upload starts",
"xau=CMD": "execute CMD after a file upload finishes",
"xiu=CMD": "execute CMD after all uploads finish and volume is idle",
"xbc=CMD": "execute CMD before a file copy",
"xac=CMD": "execute CMD after a file copy",
"xbr=CMD": "execute CMD before a file rename/move",
"xar=CMD": "execute CMD after a file rename/move",
"xbd=CMD": "execute CMD before a file delete",
@@ -222,8 +252,12 @@ flagcats = {
"grid": "show grid/thumbnails by default",
"gsel": "select files in grid by ctrl-click",
"sort": "default sort order",
"nsort": "natural-sort of leading digits in filenames",
"hsortn": "number of sort-rules to add to media URLs",
"unlist": "dont list files matching REGEX",
"html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
"tcolor=#fc0": "theme color (a hint for webbrowsers, discord, etc.)",
"nodirsz": "don't show total folder size",
"robots": "allows indexing by search engines (default)",
"norobots": "kindly asks search engines to leave",
"no_sb_md": "disable js sandbox for markdown files",
@@ -232,12 +266,40 @@ flagcats = {
"sb_lg": "enable js sandbox for prologue/epilogue (default)",
"md_sbf": "list of markdown-sandbox safeguards to disable",
"lg_sbf": "list of *logue-sandbox safeguards to disable",
"md_sba": "value of iframe allow-prop for markdown-sandbox",
"lg_sba": "value of iframe allow-prop for *logue-sandbox",
"nohtml": "return html and markdown as text/html",
},
"opengraph (discord embeds)": {
"og": "enable OG (disables hotlinking)",
"og_site": "sitename; defaults to --name, disable with '-'",
"og_desc": "description text for all files; disable with '-'",
"og_th=jf": "thumbnail format; j / jf / jf3 / w / w3 / ...",
"og_title_a": "audio title format; default: {{ artist }} - {{ title }}",
"og_title_v": "video title format; default: {{ title }}",
"og_title_i": "image title format; default: {{ title }}",
"og_title=foo": "fallback title if there's nothing in the db",
"og_s_title": "force default title; do not read from tags",
"og_tpl": "custom html; see --og-tpl in --help",
"og_no_head": "you want to add tags manually with og_tpl",
"og_ua": "if defined: only send OG html if useragent matches this regex",
},
"textfiles": {
"exp": "enable textfile expansion; see --help-exp",
"exp_md": "placeholders to expand in markdown files; see --help",
"exp_lg": "placeholders to expand in prologue/epilogue; see --help",
},
"others": {
"dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
"fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
"fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
"dk=8": 'generates per-directory accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
"dks": "per-directory accesskeys allow browsing into subdirs",
"dky": 'allow seeing files (not folders) inside a specific folder\nwith "g" perm, and does not require a valid dirkey to do so',
"rss": "allow '?rss' URL suffix (experimental)",
"ups_who=2": "restrict viewing the list of recent uploads",
"zip_who=2": "restrict access to download-as-zip/tar",
"nopipe": "disable race-the-beam (download unfinished uploads)",
"mv_retry": "ms-windows: timeout for renaming busy files",
"rm_retry": "ms-windows: timeout for deleting busy files",
"davauth": "ask webdav clients to login for all folders",
@@ -247,3 +309,10 @@ flagcats = {
flagdescs = {k.split("=")[0]: v for tab in flagcats.values() for k, v in tab.items()}
if True: # so it gets removed in release-builds
for fun in [vf_bmap, vf_cmap, vf_vmap]:
for k in fun().values():
if k not in flagdescs:
raise Exception("undocumented volflag: " + k)

View File

@@ -1,3 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import importlib
import sys
import xml.etree.ElementTree as ET
@@ -8,6 +11,10 @@ if True: # pylint: disable=using-constant-test
from typing import Any, Optional
class BadXML(Exception):
pass
def get_ET() -> ET.XMLParser:
pn = "xml.etree.ElementTree"
cn = "_elementtree"
@@ -34,7 +41,7 @@ def get_ET() -> ET.XMLParser:
XMLParser: ET.XMLParser = get_ET()
class DXMLParser(XMLParser): # type: ignore
class _DXMLParser(XMLParser): # type: ignore
def __init__(self) -> None:
tb = ET.TreeBuilder()
super(DXMLParser, self).__init__(target=tb)
@@ -49,8 +56,12 @@ class DXMLParser(XMLParser): # type: ignore
raise BadXML("{}, {}".format(a, ka))
class BadXML(Exception):
pass
class _NG(XMLParser): # type: ignore
def __int__(self) -> None:
raise BadXML("dxml selftest failed")
DXMLParser = _DXMLParser
def parse_xml(txt: str) -> ET.Element:
@@ -59,6 +70,40 @@ def parse_xml(txt: str) -> ET.Element:
return parser.close() # type: ignore
def selftest() -> bool:
qbe = r"""<!DOCTYPE d [
<!ENTITY a "nice_bakuretsu">
]>
<root>&a;&a;&a;</root>"""
emb = r"""<!DOCTYPE d [
<!ENTITY a SYSTEM "file:///etc/hostname">
]>
<root>&a;</root>"""
# future-proofing; there's never been any known vulns
# regarding DTDs and ET.XMLParser, but might as well
# block them since webdav-clients don't use them
dtd = r"""<!DOCTYPE d SYSTEM "a.dtd">
<root>a</root>"""
for txt in (qbe, emb, dtd):
try:
parse_xml(txt)
t = "WARNING: dxml selftest failed:\n%s\n"
print(t % (txt,), file=sys.stderr)
return False
except BadXML:
pass
return True
DXML_OK = selftest()
if not DXML_OK:
DXMLParser = _NG
def mktnod(name: str, text: str) -> ET.Element:
el = ET.Element(name)
el.text = text

View File

@@ -42,14 +42,14 @@ class Fstab(object):
self.cache = {}
fs = "ext4"
msg = "failed to determine filesystem at [{}]; assuming {}\n{}"
msg = "failed to determine filesystem at %r; assuming %s\n%s"
if ANYWIN:
fs = "vfat"
try:
path = self._winpath(path)
except:
self.log(msg.format(path, fs, min_ex()), 3)
self.log(msg % (path, fs, min_ex()), 3)
return fs
path = undot(path)
@@ -61,11 +61,11 @@ class Fstab(object):
try:
fs = self.get_w32(path) if ANYWIN else self.get_unix(path)
except:
self.log(msg.format(path, fs, min_ex()), 3)
self.log(msg % (path, fs, min_ex()), 3)
fs = fs.lower()
self.cache[path] = fs
self.log("found {} at {}".format(fs, path))
self.log("found %s at %r" % (fs, path))
return fs
def _winpath(self, path: str) -> str:

View File

@@ -296,6 +296,7 @@ class FtpFs(AbstractedFS):
self.uname,
not self.args.no_scandir,
[[True, False], [False, True]],
throw=True,
)
vfs_ls = [x[0] for x in vfs_ls1]
vfs_ls.extend(vfs_virt.keys())

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,7 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import hashlib
import math
import os
import re
@@ -80,6 +81,7 @@ from .util import (
)
if TYPE_CHECKING:
from .authsrv import VFS
from .broker_util import BrokerCli
from .ssdp import SSDPr
@@ -129,6 +131,12 @@ class HttpSrv(object):
self.bans: dict[str, int] = {}
self.aclose: dict[str, int] = {}
dli: dict[str, tuple[float, int, "VFS", str, str]] = {} # info
dls: dict[str, tuple[float, int]] = {} # state
self.dli = self.tdli = dli
self.dls = self.tdls = dls
self.iiam = '<img src="%s.cpr/iiam.gif?cache=i" />' % (self.args.SRS,)
self.bound: set[tuple[str, int]] = set()
self.name = "hsrv" + nsuf
self.mutex = threading.Lock()
@@ -144,6 +152,7 @@ class HttpSrv(object):
self.t_periodic: Optional[threading.Thread] = None
self.u2fh = FHC()
self.u2sc: dict[str, tuple[int, "hashlib._Hash"]] = {}
self.pipes = CachedDict(0.2)
self.metrics = Metrics(self)
self.nreq = 0
@@ -163,15 +172,16 @@ class HttpSrv(object):
env = jinja2.Environment()
env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
jn = [
"splash",
"shares",
"svcs",
"browser",
"browser2",
"msg",
"cf",
"md",
"mde",
"cf",
"msg",
"rups",
"shares",
"splash",
"svcs",
]
self.j2 = {x: env.get_template(x + ".html") for x in jn}
self.prism = has_resource(self.E, "web/deps/prism.js.gz")
@@ -185,10 +195,6 @@ class HttpSrv(object):
self.xff_nm = build_netmap(self.args.xff_src)
self.xff_lan = build_netmap("lan")
self.ptn_cc = re.compile(r"[\x00-\x1f]")
self.ptn_hsafe = re.compile(r"[\x00-\x1f<>\"'&]")
self.uparam_cc_ok = set("doc move tree".split())
self.mallow = "GET HEAD POST PUT DELETE OPTIONS".split()
if not self.args.no_dav:
zs = "PROPFIND PROPPATCH LOCK UNLOCK MKCOL COPY MOVE"
@@ -203,6 +209,9 @@ class HttpSrv(object):
self.start_threads(4)
if nid:
self.tdli = {}
self.tdls = {}
if self.args.stackmon:
start_stackmon(self.args.stackmon, nid)
@@ -577,3 +586,32 @@ class HttpSrv(object):
ident += "a"
self.u2idx_free[ident] = u2idx
def read_dls(
self,
) -> tuple[
dict[str, tuple[float, int, str, str, str]], dict[str, tuple[float, int]]
]:
"""
mp-broker asking for local dl-info + dl-state;
reduce overhead by sending just the vfs vpath
"""
dli = {k: (a, b, c.vpath, d, e) for k, (a, b, c, d, e) in self.dli.items()}
return (dli, self.dls)
def write_dls(
self,
sdli: dict[str, tuple[float, int, str, str, str]],
dls: dict[str, tuple[float, int]],
) -> None:
"""
mp-broker pushing total dl-info + dl-state;
swap out the vfs vpath with the vfs node
"""
dli: dict[str, tuple[float, int, "VFS", str, str]] = {}
for k, (a, b, c, d, e) in sdli.items():
vn = self.asrv.vfs.all_nodes[c]
dli[k] = (a, b, vn, d, e)
self.tdli = dli
self.tdls = dls

View File

@@ -25,6 +25,7 @@ from .stolen.dnslib import (
DNSHeader,
DNSQuestion,
DNSRecord,
set_avahi_379,
)
from .util import CachedSet, Daemon, Netdev, list_ips, min_ex
@@ -72,6 +73,9 @@ class MDNS(MCast):
self.ngen = ngen
self.ttl = 300
if not self.args.zm_nwa_1:
set_avahi_379()
zs = self.args.name + ".local."
zs = zs.encode("ascii", "replace").decode("ascii", "replace")
self.hn = "-".join(x for x in zs.split("?") if x) or (
@@ -336,6 +340,9 @@ class MDNS(MCast):
self.log("stopped", 2)
return
if self.args.zm_no_pe:
continue
t = "{} {} \033[33m|{}| {}\n{}".format(
self.srv[sck].name, addr, len(buf), repr(buf)[2:-1], min_ex()
)

View File

@@ -18,7 +18,7 @@ class Metrics(object):
def tx(self, cli: "HttpCli") -> bool:
if not cli.avol:
raise Pebkac(403, "not allowed for user " + cli.uname)
raise Pebkac(403, "'stats' not allowed for user " + cli.uname)
args = cli.args
if not args.stats:
@@ -72,6 +72,9 @@ class Metrics(object):
v = "{:.3f}".format(self.hsrv.t0)
addug("cpp_boot_unixtime", "seconds", v, t)
t = "number of active downloads"
addg("cpp_active_dl", str(len(self.hsrv.tdls)), t)
t = "number of open http(s) client connections"
addg("cpp_http_conns", str(self.hsrv.ncli), t)
@@ -128,7 +131,7 @@ class Metrics(object):
addbh("cpp_disk_size_bytes", "total HDD size of volume")
addbh("cpp_disk_free_bytes", "free HDD space in volume")
for vpath, vol in allvols:
free, total = get_df(vol.realpath)
free, total, _ = get_df(vol.realpath, False)
if free is None or total is None:
continue

View File

@@ -4,6 +4,7 @@ from __future__ import print_function, unicode_literals
import argparse
import json
import os
import re
import shutil
import subprocess as sp
import sys
@@ -62,6 +63,9 @@ def have_ff(scmd: str) -> bool:
HAVE_FFMPEG = not os.environ.get("PRTY_NO_FFMPEG") and have_ff("ffmpeg")
HAVE_FFPROBE = not os.environ.get("PRTY_NO_FFPROBE") and have_ff("ffprobe")
CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif".split())
CBZ_01 = re.compile(r"(^|[^0-9v])0+[01]\b")
class MParser(object):
def __init__(self, cmdline: str) -> None:
@@ -126,6 +130,7 @@ def au_unpk(
log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
) -> str:
ret = ""
maxsz = 1024 * 1024 * 64
try:
ext = abspath.split(".")[-1].lower()
au, pk = fmt_map[ext].split(".")
@@ -148,24 +153,48 @@ def au_unpk(
zf = zipfile.ZipFile(abspath, "r")
zil = zf.infolist()
zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
if not zil:
raise Exception("no audio inside zip")
fi = zf.open(zil[0])
elif pk == "cbz":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
znil = [(x.filename.lower(), x) for x in zf.infolist()]
nf = len(znil)
znil = [x for x in znil if x[0].split(".")[-1] in CBZ_PICS]
znil = [x for x in znil if "cover" in x[0]] or znil
znil = [x for x in znil if CBZ_01.search(x[0])] or znil
t = "cbz: %d files, %d hits" % (nf, len(znil))
if znil:
t += ", using " + znil[0][1].filename
log(t)
if not znil:
raise Exception("no images inside cbz")
fi = zf.open(znil[0][1])
else:
raise Exception("unknown compression %s" % (pk,))
fsz = 0
with os.fdopen(fd, "wb") as fo:
while True:
buf = fi.read(32768)
if not buf:
break
fsz += len(buf)
if fsz > maxsz:
raise Exception("zipbomb defused")
fo.write(buf)
return ret
except Exception as ex:
if ret:
t = "failed to decompress audio file [%s]: %r"
t = "failed to decompress audio file %r: %r"
log(t % (abspath, ex))
wunlink(log, ret, vn.flags if vn else VF_CAREFUL)
@@ -553,7 +582,7 @@ class MTag(object):
raise Exception()
except Exception as ex:
if self.args.mtag_v:
self.log("mutagen-err [{}] @ [{}]".format(ex, abspath), "90")
self.log("mutagen-err [%s] @ %r" % (ex, abspath), "90")
return self.get_ffprobe(abspath) if self.can_ffprobe else {}
@@ -670,8 +699,8 @@ class MTag(object):
ret[tag] = zj[tag]
except:
if self.args.mtag_v:
t = "mtag error: tagname {}, parser {}, file {} => {}"
self.log(t.format(tagname, parser.bin, abspath, min_ex()))
t = "mtag error: tagname %r, parser %r, file %r => %r"
self.log(t % (tagname, parser.bin, abspath, min_ex()), 6)
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)

View File

@@ -163,6 +163,7 @@ class MCast(object):
sck.settimeout(None)
sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
# safe for this purpose; https://lwn.net/Articles/853637/
sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
except:
pass

View File

@@ -24,17 +24,13 @@ class PWHash(object):
def __init__(self, args: argparse.Namespace):
self.args = args
try:
alg, ac = args.ah_alg.split(",")
except:
alg = args.ah_alg
ac = {}
zsl = args.ah_alg.split(",")
alg = zsl[0]
if alg == "none":
alg = ""
self.alg = alg
self.ac = ac
self.ac = zsl[1:]
if not alg:
self.on = False
self.hash = unicode
@@ -90,17 +86,23 @@ class PWHash(object):
its = 2
blksz = 8
para = 4
ramcap = 0 # openssl 1.1 = 32 MiB
try:
cost = 2 << int(self.ac[0])
its = int(self.ac[1])
blksz = int(self.ac[2])
para = int(self.ac[3])
ramcap = int(self.ac[4]) * 1024 * 1024
except:
pass
cfg = {"salt": self.salt, "n": cost, "r": blksz, "p": para, "dklen": 24}
if ramcap:
cfg["maxmem"] = ramcap
ret = plain.encode("utf-8")
for _ in range(its):
ret = hashlib.scrypt(ret, salt=self.salt, n=cost, r=blksz, p=para, dklen=24)
ret = hashlib.scrypt(ret, **cfg)
return "+" + base64.urlsafe_b64encode(ret).decode("utf-8")

View File

@@ -263,7 +263,7 @@ class SMB(object):
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
yeet("blocked by xbu server config: %r" % (vpath,))
ret = bos.open(ap, flags, *a, mode=chmod, **ka)
if wr:

View File

@@ -8,7 +8,7 @@ from itertools import chain
from .bimap import Bimap, BimapError
from .bit import get_bits, set_bits
from .buffer import BufferError
from .label import DNSBuffer, DNSLabel
from .label import DNSBuffer, DNSLabel, set_avahi_379
from .ranges import IP4, IP6, H, I, check_bytes
@@ -426,7 +426,7 @@ class RR(object):
if rdlength:
rdata = RDMAP.get(QTYPE.get(rtype), RD).parse(buffer, rdlength)
else:
rdata = ""
rdata = RD(b"a")
return cls(rname, rtype, rclass, ttl, rdata)
except (BufferError, BimapError) as e:
raise DNSError("Error unpacking RR [offset=%d]: %s" % (buffer.offset, e))

View File

@@ -11,6 +11,23 @@ LDH = set(range(33, 127))
ESCAPE = re.compile(r"\\([0-9][0-9][0-9])")
avahi_379 = 0
def set_avahi_379():
global avahi_379
avahi_379 = 1
def log_avahi_379(args):
global avahi_379
if avahi_379 == 2:
return
avahi_379 = 2
t = "Invalid pointer in DNSLabel [offset=%d,pointer=%d,length=%d];\n\033[35m NOTE: this is probably avahi-bug #379, packet corruption in Avahi's mDNS-reflection feature. Copyparty has a workaround and is OK, but other devices need either --zm4 or --zm6"
raise BufferError(t % args)
class DNSLabelError(Exception):
pass
@@ -96,8 +113,11 @@ class DNSBuffer(Buffer):
)
if pointer < self.offset:
self.offset = pointer
elif avahi_379:
log_avahi_379((self.offset, pointer, len(self.data)))
label.extend(b"a")
break
else:
raise BufferError(
"Invalid pointer in DNSLabel [offset=%d,pointer=%d,length=%d]"
% (self.offset, pointer, len(self.data))

View File

@@ -110,7 +110,7 @@ def errdesc(
report = ["copyparty failed to add the following files to the archive:", ""]
for fn, err in errors:
report.extend([" file: {}".format(fn), "error: {}".format(err), ""])
report.extend([" file: %r" % (fn,), "error: %s" % (err,), ""])
btxt = "\r\n".join(report).encode("utf-8", "replace")
btxt = vol_san(list(vfs.all_vols.values()), btxt)

View File

@@ -50,6 +50,8 @@ from .util import (
FFMPEG_URL,
HAVE_PSUTIL,
HAVE_SQLITE3,
HAVE_ZMQ,
URL_BUG,
UTC,
VERSIONS,
Daemon,
@@ -60,6 +62,7 @@ from .util import (
alltrace,
ansi_re,
build_netmap,
expat_ver,
load_ipu,
min_ex,
mp,
@@ -112,7 +115,7 @@ class SvcHub(object):
self.stopping = False
self.stopped = False
self.reload_req = False
self.reloading = 0
self.reload_mutex = threading.Lock()
self.stop_cond = threading.Condition()
self.nsigs = 3
self.retcode = 0
@@ -211,6 +214,15 @@ class SvcHub(object):
t = "WARNING: --s-rd-sz (%d) is larger than --iobuf (%d); this may lead to reduced performance"
self.log("root", t % (args.s_rd_sz, args.iobuf), 3)
zs = ""
if args.th_ram_max < 0.22:
zs = "generate thumbnails"
elif args.th_ram_max < 1:
zs = "generate audio waveforms or spectrograms"
if zs:
t = "WARNING: --th-ram-max is very small (%.2f GiB); will not be able to %s"
self.log("root", t % (args.th_ram_max, zs), 3)
if args.chpw and args.idp_h_usr:
t = "ERROR: user-changeable passwords is incompatible with IdP/identity-providers; you must disable either --chpw or --idp-h-usr"
self.log("root", t, 1)
@@ -223,13 +235,14 @@ class SvcHub(object):
args.chpw_no = noch
if args.ipu:
iu, nm = load_ipu(self.log, args.ipu)
iu, nm = load_ipu(self.log, args.ipu, True)
setattr(args, "ipu_iu", iu)
setattr(args, "ipu_nm", nm)
if not self.args.no_ses:
self.setup_session_db()
args.shr1 = ""
if args.shr:
self.setup_share_db()
@@ -378,6 +391,14 @@ class SvcHub(object):
self.broker = Broker(self)
# create netmaps early to avoid firewall gaps,
# but the mutex blocks multiprocessing startup
for zs in "ipu_iu ftp_ipa_nm tftp_ipa_nm".split():
try:
getattr(args, zs).mutex = threading.Lock()
except:
pass
def setup_session_db(self) -> None:
if not HAVE_SQLITE3:
self.args.no_ses = True
@@ -452,6 +473,7 @@ class SvcHub(object):
raise Exception(t)
al.shr = "/%s/" % (al.shr,)
al.shr1 = al.shr[1:]
create = True
modified = False
@@ -620,6 +642,7 @@ class SvcHub(object):
(HAVE_FFPROBE, "ffprobe", t_ff + ", read audio/media tags"),
(HAVE_MUTAGEN, "mutagen", "read audio tags (ffprobe is better but slower)"),
(HAVE_ARGON2, "argon2", "secure password hashing (advanced users only)"),
(HAVE_ZMQ, "pyzmq", "send zeromq messages from event-hooks"),
(HAVE_HEIF, "pillow-heif", "read .heif images with pillow (rarely useful)"),
(HAVE_AVIF, "pillow-avif", "read .avif images with pillow (rarely useful)"),
]
@@ -676,6 +699,15 @@ class SvcHub(object):
if self.args.bauth_last:
self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
if not self.args.no_dav:
from .dxml import DXML_OK
if not DXML_OK:
if not self.args.no_dav:
self.args.no_dav = True
t = "WARNING:\nDisabling WebDAV support because dxml selftest failed. Please report this bug;\n%s\n...and include the following information in the bug-report:\n%s | expat %s\n"
self.log("root", t % (URL_BUG, VERSIONS, expat_ver()), 1)
def _process_config(self) -> bool:
al = self.args
@@ -737,7 +769,7 @@ class SvcHub(object):
vs = os.path.expandvars(os.path.expanduser(vs))
setattr(al, k, vs)
for k in "sus_urls nonsus_urls".split(" "):
for k in "dav_ua1 sus_urls nonsus_urls".split(" "):
vs = getattr(al, k)
if not vs or vs == "no":
setattr(al, k, None)
@@ -761,8 +793,8 @@ class SvcHub(object):
al.idp_h_grp = al.idp_h_grp.lower()
al.idp_h_key = al.idp_h_key.lower()
al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa)
al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa)
al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa, True)
al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa, True)
mte = ODict.fromkeys(DEF_MTE.split(","), True)
al.mte = odfusion(mte, al.mte)
@@ -774,7 +806,7 @@ class SvcHub(object):
al.exp_md = odfusion(exp, al.exp_md.replace(" ", ","))
al.exp_lg = odfusion(exp, al.exp_lg.replace(" ", ","))
for k in ["no_hash", "no_idx", "og_ua"]:
for k in ["no_hash", "no_idx", "og_ua", "srch_excl"]:
ptn = getattr(self.args, k)
if ptn:
setattr(self.args, k, re.compile(ptn))
@@ -809,6 +841,24 @@ class SvcHub(object):
if len(al.tcolor) == 3: # fc5 => ffcc55
al.tcolor = "".join([x * 2 for x in al.tcolor])
zs = al.u2sz
zsl = zs.split(",")
if len(zsl) not in (1, 3):
t = "invalid --u2sz; must be either one number, or a comma-separated list of three numbers (min,default,max)"
raise Exception(t)
if len(zsl) < 3:
zsl = ["1", zs, zs]
zi2 = 1
for zs in zsl:
zi = int(zs)
# arbitrary constraint (anything above 2 GiB is probably unintended)
if zi < 1 or zi > 2047:
raise Exception("invalid --u2sz; minimum is 1, max is 2047")
if zi < zi2:
raise Exception("invalid --u2sz; values must be equal or ascending")
zi2 = zi
al.u2sz = ",".join(zsl)
return True
def _ipa2re(self, txt) -> Optional[re.Pattern]:
@@ -976,41 +1026,18 @@ class SvcHub(object):
except:
self.log("root", "ssdp startup failed;\n" + min_ex(), 3)
def reload(self) -> str:
with self.up2k.mutex:
if self.reloading:
return "cannot reload; already in progress"
self.reloading = 1
Daemon(self._reload, "reloading")
return "reload initiated"
def _reload(self, rescan_all_vols: bool = True, up2k: bool = True) -> None:
with self.up2k.mutex:
if self.reloading != 1:
return
self.reloading = 2
def reload(self, rescan_all_vols: bool, up2k: bool) -> str:
t = "config has been reloaded"
with self.reload_mutex:
self.log("root", "reloading config")
self.asrv.reload(9 if up2k else 4)
if up2k:
self.up2k.reload(rescan_all_vols)
t += "; volumes are now reinitializing"
else:
self.log("root", "reload done")
self.broker.reload()
self.reloading = 0
def _reload_blocking(self, rescan_all_vols: bool = True, up2k: bool = True) -> None:
while True:
with self.up2k.mutex:
if self.reloading < 2:
self.reloading = 1
break
time.sleep(0.05)
# try to handle multiple pending IdP reloads at once:
time.sleep(0.2)
self._reload(rescan_all_vols=rescan_all_vols, up2k=up2k)
return t
def _reload_sessions(self) -> None:
with self.asrv.mutex:
@@ -1024,7 +1051,7 @@ class SvcHub(object):
if self.reload_req:
self.reload_req = False
self.reload()
self.reload(True, True)
self.shutdown()

View File

@@ -100,7 +100,7 @@ def gen_hdr(
# spec says to put zeros when !crc if bit3 (streaming)
# however infozip does actual sz and it even works on winxp
# (same reasning for z64 extradata later)
# (same reasoning for z64 extradata later)
vsz = 0xFFFFFFFF if z64 else sz
ret += spack(b"<LL", vsz, vsz)

View File

@@ -371,7 +371,7 @@ class TcpSrv(object):
if self.args.q:
print(msg)
self.hub.broker.say("listen", srv)
self.hub.broker.say("httpsrv.listen", srv)
self.srv = srvs
self.bound = bound
@@ -379,7 +379,7 @@ class TcpSrv(object):
self._distribute_netdevs()
def _distribute_netdevs(self):
self.hub.broker.say("set_netdevs", self.netdevs)
self.hub.broker.say("httpsrv.set_netdevs", self.netdevs)
self.hub.start_zeroconf()
gencert(self.log, self.args, self.netdevs)
self.hub.restart_ftpd()
@@ -402,17 +402,17 @@ class TcpSrv(object):
if not netdevs:
continue
added = "nothing"
removed = "nothing"
add = []
rem = []
for k, v in netdevs.items():
if k not in self.netdevs:
added = "{} = {}".format(k, v)
add.append("\n\033[32m added %s = %s" % (k, v))
for k, v in self.netdevs.items():
if k not in netdevs:
removed = "{} = {}".format(k, v)
rem.append("\n\033[33mremoved %s = %s" % (k, v))
t = "network change detected:\n added {}\033[0;33m\nremoved {}"
self.log("tcpsrv", t.format(added, removed), 3)
t = "network change detected:%s%s"
self.log("tcpsrv", t % ("".join(add), "".join(rem)), 3)
self.netdevs = netdevs
self._distribute_netdevs()

View File

@@ -269,6 +269,7 @@ class Tftpd(object):
"*",
not self.args.no_scandir,
[[True, False]],
throw=True,
)
dnames = set([x[0] for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)])
dirs1 = [(v.st_mtime, v.st_size, k + "/") for k, v in vfs_ls if k in dnames]
@@ -356,7 +357,7 @@ class Tftpd(object):
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
yeet("blocked by xbu server config: %r" % (vpath,))
if not self.args.tftp_nols and bos.path.isdir(ap):
return self._ls(vpath, "", 0, True)

View File

@@ -6,7 +6,7 @@ import os
from .__init__ import TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .th_srv import HAVE_WEBP, thumb_path
from .th_srv import EXTS_AC, HAVE_WEBP, thumb_path
from .util import Cooldown
if True: # pylint: disable=using-constant-test
@@ -57,13 +57,17 @@ class ThumbCli(object):
if is_vid and "dvthumb" in dbv.flags:
return None
want_opus = fmt in ("opus", "caf", "mp3")
want_opus = fmt in EXTS_AC
is_au = ext in self.fmt_ffa
is_vau = want_opus and ext in self.fmt_ffv
if is_au or is_vau:
if want_opus:
if self.args.no_acode:
return None
elif fmt == "caf" and self.args.no_caf:
fmt = "mp3"
elif fmt == "owa" and self.args.no_owa:
fmt = "mp3"
else:
if "dathumb" in dbv.flags:
return None
@@ -109,13 +113,13 @@ class ThumbCli(object):
fmt = sfmt
elif fmt[:1] == "p" and not is_au and not is_vid:
t = "cannot thumbnail [%s]: png only allowed for waveforms"
self.log(t % (rem), 6)
t = "cannot thumbnail %r: png only allowed for waveforms"
self.log(t % (rem,), 6)
return None
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)

View File

@@ -20,7 +20,6 @@ from .util import (
FFMPEG_URL,
Cooldown,
Daemon,
Pebkac,
afsenc,
fsenc,
min_ex,
@@ -33,7 +32,7 @@ from .util import (
)
if True: # pylint: disable=using-constant-test
from typing import Optional, Union
from typing import Any, Optional, Union
if TYPE_CHECKING:
from .svchub import SvcHub
@@ -47,6 +46,9 @@ HAVE_HEIF = False
HAVE_AVIF = False
HAVE_WEBP = False
EXTS_TH = set(["jpg", "webp", "png"])
EXTS_AC = set(["opus", "owa", "caf", "mp3"])
try:
if os.environ.get("PRTY_NO_PIL"):
raise Exception()
@@ -140,7 +142,7 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
h = hashlib.sha512(afsenc(fn)).digest()
fn = ub64enc(h).decode("ascii")[:24]
if fmt in ("opus", "caf", "mp3"):
if fmt in EXTS_AC:
cat = "ac"
else:
fc = fmt[:1]
@@ -164,6 +166,7 @@ class ThumbSrv(object):
self.ram: dict[str, float] = {}
self.memcond = threading.Condition(self.mutex)
self.stopping = False
self.rm_nullthumbs = True # forget failed conversions on startup
self.nthr = max(1, self.args.th_mt)
self.q: Queue[Optional[tuple[str, str, str, VFS]]] = Queue(self.nthr * 4)
@@ -239,7 +242,7 @@ class ThumbSrv(object):
def get(self, ptop: str, rem: str, mtime: float, fmt: str) -> Optional[str]:
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
@@ -249,7 +252,7 @@ class ThumbSrv(object):
with self.mutex:
try:
self.busy[tpath].append(cond)
self.log("joined waiting room for %s" % (tpath,))
self.log("joined waiting room for %r" % (tpath,))
except:
thdir = os.path.dirname(tpath)
bos.makedirs(os.path.join(thdir, "w"))
@@ -266,11 +269,11 @@ class ThumbSrv(object):
allvols = list(self.asrv.vfs.all_vols.values())
vn = next((x for x in allvols if x.realpath == ptop), None)
if not vn:
self.log("ptop [{}] not in {}".format(ptop, allvols), 3)
self.log("ptop %r not in %s" % (ptop, allvols), 3)
vn = self.asrv.vfs.all_aps[0][1]
self.q.put((abspath, tpath, fmt, vn))
self.log("conv {} :{} \033[0m{}".format(tpath, fmt, abspath), c=6)
self.log("conv %r :%s \033[0m%r" % (tpath, fmt, abspath), 6)
while not self.stopping:
with self.mutex:
@@ -334,9 +337,10 @@ class ThumbSrv(object):
ap_unpk = abspath
if not bos.path.exists(tpath):
want_mp3 = tpath.endswith(".mp3")
want_opus = tpath.endswith(".opus") or tpath.endswith(".caf")
want_png = tpath.endswith(".png")
tex = tpath.rsplit(".", 1)[-1]
want_mp3 = tex == "mp3"
want_opus = tex in ("opus", "owa", "caf")
want_png = tex == "png"
want_au = want_mp3 or want_opus
for lib in self.args.th_dec:
can_au = lib == "ff" and (
@@ -375,8 +379,8 @@ class ThumbSrv(object):
fun(ap_unpk, ttpath, fmt, vn)
break
except Exception as ex:
msg = "{} could not create thumbnail of {}\n{}"
msg = msg.format(fun.__name__, abspath, min_ex())
msg = "%s could not create thumbnail of %r\n%s"
msg = msg % (fun.__name__, abspath, min_ex())
c: Union[str, int] = 1 if "<Signals.SIG" in msg else "90"
self.log(msg, c)
if getattr(ex, "returncode", 0) != 321:
@@ -754,47 +758,102 @@ class ThumbSrv(object):
if "ac" not in tags:
raise Exception("not audio")
sq = "%dk" % (self.args.q_opus,)
bq = sq.encode("ascii")
if tags["ac"][1] == "opus":
enc = "-c:a copy"
else:
enc = "-c:a libopus -b:a " + sq
fun = self._conv_caf if fmt == "caf" else self._conv_owa
fun(abspath, tpath, tags, rawtags, enc, bq, vn)
def _conv_owa(
self,
abspath: str,
tpath: str,
tags: dict[str, tuple[int, Any]],
rawtags: dict[str, list[Any]],
enc: str,
bq: bytes,
vn: VFS,
) -> None:
if tpath.endswith(".owa"):
container = b"webm"
tagset = [b"-map_metadata", b"-1"]
else:
container = b"opus"
tagset = self.big_tags(rawtags)
self.log("conv2 %s [%s]" % (container, enc), 6)
benc = enc.encode("ascii").split(b" ")
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
] + tagset + [
b"-map", b"0:a:0",
] + benc + [
b"-f", container,
fsenc(tpath)
]
# fmt: on
self._run_ff(cmd, vn, oom=300)
def _conv_caf(
self,
abspath: str,
tpath: str,
tags: dict[str, tuple[int, Any]],
rawtags: dict[str, list[Any]],
enc: str,
bq: bytes,
vn: VFS,
) -> None:
tmp_opus = tpath + ".opus"
try:
wunlink(self.log, tmp_opus, vn.flags)
except:
pass
try:
dur = tags[".dur"][1]
except:
dur = 0
src_opus = abspath.lower().endswith(".opus") or tags["ac"][1] == "opus"
want_caf = tpath.endswith(".caf")
tmp_opus = tpath
if want_caf:
tmp_opus = tpath + ".opus"
try:
wunlink(self.log, tmp_opus, vn.flags)
except:
pass
self.log("conv2 caf-tmp [%s]" % (enc,), 6)
benc = enc.encode("ascii").split(b" ")
caf_src = abspath if src_opus else tmp_opus
bq = ("%dk" % (self.args.q_opus,)).encode("ascii")
if not want_caf or not src_opus:
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
] + self.big_tags(rawtags) + [
b"-map", b"0:a:0",
b"-c:a", b"libopus",
b"-b:a", bq,
fsenc(tmp_opus)
]
# fmt: on
self._run_ff(cmd, vn, oom=300)
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
b"-map_metadata", b"-1",
b"-map", b"0:a:0",
] + benc + [
b"-f", b"opus",
fsenc(tmp_opus)
]
# fmt: on
self._run_ff(cmd, vn, oom=300)
# iOS fails to play some "insufficiently complex" files
# (average file shorter than 8 seconds), so of course we
# fix that by mixing in some inaudible pink noise :^)
# 6.3 sec seems like the cutoff so lets do 7, and
# 7 sec of psyqui-musou.opus @ 3:50 is 174 KiB
if want_caf and (dur < 20 or bos.path.getsize(caf_src) < 256 * 1024):
sz = bos.path.getsize(tmp_opus)
if dur < 20 or sz < 256 * 1024:
zs = bq.decode("ascii")
self.log("conv2 caf-transcode; dur=%d sz=%d q=%s" % (dur, sz, zs), 6)
# fmt: off
cmd = [
b"ffmpeg",
@@ -813,15 +872,16 @@ class ThumbSrv(object):
# fmt: on
self._run_ff(cmd, vn, oom=300)
elif want_caf:
else:
# simple remux should be safe
self.log("conv2 caf-remux; dur=%d sz=%d" % (dur, sz), 6)
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath if src_opus else tmp_opus),
b"-i", fsenc(tmp_opus),
b"-map_metadata", b"-1",
b"-map", b"0:a:0",
b"-c:a", b"copy",
@@ -831,11 +891,10 @@ class ThumbSrv(object):
# fmt: on
self._run_ff(cmd, vn, oom=300)
if tmp_opus != tpath:
try:
wunlink(self.log, tmp_opus, vn.flags)
except:
pass
try:
wunlink(self.log, tmp_opus, vn.flags)
except:
pass
def big_tags(self, raw_tags: dict[str, list[str]]) -> list[bytes]:
ret = []
@@ -862,7 +921,6 @@ class ThumbSrv(object):
def cleaner(self) -> None:
interval = self.args.th_clean
while True:
time.sleep(interval)
ndirs = 0
for vol, histpath in self.asrv.vfs.histtab.items():
if histpath.startswith(vol):
@@ -876,6 +934,8 @@ class ThumbSrv(object):
self.log("\033[Jcln err in %s: %r" % (histpath, ex), 3)
self.log("\033[Jcln ok; rm {} dirs".format(ndirs))
self.rm_nullthumbs = False
time.sleep(interval)
def clean(self, histpath: str) -> int:
ret = 0
@@ -890,13 +950,15 @@ class ThumbSrv(object):
def _clean(self, cat: str, thumbpath: str) -> int:
# self.log("cln {}".format(thumbpath))
exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf", "mp3"]
exts = EXTS_TH if cat == "th" else EXTS_AC
maxage = getattr(self.args, cat + "_maxage")
now = time.time()
prev_b64 = None
prev_fp = ""
try:
t1 = statdir(self.log_func, not self.args.no_scandir, False, thumbpath)
t1 = statdir(
self.log_func, not self.args.no_scandir, False, thumbpath, False
)
ents = sorted(list(t1))
except:
return 0
@@ -937,6 +999,10 @@ class ThumbSrv(object):
continue
if self.rm_nullthumbs and not inf.st_size:
bos.unlink(fp)
continue
if b64 == prev_b64:
self.log("rm replaced [{}]".format(fp))
bos.unlink(prev_fp)

View File

@@ -70,6 +70,9 @@ class U2idx(object):
self.log_func("u2idx", msg, c)
def shutdown(self) -> None:
if not HAVE_SQLITE3:
return
for cur in self.cur.values():
db = cur.connection
try:
@@ -80,6 +83,12 @@ class U2idx(object):
cur.close()
db.close()
for cur in (self.mem_cur, self.sh_cur):
if cur:
db = cur.connection
cur.close()
db.close()
def fsearch(
self, uname: str, vols: list[VFS], body: dict[str, Any]
) -> list[dict[str, Any]]:
@@ -95,7 +104,7 @@ class U2idx(object):
uv: list[Union[str, int]] = [wark[:16], wark]
try:
return self.run_query(uname, vols, uq, uv, False, 99999)[0]
return self.run_query(uname, vols, uq, uv, False, True, 99999)[0]
except:
raise Pebkac(500, min_ex())
@@ -127,7 +136,7 @@ class U2idx(object):
ptop = vn.realpath
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
db_path = os.path.join(histpath, "up2k.db")
@@ -142,7 +151,7 @@ class U2idx(object):
db = sqlite3.connect(uri, timeout=2, uri=True, check_same_thread=False)
cur = db.cursor()
cur.execute('pragma table_info("up")').fetchone()
self.log("ro: {}".format(db_path))
self.log("ro: %r" % (db_path,))
except:
self.log("could not open read-only: {}\n{}".format(uri, min_ex()))
# may not fail until the pragma so unset it
@@ -152,7 +161,7 @@ class U2idx(object):
# on windows, this steals the write-lock from up2k.deferred_init --
# seen on win 10.0.17763.2686, py 3.10.4, sqlite 3.37.2
cur = sqlite3.connect(db_path, timeout=2, check_same_thread=False).cursor()
self.log("opened {}".format(db_path))
self.log("opened %r" % (db_path,))
self.cur[ptop] = cur
return cur
@@ -301,7 +310,7 @@ class U2idx(object):
q += " lower({}) {} ? ) ".format(field, oper)
try:
return self.run_query(uname, vols, q, va, have_mt, lim)
return self.run_query(uname, vols, q, va, have_mt, True, lim)
except Exception as ex:
raise Pebkac(500, repr(ex))
@@ -312,9 +321,11 @@ class U2idx(object):
uq: str,
uv: list[Union[str, int]],
have_mt: bool,
sort: bool,
lim: int,
) -> tuple[list[dict[str, Any]], list[str], bool]:
if self.args.srch_dbg:
dbg = self.args.srch_dbg
if dbg:
t = "searching across all %s volumes in which the user has 'r' (full read access):\n %s"
zs = "\n ".join(["/%s = %s" % (x.vpath, x.realpath) for x in vols])
self.log(t % (len(vols), zs), 5)
@@ -357,14 +368,14 @@ class U2idx(object):
if not cur:
continue
excl = []
for vp2 in self.asrv.vfs.all_vols.keys():
if vp2.startswith((vtop + "/").lstrip("/")) and vtop != vp2:
excl.append(vp2[len(vtop) :].lstrip("/"))
dots = flags.get("dotsrch") and uname in vol.axs.udot
zs = "srch_re_dots" if dots else "srch_re_nodot"
rex: re.Pattern = flags.get(zs) # type: ignore
if self.args.srch_dbg:
t = "searching in volume /%s (%s), excludelist %s"
self.log(t % (vtop, ptop, excl), 5)
if dbg:
t = "searching in volume /%s (%s), excluding %s"
self.log(t % (vtop, ptop, rex.pattern), 5)
rex_cfg: Optional[re.Pattern] = flags.get("srch_excl")
self.active_cur = cur
@@ -377,7 +388,6 @@ class U2idx(object):
sret = []
fk = flags.get("fk")
dots = flags.get("dotsrch") and uname in vol.axs.udot
fk_alg = 2 if "fka" in flags else 1
c = cur.execute(uq, tuple(vuv))
for hit in c:
@@ -386,20 +396,23 @@ class U2idx(object):
if rd.startswith("//") or fn.startswith("//"):
rd, fn = s3dec(rd, fn)
if rd in excl or any([x for x in excl if rd.startswith(x + "/")]):
if self.args.srch_dbg:
zs = vjoin(vjoin(vtop, rd), fn)
t = "database inconsistency in volume '/%s'; ignoring: %s"
self.log(t % (vtop, zs), 1)
vp = vjoin(vjoin(vtop, rd), fn)
if vp in seen_rps:
continue
rp = quotep("/".join([x for x in [vtop, rd, fn] if x]))
if not dots and "/." in ("/" + rp):
continue
if rp in seen_rps:
if rex.search(vp):
if dbg:
if rex_cfg and rex_cfg.search(vp): # type: ignore
self.log("filtered by srch_excl: %s" % (vp,), 6)
elif not dots and "/." in ("/" + vp):
pass
else:
t = "database inconsistency in volume '/%s'; ignoring: %s"
self.log(t % (vtop, vp), 1)
continue
rp = quotep(vp)
if not fk:
suf = ""
else:
@@ -421,7 +434,7 @@ class U2idx(object):
if lim < 0:
break
if self.args.srch_dbg:
if dbg:
t = "in volume '/%s': hit: %s"
self.log(t % (vtop, rp), 5)
@@ -451,14 +464,15 @@ class U2idx(object):
ret.extend(sret)
# print("[{}] {}".format(ptop, sret))
if self.args.srch_dbg:
if dbg:
t = "in volume '/%s': got %d hits, %d total so far"
self.log(t % (vtop, len(sret), len(ret)), 5)
done_flag.append(True)
self.active_id = ""
ret.sort(key=itemgetter("rp"))
if sort:
ret.sort(key=itemgetter("rp"))
return ret, list(taglist.keys()), lim < 0 and not clamped

File diff suppressed because it is too large Load Diff

View File

@@ -120,6 +120,13 @@ try:
except:
HAVE_SQLITE3 = False
try:
import importlib.util
HAVE_ZMQ = bool(importlib.util.find_spec("zmq"))
except:
HAVE_ZMQ = False
try:
if os.environ.get("PRTY_NO_PSUTIL"):
raise Exception()
@@ -213,6 +220,9 @@ except:
ansi_re = re.compile("\033\\[[^mK]*[mK]")
BOS_SEP = ("%s" % (os.sep,)).encode("ascii")
surrogateescape.register_surrogateescape()
if WINDOWS and PY2:
FS_ENCODING = "utf-8"
@@ -226,9 +236,14 @@ META_NOBOTS = '<meta name="robots" content="noindex, nofollow">\n'
FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"
URL_PRJ = "https://github.com/9001/copyparty"
URL_BUG = URL_PRJ + "/issues/new?labels=bug&template=bug_report.md"
HTTPCODE = {
200: "OK",
201: "Created",
202: "Accepted",
204: "No Content",
206: "Partial Content",
207: "Multi-Status",
@@ -316,6 +331,7 @@ DAV_ALLPROPS = set(DAV_ALLPROP_L)
MIMES = {
"opus": "audio/ogg; codecs=opus",
"owa": "audio/webm; codecs=opus",
}
@@ -433,6 +449,27 @@ UNHUMANIZE_UNITS = {
VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
def read_ram() -> tuple[float, float]:
a = b = 0
try:
with open("/proc/meminfo", "rb", 0x10000) as f:
zsl = f.read(0x10000).decode("ascii", "replace").split("\n")
p = re.compile("^MemTotal:.* kB")
zs = next((x for x in zsl if p.match(x)))
a = int((int(zs.split()[1]) / 0x100000) * 100) / 100
p = re.compile("^MemAvailable:.* kB")
zs = next((x for x in zsl if p.match(x)))
b = int((int(zs.split()[1]) / 0x100000) * 100) / 100
except:
pass
return a, b
RAM_TOTAL, RAM_AVAIL = read_ram()
pybin = sys.executable or ""
if EXE:
pybin = ""
@@ -467,6 +504,15 @@ def py_desc() -> str:
)
def expat_ver() -> str:
try:
import pyexpat
return ".".join([str(x) for x in pyexpat.version_info])
except:
return "?"
def _sqlite_ver() -> str:
assert sqlite3 # type: ignore # !rm
try:
@@ -666,13 +712,20 @@ class HLog(logging.Handler):
class NetMap(object):
def __init__(
self, ips: list[str], cidrs: list[str], keep_lo=False, strict_cidr=False
self,
ips: list[str],
cidrs: list[str],
keep_lo=False,
strict_cidr=False,
defer_mutex=False,
) -> None:
"""
ips: list of plain ipv4/ipv6 IPs, not cidr
cidrs: list of cidr-notation IPs (ip/prefix)
"""
self.mutex = threading.Lock()
# fails multiprocessing; defer assignment
self.mutex: Optional[threading.Lock] = None if defer_mutex else threading.Lock()
if "::" in ips:
ips = [x for x in ips if x != "::"] + list(
@@ -711,6 +764,9 @@ class NetMap(object):
try:
return self.cache[ip]
except:
# intentionally crash the calling thread if unset:
assert self.mutex # type: ignore # !rm
with self.mutex:
return self._map(ip)
@@ -990,7 +1046,7 @@ class ProgressPrinter(threading.Thread):
now = time.time()
if msg and now - tp > 10:
tp = now
self.log("progress: %s" % (msg,), 6)
self.log("progress: %r" % (msg,), 6)
if no_stdout:
continue
@@ -1017,6 +1073,7 @@ class MTHash(object):
self.sz = 0
self.csz = 0
self.stop = False
self.readsz = 1024 * 1024 * (2 if (RAM_AVAIL or 2) < 1 else 12)
self.omutex = threading.Lock()
self.imutex = threading.Lock()
self.work_q: Queue[int] = Queue()
@@ -1092,7 +1149,7 @@ class MTHash(object):
while chunk_rem > 0:
with self.imutex:
f.seek(ofs)
buf = f.read(min(chunk_rem, 1024 * 1024 * 12))
buf = f.read(min(chunk_rem, self.readsz))
if not buf:
raise Exception("EOF at " + str(ofs))
@@ -1591,7 +1648,7 @@ class MultipartParser(object):
(only the fallback non-js uploader relies on these filenames)
"""
for ln in read_header(self.sr, 2, 2592000):
self.log(ln)
self.log(repr(ln))
m = self.re_ctype.match(ln)
if m:
@@ -1882,11 +1939,11 @@ def gen_filekey_dbg(
if p2 != fspath:
raise Exception()
except:
t = "maybe wrong abspath for filekey;\norig: {}\nreal: {}"
log(t.format(fspath, p2), 1)
t = "maybe wrong abspath for filekey;\norig: %r\nreal: %r"
log(t % (fspath, p2), 1)
t = "fk({}) salt({}) size({}) inode({}) fspath({}) at({})"
log(t.format(ret[:8], salt, fsize, inode, fspath, ctx), 5)
t = "fk(%s) salt(%s) size(%d) inode(%d) fspath(%r) at(%s)"
log(t % (ret[:8], salt, fsize, inode, fspath, ctx), 5)
return ret
@@ -2191,6 +2248,23 @@ def unquotep(txt: str) -> str:
return w8dec(unq2)
def vroots(vp1: str, vp2: str) -> tuple[str, str]:
"""
input("q/w/e/r","a/s/d/e/r") output("/q/w/","/a/s/d/")
"""
while vp1 and vp2:
zt1 = vp1.rsplit("/", 1) if "/" in vp1 else ("", vp1)
zt2 = vp2.rsplit("/", 1) if "/" in vp2 else ("", vp2)
if zt1[1] != zt2[1]:
break
vp1 = zt1[0]
vp2 = zt2[0]
return (
"/%s/" % (vp1,) if vp1 else "/",
"/%s/" % (vp2,) if vp2 else "/",
)
def vsplit(vpath: str) -> tuple[str, str]:
if "/" not in vpath:
return "", vpath
@@ -2225,7 +2299,7 @@ def log_reloc(
rem: str,
) -> None:
nap, nvp, nfn, (nvn, nrem) = pm
t = "reloc %s:\nold ap [%s]\nnew ap [%s\033[36m/%s\033[0m]\nold vp [%s]\nnew vp [%s\033[36m/%s\033[0m]\nold fn [%s]\nnew fn [%s]\nold vfs [%s]\nnew vfs [%s]\nold rem [%s]\nnew rem [%s]"
t = "reloc %s:\nold ap %r\nnew ap %r\033[36m/%r\033[0m\nold vp %r\nnew vp %r\033[36m/%r\033[0m\nold fn %r\nnew fn %r\nold vfs %r\nnew vfs %r\nold rem %r\nnew rem %r"
log(t % (re, ap, nap, nfn, vp, nvp, nfn, fn, nfn, vn.vpath, nvn.vpath, rem, nrem))
@@ -2396,7 +2470,7 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
try:
rc, so, se = runcmd([b"lsof", b"-R", fsenc(abspath)], timeout=45)
zs = (so.strip() + "\n" + se.strip()).strip()
log("lsof {} = {}\n{}".format(abspath, rc, zs), 3)
log("lsof %r = %s\n%s" % (abspath, rc, zs), 3)
except:
log("lsof failed; " + min_ex(), 3)
@@ -2432,17 +2506,17 @@ def _fs_mvrm(
for attempt in range(90210):
try:
if ino and os.stat(bsrc).st_ino != ino:
t = "src inode changed; aborting %s %s"
t = "src inode changed; aborting %s %r"
log(t % (act, src), 1)
return False
if (dst and not atomic) and os.path.exists(bdst):
t = "something appeared at dst; aborting rename [%s] ==> [%s]"
t = "something appeared at dst; aborting rename %r ==> %r"
log(t % (src, dst), 1)
return False
osfun(*args)
if attempt:
now = time.time()
t = "%sd in %.2f sec, attempt %d: %s"
t = "%sd in %.2f sec, attempt %d: %r"
log(t % (act, now - t0, attempt + 1, src))
return True
except OSError as ex:
@@ -2454,7 +2528,7 @@ def _fs_mvrm(
if not attempt:
if not PY2:
ino = os.stat(bsrc).st_ino
t = "%s failed (err.%d); retrying for %d sec: [%s]"
t = "%s failed (err.%d); retrying for %d sec: %r"
log(t % (act, ex.errno, maxtime + 0.99, src))
time.sleep(chill)
@@ -2492,23 +2566,28 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
return _fs_mvrm(log, abspath, "", False, flags)
def get_df(abspath: str) -> tuple[Optional[int], Optional[int]]:
def get_df(abspath: str, prune: bool) -> tuple[Optional[int], Optional[int], str]:
try:
# some fuses misbehave
assert ctypes # type: ignore # !rm
ap = fsenc(abspath)
while prune and not os.path.isdir(ap) and BOS_SEP in ap:
# strip leafs until it hits an existing folder
ap = ap.rsplit(BOS_SEP, 1)[0]
if ANYWIN:
assert ctypes # type: ignore # !rm
abspath = fsdec(ap)
bfree = ctypes.c_ulonglong(0)
ctypes.windll.kernel32.GetDiskFreeSpaceExW( # type: ignore
ctypes.c_wchar_p(abspath), None, None, ctypes.pointer(bfree)
)
return (bfree.value, None)
return (bfree.value, None, "")
else:
sv = os.statvfs(fsenc(abspath))
sv = os.statvfs(ap)
free = sv.f_frsize * sv.f_bfree
total = sv.f_frsize * sv.f_blocks
return (free, total)
except:
return (None, None)
return (free, total, "")
except Exception as ex:
return (None, None, repr(ex))
if not ANYWIN and not MACOS:
@@ -2646,18 +2725,35 @@ def list_ips() -> list[str]:
return list(ret)
def build_netmap(csv: str):
def build_netmap(csv: str, defer_mutex: bool = False):
csv = csv.lower().strip()
if csv in ("any", "all", "no", ",", ""):
return None
if csv in ("lan", "local", "private", "prvt"):
csv = "10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fd00::/8" # lan
csv += ", 169.254.0.0/16, fe80::/10" # link-local
csv += ", 127.0.0.0/8, ::1/128" # loopback
srcs = [x.strip() for x in csv.split(",") if x.strip()]
expanded_shorthands = False
for shorthand in ("lan", "local", "private", "prvt"):
if shorthand in srcs:
if not expanded_shorthands:
srcs += [
# lan:
"10.0.0.0/8",
"172.16.0.0/12",
"192.168.0.0/16",
"fd00::/8",
# link-local:
"169.254.0.0/16",
"fe80::/10",
# loopback:
"127.0.0.0/8",
"::1/128",
]
expanded_shorthands = True
srcs.remove(shorthand)
if not HAVE_IPV6:
srcs = [x for x in srcs if ":" not in x]
@@ -2681,10 +2777,12 @@ def build_netmap(csv: str):
cidrs.append(zs)
ips = [x.split("/")[0] for x in cidrs]
return NetMap(ips, cidrs, True)
return NetMap(ips, cidrs, True, False, defer_mutex)
def load_ipu(log: "RootLogger", ipus: list[str]) -> tuple[dict[str, str], NetMap]:
def load_ipu(
log: "RootLogger", ipus: list[str], defer_mutex: bool = False
) -> tuple[dict[str, str], NetMap]:
ip_u = {"": "*"}
cidr_u = {}
for ipu in ipus:
@@ -2701,7 +2799,7 @@ def load_ipu(log: "RootLogger", ipus: list[str]) -> tuple[dict[str, str], NetMap
cidr_u[cidr] = uname
ip_u[cip] = uname
try:
nm = NetMap(["::"], list(cidr_u.keys()), True, True)
nm = NetMap(["::"], list(cidr_u.keys()), True, True, defer_mutex)
except Exception as ex:
t = "failed to translate --ipu into netmap, probably due to invalid config: %r"
log("root", t % (ex,), 1)
@@ -2720,13 +2818,35 @@ def yieldfile(fn: str, bufsz: int) -> Generator[bytes, None, None]:
yield buf
def justcopy(
fin: Generator[bytes, None, None],
fout: Union[typing.BinaryIO, typing.IO[Any]],
hashobj: Optional["hashlib._Hash"],
max_sz: int,
slp: float,
) -> tuple[int, str, str]:
tlen = 0
for buf in fin:
tlen += len(buf)
if max_sz and tlen > max_sz:
continue
fout.write(buf)
if slp:
time.sleep(slp)
return tlen, "checksum-disabled", "checksum-disabled"
def hashcopy(
fin: Generator[bytes, None, None],
fout: Union[typing.BinaryIO, typing.IO[Any]],
slp: float = 0,
max_sz: int = 0,
hashobj: Optional["hashlib._Hash"],
max_sz: int,
slp: float,
) -> tuple[int, str, str]:
hashobj = hashlib.sha512()
if not hashobj:
hashobj = hashlib.sha512()
tlen = 0
for buf in fin:
tlen += len(buf)
@@ -2752,7 +2872,10 @@ def sendfile_py(
bufsz: int,
slp: float,
use_poll: bool,
dls: dict[str, tuple[float, int]],
dl_id: str,
) -> int:
sent = 0
remains = upper - lower
f.seek(lower)
while remains > 0:
@@ -2769,6 +2892,10 @@ def sendfile_py(
except:
return remains
if dl_id:
sent += len(buf)
dls[dl_id] = (time.time(), sent)
return 0
@@ -2781,6 +2908,8 @@ def sendfile_kern(
bufsz: int,
slp: float,
use_poll: bool,
dls: dict[str, tuple[float, int]],
dl_id: str,
) -> int:
out_fd = s.fileno()
in_fd = f.fileno()
@@ -2793,7 +2922,7 @@ def sendfile_kern(
while ofs < upper:
stuck = stuck or time.time()
try:
req = min(2 ** 30, upper - ofs)
req = min(0x2000000, upper - ofs) # 32 MiB
if use_poll:
poll.poll(10000)
else:
@@ -2817,13 +2946,16 @@ def sendfile_kern(
return upper - ofs
ofs += n
if dl_id:
dls[dl_id] = (time.time(), ofs - lower)
# print("sendfile: ok, sent {} now, {} total, {} remains".format(n, ofs - lower, upper - ofs))
return 0
def statdir(
logger: Optional["RootLogger"], scandir: bool, lstat: bool, top: str
logger: Optional["RootLogger"], scandir: bool, lstat: bool, top: str, throw: bool
) -> Generator[tuple[str, os.stat_result], None, None]:
if lstat and ANYWIN:
lstat = False
@@ -2859,6 +2991,12 @@ def statdir(
logger(src, "[s] {} @ {}".format(repr(ex), fsdec(abspath)), 6)
except Exception as ex:
if throw:
zi = getattr(ex, "errno", 0)
if zi == errno.ENOENT:
raise Pebkac(404, str(ex))
raise
t = "{} @ {}".format(repr(ex), top)
if logger:
logger(src, t, 1)
@@ -2867,7 +3005,7 @@ def statdir(
def dir_is_empty(logger: "RootLogger", scandir: bool, top: str):
for _ in statdir(logger, scandir, False, top):
for _ in statdir(logger, scandir, False, top, False):
return False
return True
@@ -2880,7 +3018,7 @@ def rmdirs(
top = os.path.dirname(top)
depth -= 1
stats = statdir(logger, scandir, lstat, top)
stats = statdir(logger, scandir, lstat, top, False)
dirs = [x[0] for x in stats if stat.S_ISDIR(x[1].st_mode)]
dirs = [os.path.join(top, x) for x in dirs]
ok = []
@@ -3270,6 +3408,7 @@ def _parsehook(
def runihook(
log: Optional["NamedLogger"],
verbose: bool,
cmd: str,
vol: "VFS",
ups: list[tuple[str, int, int, str, str, str, int]],
@@ -3299,6 +3438,17 @@ def runihook(
else:
sp_ka["sin"] = b"\n".join(fsenc(x) for x in aps)
if acmd[0].startswith("zmq:"):
try:
msg = sp_ka["sin"].decode("utf-8", "replace")
_zmq_hook(log, verbose, "xiu", acmd[0][4:].lower(), msg, wait, sp_ka)
if verbose and log:
log("hook(xiu) %r OK" % (cmd,), 6)
except Exception as ex:
if log:
log("zeromq failed: %r" % (ex,))
return True
t0 = time.time()
if fork:
Daemon(runcmd, cmd, bcmd, ka=sp_ka)
@@ -3308,15 +3458,126 @@ def runihook(
retchk(rc, bcmd, err, log, 5)
return False
wait -= time.time() - t0
if wait > 0:
time.sleep(wait)
if wait:
wait -= time.time() - t0
if wait > 0:
time.sleep(wait)
return True
ZMQ = {}
ZMQ_DESC = {
"pub": "fire-and-forget to all/any connected SUB-clients",
"push": "fire-and-forget to one of the connected PULL-clients",
"req": "send messages to a REP-server and blocking-wait for ack",
}
def _zmq_hook(
log: Optional["NamedLogger"],
verbose: bool,
src: str,
cmd: str,
msg: str,
wait: float,
sp_ka: dict[str, Any],
) -> tuple[int, str]:
import zmq
try:
mtx = ZMQ["mtx"]
except:
ZMQ["mtx"] = threading.Lock()
time.sleep(0.1)
mtx = ZMQ["mtx"]
ret = ""
nret = 0
t0 = time.time()
if verbose and log:
log("hook(%s) %r entering zmq-main-lock" % (src, cmd), 6)
with mtx:
try:
mode, sck, mtx = ZMQ[cmd]
except:
mode, uri = cmd.split(":", 1)
try:
desc = ZMQ_DESC[mode]
if log:
t = "libzmq(%s) pyzmq(%s) init(%s); %s"
log(t % (zmq.zmq_version(), zmq.__version__, cmd, desc))
except:
raise Exception("the only supported ZMQ modes are REQ PUB PUSH")
try:
ctx = ZMQ["ctx"]
except:
ctx = ZMQ["ctx"] = zmq.Context()
timeout = sp_ka["timeout"]
if mode == "pub":
sck = ctx.socket(zmq.PUB)
sck.setsockopt(zmq.LINGER, 0)
sck.bind(uri)
time.sleep(1) # give clients time to connect; avoids losing first msg
elif mode == "push":
sck = ctx.socket(zmq.PUSH)
if timeout:
sck.SNDTIMEO = int(timeout * 1000)
sck.setsockopt(zmq.LINGER, 0)
sck.bind(uri)
elif mode == "req":
sck = ctx.socket(zmq.REQ)
if timeout:
sck.RCVTIMEO = int(timeout * 1000)
sck.setsockopt(zmq.LINGER, 0)
sck.connect(uri)
else:
raise Exception()
mtx = threading.Lock()
ZMQ[cmd] = (mode, sck, mtx)
if verbose and log:
log("hook(%s) %r entering socket-lock" % (src, cmd), 6)
with mtx:
if verbose and log:
log("hook(%s) %r sending |%d|" % (src, cmd, len(msg)), 6)
sck.send_string(msg) # PUSH can safely timeout here
if mode == "req":
if verbose and log:
log("hook(%s) %r awaiting ack from req" % (src, cmd), 6)
try:
ret = sck.recv().decode("utf-8", "replace")
if ret.startswith("return "):
m = re.search("^return ([0-9]+)", ret[:12])
if m:
nret = int(m.group(1))
except:
sck.close()
del ZMQ[cmd] # bad state; must reset
raise Exception("ack timeout; zmq socket killed")
if ret and log:
log("hook(%s) %r ACK: %r" % (src, cmd, ret), 6)
if wait:
wait -= time.time() - t0
if wait > 0:
time.sleep(wait)
return nret, ret
def _runhook(
log: Optional["NamedLogger"],
verbose: bool,
src: str,
cmd: str,
ap: str,
@@ -3357,6 +3618,12 @@ def _runhook(
else:
arg = txt or ap
if acmd[0].startswith("zmq:"):
zi, zs = _zmq_hook(log, verbose, src, acmd[0][4:].lower(), arg, wait, sp_ka)
if zi:
raise Exception("zmq says %d" % (zi,))
return {"rc": 0, "stdout": zs}
acmd += [arg]
if acmd[0].endswith(".py"):
acmd = [pybin] + acmd
@@ -3385,9 +3652,10 @@ def _runhook(
except:
ret = {"rc": rc, "stdout": v}
wait -= time.time() - t0
if wait > 0:
time.sleep(wait)
if wait:
wait -= time.time() - t0
if wait > 0:
time.sleep(wait)
return ret
@@ -3410,17 +3678,17 @@ def runhook(
txt: str,
) -> dict[str, Any]:
assert broker or up2k # !rm
asrv = (broker or up2k).asrv
args = (broker or up2k).args
verbose = args.hook_v
vp = vp.replace("\\", "/")
ret = {"rc": 0}
for cmd in cmds:
try:
hr = _runhook(
log, src, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt
log, verbose, src, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt
)
if log and args.hook_v:
log("hook(%s) [%s] => \033[32m%s" % (src, cmd, hr), 6)
if verbose and log:
log("hook(%s) %r => \033[32m%s" % (src, cmd, hr), 6)
if not hr:
return {}
for k, v in hr.items():
@@ -3435,6 +3703,8 @@ def runhook(
elif k in ret:
if k == "rc" and v:
ret[k] = v
elif k == "stdout" and v and not ret[k]:
ret[k] = v
else:
ret[k] = v
except Exception as ex:

View File

@@ -32,7 +32,7 @@ window.baguetteBox = (function () {
scrollCSS = ['', ''],
scrollTimer = 0,
re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4|m4v|mov)(\?|$)/i,
anims = ['slideIn', 'fadeIn', 'none'],
data = {}, // all galleries
imagesElements = [],
@@ -633,6 +633,9 @@ window.baguetteBox = (function () {
catch (ex) { }
isFullscreen = false;
if (toast.tag == 'bb-ded')
toast.hide();
if (dtor || overlay.style.display === 'none')
return;
@@ -668,6 +671,7 @@ window.baguetteBox = (function () {
if (v == keep)
continue;
unbind(v, 'error', lerr);
v.src = '';
v.load();
@@ -695,6 +699,28 @@ window.baguetteBox = (function () {
}
}
function lerr() {
var t;
try {
t = this.getAttribute('src');
t = uricom_dec(t.split('/').pop().split('?')[0]);
}
catch (ex) { }
t = 'Failed to open ' + (t?t:'file');
console.log('bb-ded', t);
t += '\n\nEither the file is corrupt, or your browser does not understand the file format or codec';
try {
t += "\n\nerr#" + this.error.code + ", " + this.error.message;
}
catch (ex) { }
this.ded = esc(t);
if (this === vidimg())
toast.err(20, this.ded, 'bb-ded');
}
function loadImage(index, callback) {
var imageContainer = imagesElements[index];
var galleryItem = currentGallery[index];
@@ -739,7 +765,8 @@ window.baguetteBox = (function () {
var image = mknod(is_vid ? 'video' : 'img');
clmod(imageContainer, 'vid', is_vid);
image.addEventListener(is_vid ? 'loadedmetadata' : 'load', function () {
bind(image, 'error', lerr);
bind(image, is_vid ? 'loadedmetadata' : 'load', function () {
// Remove loader element
qsr('#baguette-img-' + index + ' .bbox-spinner');
if (!options.async && callback)
@@ -816,6 +843,12 @@ window.baguetteBox = (function () {
});
updateOffset();
var im = vidimg();
if (im && im.ded)
toast.err(20, im.ded, 'bb-ded');
else if (toast.tag == 'bb-ded')
toast.hide();
if (options.animation == 'none')
unvid(vid());
else

View File

@@ -188,7 +188,6 @@ html.y {
--srv-1: #555;
--srv-2: #c83;
--srv-3: #c0a;
--srv-3b: rgba(255,68,204,0.6);
--tree-bg: #fff;
@@ -286,6 +285,7 @@ html.bz {
--f-h-b1: #34384e;
--mp-sh: #11121d;
/*--mp-b-bg: #2c3044;*/
--f-play-bg: var(--btn-1-bg);
}
html.by {
--bg: #f2f2f2;
@@ -389,8 +389,6 @@ html.cy {
}
html.dz {
--fg: #4d4;
--fg-max: #fff;
--fg2-max: #fff;
--fg-weak: #2a2;
--bg-u6: #020;
@@ -400,11 +398,9 @@ html.dz {
--bg-u2: #020;
--bg-u1: #020;
--bg: #010;
--bgg: var(--bg);
--bg-d1: #000;
--bg-d2: #020;
--bg-d3: #000;
--bg-max: #000;
--tab-alt: #6f6;
--row-alt: #030;
@@ -417,45 +413,21 @@ html.dz {
--a-dark: #afa;
--a-gray: #2a2;
--btn-fg: var(--a);
--btn-bg: rgba(64,128,64,0.15);
--btn-h-fg: var(--a-hil);
--btn-h-bg: #050;
--btn-1-fg: #000;
--btn-1-bg: #4f4;
--btn-1h-fg: var(--btn-1-fg);
--btn-1h-bg: #3f3;
--btn-bs: 0 0 0 .1em #080 inset;
--btn-1-bs: a;
--chk-fg: var(--tab-alt);
--txt-sh: var(--bg-d2);
--txt-bg: var(--btn-bg);
--op-aa-fg: var(--a);
--op-aa-bg: var(--bg-d2);
--op-a-sh: rgba(0,0,0,0.5);
--u2-btn-b1: var(--fg-weak);
--u2-sbtn-b1: var(--fg-weak);
--u2-txt-bg: var(--bg-u5);
--u2-tab-bg: linear-gradient(to bottom, var(--bg), var(--bg-u1));
--u2-tab-b1: var(--fg-weak);
--u2-tab-1-fg: #fff;
--u2-tab-1-bg: linear-gradient(to bottom, #151, var(--bg) 80%);
--u2-tab-1-b1: #7c5;
--u2-tab-1-b2: #583;
--u2-tab-1-sh: #280;
--u2-b-fg: #fff;
--u2-b1-bg: #3a3;
--u2-b2-bg: #3a3;
--u2-inf-bg: #07a;
--u2-inf-b1: #0be;
--u2-ok-bg: #380;
--u2-ok-b1: #8e4;
--u2-err-bg: #900;
--u2-err-b1: #d06;
--ud-b1: #888;
--sort-1: #fff;
--sort-2: #3f3;
@@ -467,47 +439,12 @@ html.dz {
--tree-bg: #010;
--g-play-bg: #750;
--g-play-b1: #c90;
--g-play-b2: #da4;
--g-play-sh: #b83;
--g-sel-fg: #fff;
--g-sel-bg: #925;
--g-sel-b1: #c37;
--g-sel-sh: #b36;
--g-fsel-bg: #d39;
--g-fsel-b1: #d48;
--g-fsel-ts: #804;
--g-fg: var(--a-hil);
--g-bg: var(--bg-u2);
--g-b1: var(--bg-u4);
--g-b2: var(--bg-u5);
--g-g1: var(--bg-u2);
--g-g2: var(--bg-u5);
--g-f-bg: var(--bg-u4);
--g-f-b1: var(--bg-u5);
--g-f-fg: var(--a-hil);
--g-sh: rgba(0,0,0,0.3);
--f-sh1: 0.33;
--f-sh2: 0.02;
--f-sh3: 0.2;
--f-h-b1: #3b3;
--f-play-bg: #fc5;
--f-play-fg: #000;
--f-sel-sh: #fc0;
--f-gray: #999;
--fm-off: #f6c;
--mp-sh: var(--bg-d3);
--err-fg: #fff;
--err-bg: #a20;
--err-b1: #f00;
--err-ts: #500;
text-shadow: none;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
@@ -1710,6 +1647,18 @@ html.dz .btn {
background: var(--btn-1-bg);
text-shadow: none;
}
#tree ul a.ld::before {
font-weight: bold;
font-family: sans-serif;
display: inline-block;
text-align: center;
width: 1em;
margin: 0 .3em 0 -1.3em;
color: var(--fg-max);
opacity: 0;
content: '◠';
animation: .5s linear infinite forwards spin, ease .25s 1 forwards fadein;
}
#tree ul a.par {
color: var(--fg-max);
}
@@ -1746,15 +1695,24 @@ html.y #tree.nowrap .ntree a+a:hover {
line-height: 0;
}
.dumb_loader_thing {
display: inline-block;
display: block;
margin: 1em .3em 1em 1em;
padding: 0 1.2em 0 0;
font-size: 4em;
min-width: 1em;
min-height: 1em;
opacity: 0;
animation: 1s linear .15s infinite forwards spin, .2s ease .15s 1 forwards fadein;
position: absolute;
position: fixed;
top: .3em;
z-index: 9;
}
#dlt_t {
left: 0;
}
#dlt_f {
right: .5em;
}
#files .cfg {
display: none;
font-size: 2em;
@@ -1931,11 +1889,10 @@ html.y #tree.nowrap .ntree a+a:hover {
#rn_f.m td+td {
width: 50%;
}
#rn_f .err td {
background: var(--err-bg);
color: var(--fg-max);
}
#rn_f .err input[readonly] {
#rn_f .err td,
#rn_f .err input[readonly],
#rui .ng input[readonly] {
color: var(--err-fg);
background: var(--err-bg);
}
#rui input[readonly] {
@@ -2837,6 +2794,7 @@ html.b #u2conf a.b:hover {
padding-left: .2em;
}
.fsearch_explain {
color: var(--a-dark);
padding-left: .7em;
font-size: 1.1em;
line-height: 0;

View File

@@ -124,24 +124,21 @@
</div>
{%- if srv_info %}
<div id="srv_info"><span>{{ srv_info }}</span></div>
{%- endif %}
<div id="widget"></div>
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
CGV1 = {{ cgv1 }},
CGV = {{ cgv|tojson }},
TS = "{{ ts }}",
dtheme = "{{ dtheme }}",
srvinf = "{{ srv_info }}",
s_name = "{{ s_name }}",
lang = "{{ lang }}",
dfavico = "{{ favico }}",
have_tags_idx = {{ have_tags_idx|tojson }},
have_tags_idx = {{ have_tags_idx }},
sb_lg = "{{ sb_lg }}",
txt_ext = "{{ txt_ext }}",
logues = {{ logues|tojson if sb_lg else "[]" }},
ls0 = {{ ls0|tojson }};

File diff suppressed because it is too large Load Diff

BIN
copyparty/web/iiam.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 B

View File

@@ -128,9 +128,9 @@ write markdown (most html is 🙆 too)
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
last_modified = {{ lastmod }},
have_emp = {{ have_emp|tojson }},
have_emp = {{ "true" if have_emp else "false" }},
dfavico = "{{ favico }}";
var md_opt = {

View File

@@ -17,14 +17,13 @@ var chromedbg = function () { console.log(arguments); }
var dbg = function () { };
// replace dbg with the real deal here or in the console:
// dbg = chromedbg
// dbg = console.log
// dbg = chromedbg;
// dbg = console.log;
// dodge browser issues
(function () {
var ua = navigator.userAgent;
if (ua.indexOf(') Gecko/') !== -1 && /Linux| Mac /.exec(ua)) {
if (UA.indexOf(') Gecko/') !== -1 && /Linux| Mac /.exec(UA)) {
// necessary on ff-68.7 at least
var s = mknod('style');
s.innerHTML = '@page { margin: .5in .6in .8in .6in; }';

View File

@@ -450,7 +450,7 @@ function savechk_cb() {
// firefox bug: initial selection offset isn't cleared properly through js
var ff_clearsel = (function () {
if (navigator.userAgent.indexOf(') Gecko/') === -1)
if (UA.indexOf(') Gecko/') === -1)
return function () { }
return function () {
@@ -1078,26 +1078,28 @@ action_stack = (function () {
var p1 = from.length,
p2 = to.length;
while (p1-- > 0 && p2-- > 0)
while (p1 --> 0 && p2 --> 0)
if (from[p1] != to[p2])
break;
if (car > ++p1) {
if (car > ++p1)
car = p1;
}
var txt = from.substring(car, p1)
return {
car: car,
cdr: ++p2,
cdr: p2 + (car && 1),
txt: txt,
cpos: cpos
};
}
var undiff = function (from, change) {
var t1 = from.substring(0, change.car),
t2 = from.substring(change.cdr);
return {
txt: from.substring(0, change.car) + change.txt + from.substring(change.cdr),
txt: t1 + change.txt + t2,
cpos: change.cpos
};
}

View File

@@ -26,9 +26,9 @@
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
last_modified = {{ lastmod }},
have_emp = {{ have_emp|tojson }},
have_emp = {{ "true" if have_emp else "false" }},
dfavico = "{{ favico }}";
var md_opt = {

107
copyparty/web/rups.css Normal file
View File

@@ -0,0 +1,107 @@
html {
color: #333;
background: #f7f7f7;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
touch-action: manipulation;
}
#wrap {
margin: 2em auto;
padding: 0 1em 3em 1em;
line-height: 2.3em;
}
a {
color: #047;
background: #fff;
text-decoration: none;
border-bottom: 1px solid #8ab;
border-radius: .2em;
padding: .2em .6em;
margin: 0 .3em;
}
#wrap td a {
margin: 0;
line-height: 1em;
display: inline-block;
white-space: initial;
font-family: var(--font-main), sans-serif;
}
#repl {
border: none;
background: none;
color: inherit;
padding: 0;
position: fixed;
bottom: .25em;
left: .2em;
}
#wrap table {
border-collapse: collapse;
position: relative;
margin-top: 2em;
}
#wrap th {
top: -1px;
position: sticky;
background: #f7f7f7;
}
#wrap td {
font-family: var(--font-mono), monospace, monospace;
white-space: pre; /*date*/
overflow: hidden; /*ipv6*/
}
#wrap th:first-child,
#wrap td:first-child {
text-align: right;
}
#wrap td,
#wrap th {
text-align: left;
padding: .3em .6em;
max-width: 30vw;
}
#wrap tr:hover td {
background: #ddd;
box-shadow: 0 -1px 0 rgba(128, 128, 128, 0.5) inset;
}
#wrap th:first-child,
#wrap td:first-child {
border-radius: .5em 0 0 .5em;
}
#wrap th:last-child,
#wrap td:last-child {
border-radius: 0 .5em .5em 0;
}
html.z {
background: #222;
color: #ccc;
}
html.bz {
background: #11121d;
color: #bbd;
}
html.z a {
color: #fff;
background: #057;
border-color: #37a;
}
html.z input[type=text] {
color: #ddd;
background: #223;
border: none;
border-bottom: 1px solid #fc5;
border-radius: .2em;
padding: .2em .3em;
}
html.z #wrap th {
background: #222;
}
html.bz #wrap th {
background: #223;
}
html.z #wrap tr:hover td {
background: #000;
}

50
copyparty/web/rups.html Normal file
View File

@@ -0,0 +1,50 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="robots" content="noindex, nofollow">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/rups.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
</head>
<body>
<div id="wrap">
<a href="#" id="re">refresh</a>
<a href="{{ r }}/?h">control-panel</a>
&nbsp; Filter: <input type="text" id="filter" size="20" placeholder="documents/passwords" />
&nbsp; <span id="hits"></span>
<table id="tab"><thead><tr>
<th>size</th>
<th>who</th>
<th>when</th>
<th>age</th>
<th>dir</th>
<th>file</th>
</tr></thead><tbody id="tb"></tbody></table>
</div>
<a href="#" id="repl">π</a>
<script>
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script>var V={{ v }};</script>
<script src="{{ r }}/.cpr/rups.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

66
copyparty/web/rups.js Normal file
View File

@@ -0,0 +1,66 @@
function render() {
var ups = V.ups, now = V.now, html = [];
ebi('filter').value = V.filter;
ebi('hits').innerHTML = 'showing ' + ups.length + ' files';
for (var a = 0; a < ups.length; a++) {
var f = ups[a],
vsp = vsplit(f.vp.split('?')[0]),
dn = esc(uricom_dec(vsp[0])),
fn = esc(uricom_dec(vsp[1])),
at = f.at,
td = now - f.at,
ts = !at ? '(?)' : unix2iso(at),
sa = !at ? '(?)' : td > 60 ? shumantime(td) : (td + 's'),
sz = ('' + f.sz).replace(/\B(?=(\d{3})+(?!\d))/g, " ");
html.push('<tr><td>' + sz +
'</td><td>' + f.ip +
'</td><td>' + ts +
'</td><td>' + sa +
'</td><td><a href="' + vsp[0] + '">' + dn +
'</a></td><td><a href="' + f.vp + '">' + fn +
'</a></td></tr>');
}
if (!ups.length) {
var t = V.filter ? ' matching the filter' : '';
html = ['<tr><td colspan="6">there are no uploads' + t + '</td></tr>'];
}
ebi('tb').innerHTML = html.join('');
}
render();
var ti;
function ask(e) {
ev(e);
clearTimeout(ti);
ebi('hits').innerHTML = 'Loading...';
var xhr = new XHR(),
filter = unsmart(ebi('filter').value);
hist_replace(get_evpath().split('?')[0] + '?ru&filter=' + uricom_enc(filter));
xhr.onload = xhr.onerror = function () {
try {
V = JSON.parse(this.responseText)
}
catch (ex) {
ebi('tb').innerHTML = '<tr><td colspan="6">failed to decode server response as json: <pre>' + esc(this.responseText) + '</pre></td></tr>';
return;
}
render();
};
xhr.open('GET', SR + '/?ru&j&filter=' + uricom_enc(filter), true);
xhr.send();
}
ebi('re').onclick = ask;
ebi('filter').oninput = function () {
clearTimeout(ti);
ti = setTimeout(ask, 500);
ebi('hits').innerHTML = '...';
};
ebi('filter').onkeydown = function (e) {
if (('' + e.key).endsWith('Enter'))
ask();
};

View File

@@ -44,9 +44,10 @@ a {
bottom: .25em;
left: .2em;
}
table {
#wrap table {
border-collapse: collapse;
position: relative;
position: relative;
margin-top: 2em;
}
th {
top: -1px;
@@ -62,6 +63,14 @@ th {
#wrap td+td+td+td+td+td+td+td {
font-family: var(--font-mono), monospace, monospace;
}
#wrap th:first-child,
#wrap td:first-child {
border-radius: .5em 0 0 .5em;
}
#wrap th:last-child,
#wrap td:last-child {
border-radius: 0 .5em .5em 0;
}
@@ -81,3 +90,6 @@ html.bz {
color: #bbd;
background: #11121d;
}
html.bz th {
background: #223;
}

View File

@@ -6,6 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="robots" content="noindex, nofollow">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/shares.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
@@ -14,8 +15,8 @@
<body>
<div id="wrap">
<a id="a" href="{{ r }}/?shares" class="af">refresh</a>
<a id="a" href="{{ r }}/?h" class="af">control-panel</a>
<a href="{{ r }}/?shares">refresh</a>
<a href="{{ r }}/?h">control-panel</a>
<span>axs = perms (read,write,move,delet)</span>
<span>nf = numFiles (0=dir)</span>
@@ -58,9 +59,11 @@
{% if not rows %}
(you don't have any active shares btw)
{% endif %}
</div>
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
shr="{{ shr }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -45,7 +45,7 @@ function qr(e) {
function showqr(href) {
var vhref = href.replace('?qr&', '?').replace('?qr', '');
modal.alert(esc(vhref) + '<img class="b64" src="' + href + '" />');
modal.alert(esc(vhref) + '<img class="b64" width="100" height="100" src="' + href + '" />');
}
(function() {
@@ -71,7 +71,7 @@ function showqr(href) {
tr[a].cells[11].innerHTML =
'<button value="1">1min</button> ' +
'<button value="60">1h</button>';
var btns = QSA('td button'), aa = btns.length;
for (var a = 0; a < aa; a++)
btns[a].onclick = bump;

View File

@@ -90,6 +90,13 @@ table {
text-align: left;
white-space: nowrap;
}
.vols td:empty,
.vols th:empty {
padding: 0;
}
.vols img {
margin: -4px 0;
}
.num {
border-right: 1px solid #bbb;
}
@@ -222,3 +229,6 @@ html.bz {
color: #bbd;
background: #11121d;
}
html.bz .vols img {
filter: sepia(0.8) hue-rotate(180deg);
}

View File

@@ -44,6 +44,18 @@
</table>
{%- endif %}
{%- if dls %}
<h1 id="ae">active downloads:</h1>
<table class="vols">
<thead><tr><th>%</th><th>sent</th><th>speed</th><th>eta</th><th>idle</th><th></th><th>dir</th><th>file</th></tr></thead>
<tbody>
{% for u in dls %}
<tr><td>{{ u[0] }}</td><td>{{ u[1] }}</td><td>{{ u[2] }}</td><td>{{ u[3] }}</td><td>{{ u[4] }}</td><td>{{ u[5] }}</td><td><a href="{{ u[6] }}">{{ u[7]|e }}</a></td><td>{{ u[8] }}</td></tr>
{% endfor %}
</tbody>
</table>
{%- endif %}
{%- if avol %}
<h1>admin panel:</h1>
<table><tr><td> <!-- hehehe -->
@@ -129,13 +141,23 @@
{% if k304 or k304vis %}
{% if k304 %}
<li><a id="h" href="{{ r }}/?k304=n">disable k304</a> (currently enabled)
<li><a id="h" href="{{ r }}/?cc&setck=k304=n">disable k304</a> (currently enabled)
{%- else %}
<li><a id="i" href="{{ r }}/?k304=y" class="r">enable k304</a> (currently disabled)
<li><a id="i" href="{{ r }}/?cc&setck=k304=y" class="r">enable k304</a> (currently disabled)
{% endif %}
<blockquote id="j">enabling this will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
<blockquote id="j">enabling k304 will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
{% endif %}
{% if no304 or no304vis %}
{% if no304 %}
<li><a id="ab" href="{{ r }}/?cc&setck=no304=n">disable no304</a> (currently enabled)
{%- else %}
<li><a id="ac" href="{{ r }}/?cc&setck=no304=y" class="r">enable no304</a> (currently disabled)
{% endif %}
<blockquote id="ad">enabling no304 will disable all caching; try this if k304 wasn't enough. This will waste a huge amount of network traffic!</blockquote></li>
{% endif %}
<li><a id="af" href="{{ r }}/?ru">show recent uploads</a></li>
<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
</ul>
@@ -146,7 +168,7 @@
{%- endif %}
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -34,6 +34,11 @@ var Ls = {
"ta2": "gjenta for å bekrefte nytt passord:",
"ta3": "fant en skrivefeil; vennligst prøv igjen",
"aa1": "innkommende:",
"ab1": "skru av no304",
"ac1": "skru på no304",
"ad1": "no304 stopper all bruk av cache. Hvis ikke k304 var nok, prøv denne. Vil mangedoble dataforbruk!",
"ae1": "utgående:",
"af1": "vis nylig opplastede filer",
},
"eng": {
"d2": "shows the state of all active threads",
@@ -80,6 +85,11 @@ var Ls = {
"ta2": "重复以确认新密码:",
"ta3": "发现拼写错误;请重试",
"aa1": "正在接收的文件:", //m
"ab1": "关闭 k304",
"ac1": "开启 k304",
"ad1": "启用 no304 将禁用所有缓存;如果 k304 不够,可以尝试此选项。这将消耗大量的网络流量!", //m
"ae1": "正在下载:", //m
"af1": "显示最近上传的文件", //m
}
};

View File

@@ -9,7 +9,7 @@
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<style>ul{padding-left:1.3em}li{margin:.4em 0}</style>
<style>ul{padding-left:1.3em}li{margin:.4em 0}.txa{float:right;margin:0 0 0 1em}</style>
{{ html_head }}
</head>
@@ -31,15 +31,22 @@
<br />
<span class="os win lin mac">placeholders:</span>
<span class="os win">
{% if accs %}<code><b>{{ pw }}</b></code>=password, {% endif %}<code><b>W:</b></code>=mountpoint
{% if accs %}<code><b id="pw0">{{ pw }}</b></code>=password, {% endif %}<code><b>W:</b></code>=mountpoint
</span>
<span class="os lin mac">
{% if accs %}<code><b>{{ pw }}</b></code>=password, {% endif %}<code><b>mp</b></code>=mountpoint
{% if accs %}<code><b id="pw0">{{ pw }}</b></code>=password, {% endif %}<code><b>mp</b></code>=mountpoint
</span>
<a href="#" id="setpw">use real password</a>
</p>
{% if args.idp_h_usr %}
<p style="line-height:2em"><b>WARNING:</b> this server is using IdP-based authentication, so this stuff may not work as advertised. Depending on server config, these commands can probably only be used to access areas which don't require authentication, unless you auth using any non-IdP accounts defined in the copyparty config. Please see <a href="https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients">the IdP docs</a></p>
{% endif %}
{% if not args.no_dav %}
<h1>WebDAV</h1>
@@ -53,7 +60,6 @@
{% if s %}
<li>running <code>rclone mount</code> on LAN (or just dont have valid certificates)? add <code>--no-check-certificate</code></li>
{% endif %}
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
@@ -137,7 +143,6 @@
{% if args.ftps %}
<li>running on LAN (or just dont have valid certificates)? add <code>no_check_certificate=true</code> to the config command</li>
{% endif %}
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
<p>if you want to use the native FTP client in windows instead (please dont), press <code>win+R</code> and run this command:</p>
@@ -231,11 +236,65 @@
<div class="os win">
<h1>ShareX</h1>
<p>to upload screenshots using ShareX <a href="https://github.com/ShareX/ShareX/releases/tag/v12.1.1">v12</a> or <a href="https://getsharex.com/">v15+</a>, save this as <code>copyparty.sxcu</code> and run it:</p>
<pre class="dl" name="copyparty.sxcu">
{ "Name": "copyparty",
"RequestURL": "http{{ s }}://{{ ep }}/{{ rvp }}",
"Headers": {
{% if accs %}"pw": "<b>{{ pw }}</b>",{% endif %}
"accept": "url"
},
"DestinationType": "ImageUploader, TextUploader, FileUploader",
"FileFormName": "f" }
</pre>
</div>
<div class="os mac">
<h1>ishare</h1>
<p>to upload screenshots using <a href="https://isharemac.app/">ishare</a>, save this as <code>copyparty.iscu</code> and run it:</p>
<pre class="dl" name="copyparty.iscu">
{ "Name": "copyparty",
"RequestURL": "http{{ s }}://{{ ep }}/{{ rvp }}",
"Headers": {
{% if accs %}"pw": "<b>{{ pw }}</b>",{% endif %}
"accept": "json"
},
"ResponseURL": "{{ '{{fileurl}}' }}",
"FileFormName": "f" }
</pre>
</div>
<div class="os lin">
<h1>flameshot</h1>
<p>to upload screenshots using <a href="https://flameshot.org/">flameshot</a>, save this as <code>flameshot.sh</code> and run it:</p>
<pre class="dl" name="flameshot.sh">
#!/bin/bash
pw="<b>{{ pw }}</b>"
url="http{{ s }}://{{ ep }}/{{ rvp }}"
filename="$(date +%Y-%m%d-%H%M%S).png"
flameshot gui -s -r | curl -sT- "$url$filename?want=url&pw=$pw" | xsel -ib
</pre>
</div>
</div>
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -1,11 +1,3 @@
function QSA(x) {
return document.querySelectorAll(x);
}
var LINUX = /Linux/.test(navigator.userAgent),
MACOS = /[^a-z]mac ?os/i.test(navigator.userAgent),
WINDOWS = /Windows/.test(navigator.userAgent);
var oa = QSA('pre');
for (var a = 0; a < oa.length; a++) {
var html = oa[a].innerHTML,
@@ -15,6 +7,21 @@ for (var a = 0; a < oa.length; a++) {
oa[a].innerHTML = html.replace(rd, '$1').replace(/[ \r\n]+$/, '').replace(/\r?\n/g, '<br />');
}
function add_dls() {
oa = QSA('pre.dl');
for (var a = 0; a < oa.length; a++) {
var an = 'ta' + a,
o = ebi(an) || mknod('a', an, 'download');
oa[a].setAttribute('id', 'tx' + a);
oa[a].parentNode.insertBefore(o, oa[a]);
o.setAttribute('download', oa[a].getAttribute('name'));
o.setAttribute('href', 'data:text/plain;charset=utf-8,' + encodeURIComponent(oa[a].innerText));
clmod(o, 'txa', 1);
}
}
add_dls();
oa = QSA('.ossel a');
for (var a = 0; a < oa.length; a++)
@@ -40,3 +47,21 @@ function setos(os) {
}
setos(WINDOWS ? 'win' : LINUX ? 'lin' : MACOS ? 'mac' : 'idk');
ebi('setpw').onclick = function (e) {
ev(e);
modal.prompt('password:', '', function (v) {
if (!v)
return;
var pw0 = ebi('pw0').innerHTML,
oa = QSA('b');
for (var a = 0; a < oa.length; a++)
if (oa[a].innerHTML == pw0)
oa[a].textContent = v;
add_dls();
});
}

View File

@@ -73,9 +73,9 @@ html {
position: absolute;
height: 1px;
top: 1px;
right: 1%;
width: 99%;
animation: toastt var(--tmtime) steps(var(--tmstep)) forwards;
right: 1px;
left: 1px;
animation: toastt var(--tmtime) 0.07s steps(var(--tmstep)) forwards;
transform-origin: right;
}
@keyframes toastt {
@@ -322,6 +322,8 @@ html.y #tth {
margin: .1em auto;
width: 60%;
height: 60%;
background: #999;
background: rgba(128,128,128,0.2);
}
#modalb {
position: sticky;

View File

@@ -17,10 +17,14 @@ function goto_up2k() {
var up2k = null,
up2k_hooks = [],
hws = [],
hws_ok = 0,
hws_ng = false,
sha_js = WebAssembly ? 'hw' : 'ac', // ff53,c57,sa11
m = 'will use ' + sha_js + ' instead of native sha512 due to';
try {
if (sread('nosubtle') || window.nosubtle)
throw 'chickenbit';
var cf = crypto.subtle || crypto.webkitSubtle;
cf.digest('SHA-512', new Uint8Array(1)).then(
function (x) { console.log('sha-ok'); up2k = up2k_init(cf); },
@@ -242,7 +246,7 @@ function U2pvis(act, btns, uc, st) {
p = bd * 100.0 / sz,
nb = bd - bd0,
spd = nb / (td / 1000),
eta = (sz - bd) / spd;
eta = spd ? (sz - bd) / spd : 3599;
return [p, s2ms(eta), spd / (1024 * 1024)];
};
@@ -691,8 +695,9 @@ function Donut(uc, st) {
}
if (++r.tc >= 10) {
var s = r.eta === null ? 'paused' : r.eta > 60 ? shumantime(r.eta) : (r.eta + 's');
wintitle("{0}%, {1}, #{2}, ".format(
f2f(v * 100 / t, 1), shumantime(r.eta), st.files.length - st.nfile.upload), true);
f2f(v * 100 / t, 1), s, st.files.length - st.nfile.upload), true);
r.tc = 0;
}
@@ -853,8 +858,13 @@ function up2k_init(subtle) {
setmsg(suggest_up2k, 'msg');
var u2szs = u2sz.split(','),
u2sz_min = parseInt(u2szs[0]),
u2sz_tgt = parseInt(u2szs[1]),
u2sz_max = parseInt(u2szs[2]);
var parallel_uploads = ebi('nthread').value = icfg_get('nthread', u2j),
stitch_tgt = ebi('u2szg').value = icfg_get('u2sz', u2sz.split(',')[1]),
stitch_tgt = ebi('u2szg').value = icfg_get('u2sz', u2sz_tgt),
uc = {},
fdom_ctr = 0,
biggest_file = 0;
@@ -871,10 +881,29 @@ function up2k_init(subtle) {
bcfg_bind(uc, 'turbo', 'u2turbo', turbolvl > 1, draw_turbo);
bcfg_bind(uc, 'datechk', 'u2tdate', turbolvl < 3, null);
bcfg_bind(uc, 'az', 'u2sort', u2sort.indexOf('n') + 1, set_u2sort);
bcfg_bind(uc, 'hashw', 'hashw', !!WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
bcfg_bind(uc, 'hashw', 'hashw', !!WebAssembly && !(CHROME && MOBILE) && (!subtle || !CHROME), set_hashw);
bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);
uc.ow = parseInt(sread('u2ow', ['0', '1', '2']) || u2ow);
uc.owt = ['🛡️', '🕒', '♻️'];
function set_ow() {
QS('label[for="u2ow"]').innerHTML = uc.owt[uc.ow];
ebi('u2ow').checked = true; //cosmetic
}
ebi('u2ow').onclick = function (e) {
ev(e);
if (++uc.ow > 2)
uc.ow = 0;
swrite('u2ow', uc.ow);
set_ow();
if (uc.ow && !has(perms, 'delete'))
toast.warn(10, L.u_enoow, 'noow');
else if (toast.tag == 'noow')
toast.hide();
};
set_ow();
var st = {
"files": [],
"nfile": {
@@ -959,7 +988,7 @@ function up2k_init(subtle) {
ud = function () { ebi('dir' + fdom_ctr).click(); };
// too buggy on chrome <= 72
var m = / Chrome\/([0-9]+)\./.exec(navigator.userAgent);
var m = / Chrome\/([0-9]+)\./.exec(UA);
if (m && parseInt(m[1]) < 73)
return uf();
@@ -1290,7 +1319,7 @@ function up2k_init(subtle) {
if (bad_files.length) {
var msg = L.u_badf.format(bad_files.length, ntot);
for (var a = 0, aa = Math.min(20, bad_files.length); a < aa; a++)
msg += '-- ' + bad_files[a][1] + '\n';
msg += '-- ' + esc(bad_files[a][1]) + '\n';
msg += L.u_just1;
return modal.alert(msg, function () {
@@ -1302,7 +1331,7 @@ function up2k_init(subtle) {
if (nil_files.length) {
var msg = L.u_blankf.format(nil_files.length, ntot);
for (var a = 0, aa = Math.min(20, nil_files.length); a < aa; a++)
msg += '-- ' + nil_files[a][1] + '\n';
msg += '-- ' + esc(nil_files[a][1]) + '\n';
msg += L.u_just1;
return modal.confirm(msg, function () {
@@ -1350,9 +1379,21 @@ function up2k_init(subtle) {
draw_each = good_files.length < 50;
if (WebAssembly && !hws.length) {
for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
var nw = Math.min(navigator.hardwareConcurrency || 4, 16);
if (CHROME) {
// chrome-bug 383568268 // #124
nw = Math.max(1, (nw > 4 ? 4 : (nw - 1)));
nw = (subtle && !MOBILE && nw > 2) ? 2 : nw;
}
for (var a = 0; a < nw; a++)
hws.push(new Worker(SR + '/.cpr/w.hash.js?_=' + TS));
if (!subtle)
for (var a = 0; a < hws.length; a++)
hws[a].postMessage('nosubtle');
console.log(hws.length + " hashers");
}
@@ -1368,7 +1409,7 @@ function up2k_init(subtle) {
name = good_files[a][1],
fdir = evpath,
now = Date.now(),
lmod = uc.u2ts ? (fobj.lastModified || now) : 0,
lmod = (uc.u2ts && fobj.lastModified) || 0,
ofs = name.lastIndexOf('/') + 1;
if (ofs) {
@@ -1543,8 +1584,10 @@ function up2k_init(subtle) {
if (nhash) {
st.time.hashing += td;
t.push(['u2etah', st.bytes.hashed, st.bytes.hashed, st.time.hashing]);
if (uc.fsearch)
if (uc.fsearch) {
st.time.busy += td;
t.push(['u2etat', st.bytes.hashed, st.bytes.hashed, st.time.hashing]);
}
}
var b_up = st.bytes.inflight + st.bytes.uploaded,
@@ -1863,10 +1906,12 @@ function up2k_init(subtle) {
function chill(t) {
var now = Date.now();
if ((t.coolmul || 0) < 2 || now - t.cooldown < t.coolmul * 700)
if ((t.coolmul || 0) < 5 || now - t.cooldown < t.coolmul * 700)
t.coolmul = Math.min((t.coolmul || 0.5) * 2, 32);
t.cooldown = Math.max(t.cooldown || 1, Date.now() + t.coolmul * 1000);
var cd = now + 1000 * (t.coolmul + Math.random() * 4 + 2);
t.cooldown = Math.floor(Math.max(cd, t.cooldown || 1));
return t;
}
/////
@@ -1946,38 +1991,90 @@ function up2k_init(subtle) {
nchunk = 0,
chunksize = get_chunksize(t.size),
nchunks = Math.ceil(t.size / chunksize),
csz_mib = chunksize / 1048576,
tread = t.t_hashing,
cache_buf = null,
cache_car = 0,
cache_cdr = 0,
hashers = 0,
hashtab = {};
// resolving subtle.digest w/o worker takes 1sec on blur if the actx hack breaks
var use_workers = hws.length && !hws_ng && uc.hashw && (nchunks > 1 || document.visibilityState == 'hidden'),
hash_par = (!subtle && !use_workers) ? 0 : csz_mib < 48 ? 2 : csz_mib < 96 ? 1 : 0;
pvis.setab(t.n, nchunks);
pvis.move(t.n, 'bz');
if (hws.length && uc.hashw && (nchunks > 1 || document.visibilityState == 'hidden'))
// resolving subtle.digest w/o worker takes 1sec on blur if the actx hack breaks
if (use_workers)
return wexec_hash(t, chunksize, nchunks);
var segm_next = function () {
if (nchunk >= nchunks || bpend)
return false;
var reader = new FileReader(),
nch = nchunk++,
var nch = nchunk++,
car = nch * chunksize,
cdr = Math.min(chunksize + car, t.size);
st.bytes.hashed += cdr - car;
st.etac.h++;
var orz = function (e) {
bpend--;
segm_next();
hash_calc(nch, e.target.result);
if (MOBILE && CHROME && st.slow_io === null && nch == 1 && cdr - car >= 1024 * 512) {
var spd = Math.floor((cdr - car) / (Date.now() + 1 - tread));
st.slow_io = spd < 40 * 1024;
console.log('spd {0}, slow: {1}'.format(spd, st.slow_io));
}
if (cdr <= cache_cdr && car >= cache_car) {
try {
var ofs = car - cache_car,
ofs2 = ofs + (cdr - car),
buf = cache_buf.subarray(ofs, ofs2);
hash_calc(nch, buf);
}
catch (ex) {
vis_exh(ex + '', 'up2k.js', '', '', ex);
}
return;
}
var reader = new FileReader(),
fr_cdr = cdr;
if (st.slow_io) {
var step = cdr - car,
tgt = 48 * 1048576;
while (step && fr_cdr - car < tgt)
fr_cdr += step;
if (fr_cdr - car > tgt && fr_cdr > cdr)
fr_cdr -= step;
if (fr_cdr > t.size)
fr_cdr = t.size;
}
var orz = function (e) {
bpend = 0;
var buf = e.target.result;
if (fr_cdr > cdr) {
cache_buf = new Uint8Array(buf);
cache_car = car;
cache_cdr = fr_cdr;
buf = cache_buf.subarray(0, cdr - car);
}
if (hashers < hash_par)
segm_next();
hash_calc(nch, buf);
};
reader.onload = function (e) {
try { orz(e); } catch (ex) { vis_exh(ex + '', 'up2k.js', '', '', ex); }
};
reader.onerror = function () {
var err = reader.error + '';
var handled = false;
var err = esc('' + reader.error),
handled = false;
if (err.indexOf('NotReadableError') !== -1 || // win10-chrome defender
err.indexOf('NotFoundError') !== -1 // macos-firefox permissions
@@ -1998,17 +2095,20 @@ function up2k_init(subtle) {
toast.err(0, 'y o u b r o k e i t\nfile: ' + esc(t.name + '') + '\nerror: ' + err);
};
bpend++;
reader.readAsArrayBuffer(t.fobj.slice(car, cdr));
bpend = 1;
tread = Date.now();
reader.readAsArrayBuffer(t.fobj.slice(car, fr_cdr));
return true;
};
var hash_calc = function (nch, buf) {
hashers++;
var orz = function (hashbuf) {
var hslice = new Uint8Array(hashbuf).subarray(0, 33),
b64str = buf2b64(hslice);
hashers--;
hashtab[nch] = b64str;
t.hash.push(nch);
pvis.hashed(t);
@@ -2060,16 +2160,27 @@ function up2k_init(subtle) {
free = [],
busy = {},
nbusy = 0,
init = 0,
hashtab = {},
mem = (MOBILE ? 128 : 256) * 1024 * 1024;
if (!hws_ok)
init = setTimeout(function() {
hws_ng = true;
toast.warn(30, 'webworkers failed to start\n\nwill be a bit slower due to\nhashing on main-thread');
apop(st.busy.hash, t);
st.todo.hash.unshift(t);
exec_hash();
}, 5000);
for (var a = 0; a < hws.length; a++) {
var w = hws[a];
free.push(w);
w.onmessage = onmsg;
if (init)
w.postMessage('ping');
if (mem > 0)
free.push(w);
mem -= chunksize;
if (mem <= 0)
break;
}
function go_next() {
@@ -2099,6 +2210,12 @@ function up2k_init(subtle) {
d = d.data;
var k = d[0];
if (k == "pong")
if (++hws_ok == hws.length) {
clearTimeout(init);
go_next();
}
if (k == "panic")
return vis_exh(d[1], 'up2k.js', '', '', d[1]);
@@ -2161,7 +2278,8 @@ function up2k_init(subtle) {
tasker();
}
}
go_next();
if (!init)
go_next();
}
/////
@@ -2180,7 +2298,7 @@ function up2k_init(subtle) {
xhr.onerror = xhr.ontimeout = function () {
console.log('head onerror, retrying', t.name, t);
if (!toast.visible)
toast.warn(9.98, L.u_enethd + "\n\nfile: " + t.name, t);
toast.warn(9.98, L.u_enethd + "\n\nfile: " + esc(t.name), t);
apop(st.busy.head, t);
st.todo.head.unshift(t);
@@ -2255,12 +2373,11 @@ function up2k_init(subtle) {
return console.log('zombie handshake onerror', t.name, t);
if (!toast.visible)
toast.warn(9.98, L.u_eneths + "\n\nfile: " + t.name, t);
toast.warn(9.98, L.u_eneths + "\n\nfile: " + esc(t.name), t);
console.log('handshake onerror, retrying', t.name, t);
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
st.todo.handshake.unshift(chill(t));
t.keepalive = keepalive;
};
var orz = function (e) {
@@ -2273,8 +2390,7 @@ function up2k_init(subtle) {
}
catch (ex) {
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
st.todo.handshake.unshift(chill(t));
var txt = t.t_uploading ? L.u_ehsfin : t.srch ? L.u_ehssrch : L.u_ehsinit;
return toast.err(0, txt + '\n\n' + L.badreply + ':\n\n' + unpre(xhr.responseText));
}
@@ -2362,7 +2478,7 @@ function up2k_init(subtle) {
var idx = t.hash.indexOf(missing[a]);
if (idx < 0)
return modal.alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
missing[a], JSON.stringify(t)));
missing[a], esc(JSON.stringify(t))));
t.postlist.push(idx);
cbd[idx] = 0;
@@ -2375,6 +2491,9 @@ function up2k_init(subtle) {
msg = 'done';
if (t.postlist.length) {
if (t.rechecks && QS('#opa_del.act'))
toast.inf(30, L.u_started, L.u_unpt);
var arr = st.todo.upload,
sort = arr.length && arr[arr.length - 1].nfile > t.n;
@@ -2453,6 +2572,7 @@ function up2k_init(subtle) {
else {
pvis.seth(t.n, 1, "ERROR");
pvis.seth(t.n, 2, L.u_ehstmp, t);
apop(st.busy.handshake, t);
var err = "",
cls = "ERROR",
@@ -2466,7 +2586,6 @@ function up2k_init(subtle) {
var penalty = rsp.replace(/.*rate-limit /, "").split(' ')[0];
console.log("rate-limit: " + penalty);
t.cooldown = Date.now() + parseFloat(penalty) * 1000;
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
return;
}
@@ -2485,12 +2604,15 @@ function up2k_init(subtle) {
if (!t.rechecks && (err_pend || err_srcb)) {
t.rechecks = 0;
t.want_recheck = true;
err = L.u_dupdefer;
cls = 'defer';
if (st.busy.upload.length || st.busy.handshake.length || st.bytes.uploaded) {
err = L.u_dupdefer;
cls = 'defer';
}
}
if (err_pend) {
err += ' <a href="#" onclick="toast.inf(60, L.ue_ab);" class="fsearch_explain">(' + L.u_expl + ')</a>';
}
}
if (rsp.indexOf('server HDD is full') + 1)
return toast.err(0, L.u_ehsdf + "\n\n" + rsp.replace(/.*; /, ''));
if (err != "") {
if (!t.t_uploading)
@@ -2500,12 +2622,17 @@ function up2k_init(subtle) {
pvis.seth(t.n, 2, err);
pvis.move(t.n, 'ng');
apop(st.busy.handshake, t);
tasker();
return;
}
st.todo.handshake.unshift(chill(t));
if (rsp.indexOf('server HDD is full') + 1)
return toast.err(0, L.u_ehsdf + "\n\n" + rsp.replace(/.*; /, ''));
err = t.t_uploading ? L.u_ehsfin : t.srch ? L.u_ehssrch : L.u_ehsinit;
xhrchk(xhr, err + "\n\nfile: " + t.name + "\n\nerror ", "404, target folder not found", "warn", t);
xhrchk(xhr, err + "\n\nfile: " + esc(t.name) + "\n\nerror ", "404, target folder not found", "warn", t);
}
}
xhr.onload = function (e) {
@@ -2526,6 +2653,13 @@ function up2k_init(subtle) {
else if (t.umod)
req.umod = true;
if (!t.srch) {
if (uc.ow == 1)
req.replace = 'mt';
if (uc.ow == 2)
req.replace = true;
}
xhr.open('POST', t.purl, true);
xhr.responseType = 'text';
xhr.timeout = 42000 + (t.srch || t.t_uploaded ? 0 :
@@ -2574,8 +2708,7 @@ function up2k_init(subtle) {
nparts = upt.nparts,
pcar = nparts[0],
pcdr = nparts[nparts.length - 1],
snpart = pcar == pcdr ? pcar : ('' + pcar + '~' + pcdr),
tries = 0;
maxsz = (u2sz_max > 1 ? u2sz_max : 2040) * 1024 * 1024;
if (t.done)
return console.log('done; skip chunk', t.name, t);
@@ -2595,6 +2728,30 @@ function up2k_init(subtle) {
if (cdr >= t.size)
cdr = t.size;
if (cdr - car <= maxsz)
return upload_sub(t, upt, pcar, pcdr, car, cdr, chunksize, car, []);
var car0 = car, subs = [];
while (car < cdr) {
subs.push([car, Math.min(cdr, car + maxsz)]);
car += maxsz;
}
upload_sub(t, upt, pcar, pcdr, 0, 0, chunksize, car0, subs);
}
function upload_sub(t, upt, pcar, pcdr, car, cdr, chunksize, car0, subs) {
var nparts = upt.nparts,
is_sub = subs.length;
if (is_sub) {
var x = subs.shift();
car = x[0];
cdr = x[1];
}
var snpart = is_sub ? ('' + pcar + '(' + (car-car0) +'+'+ (cdr-car)) :
pcar == pcdr ? pcar : ('' + pcar + '~' + pcdr);
var orz = function (xhr) {
st.bytes.inflight -= xhr.bsent;
var txt = unpre((xhr.response && xhr.response.err) || xhr.responseText);
@@ -2608,6 +2765,10 @@ function up2k_init(subtle) {
return;
}
if (xhr.status == 200) {
car = car0;
if (subs.length)
return upload_sub(t, upt, pcar, pcdr, 0, 0, chunksize, car0, subs);
var bdone = cdr - car;
for (var a = pcar; a <= pcdr; a++) {
pvis.prog(t, a, Math.min(bdone, chunksize));
@@ -2616,6 +2777,7 @@ function up2k_init(subtle) {
st.bytes.finished += cdr - car;
st.bytes.uploaded += cdr - car;
t.bytes_uploaded += cdr - car;
t.cooldown = t.coolmul = 0;
st.etac.u++;
st.etac.t++;
}
@@ -2627,7 +2789,7 @@ function up2k_init(subtle) {
toast.inf(10, L.u_cbusy);
}
else {
xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), t.name), "404, target folder not found (???)", "warn", t);
xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), "404, target folder not found (???)", "warn", t);
chill(t);
}
orz2(xhr);
@@ -2671,10 +2833,10 @@ function up2k_init(subtle) {
xhr.bsent = 0;
if (!toast.visible)
toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), t.name), t);
toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), t);
t.nojoin = t.nojoin || t.postlist.length; // maybe rproxy postsize limit
console.log('chunkpit onerror,', ++tries, t.name, t);
console.log('chunkpit onerror,', t.name, t);
orz2(xhr);
};
@@ -2692,9 +2854,13 @@ function up2k_init(subtle) {
xhr.open('POST', t.purl, true);
xhr.setRequestHeader("X-Up2k-Hash", ctxt);
xhr.setRequestHeader("X-Up2k-Wark", t.wark);
if (is_sub)
xhr.setRequestHeader("X-Up2k-Subc", car - car0);
xhr.setRequestHeader("X-Up2k-Stat", "{0}/{1}/{2}/{3} {4}/{5} {6}".format(
pvis.ctr.ok, pvis.ctr.ng, pvis.ctr.bz, pvis.ctr.q, btot, btot - bfin,
st.eta.t.split(' ').pop()));
st.eta.t.indexOf('/s, ')+1 ? st.eta.t.split(' ').pop() : 'x'));
xhr.setRequestHeader('Content-Type', 'application/octet-stream');
if (xhr.overrideMimeType)
xhr.overrideMimeType('Content-Type', 'application/octet-stream');
@@ -2812,13 +2978,13 @@ function up2k_init(subtle) {
}
var read_u2sz = function () {
var el = ebi('u2szg'), n = parseInt(el.value), dv = u2sz.split(',');
var el = ebi('u2szg'), n = parseInt(el.value);
stitch_tgt = n = (
isNaN(n) ? dv[1] :
n < dv[0] ? dv[0] :
n > dv[2] ? dv[2] : n
isNaN(n) ? u2sz_tgt :
n < u2sz_min ? u2sz_min :
n > u2sz_max ? u2sz_max : n
);
if (n == dv[1]) sdrop('u2sz'); else swrite('u2sz', n);
if (n == u2sz_tgt) sdrop('u2sz'); else swrite('u2sz', n);
if (el.value != n) el.value = n;
};
ebi('u2szg').addEventListener('blur', read_u2sz);
@@ -2959,7 +3125,7 @@ function up2k_init(subtle) {
new_state = false;
fixed = true;
}
if (new_state === undefined)
if (new_state === undefined && preferred === undefined)
new_state = can_write ? false : have_up2k_idx ? true : undefined;
}

View File

@@ -5,10 +5,17 @@ if (!window.console || !console.log)
"log": function (msg) { }
};
if (!Object.assign)
Object.assign = function (a, b) {
for (var k in b)
a[k] = b[k];
};
if (window.CGV1)
Object.assign(window, window.CGV1);
if (window.CGV)
for (var k in CGV)
window[k] = CGV[k];
Object.assign(window, window.CGV);
var wah = '',
@@ -22,14 +29,17 @@ var wah = '',
HTTPS = ('' + location).indexOf('https:') === 0,
TOUCH = 'ontouchstart' in window,
MOBILE = TOUCH,
CHROME = !!window.chrome,
CHROME = !!window.chrome, // safari=false
VCHROME = CHROME ? 1 : 0,
IE = /Trident\//.test(navigator.userAgent),
FIREFOX = ('netscape' in window) && / rv:/.test(navigator.userAgent),
IPHONE = TOUCH && /iPhone|iPad|iPod/i.test(navigator.userAgent),
LINUX = /Linux/.test(navigator.userAgent),
MACOS = /[^a-z]mac ?os/i.test(navigator.userAgent),
WINDOWS = /Windows/.test(navigator.userAgent);
UA = '' + navigator.userAgent,
IE = /Trident\//.test(UA),
FIREFOX = ('netscape' in window) && / rv:/.test(UA),
IPHONE = TOUCH && /iPhone|iPad|iPod/i.test(UA),
LINUX = /Linux/.test(UA),
MACOS = /Macintosh/.test(UA),
WINDOWS = /Windows/.test(UA),
APPLE = IPHONE || MACOS,
APPLEM = TOUCH && APPLE;
if (!window.WebAssembly || !WebAssembly.Memory)
window.WebAssembly = false;
@@ -189,7 +199,7 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
'<p style="font-size:1.3em;margin:0;line-height:2em">try to <a href="#" onclick="localStorage.clear();location.reload();">reset copyparty settings</a> if you are stuck here, or <a href="#" onclick="ignex();">ignore this</a> / <a href="#" onclick="ignex(true);">ignore all</a> / <a href="?b=u">basic</a></p>',
'<p style="color:#fff">please send me a screenshot arigathanks gozaimuch: <a href="<ghi>" target="_blank">new github issue</a></p>',
'<p class="b">' + esc(url + ' @' + lineNo + ':' + columnNo), '<br />' + esc(msg).replace(/\n/g, '<br />') + '</p>',
'<p><b>UA:</b> ' + esc(navigator.userAgent + '')
'<p><b>UA:</b> ' + esc(UA)
];
try {
@@ -424,7 +434,7 @@ function import_js(url, cb, ecb) {
function unsmart(txt) {
return !IPHONE ? txt : (txt.
return !APPLEM ? txt : (txt.
replace(/[\u2014]/g, "--").
replace(/[\u2022]/g, "*").
replace(/[\u2018\u2019]/g, "'").
@@ -571,7 +581,9 @@ function yscroll() {
function showsort(tab) {
var v, vn, v1, v2, th = tab.tHead,
sopts = jread('fsort', jcp(dsort));
sopts = jread('fsort');
sopts = sopts && sopts.length ? sopts : dsort;
th && (th = th.rows[0]) && (th = th.cells);
@@ -608,10 +620,13 @@ function sortTable(table, col, cb) {
tr = Array.prototype.slice.call(tb.rows, 0),
i, reverse = /s0[^r]/.exec(th[col].className + ' ') ? -1 : 1;
var stype = th[col].getAttribute('sort');
var kname = th[col].getAttribute('name'),
stype = th[col].getAttribute('sort');
try {
var nrules = [], rules = jread("fsort", []);
rules.unshift([th[col].getAttribute('name'), reverse, stype || '']);
var nrules = [],
rules = kname == 'href' ? [] : jread("fsort", []);
rules.unshift([kname, reverse, stype || '']);
for (var a = 0; a < rules.length; a++) {
var add = true;
for (var b = 0; b < a; b++)
@@ -874,6 +889,11 @@ if (window.Number && Number.isFinite)
function f2f(val, nd) {
// 10.toFixed(1) returns 10.00 for certain values of 10
if (!isNum(val)) {
val = parseFloat(val);
if (!isNum(val))
val = 999;
}
val = (val * Math.pow(10, nd)).toFixed(0).split('.')[0];
return nd ? (val.slice(0, -nd) || '0') + '.' + val.slice(-nd) : val;
}
@@ -970,11 +990,33 @@ function apop(arr, v) {
}
function jcp(obj) {
function jcp1(obj) {
return JSON.parse(JSON.stringify(obj));
}
function jcp2(src) {
if (Array.isArray(src)) {
var ret = [];
for (var a = 0; a < src.length; ++a) {
var sub = src[a];
ret.push((sub === null) ? sub : (sub instanceof Date) ? new Date(sub.valueOf()) : (typeof sub === 'object') ? jcp2(sub) : sub);
}
} else {
var ret = {};
for (var key in src) {
var sub = src[key];
ret[key] = sub === null ? sub : (sub instanceof Date) ? new Date(sub.valueOf()) : (typeof sub === 'object') ? jcp2(sub) : sub;
}
}
return ret;
};
// jcp1 50% faster on android-chrome, jcp2 7x everywhere else
var jcp = MOBILE && CHROME ? jcp1 : jcp2;
function sdrop(key) {
try {
STG.removeItem(key);
@@ -1317,7 +1359,7 @@ var tt = (function () {
};
r.getmsg = function (el) {
if (IPHONE && QS('body.bbox-open'))
if (APPLEM && QS('body.bbox-open'))
return;
var cfg = sread('tooltips');
@@ -1527,21 +1569,26 @@ var toast = (function () {
if (sec)
te = setTimeout(r.hide, sec * 1000);
var tb = ebi('toastt');
if (same && delta < 1000 && tb) {
tb.style.animation = 'none';
tb.offsetHeight;
tb.style.animation = null;
if (same && delta < 1000) {
var tb = ebi('toastt');
if (tb) {
tb.style.animation = 'none';
tb.offsetHeight;
tb.style.animation = null;
}
return;
}
if (txt.indexOf('<body>') + 1)
txt = txt.slice(0, txt.indexOf('<')) + ' [...]';
setcvar('--tmtime', sec + 's');
setcvar('--tmstep', sec * 15);
obj.innerHTML = '<div id="toastt"></div><a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
var html = '';
if (sec) {
setcvar('--tmtime', (sec - 0.15) + 's');
setcvar('--tmstep', Math.floor(sec * 20));
html += '<div id="toastt"></div>';
}
obj.innerHTML = html + '<a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
obj.className = cl;
sec += obj.offsetWidth;
obj.className += ' vis';

View File

@@ -20,6 +20,7 @@ catch (ex) {
function load_fb() {
subtle = null;
importScripts('deps/sha512.hw.js');
console.log('using fallback hasher');
}
@@ -29,6 +30,12 @@ var reader = null,
onmessage = (d) => {
if (d.data == 'nosubtle')
return load_fb();
if (d.data == 'ping')
return postMessage(['pong']);
if (busy)
return postMessage(["panic", 'worker got another task while busy']);
@@ -57,7 +64,7 @@ onmessage = (d) => {
};
reader.onerror = function () {
busy = false;
var err = reader.error + '';
var err = esc('' + reader.error);
if (err.indexOf('NotReadableError') !== -1 || // win10-chrome defender
err.indexOf('NotFoundError') !== -1 // macos-firefox permissions

View File

@@ -25,6 +25,9 @@
## [`changelog.md`](changelog.md)
* occasionally grabbed from github release notes
## [`synology-dsm.md`](synology-dsm.md)
* running copyparty on a synology nas
## [`devnotes.md`](devnotes.md)
* technical stuff

View File

@@ -1,3 +1,623 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0225-0017 `v1.16.15` fix low-severity vuln
<img src="https://github.com/9001/copyparty/raw/hovudstraum/docs/logo.svg" width="250" align="right"/>
* read-only demo server at https://a.ocv.me/pub/demo/
* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) [client testbed](https://cd.ocv.me/b/)
## ⚠️ this fixes a minor vulnerability; CVE-score `3.6`/`10`
[GHSA-m2jw-cj8v-937r](https://github.com/9001/copyparty/security/advisories/GHSA-m2jw-cj8v-937r) aka [CVE-2025-27145](https://www.cve.org/CVERecord?id=CVE-2025-27145) could let an attacker run arbitrary javascript by tricking an authenticated user into uploading files with malicious filenames
* ...but it required some clever social engineering, and is **not likely** to be a cause for concern... ah, better safe than sorry
there is a [discord server](https://discord.gg/25J8CdTT6G) with an `@everyone` in case of future important updates, such as [vulnerabilities](https://github.com/9001/copyparty/security) (most recently 2025-02-25)
## recent important news
* [v1.15.0 (2024-09-08)](https://github.com/9001/copyparty/releases/tag/v1.15.0) changed upload deduplication to be default-disabled
* [v1.14.3 (2024-08-30)](https://github.com/9001/copyparty/releases/tag/v1.14.3) fixed a bug that was introduced in v1.13.8 (2024-08-13); this bug could lead to **data loss** -- see the v1.14.3 release-notes for details
## 🧪 new features
* nothing this time
## 🩹 bugfixes
* fix [GHSA-m2jw-cj8v-937r](https://github.com/9001/copyparty/security/advisories/GHSA-m2jw-cj8v-937r) / [CVE-2025-27145](https://www.cve.org/CVERecord?id=CVE-2025-27145) in 438ea6cc
* when trying to upload an empty files by dragging it into the browser, the filename would be rendered as HTML, allowing javascript injection if the filename was malicious
* issue discovered and reported by @JayPatel48 (thx!)
* related issues in errorhandling of uploads 499ae1c7 36866f1d
* these all had the same consequences as the GHSA above, but a network outage was necessary to trigger them
* which would probably have the lucky side-effect of blocking the javascript download, nice
* paranoid fixing of probably-not-even-issues 3adbb2ff
* fix some markdown / texteditor bugs 407531bc
* only indicate file-versions for markdown files in listings, since it's tricky to edit non-textfiles otherwise
* CTRL-C followed by CTRL-V and CTRL-Z in a single-line file would make a character fall off
* ensure safety of extensions
## 🔧 other changes
* readme:
* mention support for running the server on risc-v 6d102fc8
* mention that the [sony psp](https://github.com/user-attachments/assets/9d21f020-1110-4652-abeb-6fc09c533d4f) can browse and upload 598a29a7
----
# 💾 what to download?
| download link | is it good? | description |
| -- | -- | -- |
| **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** | ✅ the best 👍 | runs anywhere! only needs python |
| [a docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker/README.md) | it's ok | good if you prefer docker 🐋 |
| [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) | ⚠️ [acceptable](https://github.com/9001/copyparty#copypartyexe) | for [win8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png) or later; built-in thumbnailer |
| [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.16.14/u2c.exe) | ⚠️ acceptable | [CLI uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/u2c.py) as a win7+ exe ([video](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm)) |
| [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) | ⚠️ acceptable | similar to the regular sfx, [mostly worse](https://github.com/9001/copyparty#zipapp) |
| [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) | ⛔️ [dangerous](https://github.com/9001/copyparty#copypartyexe) | for [win7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png) -- never expose to the internet! |
| [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.16.5/copyparty-winpe64.exe) | ⛔️ dangerous | runs on [64bit WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png), otherwise useless |
* except for [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.16.14/u2c.exe), all of the options above are mostly equivalent
* the zip and tar.gz files below are just source code
* python packages are available at [PyPI](https://pypi.org/project/copyparty/#files)
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0219-2309 `v1.16.14` overwrite by upload
## 🧪 new features
* #139 overwrite existing files by uploading over them e9f78ea7
* default-disabled; a new togglebutton in the upload-UI configures it
* can optionally compare last-modified-time and only overwrite older files
* [GDPR compliance](https://github.com/9001/copyparty#GDPR-compliance) (maybe/probably) 4be0d426
## 🩹 bugfixes
* some cosmetic volflag stuff, all harmless b190e676
* disabling a volflag `foo` with `-foo` shows a warning that `-foo` was not a recognized volflag, but it still does the right thing
* some volflags give the *"unrecognized volflag, will ignore"* warning, but not to worry, they still work just fine:
* `xz` to allow serverside xz-compression of uploaded files
* the option to customize the loader-spinner would glitch out during the initial page load 7d7d5d6c
## 🔧 other changes
* [randpic.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/handlers/randpic.py), new 404-handler example, returns a random pic from a folder 60d5f271
* readme: [howto permanent cloudflare tunnel](https://github.com/9001/copyparty#permanent-cloudflare-tunnel) for easy hosting from home 2beb2acc
* [synology-dsm](https://github.com/9001/copyparty/blob/hovudstraum/docs/synology-dsm.md): mention how to update the docker image 56ce5919
* spinner improvements 6858cb06
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0213-2057 `v1.16.13` configure with confidence
## 🧪 new features
* make the config-parser more helpful regarding volflags a255db70
* if an unrecognized volflag is specified, print a warning instead of silently ignoring it
* understand volflag-names with Uppercase and/or kebab-case (dashes), and not just snake_case (underscores)
* improve `--help-flags` to mention and explain all available flags
* #136 WebDAV: support COPY 62ee7f69
* also support overwrite of existing target files (default-enabled according to the spec)
* the user must have the delete-permission to actually replace files
* option to specify custom icons for certain file extensions 7e4702cf
* see `--ext-th` mentioned briefly in the [thumbnails section](https://github.com/9001/copyparty/#thumbnails)
* option to replace the loading-spinner animation 685f0869
* including how to [make it exceptionally normal-looking](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner)
## 🩹 bugfixes
* #136 WebDAV fixes 62ee7f69
* COPY/MOVE/MKCOL: challenge clients to provide the password as necessary
* most clients only need this in PROPFIND, but KDE-Dolphin is more picky
* MOVE: support `webdav://` Destination prefix as used by Dolphin, probably others
* #136 WebDAV: improve support for KDE-Dolphin as client 9d769027
* it masquerades as a graphical browser yet still expects 401, so special-case it with a useragent scan
## 🔧 other changes
* Docker-only: quick hacky fix for the [musl CVE](https://www.openwall.com/lists/musl/2025/02/13/1) until the official fix is out 4d6626b0
* the docker images will be rebuilt when `musl-1.2.5-r9.apk` is released, in 6~24h or so
* until then, there is no support for reading korean XML files when running in docker
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0209-2331 `v1.16.12` RTT
## 🧪 new features
* show rtt (network latency to server, including request processing time) in the top status text d27f1104
* and log the client-reported RTT to serverlog 20ddeb6e
* remember file selection when changing folders c7db08ed
* good for when you accidentally navigate elsewhere
* option to restrict download-as-zip/tar to admins-only c87af9e8
* #135 add [bubbleparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/README.md#bubblepartysh), thx @coderofsalvation! 3582a100
* runs copyparty in a [sandbox](https://github.com/containers/bubblewrap), making it harder to gain unintended access through bugs in python or copyparty
* better alternative to [prisonparty](https://github.com/9001/copyparty/tree/hovudstraum/bin#prisonpartysh), more similar to [the sandboxing in the nixos package](https://github.com/9001/copyparty/blob/7dda77dcb/contrib/nixos/modules/copyparty.nix#L232-L272)
* new plugin: [quickmove](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/quickmove.js) 46f9e9ef
* adds hotkey `W` to quickly move selected files into a subfolder
* #133 new plugin: [graft-thumbs.js](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/graft-thumbs.js) 6c202eff
* in folders with foobar.mp3 and foobar.png, can copy the thumbnail from the png to the jpg (and then hide the png)
* handlers: add [http-redirect example](https://github.com/9001/copyparty/blob/hovudstraum/bin/handlers/redirect.py) 22cbd2db
* add [ping.html](https://github.com/9001/copyparty/blob/hovudstraum/srv/ping.html) 7de9d15a 910797cc
## 🩹 bugfixes
* improve iPad detection so they get opus instead of mp3 12dcea4f
## 🔧 other changes
* safeguard against accidental config loss cd71b505
* while no copyparty servers have ended up in this unfortunate situation yet (afaik), be proactive and borrow some experience from other docker-based services
* readme: improve config examples 32e90859
* improve serverlog entries regarding 403s b020fd4a
* #132 mention fuse permissions in readme d9d2a092
* traefik-example: fix disconnect during big uploads 6a9ffe7e
* try to show an appropriate warning for media that the browser doesn't support playing 4ef35263
* was an attempt at detecting iphones failing to play high-color-precision webm files, but safari doesn't seem to realize itself that playback has failed, ah well
* copyparty.exe: update to python 3.12.9
* update deps: dompurify 3.2.4
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0127-0140 `v1.16.11` fix no-acode
## 🧪 new features
* u2c (commandline uploader): print download-links for uploaded files 1fe30363
* `-u` prints a list after all uploads finished
* `-ud` print during upload, after each file
* `-uf a.txt` writes them to `a.txt`
## 🩹 bugfixes
* [previous ver](https://github.com/9001/copyparty/releases/tag/v1.16.10) broke `--no-acode` (disable audio transcoding) by showing javascript errors 54a7256c
* reported on discord (thx)
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0125-1809 `v1.16.10` iOS9 is fine too
## 🧪 new features
* support audio playback on *really old* apple devices c9eba39e
* will now transcode to mp3 when necessary, since iOS didn't support opus-in-caf before iOS 11
* support audio playback on *future* apple devices 28c9de3f 95390b65
* iOS 17.5 introduced support for opus-in-weba (like webp just audio instead) and, unlike caf, this intentionally supports vbr-opus (awesome)
* ...but the current code in iOS is too buggy, so this new format is default-disabled and we'll stick to caf for now fff38f48
* ZeroMQ event-hooks can reject uploads 3a5c1d9f
* see [the example zmq listener](https://github.com/9001/copyparty/blob/1dace720/bin/zmq-recv.py#L26-L28)
* chat with ZeroMQ event-hooks from javascript cdd3b67a
* replies from ZMQ REP servers are included in the msg-to-log responses
* which makes [this joke](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/usb-eject.py) possible f38c7543
## 🩹 bugfixes
* nope
## 🔧 other changes
* option to restrict the recent-uploads listing to admins-only b8b5214f
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0122-2326 `v1.16.9` ZeroMQ says hello
## 🧪 new features
* event-hooks can send zeromq / zmq / 0mq messages; see [readme](https://github.com/9001/copyparty#zeromq) or `--help-hooks` for examples d9db1534
* new volflags to specify the [allow-tag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy#iframes) of the markdown/logue sandbox, to allow fullscreen and such (see `--help-flags`) 6a0aaaf0
* new volflag `nosparse` for possibly-better performance in very rare and specific scenarios 917380dd
* only enable this if you're uploading to s3 or something like that, and do plenty of benchmarking to make sure that it actually improved performance instead of making it worse
## 🩹 bugfixes
* restrict max-length of filekeys to 72 characters e0cac6fd
* the hash-calculator mode of the commandline uploader produced incorrect whole-file hashes 4c04798a
* each chunk (`--chs`) was okay, but the final sum was not
## 🔧 other changes
* selftest the xml-parser on startup with malicious xml b2e8bf6e
* just in case a future python-version suddenly makes it unsafe somehow
* disable some features if a dangerously misconfigured reverseproxy is detected 3f84b0a0
* the download-as-zip feature now defaults to utf8 filenames 1231ce19
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2025-0111-1611 `v1.16.8` android boost
## 🧪 new features
* 10x faster file hashing in android-chrome ec507889
* on a recent pixel, speed went from 13 to 139 MiB/s
* android's sandboxing makes small reads expensive, so do bigger reads instead
* so the browser-tab will use more RAM on android now, maybe around 200 MiB
* this only affects chrome-based browsers on android, not firefox
* PUT/multipart uploads: request-header `Accept: json` makes it return json instead of html, just like `?j` ce0e5be4
* add config examples for [ishare](https://isharemac.app/), a MacOS screenshot utility inspired by ShareX 0c0d6b2b
* also includes a bug-workaround for [ishare#107](https://github.com/castdrian/ishare/issues/107) - copyparty will now include a toplevel json property `fileurl` in the response if exactly one file was uploaded
* the [connect-page](https://a.ocv.me/?hc) generates an appropriate `copyparty.iscu` for ishare; [it looks like this](https://github.com/user-attachments/assets/820730ad-2319-4912-8eb2-733755a4cf54)
## 🩹 bugfixes
* fix a potential upload deadlock when...
* ...the database (`-e2d`) is **not** enabled for any volume, and...
* ...either the shares feature, or user-changeable passwords, is enabled 9e542cf8
* when loading the partial-uploads registry on startup, a cosmetic desync could occur 467acb47
## 🔧 other changes
* remove some deprecated properties in partial-upload metadata aa2a8fa2
* v1.15.7 is now the oldest version which still has any chance of reading a modern up2k.snap
* #129 added howto: [using webdav when copyparty is behind IdP](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients) -- thanks @wuast94 !
* added howto: [install copyparty on a synology nas](https://github.com/9001/copyparty/blob/hovudstraum/docs/synology-dsm.md) 21f93042
* more examples in the connect-page: 278258ee fb139697
* config-file for sharex on windows
* config-file for ishare on macos
* script for flameshot on linux
* #75 add recommendation to use the [kamelåså project](https://github.com/steinuil/kameloso) instead of copyparty's [very-bad-idea.py](https://github.com/9001/copyparty/tree/hovudstraum/bin/mtag#dangerous-plugins) 9f84dc42
* more reverse-proxy examples (haproxy, lighttpd, traefik, caddy) and improved nginx performance ac0a2da3
* readme has a [performance comparison](https://github.com/9001/copyparty?tab=readme-ov-file#reverse-proxy-performance) -- `haproxy > caddy > traefik > nginx > apache > lighttpd`
* copyparty.exe: updated pillow 244e952f
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1223-0005 `v1.16.7` an idp fix for xmas
# ☃️🎄 **there is still time** 🎅🎁
❄️❄️❄️ please [enjoy some appropriate music](https://a.ocv.me/pub/demo/music/.bonus/#af-55d4554d) -- you'll probably like this more than the idp thing honestly ❄️❄️❄️
## 🧪 new features
* more improvements to the recent-uploads feature 87598dcd
* move html rendering to clientside
* any changes to the filter-text applies in real-time
* loads 50% faster, reduces server-load by 30%
* inhibits search engines from indexing it
## 🩹 bugfixes
* using idp without e2d could mess with uploads dd6e9ea7
* u2c (commandline uploader): fix window title 946a8c5b
* mDNS/SSDP: fix incorrect log colors when multiple primary IPs are lost 552897ab
## 🔧 other changes
* ui: make it more obvious that the volume-control is a volume-control 7f044372
* copyparty.exe: update deps (jinja2, markupsafe, pyinstaller) c0dacbc4
* improve safety of custom plugins 988a7223
* if you've made your own plugins which expect certain values (host-header, filekeys) to be html-safe, then you'll want to upgrade
* also fixes rss-feed xml if password contains special characters
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1219-0037 `v1.16.6` merry \x58mas
# ☃️🎄 **it is time** 🎅🎁
❄️❄️❄️ please [enjoy some appropriate music](https://a.ocv.me/pub/demo/music/.bonus/#af-55d4554d) (trust me on this one, you won't regret it) ❄️❄️❄️
## 🧪 new features
* [list of recent uploads](https://a.ocv.me/?ru) eaa4b04a
* new button in the controlpanel; can be disabled with `--no-ups-page`
* only users with the dot-permission can see dotfiles
* only admins can see uploader-ip and upload-times
* enable `--ups-when` to let all users see upload-times
* #125 log decoded request-URLs 73f7249c
* non-ascii filenames would make the accesslog a wall of `%E5%B9%BB%E6%83%B3%E9%83%B7` so print [the decoded URL](https://github.com/user-attachments/assets/9d411183-30f3-4cb2-a880-84cf18011183) in addition to the original one, which is left as-is for debugging purposes
## 🩹 bugfixes
* #126 improve dotfile handling 4c4e48ba
* was impossible to delete a folder which contained hidden files if the user did not have the permission to see hidden files
* would also affect moving, renaming, copying folders, in which case the dotfiles would not be carried over to the new location
* now, dotfiles are always deleted, and always moved/copied into a new destination, on the condition that this is safe -- if the user has the dotfile permission in the target loocation but not in the source location, the dotfiles will be left behind to avoid accidentally making then browsable
* ux: cosmetic eta/idle-timer fixes 01a3eb29
## 🔧 other changes
* warn on ambiguous comments in config files da5ad2ab
* avoid writing mojibake to the log 3051b131
* use `\x`-encoding for unprintable text
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1211-2236 `v1.16.5` 4chrome
## 🧪 new features
* #124 add workaround for a chrome bug (crash during upload) 24ce46b3
* chrome and chromium-based browsers could OOM
* https://issues.chromium.org/issues/383568268
* #122 "hybrid IdP", regular users can still auth while [IdP](https://github.com/9001/copyparty#identity-providers) is enabled 64501fd7
* previously, enabling IdP would entirely disable password-based login
* now, password-auth is attempted for requests without a valid IdP header
## 🩹 bugfixes
* the terminal window title would only change if `--no-ansi` was specified, which is exactly the opposite of what it should be (and now is) doing db3c0b09
## 🔧 other changes
* mDNS: better log messages when several IPs are added/removed a49bf81f
* webdeps: update dompurify 06868606
----
this release includes a build of [copyparty-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.16.5/copyparty-winpe64.exe) since the last one was [almost a year ago](https://github.com/9001/copyparty/releases/tag/v1.10.1)
* winpe64.exe is only for *very* specific usecases, you almost definitely *do not* want to download it, please just grab the regular [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) instead (works on all 64bit machines running win8 or newer)
* the only difference between winpe64.exe and [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) is that winpe64.exe works in the win7x64 PE (rescue-env), which makes it *almost* entirely useless, and every bit as dangerous to use as copyparty32.exe
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1207-0024 `v1.16.4` ux is hard
## 🧪 new features
* improve the upload ui so it explains how to abort an unfinished upload when someone uploads to the wrong folder by accident be6afe2d
* also reduces serverload slightly when cloning an incoming file to multiple destinations
* u2c (commandline uploader): windows improvements 91637800
* now supports globbing (filename wildcards) on windows
* progressbar in the windows taskbar (requires conemu or the "new windows terminal")
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1204-0003 `v1.16.3` 120%
## 🧪 new features
* #120 add option `--srch-excl` and volflag `srch_excl` for excluding certain paths from search results 697a4fa8
* mDNS: add workaround for https://github.com/avahi/avahi/issues/379 6c1cf68b 94d1924f
* Avahi mDNS Reflection, sometimes used in intricate LAN setups, doesn't understand NSEC records and corrupts them
* the workaround makes copyparty able to read the corrupted packets, but clients without a similar workaround will require either `--zm4` or `--zm6` so copyparty doesn't include the usual NSEC records
* this is mentioned in a very loud warning in the logs when necessary
* mDNS: option to silently ignore buggy devices instead of spamming the log with parser errors 395af051
* webdav: support listing unmapped root with infinite recursion (Depth:0) 21a3f369
* embed current sort config into media URLs (gallery/music) 0f257c93 4cfdc4c5 01670827
* ensures that anyone clicking your link will see the files in the same order as you
* can be confgured serverside (`--hsortn`, volflag `hsortn`) and clientside (`#sort` in settings)
* URL and UI options to disable checksum calculation of PUT, bup, basic uploads c5a000d2
* also allows [choosing either md5, sha1, sha256, or blake2](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#write) instead of the default sha512
* can give uploads a nice speed boost when copyparty is running on a potato
## 🩹 bugfixes
* webdav: more correct login challenge 2ce82339
* the previous behavior could make some clients reluctant to send the password
* #120 forget metadata of all files (including uploads) when shadowed d168b2ac
* thanks to @Gremious for all the debugging to narrow this down!
* #120 drop volume caches if relevant config is changed (mainly indexing filters) 2f83c6c7
* #121 couldn't access arbitrary toplevel files from accounts with `h` permission 1f5f42f2
## 🔧 other changes
* exclude thumbnails from accesslog by default 9082c470
* filesearch: show a final summary of time-elapsed and average hashing speed 8a631f04
* improve phrasing of debug messages during indexing at startup 127f414e
* `--license` no longer depends on opensource.org at build time 33c4ccff
* update deps 6cedcfbf
* copyparty.exe: python 3.12.7 => 3.12.8
* webdeps: hashwasm, dompurify
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1123-2336 `v1.16.2` webdav upload fix
## 🧪 new features
* add `--nsort` and volflag `nsort` to default-enable natural sort of filenames with leading digits 8f7ffcf3
* video-player: support `.mov` files which contain browser-native codecs 2d0cbdf1
## 🩹 bugfixes
* #119 v1.16.0 broke webdav uploads from rclone and possibly other clients 7dfbfc72
* a collection of webdav unittests will be added soon to prevent similar issues in the future
* #118 ip-ranges can be mixed with `lan` when specifying the list of trusted proxies for `x-forwarded-for` with `--xff-src`
* found and fixed by @codemicro (thx!) 0e31cfa7
* ux:
* in the grid-view, markdown files would open in the generic text viewer 520ac8f4
* qr-codes (create-share, view-share) didn't render on chrome db069c3d
* qr-codes could cause layout-shifting 5afb562a
* fix layout-shifting for ongoing downloads in controlpanel 9c8507a0
* cosmetic eta jank b10843d0
## 🔧 other changes
* upto 7% faster folder listings due to refactoring for more ux knobs 0c43b592
* fix resource leaks (only affected tests/debug) 2ab8924e
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1115-2218 `v1.16.1` cbz thumbnails
## 🧪 new features
* thumbnails of .cbz manga archives 4d15dd6e
## 🩹 bugfixes
* when running with `-j0`, download-ETA could break in complex volume layouts 10fc4768
* linking to the image gallery didn't quite work if multiselect was enabled 56a04996
* password-hashing parameters (cpu/ram cost) could not be customized 1f177528
* the defaults must be perfect considering nobody ever tried changing them ¯\\_(ツ)_/¯
## 🔧 other changes
* add intentional crash on startup if two volumes are configured to use the same histpath 2b63d7d1
* prevents funky deadlocks and an eventual database loss in case of a no-thoughts-head-empty moment, purely hypothetical of course 🗿
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1110-1932 `v1.16.0` COPYparty
## 🧪 new features
* #46 #115 copy/paste files and folders cacec9c1
* cut/paste still exists, but now you can copy too
* with a UI to rename files in case of filename collisions 56317b00
* files are created according to the dedup settings in the target volume (either full copies or symlinks/hardlinks)
* show currently active downloads in the controlpanel 8aba5aed
* can be made admin-only with `--dl-list=1` or disabled with `--dl-list=0`
* hides filenames of hidden files, and files from volumes where the viewer doesn't have access
* #114 async reinit on new [IdP users](https://github.com/9001/copyparty#identity-providers) 44ee07f0
* new IdP users can now always auth, even while a filesystem reindex is running
* ux:
* remember batch-rename settings from last time 6a8d5e17
* URL parameters to force grid/thumbs on/off 5718caa9
## 🩹 bugfixes
* folders that fail to list due to a corrupt HDD/filesystem will now return a 404 instead of an empty listing 119e88d8
* also fixes similar issues in u2c and partyfuse
* u2c (commandline uploader): detect and adapt to proxies with short connection keepalives c784e528
* ui/ux:
* show the "switch-to-https" button in 404-messages too efd8a32e
* the folder-loading indicator could steal keyboard focus d9962f65
* hotkey-help was very trigger-happy 71d9e010
## 🔧 other changes
* choose more conservative defaults when server has less than 1 GiB RAM 2bf9055c
* runs okay down to 128 MiB, but thumbnails die below 256 MiB
* update the [comparison to similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) after years of optimizations on both sides 0ce7cf5e
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1027-0751 `v1.15.10` temporary upload links
## 🧪 new features
* [shares](https://github.com/9001/copyparty#shares) can now be uploaded into, and unpost works too 4bdcbc1c
* useful to create temporary URLs for other people to upload to
* shares can be write-only, so visitors can't browse or see any files
* #110 HTTP 304 (caching):
* support `If-Range` for HTTP 206 159f51b1
* add server-side and client-side options to force-disable cache dd6dbdd9
* `--no304=1` shows a button in the controlpanel to disable caching
* `--no304=2` makes that button auto-enabled
* even when `--no304` is not specified, accessing the URL `/?setck=no304=y` force-disables cache
* when cache is force-disabled, browsers will waste a lot of network traffic / data usage
* might help to avoid bugs in browsers or proxies, for example if media files suddenly stop loading
* but such bugs should be exceedingly rare, so do not enable this unless actually necessary
## 🩹 bugfixes
* #110 HTTP 304 (caching):
* remove `Content-Length` and `Content-Type` response headers from 304 replies 91240236
* browsers don't need these, and some middlewares might get confused if they're present
* #113 fix crash on startup if `-j0` was combined with `--ipa` or `--ipu` 3a0d882c
* #111 fix javascript crash if `--u2sz` was set to an invalid value b13899c6
## 🔧 other changes
* #110 HTTP 304 (caching):
* never automatically enable k304 because the `Vary` header killed support for caching in msie anyways 63013cc5
* change time comparison for `If-Modified-Since` to require an exact timestamp match, instead of the intended "modified since". This technically violates the http-spec, but should be safer for backdating file mtimes 159f51b1
* new option `--ohead` to log response headers 7678a91b
* added [nintendo 3ds](https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853) to the [list of supported browsers](https://github.com/9001/copyparty#browser-support) cb81f0ad
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1018-2342 `v1.15.9` rss server
## 🧪 new features
* #109 [rss feed generator](https://github.com/9001/copyparty#rss-feeds) 7ffd805a
* monitor folders recursively with RSS readers
## 🩹 bugfixes
* #107 `--df` diskspace limits was incompatible with webdav 2a570bb4
* #108 up2k javascript crash (only affected the Chinese translation) a7e2a0c9
## 🔧 other changes
* up2k: detect buggy webworkers 5ca8f070
* up2k: improve upload retry/timeout logic a9b4436c
* js: make handshake retries more aggressive
* u2c: reduce chunks timeout + ^
* main: reduce tcp timeout to 128sec (js is 42s)
* httpcli: less confusing log messages
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1016-2153 `v1.15.8` the sky is the limit
## 🧪 new features
* subchunks; avoid the Cloudflare filesize limit entirely fc8298c4 48147c07
* the previous max filesize was `383.9 GiB`, now only the sky is the limit
* if you're using another proxy with a more restrictive limit than Cloudflare's 100 MiB, for example 64 MiB, then `--u2sz 1,64,64`
* m4v videos can be played in the gallery ff0a71f2
## 🩹 bugfixes
* up2k: uploading duplicate files could initially fail (but would succeed after a few automatic retries) due to a toctou 114b71b7
* [u2c](https://github.com/9001/copyparty/blob/hovudstraum/bin/README.md#u2cpy) / commandline uploader:
* directory scanner got stuck if it found a FIFO cba1878b
* excessive number of FDs when uploading large files 65a2b6a2
* chunksize calculation; only affected files exactly 128 GiB large a2e037d6
* support filenames with newlines and invalid utf-8 b2770a20
* invalid utf-8 is replaced by `?` when they hit the server
## 🔧 other changes
* don't show the toast countdown bar if duration is infinite 22dfc6ec
* chickenbit to disable the browser's built-in sha512 implementation and force the bundled wasm instead d715479e
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1013-2244 `v1.15.7` the 'a' in "ip address" stands for authentication
## 🧪 new features
* [cidr-based autologin](https://github.com/9001/copyparty#ip-auth) b7f9bf5a
* map a cidr ip-range to a username; anyone connecting from that ip-range will autologin as that user
* thx to @byteturtle for the idea!
* [u2c](https://github.com/9001/copyparty/blob/hovudstraum/bin/README.md#u2cpy) / commandline uploader:
* option `--chs` to list individual chunk hashes cf1b7562
* fix progress indicator when resuming an upload 53ffd245
* up2k: verbose logging of detected/corrected bitflips ee628363
* *foreshadowing intensifies* (story still developing)
## 🩹 bugfixes
* up2k with database disabled / running without `-e2d` 705f598b
* respect `noforget` when loading snaps
* ...but actually forget deleted files otherwise
* snap-loader adds empty need/hash entries as necessary
## 🔧 other changes
* authed users can now unpost recent uploads of unauthed users from the same IP 22b58e31
* would have become problematic now that cidr-based autologin is a thing
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1011-2256 `v1.15.6` preadme

48
docs/chunksizes.py Executable file
View File

@@ -0,0 +1,48 @@
#!/usr/bin/env python3
# there's far better ways to do this but its 4am and i dont wanna think
# just pypy it my dude
import math
def humansize(sz, terse=False):
for unit in ["B", "KiB", "MiB", "GiB", "TiB"]:
if sz < 1024:
break
sz /= 1024.0
ret = " ".join([str(sz)[:4].rstrip("."), unit])
if not terse:
return ret
return ret.replace("iB", "").replace(" ", "")
def up2k_chunksize(filesize):
chunksize = 1024 * 1024
stepsize = 512 * 1024
while True:
for mul in [1, 2]:
nchunks = math.ceil(filesize * 1.0 / chunksize)
if nchunks <= 256 or (chunksize >= 32 * 1024 * 1024 and nchunks <= 4096):
return chunksize
chunksize += stepsize
stepsize *= mul
def main():
prev = 1048576
n = n0 = 524288
while True:
csz = up2k_chunksize(n)
if csz > prev:
print(f"| {n-n0:>18_} | {humansize(n-n0):>8} | {prev:>13_} | {humansize(prev):>8} |".replace("_", " "))
prev = csz
n += n0
main()

View File

@@ -6,6 +6,7 @@
* [up2k](#up2k) - quick outline of the up2k protocol
* [why not tus](#why-not-tus) - I didn't know about [tus](https://tus.io/)
* [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
* [list of chunk-sizes](#list-of-chunk-sizes) - specific chunksizes are enforced
* [hashed passwords](#hashed-passwords) - regarding the curious decisions
* [http api](#http-api)
* [read](#read)
@@ -95,6 +96,44 @@ hashwasm would solve the streaming issue but reduces hashing speed for sha512 (x
* blake2 might be a better choice since xxh is non-cryptographic, but that gets ~15 MiB/s on slower androids
### list of chunk-sizes
specific chunksizes are enforced depending on total filesize
each pair of filesize/chunksize is the largest filesize which will use its listed chunksize; a 512 MiB file will use chunksize 2 MiB, but if the file is one byte larger than 512 MiB then it becomes 3 MiB
for the purpose of performance (or dodging arbitrary proxy limitations), it is possible to upload combined and/or partial chunks using stitching and/or subchunks respectively
| filesize | filesize | chunksize | chunksz |
| -----------------: | -------: | ------------: | ------: |
| 268 435 456 | 256 MiB | 1 048 576 | 1.0 MiB |
| 402 653 184 | 384 MiB | 1 572 864 | 1.5 MiB |
| 536 870 912 | 512 MiB | 2 097 152 | 2.0 MiB |
| 805 306 368 | 768 MiB | 3 145 728 | 3.0 MiB |
| 1 073 741 824 | 1.0 GiB | 4 194 304 | 4.0 MiB |
| 1 610 612 736 | 1.5 GiB | 6 291 456 | 6.0 MiB |
| 2 147 483 648 | 2.0 GiB | 8 388 608 | 8.0 MiB |
| 3 221 225 472 | 3.0 GiB | 12 582 912 | 12 MiB |
| 4 294 967 296 | 4.0 GiB | 16 777 216 | 16 MiB |
| 6 442 450 944 | 6.0 GiB | 25 165 824 | 24 MiB |
| 137 438 953 472 | 128 GiB | 33 554 432 | 32 MiB |
| 206 158 430 208 | 192 GiB | 50 331 648 | 48 MiB |
| 274 877 906 944 | 256 GiB | 67 108 864 | 64 MiB |
| 412 316 860 416 | 384 GiB | 100 663 296 | 96 MiB |
| 549 755 813 888 | 512 GiB | 134 217 728 | 128 MiB |
| 824 633 720 832 | 768 GiB | 201 326 592 | 192 MiB |
| 1 099 511 627 776 | 1.0 TiB | 268 435 456 | 256 MiB |
| 1 649 267 441 664 | 1.5 TiB | 402 653 184 | 384 MiB |
| 2 199 023 255 552 | 2.0 TiB | 536 870 912 | 512 MiB |
| 3 298 534 883 328 | 3.0 TiB | 805 306 368 | 768 MiB |
| 4 398 046 511 104 | 4.0 TiB | 1 073 741 824 | 1.0 GiB |
| 6 597 069 766 656 | 6.0 TiB | 1 610 612 736 | 1.5 GiB |
| 8 796 093 022 208 | 8.0 TiB | 2 147 483 648 | 2.0 GiB |
| 13 194 139 533 312 | 12.0 TiB | 3 221 225 472 | 3.0 GiB |
| 17 592 186 044 416 | 16.0 TiB | 4 294 967 296 | 4.0 GiB |
| 26 388 279 066 624 | 24.0 TiB | 6 442 450 944 | 6.0 GiB |
| 35 184 372 088 832 | 32.0 TiB | 8 589 934 592 | 8.0 GiB |
# hashed passwords
@@ -133,15 +172,19 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?tar=xz:9` | ...as an xz-level-9 gnu-tar file |
| GET | `?tar=pax` | ...as a pax-tar file |
| GET | `?tar=pax,xz` | ...as an xz-level-1 pax-tar file |
| GET | `?zip=utf-8` | ...as a zip file |
| GET | `?zip` | ...as a WinXP-compatible zip file |
| GET | `?zip` | ...as a zip file |
| GET | `?zip=dos` | ...as a WinXP-compatible zip file |
| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
| GET | `?tar&w` | pregenerate webp thumbnails |
| GET | `?tar&j` | pregenerate jpg thumbnails |
| GET | `?tar&p` | pregenerate audio waveforms |
| GET | `?shares` | list your shared files/folders |
| GET | `?dls` | show active downloads (do this as admin) |
| GET | `?ups` | show recent uploads from your IP |
| GET | `?ups&filter=f` | ...where URL contains `f` |
| GET | `?ru` | show all recent uploads |
| GET | `?ru&filter=f` | ...where URL contains `f` |
| GET | `?ru&j` | ...as json |
| GET | `?mime=foo` | specify return mimetype `foo` |
| GET | `?v` | render markdown file at URL |
| GET | `?v` | open image/video/audio in mediaplayer |
@@ -163,15 +206,21 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| method | params | result |
|--|--|--|
| POST | `?copy=/foo/bar` | copy the file/folder at URL to /foo/bar |
| POST | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
| method | params | body | result |
|--|--|--|--|
| PUT | | (binary data) | upload into file at URL |
| PUT | `?j` | (binary data) | ...and reply with json |
| PUT | `?ck` | (binary data) | upload without checksum gen (faster) |
| PUT | `?ck=md5` | (binary data) | return md5 instead of sha512 |
| PUT | `?gz` | (binary data) | compress with gzip and write into file at URL |
| PUT | `?xz` | (binary data) | compress with xz and write into file at URL |
| mPOST | | `f=FILE` | upload `FILE` into the folder at URL |
| mPOST | `?j` | `f=FILE` | ...and reply with json |
| mPOST | `?ck` | `f=FILE` | ...and disable checksum gen (faster) |
| mPOST | `?ck=md5` | `f=FILE` | ...and return md5 instead of sha512 |
| mPOST | `?replace` | `f=FILE` | ...and overwrite existing files |
| mPOST | `?media` | `f=FILE` | ...and return medialink (not hotlink) |
| mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
@@ -188,8 +237,15 @@ upload modifiers:
| http-header | url-param | effect |
|--|--|--|
| `Accept: url` | `want=url` | return just the file URL |
| `Accept: json` | `want=json` | return upload info as json; same as `?j` |
| `Rand: 4` | `rand=4` | generate random filename with 4 characters |
| `Life: 30` | `life=30` | delete file after 30 seconds |
| `CK: no` | `ck` | disable serverside checksum (maybe faster) |
| `CK: md5` | `ck=md5` | return md5 checksum instead of sha512 |
| `CK: sha1` | `ck=sha1` | return sha1 checksum |
| `CK: sha256` | `ck=sha256` | return sha256 checksum |
| `CK: b2` | `ck=b2` | return blake2b checksum |
| `CK: b2s` | `ck=b2s` | return blake2s checksum |
* `life` only has an effect if the volume has a lifetime, and the volume lifetime must be greater than the file's
@@ -208,6 +264,12 @@ upload modifiers:
| method | params | result |
|--|--|--|
| GET | `?pw=x` | logout |
| GET | `?grid` | ui: show grid-view |
| GET | `?imgs` | ui: show grid-view with thumbnails |
| GET | `?grid=0` | ui: show list-view |
| GET | `?imgs=0` | ui: show list-view |
| GET | `?thumb` | ui, grid-mode: show thumbnails |
| GET | `?thumb=0` | ui, grid-mode: show icons |
# event hooks
@@ -280,6 +342,7 @@ python3 -m venv .venv
. .venv/bin/activate
pip install jinja2 strip_hints # MANDATORY
pip install argon2-cffi # password hashing
pip install pyzmq # send 0mq from hooks
pip install mutagen # audio metadata
pip install pyftpdlib # ftp server
pip install partftpy # tftp server

View File

@@ -20,3 +20,25 @@ this means that, if an IdP volume is located inside a folder that is readable by
and likewise -- if the IdP volume is inside a folder that is only accessible by certain users, but the IdP volume is configured to allow access from unauthenticated users, then the contents of the volume will NOT be accessible until it is revived
until this limitation is fixed (if ever), it is recommended to place IdP volumes inside an appropriate parent volume, so they can inherit acceptable permissions until their revival; see the "strategic volumes" at the bottom of [./examples/docker/idp/copyparty.conf](./examples/docker/idp/copyparty.conf)
## Connecting webdav clients
If you use only idp and want to connect via rclone you have to adapt a few things.
The following steps are for Authelia, but should be easy adaptable to other IdPs and clients. There may be better/smarter ways to do this, but this is a known solution.
1. Add a rule for your domain and set it to one factor
```
rules:
- domain: 'sub.domain.tld'
policy: one_factor
```
2. After you created your rclone config find its location with `rclone config file` and add the headers option to it, change the string to `username:password` base64 encoded. Make sure to set the right url location, otherwise you will get a 401 from copyparty.
```
[servername-dav]
type = webdav
url = https://sub.domain.tld/u/user/priv/
vendor = owncloud
pacer_min_sleep = 0.01ms
headers = Proxy-Authorization,basic base64encodedstring==
```

View File

@@ -259,6 +259,12 @@ for d in /usr /var; do find $d -type f -size +30M 2>/dev/null; done | while IFS=
for f in {0..255}; do echo $f; truncate -s 256M $f; b1=$(printf '%02x' $f); for o in {0..255}; do b2=$(printf '%02x' $o); printf "\x$b1\x$b2" | dd of=$f bs=2 seek=$((o*1024*1024)) conv=notrunc 2>/dev/null; done; done
# create 6.06G file with 16 bytes of unique data at start+end of each 32M chunk
sz=6509559808; truncate -s $sz f; csz=33554432; sz=$((sz/16)); step=$((csz/16)); ofs=0; while [ $ofs -lt $sz ]; do dd if=/dev/urandom of=f bs=16 count=2 seek=$ofs conv=notrunc iflag=fullblock; [ $ofs = 0 ] && ofs=$((ofs+step-1)) || ofs=$((ofs+step)); done
# same but for chunksizes 16M (3.1G), 24M (4.1G), 48M (128.1G)
sz=3321225472; csz=16777216;
sz=4394967296; csz=25165824;
sz=6509559808; csz=33554432;
sz=138438953472; csz=50331648;
f=csz-$csz; truncate -s $sz $f; sz=$((sz/16)); step=$((csz/16)); ofs=0; while [ $ofs -lt $sz ]; do dd if=/dev/urandom of=$f bs=16 count=2 seek=$ofs conv=notrunc iflag=fullblock; [ $ofs = 0 ] && ofs=$((ofs+step-1)) || ofs=$((ofs+step)); done
# py2 on osx
brew install python@2

View File

@@ -48,6 +48,20 @@ and if you want to have a monospace font in the fancy markdown editor, do this:
NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative
# boring loader spinner
replace the 🌲 with a spinning circle using commandline args:
`--spinner ',padding:0;border-radius:9em;border:.2em solid #444;border-top:.2em solid #fc0'`
or config file example:
```yaml
[global]
spinner: ,padding:0;border-radius:9em;border:.2em solid #444;border-top:.2em solid #fc0
```
# `<head>`
to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
@@ -61,6 +75,8 @@ if the value starts with `%` it will assume a jinja2 template and expand it; the
add your own translations by using the english or norwegian one from `browser.js` as a template
> ⚠ Please do not contribute translations to [RTL (Right-to-Left) languages](https://en.wikipedia.org/wiki/Right-to-left_script) for now; the javascript is [not ready](https://github.com/9001/copyparty/blob/hovudstraum/docs/rice/rtl.patch) to deal with it
the easy way is to open up and modify `browser.js` in your own installation; depending on how you installed copyparty it might be named `browser.js.gz` instead, in which case just decompress it, restart copyparty, and start editing it anyways
you will be delighted to see inline html in the translation strings; to help prevent syntax errors, there is [a very jank linux script](https://github.com/9001/copyparty/blob/hovudstraum/scripts/tlcheck.sh) which is slightly better than nothing -- just beware the false-positives, so even if it complains it's not necessarily wrong/bad

79
docs/rice/rtl.patch Normal file
View File

@@ -0,0 +1,79 @@
RTL support is not planned, but it would be
something like this (just a whole lot more)
diff --git a/copyparty/web/browser.css b/copyparty/web/browser.css
index e66279d4..2888be56 100644
--- a/copyparty/web/browser.css
+++ b/copyparty/web/browser.css
@@ -653,12 +653,10 @@ a:hover {
.s0:after,
.s1:after {
content: '⌄';
- margin-left: -.15em;
}
.s0r:after,
.s1r:after {
content: '⌃';
- margin-left: -.15em;
}
.s0:after,
.s0r:after {
@@ -668,6 +666,19 @@ a:hover {
.s1r:after {
color: var(--sort-2);
}
+.ltr .s0:after,
+.ltr .s1:after,
+.ltr .s0r:after,
+.ltr .s1r:after {
+ margin-left: -.15em;
+}
+.rtl .s0:after,
+.rtl .s1:after,
+.rtl .s0r:after,
+.rtl .s1r:after {
+ margin-left: -.5em;
+ padding: 0 .25em 0 0;
+}
#files thead th:after {
margin-right: -.5em;
}
diff --git a/copyparty/web/browser.js b/copyparty/web/browser.js
index 33965a70..bf425cc7 100644
--- a/copyparty/web/browser.js
+++ b/copyparty/web/browser.js
@@ -1797,9 +1797,13 @@ var Ls = {
"lang_set": "刷新以使更改生效?",
},
+ "foo": {
+ "tt": "Foobar",
+ "rtl": "rtl",
+ },
};
-var LANGS = ["eng", "nor", "chi"];
+var LANGS = ["eng", "nor", "chi", "foo"];
if (window.langmod)
langmod();
@@ -1819,7 +1823,7 @@ for (var a = 0; a < LANGS.length; a++) {
t2 = Ls[LANGS[i2]];
for (var k in t1)
- if (!t2[k]) {
+ if (!t2[k] && k != 'rtl') {
console.log("E missing TL", LANGS[i2], k);
t2[k] = t1[k];
}
@@ -1829,6 +1833,10 @@ for (var a = 0; a < LANGS.length; a++) {
if (!has(LANGS, lang))
alert('unsupported --lang "' + lang + '" specified in server args;\nplease use one of these: ' + LANGS);
+if (L.rtl)
+ document.documentElement.setAttribute('dir', L.rtl);
+document.documentElement.className = L.rtl || 'ltr';
+
modal.load();

150
docs/synology-dsm.md Normal file
View File

@@ -0,0 +1,150 @@
# running copyparty on synology dsm nas
![synology-dsm-container-status.png](https://ocv.me/copyparty/doc/pics/dsm.png)
this has been tested on a `Synology ds218+` NAS with 1 SHR storage-pool and 1 volume, but the same steps should work in more advanced setups too
verified on DSM 7.1 and 7.2, but not on 6.x since my flea-market ds218+ refuses to install it for some reason
# ok let's go
go to controlpanel -> shared-folders, and create the following shared-folders if you don't already have appropriate ones:
* a shared-folder for configuration files, preferably on SSD if you have one
* one or more shared-folders for your actual data/media to share
(btw, when you create the shared-folders, it asks whether you want to enable data checksum and file compression, i would recommend both)
the rest of this doc assumes that these two shared-folders are named `configs` and `media1`, and that you made an empty folder inside the `configs` shared-folder named `cpp`
* your copyparty config file (see below) should be named `something.conf` directly inside that cpp folder, for example `/configs/cpp/copyparty.conf`
* during first start, copyparty will create a folder there named `copyparty`, in other words `/configs/cpp/copyparty` which you should leave alone; that's where copyparty stores its indexes and other runtime config
## recommended copyparty config
open the Package Center and install `Text Editor` (by Synology Inc.) to create and edit your copyparty config:
![synology-text-editor-copyparty-conf.png](https://ocv.me/copyparty/doc/pics/dsm-cfg.png)
* note the `copyparty` and `hist` folders in that screenshot which are autogenerated by copyparty and to be left alone
```yaml
[global]
e2d, e2t # remember uploads & read media tags
rss, daw, ver # some other nice-to-have features
#dedup # you may want this, or maybe not
hist: /cfg/hist # don't pollute the shared-folder
name: synology # shows in the browser, can be anything
[accounts]
ed: wark # username ed, password wark
[/] # share the following at the webroot:
/w # the "/w" docker-volume (the shared-folder)
accs:
A: ed # give Admin to username ed
# hide the synology system files by creating a hidden volume
[/@eaDir]
/w/@eaDir
```
if you ever change the copyparty config file, then [restart the container](https://ocv.me/copyparty/doc/pics/dsm71-02.png) to make the changes take effect
okay now continue with one of these:
* [DSM v7.2 or newer](#dsm-v72-or-newer)
* [all older DSM versions](#dsm-v6x-dsm-v71x-or-older)
# DSM v7.2 or newer
`Docker` was replaced by `Container Manager` in DSM v7.2 but they're almost the same thing;
* open the `Package Center` and install the [Container Manager package](https://ocv.me/copyparty/doc/pics/dsm72-01.png) by `Docker Inc.`
* open the `Container Manager` app
* go to the `Registry` tab and search for `copyparty`
* [doubleclick copyparty/ac](https://ocv.me/copyparty/doc/pics/dsm72-02.png) and keep the [default `latest`](https://ocv.me/copyparty/doc/pics/dsm72-03.png) when it asks you which tag to use
* switch to the `Container` tab and click `Create`
* [choose `copyparty/ac:latest`](https://ocv.me/copyparty/doc/pics/dsm72-04.png) and click `Next`
finally, in the [Advanced Settings](https://ocv.me/copyparty/doc/pics/dsm72-05.png) window,
* under `Port Settings`, type `3923` into the `Local Port` textbox
* click `Add Folder` and select `/configs/cpp` on your nas (the `cpp` folder in the `configs` shared-folder), and change `Mount path` to `/cfg`
* click `Add Folder` and select `/media1` on your nas (the shared-folder that copyparty can share in its web-UI) and change `Mount path` to `/w`
* if you are adding multiple shared-folders for media, then the `Mount path` of the 2nd folder should be something like `/w/share2` or `/w/music`
copyparty will launch and become available at http://192.168.1.9:3923/ (assuming `192.168.1.9` is your nas ip)
# DSM v6.x, DSM v7.1.x or older
if you're using DSM 7.1 or older, then you don't have [Container Manager](https://www.synology.com/en-global/dsm/packages/ContainerManager) yet and you'll have to use [Docker](https://www.synology.com/en-global/dsm/packages/Docker?os_ver=6.2&search=docker) instead. Here's how:
* open the `Package Center` and install the [Docker package](https://ocv.me/copyparty/doc/pics/dsm71-01.png) by `Docker Inc.`
* open the `Docker` app
* go to the `Registry` tab and search for `copyparty`
* [doubleclick copyparty/ac](https://ocv.me/copyparty/doc/pics/dsm71-02.png) and keep the [default `latest`](https://ocv.me/copyparty/doc/pics/dsm71-03.png) when it asks you which tag to use
* switch to the `Container` tab and click `Create`
* [choose `copyparty/ac:latest`](https://ocv.me/copyparty/doc/pics/dsm71-04.png) and `Next`
* in the [Network](https://ocv.me/copyparty/doc/pics/dsm71-05.png) window, keep the default `Use the selected networks: [x] bridge`
* in the [General Settings](https://ocv.me/copyparty/doc/pics/dsm71-06.png) window, just keep everything default (in other words, everything disabled)
* in the [Port Settings](https://ocv.me/copyparty/doc/pics/dsm71-07.png) window, change `Local Port` to `3923` (or choose something else, but it cannot be the default `Auto`)
finally, in the [Volume Settings](https://ocv.me/copyparty/doc/pics/dsm71-08.png) window, add a docker volume for copyparty config, and at least one volume for media-files which copyparty can share in its web-UI
* click `Add Folder` and select `/configs/cpp` on your nas (the `cpp` folder in the `configs` shared-folder), and change `Mount path` to `/cfg`
* click `Add Folder` and select `/media1` on your nas (the shared-folder that copyparty can share in its web-UI) and change `Mount path` to `/w`
* if you are adding multiple shared-folders for media, then the `Mount path` of the 2nd folder should be something like `/w/share2` or `/w/music`
copyparty will launch and become available at http://192.168.1.9:3923/ (assuming `192.168.1.9` is your nas ip)
# misc notes
note that if you only want to share some folders inside your data volume, and not all of it, then you can either give copyparty the whole shared-folder anyways and control/restrict access in the copyparty config file (recommended), or you can add each folder as a new docker volume (not as flexible)
## updating
to update to a new copyparty version: `Container Manager` » `Images` » `Update available` » `Update`
* DSM checks for updates every 12h; you can force a check with `sudo /var/packages/ContainerManager/target/tool/image_upgradable_checker`
* there is no auto-update feature, and beware that watchtower does not support DSM
## regarding ram usage
the ram usage indicator in both `Docker` and `Container Manager` is misleading because it also counts the kernel disk cache which makes the number insanely high -- the synology resource monitor shows the correct values, usually less than 100 MiB
to see the actual memory usage by copyparty, see `Resource Monitor` -> `Task Manager` -> `Processes` and look at the `Private Memory` of `python3` which is probably copyparty
## regarding performance
when uploading files to the synology nas with the respective web-UIs,
* `File Station` does about 16 MiB/s,
* `Synology Drive Server` does about 50 MiB/s; deceivingly fast upload speeds at first, but when the file is fully uploaded, there is a lengthy "processing" step at the end, reducing the average speed to about 50% of the initial
* copyparty maxes the HDD write-speeds, 99 MiB/s
when uploading to the synology nas over webdav,
* `WebDAV Server` by `Synology Inc.` in the Package Center does 86 MiB/s
* copyparty does 79 MiB/s; the NAS CPU is a bottleneck because copyparty verifies the upload checksum while `WebDAV Server` doesn't

View File

@@ -58,7 +58,9 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [h5ai](#h5ai)
* [autoindex](#autoindex)
* [miniserve](#miniserve)
* [pingvin-share](#pingvin-share)
* [briefly considered](#briefly-considered)
* [notes](#notes)
# recommendations
@@ -106,6 +108,7 @@ some softwares not in the matrixes,
* [h5ai](#h5ai)
* [autoindex](#autoindex)
* [miniserve](#miniserve)
* [pingvin-share](#pingvin-share)
symbol legend,
* `█` = absolutely
@@ -128,6 +131,7 @@ symbol legend,
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ | |
| runs on Risc-V | █ | | | █ | █ | █ | | • | | █ | | | |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | | █ |
| android app | | | | █ | █ | | | | | | | | |
@@ -276,7 +280,7 @@ symbol legend,
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | | █ |
| unmap subfolders | █ | | █ | | | | █ | | | █ | | • | |
| index.html blocks list | | | | | | | █ | | | • | | | |
| write-only folders | █ | | █ | | | | | | | | █ | █ | |
| write-only folders | █ | | █ | | | | | | | | █ | █ | |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ | |
@@ -426,6 +430,10 @@ symbol legend,
| gimme-that | python | █ mit | 4.8 MB |
| ass | ts | █ isc | • |
| linx | go | ░ gpl3 | 20 MB |
| h5ai | php | █ mit | • |
| autoindex | go | █ mpl2 | 11 MB |
| miniserve | rust | █ mit | 2 MB |
| pingvin-share | go | █ bsd2 | 487 MB |
* `size` = binary (if available) or installed size of program and its dependencies
* copyparty size is for the [standalone python](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) file; the [windows exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) is **6 MiB**
@@ -458,11 +466,13 @@ symbol legend,
## [hfs3](https://rejetto.com/hfs/)
* nodejs; cross-platform
* vfs with gui config, per-volume permissions
* tested locally, v0.53.2 on archlinux
* 🔵 uploads are resumable
* ⚠️ uploads are not segmented; max upload size 100 MiB on cloudflare
* ⚠️ uploads are not accelerated (copyparty is 3x faster across the atlantic)
* ⚠️ uploads are not integrity-checked
* ⚠️ copies the file after upload; need twice filesize free disk space
* ⚠️ uploading small files is decent; `107` files per sec (copyparty does `670`/sec, 6x faster)
* ⚠️ doesn't support crazy filenames
* ✅ config GUI
* ✅ download counter
@@ -471,11 +481,12 @@ symbol legend,
## [nextcloud](https://github.com/nextcloud/server)
* php, mariadb
* tested locally, [linuxserver/nextcloud](https://hub.docker.com/r/linuxserver/nextcloud) v30.0.2 (sqlite)
* ⚠️ [isolated on-disk file hierarchy] in per-user folders
* not that bad, can probably be remedied with bindmounts or maybe symlinks
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ uploading small files is slow; `2.2` files per sec (copyparty does `87`/sec), tested locally with [linuxserver/nextcloud](https://hub.docker.com/r/linuxserver/nextcloud) (sqlite)
* ⚠️ uploading small files is slow; `4` files per sec (copyparty does `670`/sec, 160x faster)
* ⚠️ no write-only / upload-only folders
* ⚠️ http/webdav only; no ftp, zeroconf
* ⚠️ less awesome music player
@@ -491,12 +502,12 @@ symbol legend,
## [seafile](https://github.com/haiwen/seafile)
* c, mariadb
* tested locally, [official container](https://manual.seafile.com/latest/docker/deploy_seafile_with_docker/) v11.0.13
* ⚠️ [isolated on-disk file hierarchy](https://manual.seafile.com/maintain/seafile_fsck/), incompatible with other software
* *much worse than nextcloud* in that regard
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ uploading small files is slow; `2.7` files per sec (copyparty does `87`/sec), tested locally with [official container](https://manual.seafile.com/docker/deploy_seafile_with_docker/)
* ⚠️ no write-only / upload-only folders
* ⚠️ uploading small files is slow; `4.7` files per sec (copyparty does `670`/sec, 140x faster)
* ⚠️ big folders cannot be zip-downloaded
* ⚠️ http/webdav only; no ftp, zeroconf
* ⚠️ less awesome music player
@@ -519,9 +530,11 @@ symbol legend,
## [dufs](https://github.com/sigoden/dufs)
* rust; cross-platform (windows, linux, macos)
* tested locally, v0.43.0 on archlinux (plain binary)
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ across the atlantic, copyparty is 3x faster
* ⚠️ uploading small files is decent; `97` files per sec (copyparty does `670`/sec, 7x faster)
* ⚠️ doesn't support crazy filenames
* ✅ per-url access control (copyparty is per-volume)
* 🔵 basic but really snappy ui
@@ -564,10 +577,12 @@ symbol legend,
## [filebrowser](https://github.com/filebrowser/filebrowser)
* go; cross-platform (windows, linux, mac)
* tested locally, v2.31.2 on archlinux (plain binary)
* 🔵 uploads are resumable and segmented
* 🔵 multiple files are uploaded in parallel, but...
* ⚠️ big files are not accelerated (copyparty is 5x faster across the atlantic)
* ⚠️ uploads are not integrity-checked
* ⚠️ uploading small files is decent; `69` files per sec (copyparty does `670`/sec, 9x faster)
* ⚠️ http only; no webdav / ftp / zeroconf
* ⚠️ doesn't support crazy filenames
* ⚠️ no directory tree nav
@@ -578,6 +593,7 @@ symbol legend,
* ✅ user signup
* ✅ command runner / remote shell
* ✅ more efficient; can handle around twice as much simultaneous traffic
* note: keep an eye on [gtsteffaniak's fork](https://github.com/gtsteffaniak/filebrowser)
## [filegator](https://github.com/filegator/filegator)
* php; cross-platform (windows, linux, mac)
@@ -605,6 +621,7 @@ symbol legend,
* ⚠️ no zeroconf (mdns/ssdp)
* ⚠️ impractical directory URLs
* ⚠️ AGPL licensed
* 🔵 uploading small files is fast; `340` files per sec (copyparty does `670`/sec)
* 🔵 ftp, ftps, webdav
* ✅ sftp server
* ✅ settings gui
@@ -719,7 +736,31 @@ symbol legend,
* 🔵 upload, tar/zip download, qr-code
* ✅ faster at loading huge folders
## [pingvin-share](https://github.com/stonith404/pingvin-share)
* node; linux (docker)
* mainly for uploads, not a general file server
* 🔵 uploads are segmented (avoids cloudflare size limit)
* 🔵 segments are written directly to target file (HDD-friendly)
* ⚠️ uploads not resumable after a browser or laptop crash
* ⚠️ uploads are not accelerated / integrity-checked
* ⚠️ across the atlantic, copyparty is 3x faster
* measured with chunksize 96 MiB; pingvin's default 10 MiB is much slower
* ⚠️ can't upload folders with subfolders
* ⚠️ no upload ETA
* 🔵 expiration times, shares, upload-undo
* ✅ config + user-registration gui
* ✅ built-in OpenID and LDAP support
* 💾 [IdP middleware](https://github.com/9001/copyparty#identity-providers) and config-files
* ✅ probably more than one person who understands the code
# briefly considered
* [pydio](https://github.com/pydio/cells): python/agpl3, looks great, fantastic ux -- but needs mariadb, systemwide install
* [gossa](https://github.com/pldubouilh/gossa): go/mit, minimalistic, basic file upload, text editor, mkdir and rename (no delete/move)
# notes
* high-latency connections (cross-atlantic uploads) can be accurately simulated with `tc qdisc add dev eth0 root netem delay 100ms`

View File

@@ -25,6 +25,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: Jython",
"Programming Language :: Python :: Implementation :: PyPy",
@@ -51,6 +52,7 @@ ftpd = ["pyftpdlib"]
ftps = ["pyftpdlib", "pyopenssl"]
tftpd = ["partftpy>=0.4.0"]
pwhash = ["argon2-cffi"]
zeromq = ["pyzmq"]
[project.scripts]
copyparty = "copyparty.__main__:main"

View File

@@ -1,11 +1,11 @@
FROM alpine:3.18
WORKDIR /z
ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
ver_hashwasm=4.10.0 \
ver_hashwasm=4.12.0 \
ver_marked=4.3.0 \
ver_dompf=3.1.7 \
ver_dompf=3.2.4 \
ver_mde=2.18.0 \
ver_codemirror=5.65.16 \
ver_codemirror=5.65.18 \
ver_fontawesome=5.13.0 \
ver_prism=1.29.0 \
ver_zopfli=1.0.3
@@ -16,7 +16,7 @@ ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
# https://github.com/codemirror/codemirror5/releases
# https://github.com/cure53/DOMPurify/releases
# https://github.com/Daninet/hash-wasm/releases
# https://github.com/openpgpjs/asmcrypto.js
# https://github.com/openpgpjs/asmcrypto.js/commits/main/
# https://github.com/google/zopfli/tags

View File

@@ -9,7 +9,7 @@ ENV XDG_CONFIG_HOME=/cfg
RUN apk --no-cache add !pyc \
tzdata wget \
py3-jinja2 py3-argon2-cffi py3-pillow \
py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
ffmpeg
COPY i/dist/copyparty-sfx.py innvikler.sh ./

View File

@@ -12,7 +12,8 @@ COPY i/bin/mtag/audio-bpm.py /mtag/
COPY i/bin/mtag/audio-key.py /mtag/
RUN apk add -U !pyc \
tzdata wget \
py3-jinja2 py3-argon2-cffi py3-pillow py3-pip py3-cffi \
py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
py3-pip py3-cffi \
ffmpeg \
vips-jxl vips-heif vips-poppler vips-magick \
py3-numpy fftw libsndfile \

View File

@@ -9,7 +9,8 @@ ENV XDG_CONFIG_HOME=/cfg
RUN apk add -U !pyc \
tzdata wget \
py3-jinja2 py3-argon2-cffi py3-pillow py3-pip py3-cffi \
py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
py3-pip py3-cffi \
ffmpeg \
vips-jxl vips-heif vips-poppler vips-magick \
&& apk add -t .bd \

Some files were not shown because too many files have changed in this diff Show More