v1.6.14

for convenience, the password can be provided as the username
but that confuses pyftpd a little so let's do this

.github/pull_request_template.md

@@ -0,0 +1,2 @@
+Please include the following text somewhere in this PR description:  
+This PR complies with the DCO; https://developercertificate.org/  

.gitignore

@@ -21,11 +21,22 @@ copyparty.egg-info/
 # winmerge
 *.bak
 
+# apple pls
+.DS_Store
+
 # derived
 copyparty/res/COPYING.txt
 copyparty/web/deps/
 srv/
+scripts/docker/i/
+contrib/package/arch/pkg/
+contrib/package/arch/src/
 
 # state/logs
 up.*.txt
-.hist/
+.hist/
+scripts/docker/*.out
+scripts/docker/*.err
+
+# nix build output link
+result

.vscode/launch.json

@@ -8,6 +8,7 @@
             "module": "copyparty",
             "console": "integratedTerminal",
             "cwd": "${workspaceFolder}",
+            "justMyCode": false,
             "args": [
                 //"-nw",
                 "-ed",

.vscode/settings.json

@@ -52,9 +52,11 @@
         "--disable=missing-module-docstring",
         "--disable=missing-class-docstring",
         "--disable=missing-function-docstring",
+        "--disable=import-outside-toplevel",
         "--disable=wrong-import-position",
         "--disable=raise-missing-from",
         "--disable=bare-except",
+        "--disable=broad-except",
         "--disable=invalid-name",
         "--disable=line-too-long",
         "--disable=consider-using-f-string"
@@ -64,6 +66,7 @@
     "editor.formatOnSave": true,
     "[html]": {
         "editor.formatOnSave": false,
+        "editor.autoIndent": "keep",
     },
     "[css]": {
         "editor.formatOnSave": false,

README.md

@@ -1,35 +1,21 @@
-# ⇆🎉 copyparty
+# 💾🎉 copyparty
 
-* http file sharing hub (py2/py3) [(on PyPI)](https://pypi.org/project/copyparty/)
-* MIT-Licensed, 2019-05-26, ed @ irc.rizon.net
+turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser
 
+* server only needs Python (2 or 3), all dependencies optional
+* 🔌 protocols: [http](#the-browser) // [ftp](#ftp-server) // [webdav](#webdav-server) // [smb/cifs](#smb-server)
+* 📱 [android app](#android-app) // [iPhone shortcuts](#ios-shortcuts)
 
-## summary
-
-turn your phone or raspi into a portable file server with resumable uploads/downloads using *any* web browser
-
-* server only needs Python (`2.7` or `3.3+`), all dependencies optional
-* browse/upload with [IE4](#browser-support) / netscape4.0 on win3.11 (heh)
-* protocols: [http](#the-browser) // [ftp](#ftp-server) // [webdav](#webdav-server) // [smb/cifs](#smb-server)
-
-try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running from a basement in finland
+👉 **[Get started](#quickstart)!** or visit the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running from a basement in finland
 
 📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer)
 
 
-## get the app
-
-<a href="https://f-droid.org/packages/me.ocv.partyup/"><img src="https://ocv.me/fdroid.png" alt="Get it on F-Droid" height="50" /> '' <img src="https://img.shields.io/f-droid/v/me.ocv.partyup.svg" alt="f-droid version info" /></a> '' <a href="https://github.com/9001/party-up"><img src="https://img.shields.io/github/release/9001/party-up.svg?logo=github" alt="github version info" /></a>
-
-(the app is **NOT** the full copyparty server! just a basic upload client, nothing fancy yet)
-
-
 ## readme toc
 
 * top
-    * [quickstart](#quickstart) - download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
+    * [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
         * [on servers](#on-servers) - you may also want these, especially on servers
-        * [on debian](#on-debian) - recommended additional steps on debian
     * [features](#features)
     * [testimonials](#testimonials) - small collection of user feedback
 * [motivations](#motivations) - project goals / philosophy
@@ -53,11 +39,13 @@ try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running fro
         * [self-destruct](#self-destruct) - uploads can be given a lifetime
     * [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
     * [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
+    * [media player](#media-player) - plays almost every audio format there is
+        * [audio equalizer](#audio-equalizer) - bass boosted
     * [markdown viewer](#markdown-viewer) - and there are *two* editors
     * [other tricks](#other-tricks)
     * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config) - using arguments or config files, or a mix of both
-    * [zeroconf](#zeroconf) - announce enabled services on the LAN
+    * [zeroconf](#zeroconf) - announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png))
         * [mdns](#mdns) - LAN domain-name and feature announcer
         * [ssdp](#ssdp) - windows-explorer announcer
     * [qr-code](#qr-code) - print a qr-code [(screenshot)](https://user-images.githubusercontent.com/241032/194728533-6f00849b-c6ac-43c6-9359-83e454d11e00.png) for quick access
@@ -75,32 +63,35 @@ try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running fro
     * [database location](#database-location) - in-volume (`.hist/up2k.db`, default) or somewhere else
     * [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
     * [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
-    * [upload events](#upload-events) - trigger a script/program on each upload
+    * [event hooks](#event-hooks) - trigger a program on uploads, renames etc ([examples](./bin/hooks/))
+        * [upload events](#upload-events) - the older, more powerful approach ([examples](./bin/mtag/))
     * [hiding from google](#hiding-from-google) - tell search engines you dont wanna be indexed
     * [themes](#themes)
     * [complete examples](#complete-examples)
+    * [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
+    * [nix package](#nix-package) - `nix profile install github:9001/copyparty`
+    * [nixos module](#nixos-module)
 * [browser support](#browser-support) - TLDR: yes
 * [client examples](#client-examples) - interact with copyparty using non-browser clients
+    * [folder sync](#folder-sync) - sync folders to/from copyparty
     * [mount as drive](#mount-as-drive) - a remote copyparty server as a local filesystem
-* [up2k](#up2k) - quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
+* [android app](#android-app) - upload to copyparty with one tap
+* [iOS shortcuts](#iOS-shortcuts) - there is no iPhone app, but
 * [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
     * [client-side](#client-side) - when uploading files
 * [security](#security) - some notes on hardening
     * [gotchas](#gotchas) - behavior that might be unexpected
+    * [cors](#cors) - cross-site request config
+    * [https](#https) - both HTTP and HTTPS are accepted
 * [recovering from crashes](#recovering-from-crashes)
     * [client crashes](#client-crashes)
         * [frefox wsod](#frefox-wsod) - firefox 87 can crash during uploads
-* [HTTP API](#HTTP-API)
-    * [read](#read)
-    * [write](#write)
-    * [admin](#admin)
-    * [general](#general)
+* [HTTP API](#HTTP-API) - see [devnotes](./docs/devnotes.md#http-api)
 * [dependencies](#dependencies) - mandatory deps
     * [optional dependencies](#optional-dependencies) - install these to enable bonus features
-    * [install recommended deps](#install-recommended-deps)
     * [optional gpl stuff](#optional-gpl-stuff)
 * [sfx](#sfx) - the self-contained "binary"
-    * [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) or [copyparty64.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty64.exe)
+    * [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
 * [install on android](#install-on-android)
 * [reporting bugs](#reporting-bugs) - ideas for context to include in bug reports
 * [devnotes](#devnotes) - for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
@@ -108,27 +99,49 @@ try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running fro
 
 ## quickstart
 
-download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
+just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
 
-if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
+* or install through pypi (python3 only): `python3 -m pip install --user -U copyparty`
+* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
+* or [install through nix](#nix-package), or [on NixOS](#nixos-module)
+* or if you are on android, [install copyparty in termux](#install-on-android)
+* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
+  * docker has all deps built-in, so skip this step:
 
-running the sfx without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; you may want [accounts and volumes](#accounts-and-volumes)
+enable thumbnails (images/audio/video), media indexing, and audio transcoding by installing some recommended deps:
+
+* **Alpine:** `apk add py3-pillow ffmpeg`
+* **Debian:** `apt install python3-pil ffmpeg`
+* **Fedora:** `dnf install python3-pillow ffmpeg`
+* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
+* **MacOS:** `port install py-Pillow ffmpeg`
+* **MacOS** (alternative): `brew install pillow ffmpeg`
+* **Windows:** `python -m pip install --user -U Pillow`
+  * install python and ffmpeg manually; do not use `winget` or `Microsoft Store` (it breaks $PATH)
+  * copyparty.exe comes with `Pillow` and only needs `ffmpeg`
+* see [optional dependencies](#optional-dependencies) to enable even more features
+
+running copyparty without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; you may want [accounts and volumes](#accounts-and-volumes)
+
+or see [complete windows example](./docs/examples/windows.md)
 
 some recommended options:
 * `-e2dsa` enables general [file indexing](#file-indexing)
-* `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen), see [optional dependencies](#optional-dependencies) to enable thumbnails and more
+* `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen)
 * `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
   * replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
-  * see [accounts and volumes](#accounts-and-volumes) for the syntax and other permissions (`r`ead, `w`rite, `m`ove, `d`elete, `g`et, up`G`et)
+  * see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
 
 
 ### on servers
 
 you may also want these, especially on servers:
 
-* [contrib/systemd/copyparty.service](contrib/systemd/copyparty.service) to run copyparty as a systemd service
+* [contrib/systemd/copyparty.service](contrib/systemd/copyparty.service) to run copyparty as a systemd service (see guide inside)
 * [contrib/systemd/prisonparty.service](contrib/systemd/prisonparty.service) to run it in a chroot (for extra security)
-* [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to reverse-proxy behind nginx (for better https)
+* [contrib/rc/copyparty](contrib/rc/copyparty) to run copyparty on FreeBSD
+* [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to [reverse-proxy](#reverse-proxy) behind nginx (for better https)
+* [nixos module](#nixos-module) to run copyparty on NixOS hosts
 
 and remember to open the ports you want; here's a complete example including every feature copyparty has to offer:
 ```
@@ -139,18 +152,6 @@ firewall-cmd --reload
 ```
 (1900:ssdp, 3921:ftp, 3923:http/https, 3945:smb, 3990:ftps, 5353:mdns, 12000:passive-ftp)
 
-### on debian
-
-recommended additional steps on debian  which enable audio metadata and thumbnails (from images and videos):
-
-* as root, run the following:  
-  `apt install python3 python3-pip python3-dev ffmpeg`
-
-* then, as the user which will be running copyparty (so hopefully not root), run this:  
-  `python3 -m pip install --user -U Pillow pillow-avif-plugin`
-
-(skipped `pyheif-pillow-opener` because apparently debian is too old to build it)
-
 
 ## features
 
@@ -164,13 +165,18 @@ recommended additional steps on debian  which enable audio metadata and thumbnai
   * ☑ [smb/cifs server](#smb-server)
   * ☑ [qr-code](#qr-code) for quick access
   * ☑ [upnp / zeroconf / mdns / ssdp](#zeroconf)
+  * ☑ [event hooks](#event-hooks) / script runner
+  * ☑ [reverse-proxy support](https://github.com/9001/copyparty#reverse-proxy)
 * upload
   * ☑ basic: plain multipart, ie6 support
   * ☑ [up2k](#uploading): js, resumable, multithreaded
+    * unaffected by cloudflare's max-upload-size (100 MiB)
   * ☑ stash: simple PUT filedropper
+  * ☑ filename randomizer
+  * ☑ write-only folders
   * ☑ [unpost](#unpost): undo/delete accidental uploads
   * ☑ [self-destruct](#self-destruct) (specified server-side or client-side)
-  * ☑ symlink/discard existing files (content-matching)
+  * ☑ symlink/discard duplicates (content-matching)
 * download
   * ☑ single files in browser
   * ☑ [folders as zip / tar files](#zip-downloads)
@@ -178,7 +184,7 @@ recommended additional steps on debian  which enable audio metadata and thumbnai
 * browser
   * ☑ [navpane](#navpane) (directory tree sidebar)
   * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
-  * ☑ audio player (with OS media controls and opus transcoding)
+  * ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus transcoding)
   * ☑ image gallery with webm player
   * ☑ textfile browser with syntax hilighting
   * ☑ [thumbnails](#thumbnails)
@@ -191,16 +197,21 @@ recommended additional steps on debian  which enable audio metadata and thumbnai
   * ☑ [locate files by contents](#file-search)
   * ☑ search by name/path/date/size
   * ☑ [search by ID3-tags etc.](#searching)
+* client support
+  * ☑ [folder sync](#folder-sync)
+  * ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
 * markdown
   * ☑ [viewer](#markdown-viewer)
   * ☑ editor (sure why not)
 
+PS: something missing? post any crazy ideas you've got as a [feature request](https://github.com/9001/copyparty/issues/new?assignees=9001&labels=enhancement&template=feature_request.md) or [discussion](https://github.com/9001/copyparty/discussions/new?category=ideas) 🤙
+
 
 ## testimonials
 
 small collection of user feedback
 
-`good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`
+`good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`, `wow this is better than nextcloud`
 
 
 # motivations
@@ -209,8 +220,7 @@ project goals / philosophy
 
 * inverse linux philosophy -- do all the things, and do an *okay* job
   * quick drop-in service to get a lot of features in a pinch
-  * there are probably [better alternatives](https://github.com/awesome-selfhosted/awesome-selfhosted) if you have specific/long-term needs
-    * but the resumable multithreaded uploads are p slick ngl
+  * some of [the alternatives](./docs/versus.md) might be a better fit for you
 * run anywhere, support everything
   * as many web-browsers and python versions as possible
     * every browser should at least be able to browse, download, upload files
@@ -234,11 +244,14 @@ browser-specific:
 * Android-Chrome: increase "parallel uploads" for higher speed (android bug)
 * Android-Firefox: takes a while to select files (their fix for ☝️)
 * Desktop-Firefox: ~~may use gigabytes of RAM if your files are massive~~ *seems to be OK now*
-* Desktop-Firefox: may stop you from deleting files you've uploaded until you visit `about:memory` and click `Minimize memory usage`
+* Desktop-Firefox: [may stop you from unplugging USB flashdrives](https://bugzilla.mozilla.org/show_bug.cgi?id=1792598) until you visit `about:memory` and click `Minimize memory usage`
 
 server-os-specific:
 * RHEL8 / Rocky8: you can run copyparty using `/usr/libexec/platform-python`
 
+server notes:
+* pypy is supported but regular cpython is faster if you enable the database
+
 
 # bugs
 
@@ -252,27 +265,19 @@ server-os-specific:
 * Windows: if the `up2k.db` (filesystem index) is on a samba-share or network disk, you'll get unpredictable behavior if the share is disconnected for a bit
   * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db on a local disk instead
 * all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
-* [the database can get stuck](https://github.com/9001/copyparty/issues/10)
-  * has only happened once but that is once too many
-  * luckily not dangerous for file integrity and doesn't really stop uploads or anything like that
-  * but would really appreciate some logs if anyone ever runs into it again
 * probably more, pls let me know
 
 ## not my bugs
 
 * [Chrome issue 1317069](https://bugs.chromium.org/p/chromium/issues/detail?id=1317069) -- if you try to upload a folder which contains symlinks by dragging it into the browser, the symlinked files will not get uploaded
 
-* [Chrome issue 1354816](https://bugs.chromium.org/p/chromium/issues/detail?id=1354816) -- chrome may eat all RAM uploading over plaintext http with `mt` enabled
+* [Chrome issue 1352210](https://bugs.chromium.org/p/chromium/issues/detail?id=1352210) -- plaintext http may be faster at filehashing than https (but also extremely CPU-intensive)
 
-  * more amusingly, [Chrome issue 1354800](https://bugs.chromium.org/p/chromium/issues/detail?id=1354800) -- chrome may eat all RAM uploading in general (altho you probably won't run into this one)
-
-* [Chrome issue 1352210](https://bugs.chromium.org/p/chromium/issues/detail?id=1352210) -- plaintext http may be faster at filehashing than https (but also extremely CPU-intensive and likely to run into the above gc bugs)
-
-* [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- sometimes forgets to close filedescriptors during upload so the browser can crash after ~4000 files
+* [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files
 
 * iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
   * *future workaround:* enable the equalizer, make it all-zero, and set a negative boost to reduce the volume
-    * "future" because `AudioContext` is broken in the current iOS version (15.1), maybe one day...
+    * "future" because `AudioContext` can't maintain a stable playback speed in the current iOS version (15.7), maybe one day...
 
 * Windows: folders cannot be accessed if the name ends with `.`
   * python or windows bug
@@ -291,6 +296,9 @@ server-os-specific:
 
 upgrade notes
 
+* `1.6.0` (2023-01-29):
+  * http-api: delete/move is now `POST` instead of `GET`
+  * everything other than `GET` and `HEAD` must pass [cors validation](#cors)
 * `1.5.0` (2022-12-03): [new chunksize formula](https://github.com/9001/copyparty/commit/54e1c8d261df) for files larger than 128 GiB
   * **users:** upgrade to the latest [cli uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) if you use that
   * **devs:** update third-party up2k clients (if those even exist)
@@ -305,13 +313,14 @@ upgrade notes
   * you can also do this with linux filesystem permissions; `chmod 111 music` will make it possible to access files and folders inside the `music` folder but not list the immediate contents -- also works with other software, not just copyparty
 
 * can I make copyparty download a file to my server if I give it a URL?
-  * not really, but there is a [terrible hack](https://github.com/9001/copyparty/blob/hovudstraum/bin/mtag/wget.py) which makes it possible
+  * yes, using [hooks](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/wget.py)
 
 
 # accounts and volumes
 
 per-folder, per-user permissions  - if your setup is getting complex, consider making a [config file](./docs/example.conf) instead of using arguments
 * much easier to manage, and you can modify the config at runtime with `systemctl reload copyparty` or more conveniently using the `[reload cfg]` button in the control-panel (if logged in as admin)
+  * changes to the `[global]` config section requires a restart to take effect
 
 a quick summary can be seen using `--help-accounts`
 
@@ -516,11 +525,14 @@ up2k has several advantages:
 * much higher speeds than ftp/scp/tarpipe on some internet connections (mainly american ones) thanks to parallel connections
 * the last-modified timestamp of the file is preserved
 
+> it is perfectly safe to restart / upgrade copyparty while someone is uploading to it!  
+> all known up2k clients will resume just fine 💪
+
 see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
 
 ![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
 
-**protip:** you can avoid scaring away users with [contrib/plugins/minimal-up2k.html](contrib/plugins/minimal-up2k.html) which makes it look [much simpler](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
+**protip:** you can avoid scaring away users with [contrib/plugins/minimal-up2k.js](contrib/plugins/minimal-up2k.js) which makes it look [much simpler](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
 
 **protip:** if you enable `favicon` in the `[⚙️] settings` tab (by typing something into the textbox), the icon in the browser tab will indicate upload progress -- also, the `[🔔]` and/or `[🔊]` switches enable visible and/or audible notifications on upload completion
 
@@ -645,12 +657,63 @@ or a mix of both:
 the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
 
 
+## media player
+
+plays almost every audio format there is  (if the server has FFmpeg installed for on-demand transcoding)
+
+the following audio formats are usually always playable, even without FFmpeg: `aac|flac|m4a|mp3|ogg|opus|wav`
+
+some hilights:
+* OS integration; control playback from your phone's lockscreen ([windows](https://user-images.githubusercontent.com/241032/233213022-298a98ba-721a-4cf1-a3d4-f62634bc53d5.png) // [iOS](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) // [android](https://user-images.githubusercontent.com/241032/233212311-a7368590-08c7-4f9f-a1af-48ccf3f36fad.png))
+* shows the audio waveform in the seekbar
+* not perfectly gapless but can get really close (see settings + eq below); good enough to enjoy gapless albums as intended
+
+click the `play` link next to an audio file, or copy the link target to [share it](https://a.ocv.me/pub/demo/music/Ubiktune%20-%20SOUNDSHOCK%202%20-%20FM%20FUNK%20TERRROR!!/#af-1fbfba61&t=18) (optionally with a timestamp to start playing from, like that example does)
+
+open the `[🎺]` media-player-settings tab to configure it,
+* switches:
+  * `[preload]` starts loading the next track when it's about to end, reduces the silence between songs
+  * `[full]` does a full preload by downloading the entire next file; good for unreliable connections, bad for slow connections
+  * `[~s]` toggles the seekbar waveform display
+  * `[/np]` enables buttons to copy the now-playing info as an irc message
+  * `[os-ctl]` makes it possible to control audio playback from the lockscreen of your device (enables [mediasession](https://developer.mozilla.org/en-US/docs/Web/API/MediaSession))
+  * `[seek]` allows seeking with lockscreen controls (buggy on some devices)
+  * `[art]` shows album art on the lockscreen
+  * `[🎯]` keeps the playing song scrolled into view (good when using the player as a taskbar dock)
+  * `[⟎]` shrinks the playback controls
+* playback mode:
+  * `[loop]` keeps looping the folder
+  * `[next]` plays into the next folder
+* transcode:
+  * `[flac]` convers `flac` and `wav` files into opus
+  * `[aac]` converts `aac` and `m4a` files into opus
+  * `[oth]` converts all other known formats into opus
+    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
+* "tint" reduces the contrast of the playback bar
+
+
+### audio equalizer
+
+bass boosted
+
+can also boost the volume in general, or increase/decrease stereo width (like [crossfeed](https://www.foobar2000.org/components/view/foo_dsp_meiercf) just worse)
+
+has the convenient side-effect of reducing the pause between songs, so gapless albums play better with the eq enabled (just make it flat)
+
+
 ## markdown viewer
 
 and there are *two* editors
 
 ![copyparty-md-read-fs8](https://user-images.githubusercontent.com/241032/115978057-66419080-a57d-11eb-8539-d2be843991aa.png)
 
+there is a built-in extension for inline clickable thumbnails;
+* enable it by adding `<!-- th -->` somewhere in the doc
+* add thumbnails with `!th[l](your.jpg)` where `l` means left-align (`r` = right-align)
+* a single line with `---` clears the float / inlining
+* in the case of README.md being displayed below a file listing, thumbnails will open in the gallery viewer
+
+other notes,
 * the document preview has a max-width which is the same as an A4 paper when printed
 
 
@@ -696,11 +759,12 @@ using arguments or config files, or a mix of both:
 * config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf) and [./docs/example2.conf](docs/example2.conf)
 * `kill -s USR1` (same as `systemctl reload copyparty`) to reload accounts and volumes from config files without restarting
   * or click the `[reload cfg]` button in the control-panel when logged in as admin 
+  * changes to the `[global]` config section requires a restart to take effect
 
 
 ## zeroconf
 
-announce enabled services on the LAN  if you specify the `-z` option, which enables [mdns](#mdns) and [ssdp](#ssdp)
+announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png))  -- `-z` enables both [mdns](#mdns) and [ssdp](#ssdp)
 
 * `--z-on` / `--z-off`' limits the feature to certain networks
 
@@ -724,6 +788,10 @@ uses [ssdp](https://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol) to
 
 doubleclicking the icon opens the "connect" page which explains how to mount copyparty as a local filesystem
 
+if copyparty does not appear in windows explorer, use `--zsv` to see why:
+
+* maybe the discovery multicast was sent from an IP which does not intersect with the server subnets
+
 
 ## qr-code
 
@@ -750,6 +818,13 @@ an FTP server can be started using `--ftp 3921`,  and/or `--ftps` for explicit T
   * some older software (filezilla on debian-stable) cannot passive-mode with TLS
 * login with any username + your password, or put your password in the username field
 
+some recommended FTP / FTPS clients; `wark` = example password:
+* https://winscp.net/eng/download.php
+* https://filezilla-project.org/ struggles a bit with ftps in active-mode, but is fine otherwise
+* https://rclone.org/ does FTPS with `tls=false explicit_tls=true`
+* `lftp -u k,wark -p 3921 127.0.0.1 -e ls`
+* `lftp -u k,wark -p 3990 127.0.0.1 -e 'set ssl:verify-certificate no; ls'`
+
 
 ## webdav server
 
@@ -763,6 +838,8 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/
 
+in order to grant full write-access to webdav clients, the volflag `daw` must be set and the account must also have delete-access (otherwise the client won't be allowed to replace the contents of existing files, which is how webdav works)
+
 
 ### connecting to webdav from windows
 
@@ -802,7 +879,7 @@ some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
 
 and some minor issues,
 * clients only see the first ~400 files in big folders; [impacket#1433](https://github.com/SecureAuthCorp/impacket/issues/1433)
-* hot-reload of server config (`/?reload=cfg`) only works for volumes, not account passwords
+* hot-reload of server config (`/?reload=cfg`) does not include the `[global]` section (commandline args)
 * listens on the first IPv4 `-i` interface only (default = :: = 0.0.0.0 = all)
 * login doesn't work on winxp, but anonymous access is ok -- remove all accounts from copyparty config for that to work
   * win10 onwards does not allow connecting anonymously / without accounts
@@ -932,6 +1009,8 @@ some examples,
 ## other flags
 
 * `:c,magic` enables filetype detection for nameless uploads, same as `--magic`
+  * needs https://pypi.org/project/python-magic/ `python3 -m pip install --user -U python-magic`
+  * on windows grab this instead `python3 -m pip install --user -U python-magic-bin`
 
 
 ## database location
@@ -1000,9 +1079,18 @@ copyparty can invoke external programs to collect additional metadata for files
 if something doesn't work, try `--mtag-v` for verbose error messages
 
 
-## upload events
+## event hooks
 
-trigger a script/program on each upload  like so:
+trigger a program on uploads, renames etc ([examples](./bin/hooks/))
+
+you can set hooks before and/or after an event happens, and currently you can hook uploads, moves/renames, and deletes
+
+there's a bunch of flags and stuff, see `--help-hooks`
+
+
+### upload events
+
+the older, more powerful approach ([examples](./bin/mtag/)):
 
 ```
 -v /mnt/inc:inc:w:c,mte=+x1:c,mtp=x1=ad,kn,/usr/bin/notify-send
@@ -1012,11 +1100,12 @@ so filesystem location `/mnt/inc` shared at `/inc`, write-only for everyone, app
 
 that'll run the command `notify-send` with the path to the uploaded file as the first and only argument (so on linux it'll show a notification on-screen)
 
-note that it will only trigger on new unique files, not dupes
+note that this is way more complicated than the new [event hooks](#event-hooks) but this approach has the following advantages:
+* non-blocking and multithreaded; doesn't hold other uploads back
+* you get access to tags from FFmpeg and other mtp parsers
+* only trigger on new unique files, not dupes
 
-and it will occupy the parsing threads, so fork anything expensive (or set `kn` to have copyparty fork it for you) -- otoh if you want to intentionally queue/singlethread you can combine it with `--mtag-mt 1`
-
-if this becomes popular maybe there should be a less janky way to do it actually
+note that it will occupy the parsing threads, so fork anything expensive (or set `kn` to have copyparty fork it for you) -- otoh if you want to intentionally queue/singlethread you can combine it with `--mtag-mt 1`
 
 
 ## hiding from google
@@ -1055,6 +1144,8 @@ see the top of [./copyparty/web/browser.css](./copyparty/web/browser.css) where
 
 ## complete examples
 
+* [running on windows](./docs/examples/windows.md)
+
 * read-only music server  
   `python copyparty-sfx.py -v /mnt/nas/music:/music:r -e2dsa -e2ts --no-robots --force-js --theme 2`
   
@@ -1068,6 +1159,127 @@ see the top of [./copyparty/web/browser.css](./copyparty/web/browser.css) where
     `-lo log/cpp-%Y-%m%d-%H%M%S.txt.xz`
 
 
+## reverse-proxy
+
+running copyparty next to other websites  hosted on an existing webserver such as nginx, caddy, or apache
+
+you can either:
+* give copyparty its own domain or subdomain (recommended)
+* or do location-based proxying, using `--rp-loc=/stuff` to tell copyparty where it is mounted -- has a slight performance cost and higher chance of bugs
+  * if copyparty says `incorrect --rp-loc or webserver config; expected vpath starting with [...]` it's likely because the webserver is stripping away the proxy location from the request URLs -- see the `ProxyPass` in the apache example below
+
+some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which could be a nice speed boost
+
+example webserver configs:
+
+* [nginx config](contrib/nginx/copyparty.conf) -- entire domain/subdomain
+* [apache2 config](contrib/apache/copyparty.conf) -- location-based
+
+
+## nix package
+
+`nix profile install github:9001/copyparty`
+
+requires a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of nix
+
+some recommended dependencies are enabled by default; [override the package](https://github.com/9001/copyparty/blob/hovudstraum/contrib/package/nix/copyparty/default.nix#L3-L22) if you want to add/remove some features/deps
+
+`ffmpeg-full` was chosen over `ffmpeg-headless` mainly because we need `withWebp` (and `withOpenmpt` is also nice) and being able to use a cached build felt more important than optimizing for size at the time -- PRs welcome if you disagree 👍
+
+
+## nixos module
+
+for this setup, you will need a [flake-enabled](https://nixos.wiki/wiki/Flakes) installation of NixOS.
+
+```nix
+{
+  # add copyparty flake to your inputs
+  inputs.copyparty.url = "github:9001/copyparty";
+
+  # ensure that copyparty is an allowed argument to the outputs function
+  outputs = { self, nixpkgs, copyparty }: {
+    nixosConfigurations.yourHostName = nixpkgs.lib.nixosSystem {
+      modules = [
+        # load the copyparty NixOS module
+        copyparty.nixosModules.default
+        ({ pkgs, ... }: {
+          # add the copyparty overlay to expose the package to the module
+          nixpkgs.overlays = [ copyparty.overlays.default ];
+          # (optional) install the package globally
+          environment.systemPackages = [ pkgs.copyparty ];
+          # configure the copyparty module
+          services.copyparty.enable = true;
+        })
+      ];
+    };
+  };
+}
+```
+
+copyparty on NixOS is configured via `services.copyparty` options, for example:
+```nix
+services.copyparty = {
+  enable = true;
+  # directly maps to values in the [global] section of the copyparty config.
+  # see `copyparty --help` for available options
+  settings = {
+    i = "0.0.0.0";
+    # use lists to set multiple values
+    p = [ 3210 3211 ];
+    # use booleans to set binary flags
+    no-reload = true;
+    # using 'false' will do nothing and omit the value when generating a config
+    ignored-flag = false;
+  };
+
+  # create users
+  accounts = {
+    # specify the account name as the key
+    ed = {
+      # provide the path to a file containing the password, keeping it out of /nix/store
+      # must be readable by the copyparty service user
+      passwordFile = "/run/keys/copyparty/ed_password";
+    };
+    # or do both in one go
+    k.passwordFile = "/run/keys/copyparty/k_password";
+  };
+
+  # create a volume
+  volumes = {
+    # create a volume at "/" (the webroot), which will
+    "/" = {
+      # share the contents of "/srv/copyparty"
+      path = "/srv/copyparty";
+      # see `copyparty --help-accounts` for available options
+      access = {
+        # everyone gets read-access, but
+        r = "*";
+        # users "ed" and "k" get read-write
+        rw = [ "ed" "k" ];
+      };
+      # see `copyparty --help-flags` for available options
+      flags = {
+        # "fk" enables filekeys (necessary for upget permission) (4 chars long)
+        fk = 4;
+        # scan for new files every 60sec
+        scan = 60;
+        # volflag "e2d" enables the uploads database
+        e2d = true;
+        # "d2t" disables multimedia parsers (in case the uploads are malicious)
+        d2t = true;
+        # skips hashing file contents if path matches *.iso
+        nohash = "\.iso$";
+      };
+    };
+  };
+  # you may increase the open file limit for the process
+  openFilesLimit = 8192;
+};
+```
+
+the passwordFile at /run/keys/copyparty/ could for example be generated by [agenix](https://github.com/ryantm/agenix), or you could just dump it in the nix store instead if that's acceptable
+
+
 # browser support
 
 TLDR: yes
@@ -1127,11 +1339,11 @@ interact with copyparty using non-browser clients
 * curl/wget: upload some files (post=file, chunk=stdin)
   * `post(){ curl -F act=bput -F f=@"$1" http://127.0.0.1:3923/?pw=wark;}`  
     `post movie.mkv`
-  * `post(){ curl -b cppwd=wark -H rand:8 -T "$1" http://127.0.0.1:3923/;}`  
+  * `post(){ curl -H pw:wark -H rand:8 -T "$1" http://127.0.0.1:3923/;}`  
     `post movie.mkv`
-  * `post(){ wget --header='Cookie: cppwd=wark' --post-file="$1" -O- http://127.0.0.1:3923/?raw;}`  
+  * `post(){ wget --header='pw: wark' --post-file="$1" -O- http://127.0.0.1:3923/?raw;}`  
     `post movie.mkv`
-  * `chunk(){ curl -b cppwd=wark -T- http://127.0.0.1:3923/;}`  
+  * `chunk(){ curl -H pw:wark -T- http://127.0.0.1:3923/;}`  
     `chunk <movie.mkv`
 
 * bash: when curl and wget is not available or too boring
@@ -1139,7 +1351,7 @@ interact with copyparty using non-browser clients
   * `(printf 'PUT / HTTP/1.1\r\n\r\n'; cat movie.mkv) >/dev/tcp/127.0.0.1/3923`
 
 * python: [up2k.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) is a command-line up2k client [(webm)](https://ocv.me/stuff/u2cli.webm)
-  * file uploads, file-search, autoresume of aborted/broken uploads
+  * file uploads, file-search, [folder sync](#folder-sync), autoresume of aborted/broken uploads
   * can be downloaded from copyparty: controlpanel -> connect -> [up2k.py](http://127.0.0.1:3923/.cpr/a/up2k.py)
   * see [./bin/README.md#up2kpy](bin/README.md#up2kpy)
 
@@ -1155,22 +1367,32 @@ copyparty returns a truncated sha512sum of your PUT/POST as base64; you can gene
     b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
     b512 <movie.mkv
 
-you can provide passwords using cookie `cppwd=hunter2`, as a url-param `?pw=hunter2`, or with basic-authentication (either as the username or password)
+you can provide passwords using header `PW: hunter2`, cookie `cppwd=hunter2`, url-param `?pw=hunter2`, or with basic-authentication (either as the username or password)
 
 NOTE: curl will not send the original filename if you use `-T` combined with url-params! Also, make sure to always leave a trailing slash in URLs unless you want to override the filename
 
 
+## folder sync
+
+sync folders to/from copyparty
+
+the commandline uploader [up2k.py](https://github.com/9001/copyparty/tree/hovudstraum/bin#up2kpy) with `--dr` is the best way to sync a folder to copyparty; verifies checksums and does files in parallel, and deletes unexpected files on the server after upload has finished which makes file-renames really cheap (it'll rename serverside and skip uploading)
+
+alternatively there is [rclone](./docs/rclone.md) which allows for bidirectional sync and is *way* more flexible (stream files straight from sftp/s3/gcs to copyparty, ...), although there is no integrity check and it won't work with files over 100 MiB if copyparty is behind cloudflare
+
+* starting from rclone v1.63 (currently [in beta](https://beta.rclone.org/?filter=latest)), rclone will also be faster than up2k.py
+
+
 ## mount as drive
 
 a remote copyparty server as a local filesystem;  go to the control-panel and click `connect` to see a list of commands to do that
 
 alternatively, some alternatives roughly sorted by speed (unreproducible benchmark), best first:
 
-* [rclone-http](./docs/rclone.md) (25s), read-only
+* [rclone-webdav](./docs/rclone.md) (25s), read/WRITE ([v1.63-beta](https://beta.rclone.org/?filter=latest))
+* [rclone-http](./docs/rclone.md) (26s), read-only
+* [partyfuse.py](./bin/#partyfusepy) (35s), read-only
 * [rclone-ftp](./docs/rclone.md) (47s), read/WRITE
-* [rclone-webdav](./docs/rclone.md) (51s), read/WRITE
-  * copyparty-1.5.0's webdav server is faster than rclone-1.60.0 (69s)
-* [partyfuse.py](./bin/#partyfusepy) (71s), read-only
 * davfs2 (103s), read/WRITE, *very fast* on small files
 * [win10-webdav](#webdav-server) (138s), read/WRITE
 * [win10-smb2](#smb-server) (387s), read/WRITE
@@ -1178,6 +1400,27 @@ alternatively, some alternatives roughly sorted by speed (unreproducible benchma
 most clients will fail to mount the root of a copyparty server unless there is a root volume (so you get the admin-panel instead of a browser when accessing it) -- in that case, mount a specific volume instead
 
 
+# android app
+
+upload to copyparty with one tap
+
+<a href="https://f-droid.org/packages/me.ocv.partyup/"><img src="https://ocv.me/fdroid.png" alt="Get it on F-Droid" height="50" /> '' <img src="https://img.shields.io/f-droid/v/me.ocv.partyup.svg" alt="f-droid version info" /></a> '' <a href="https://github.com/9001/party-up"><img src="https://img.shields.io/github/release/9001/party-up.svg?logo=github" alt="github version info" /></a>
+
+the app is **NOT** the full copyparty server! just a basic upload client, nothing fancy yet
+
+if you want to run the copyparty server on your android device, see [install on android](#install-on-android)
+
+
+# iOS shortcuts
+
+there is no iPhone app, but  the following shortcuts are almost as good:
+
+* [upload to copyparty](https://www.icloud.com/shortcuts/41e98dd985cb4d3bb433222bc1e9e770) ([offline](https://github.com/9001/copyparty/raw/hovudstraum/contrib/ios/upload-to-copyparty.shortcut)) ([png](https://user-images.githubusercontent.com/241032/226118053-78623554-b0ed-482e-98e4-6d57ada58ea4.png)) based on the [original](https://www.icloud.com/shortcuts/ab415d5b4de3467b9ce6f151b439a5d7) by [Daedren](https://github.com/Daedren) (thx!)
+  * can strip exif, upload files, pics, vids, links, clipboard
+  * can download links and rehost the target file on copyparty (see first comment inside the shortcut)
+  * pics become lowres if you share from gallery to shortcut, so better to launch the shortcut and pick stuff from there
+
+
 # performance
 
 defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
@@ -1185,15 +1428,16 @@ defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
 below are some tweaks roughly ordered by usefulness:
 
 * `-q` disables logging and can help a bunch, even when combined with `-lo` to redirect logs to file
-* `--http-only` or `--https-only` (unless you want to support both protocols) will reduce the delay before a new connection is established
 * `--hist` pointing to a fast location (ssd) will make directory listings and searches faster when `-e2d` or `-e2t` is set
 * `--no-hash .` when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable
 * `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
-* `-j` enables multiprocessing (actual multithreading) and can make copyparty perform better in cpu-intensive workloads, for example:
-  * huge amount of short-lived connections
-  * really heavy traffic (downloads/uploads)
+* `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
+  * lots of connections (many users or heavy clients)
+  * simultaneous downloads and uploads saturating a 20gbps connection
   
   ...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
+* using [pypy](https://www.pypy.org/) instead of [cpython](https://www.python.org/) *can* be 70% faster for some workloads, but slower for many others
+  * and pypy can sometimes crash on startup with `-j0` (TODO make issue)
 
 
 ## client-side
@@ -1216,6 +1460,11 @@ when uploading files,
 
 some notes on hardening
 
+* set `--rproxy 0` if your copyparty is directly facing the internet (not through a reverse-proxy)
+  * cors doesn't work right otherwise
+
+safety profiles:
+
 * option `-s` is a shortcut to set the following options:
   * `--no-thumb` disables thumbnails and audio transcoding to stop copyparty from running `FFmpeg`/`Pillow`/`VIPS` on uploaded files, which is a [good idea](https://www.cvedetails.com/vulnerability-list.php?vendor_id=3611) if anonymous upload is enabled
   * `--no-mtag-ff` uses `mutagen` to grab music tags instead of `FFmpeg`, which is safer and faster but less accurate
@@ -1223,7 +1472,6 @@ some notes on hardening
   * `--no-robots` and `--force-js` makes life harder for crawlers, see [hiding from google](#hiding-from-google)
 
 * option `-ss` is a shortcut for the above plus:
-  * `--no-logues` and `--no-readme` disables support for readme's and prologues / epilogues in directory listings, which otherwise lets people upload arbitrary `<script>` tags
   * `--unpost 0`, `--no-del`, `--no-mv` disables all move/delete support
   * `--hardlink` creates hardlinks instead of symlinks when deduplicating uploads, which is less maintenance
     * however note if you edit one file it will also affect the other copies
@@ -1234,6 +1482,7 @@ some notes on hardening
 
 * option `-sss` is a shortcut for the above plus:
   * `--no-dav` disables webdav support
+  * `--no-logues` and `--no-readme` disables support for readme's and prologues / epilogues in directory listings, which otherwise lets people upload arbitrary (but sandboxed) `<script>` tags
   * `-lo cpp-%Y-%m%d-%H%M%S.txt.xz` enables logging to disk
   * `-ls **,*,ln,p,r` does a scan on startup for any dangerous symlinks
 
@@ -1241,6 +1490,7 @@ other misc notes:
 
 * you can disable directory listings by giving permission `g` instead of `r`, only accepting direct URLs to files
   * combine this with volflag `c,fk` to generate filekeys (per-file accesskeys); users which have full read-access will then see URLs with `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
+    * the default filekey entropy is fairly small so give `--fk-salt` around 30 characters if you want filekeys longer than 16 chars
   * permissions `wG` lets users upload files and receive their own filekeys, still without being able to see other uploads
 
 
@@ -1249,6 +1499,29 @@ other misc notes:
 behavior that might be unexpected
 
 * users without read-access to a folder can still see the `.prologue.html` / `.epilogue.html` / `README.md` contents, for the purpose of showing a description on how to use the uploader for example
+* users can submit `<script>`s which autorun for other visitors in a few ways;
+  * uploading a `README.md` -- avoid with `--no-readme`
+  * renaming `some.html` to `.epilogue.html` -- avoid with either `--no-logues` or `--no-dot-ren`
+  * the directory-listing embed is sandboxed (so any malicious scripts can't do any damage) but the markdown editor is not
+
+
+## cors
+
+cross-site request config
+
+by default, except for `GET` and `HEAD` operations, all requests must either:
+* not contain an `Origin` header at all
+* or have an `Origin` matching the server domain
+* or the header `PW` with your password as value
+
+cors can be configured with `--acao` and `--acam`, or the protections entirely disabled with `--allow-csrf`
+
+
+## https
+
+both HTTP and HTTPS are accepted  by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
+
+copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well
 
 
 # recovering from crashes
@@ -1273,7 +1546,7 @@ however you can hit `F12` in the up2k tab and use the devtools to see how far yo
 
 # HTTP API
 
-see [devnotes](#./docs/devnotes.md#http-api)
+see [devnotes](./docs/devnotes.md#http-api)
 
 
 # dependencies
@@ -1301,18 +1574,12 @@ enable [thumbnails](#thumbnails) of...
 * **AVIF pictures:** `pyvips` or `ffmpeg` or `pillow-avif-plugin`
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`
 
-enable [smb](#smb-server) support:
+enable [smb](#smb-server) support (**not** recommended):
 * `impacket==0.10.0`
 
 `pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
 
 
-## install recommended deps
-```
-python -m pip install --user -U jinja2 mutagen Pillow
-```
-
-
 ## optional gpl stuff
 
 some bundled tools have copyleft dependencies, see [./bin/#mtag](bin/#mtag)
@@ -1324,20 +1591,25 @@ these are standalone programs and will never be imported / evaluated by copypart
 
 the self-contained "binary"  [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
 
-you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](#./docs/devnotes.md#sfx-repack)
+you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
 
 
 ## copyparty.exe
 
-download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) or [copyparty64.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty64.exe)
+download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
 
-![copyparty-exe-fs8](https://user-images.githubusercontent.com/241032/194707422-cb7f66c9-41a2-4cb9-8dbc-2ab866cd4338.png)
+![copyparty-exe-fs8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png)
 
-can be convenient on old machines where installing python is problematic, however is **not recommended** and should be considered a last resort -- if possible, please use **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** instead
+can be convenient on machines where installing python is problematic, however is **not recommended** -- if possible, please use **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** instead
 
-* [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) is compatible with 32bit windows7, which means it uses an ancient copy of python (3.7.9) which cannot be upgraded and will definitely become a security hazard at some point
+* [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) runs on win8 or newer, was compiled on win10, does thumbnails + media tags, and is *currently* safe to use, but any future python/expat/pillow CVEs can only be remedied by downloading a newer version of the exe
 
-* [copyparty64.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty64.exe) is identical except 64bit so it [works in WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png)
+  * on win8 it needs [vc redist 2015](https://www.microsoft.com/en-us/download/details.aspx?id=48145), on win10 it just works
+  * some antivirus may freak out (false-positive), possibly [Avast, AVG, and McAfee](https://www.virustotal.com/gui/file/52391a1e9842cf70ad243ef83844d46d29c0044d101ee0138fcdd3c8de2237d6/detection)
+
+* dangerous: [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) is compatible with [windows7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png), which means it uses an ancient copy of python (3.7.9) which cannot be upgraded and should never be exposed to the internet (LAN is fine)
+
+* dangerous and deprecated: [copyparty-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.6.8/copyparty-winpe64.exe) lets you [run copyparty in WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png) and is otherwise completely useless
 
 meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) instead relies on your system python which gives better performance and will stay safe as long as you keep your python install up-to-date
 

SECURITY.md

@@ -0,0 +1,9 @@
+# Security Policy
+
+if you hit something extra juicy pls let me know on either of the following
+* email -- `copyparty@ocv.ze` except `ze` should be `me`
+* [mastodon dm](https://layer8.space/@tripflag) -- `@tripflag@layer8.space`
+* [github private vulnerability report](https://github.com/9001/copyparty/security/advisories/new), wow that form is complicated
+* [twitter dm](https://twitter.com/tripflag) (if im somehow not banned yet)
+
+no bug bounties sorry! all i can offer is greetz in the release notes

bin/README.md

@@ -1,7 +1,8 @@
 # [`up2k.py`](up2k.py)
 * command-line up2k client [(webm)](https://ocv.me/stuff/u2cli.webm)
 * file uploads, file-search, autoresume of aborted/broken uploads
-* faster than browsers
+* sync local folder to server
+* generally faster than browsers
 * if something breaks just restart it
 
 

bin/hooks/README.md

@@ -0,0 +1,29 @@
+standalone programs which are executed by copyparty when an event happens (upload, file rename, delete, ...)
+
+these programs either take zero arguments, or a filepath (the affected file), or a json message with filepath + additional info
+
+run copyparty with `--help-hooks` for usage details / hook type explanations (xbu/xau/xiu/xbr/xar/xbd/xad)
+
+> **note:** in addition to event hooks (the stuff described here), copyparty has another api to run your programs/scripts while providing way more information such as audio tags / video codecs / etc and optionally daisychaining data between scripts in a processing pipeline; if that's what you want then see [mtp plugins](../mtag/) instead
+
+
+# after upload
+* [notify.py](notify.py) shows a desktop notification ([example](https://user-images.githubusercontent.com/241032/215335767-9c91ed24-d36e-4b6b-9766-fb95d12d163f.png))
+  * [notify2.py](notify2.py) uses the json API to show more context
+* [image-noexif.py](image-noexif.py) removes image exif by overwriting / directly editing the uploaded file
+* [discord-announce.py](discord-announce.py) announces new uploads on discord using webhooks ([example](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png))
+* [reject-mimetype.py](reject-mimetype.py) rejects uploads unless the mimetype is acceptable
+
+
+# upload batches
+these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every single file), `xiu` hooks are given a list of recent uploads on STDIN after the server has gone idle for N seconds, reducing server load + providing more context
+* [xiu.py](xiu.py) is a "minimal" example showing a list of filenames + total filesize
+* [xiu-sha.py](xiu-sha.py) produces a sha512 checksum list in the volume root
+
+
+# before upload
+* [reject-extension.py](reject-extension.py) rejects uploads if they match a list of file extensions
+
+
+# on message
+* [wget.py](wget.py) lets you download files by POSTing URLs to copyparty

bin/hooks/discord-announce.py

@@ -0,0 +1,68 @@
+#!/usr/bin/env python3
+
+import sys
+import json
+import requests
+from copyparty.util import humansize, quotep
+
+
+_ = r"""
+announces a new upload on discord
+
+example usage as global config:
+    --xau f,t5,j,bin/hooks/discord-announce.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xau=f,t5,j,bin/hooks/discord-announce.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params listed below)
+
+parameters explained,
+    xbu = execute after upload
+    f  = fork; don't wait for it to finish
+    t5 = timeout if it's still running after 5 sec
+    j  = provide upload information as json; not just the filename
+
+replace "xau" with "xbu" to announce Before upload starts instead of After completion
+
+# how to discord:
+first create the webhook url; https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks
+then use this to design your message: https://discohook.org/
+"""
+
+
+def main():
+    WEBHOOK = "https://discord.com/api/webhooks/1234/base64"
+    WEBHOOK = "https://discord.com/api/webhooks/1066830390280597718/M1TDD110hQA-meRLMRhdurych8iyG35LDoI1YhzbrjGP--BXNZodZFczNVwK4Ce7Yme5"
+
+    # read info from copyparty
+    inf = json.loads(sys.argv[1])
+    vpath = inf["vp"]
+    filename = vpath.split("/")[-1]
+    url = f"https://{inf['host']}/{quotep(vpath)}"
+
+    # compose the message to discord
+    j = {
+        "title": filename,
+        "url": url,
+        "description": url.rsplit("/", 1)[0],
+        "color": 0x449900,
+        "fields": [
+            {"name": "Size", "value": humansize(inf["sz"])},
+            {"name": "User", "value": inf["user"]},
+            {"name": "IP", "value": inf["ip"]},
+        ],
+    }
+
+    for v in j["fields"]:
+        v["inline"] = True
+
+    r = requests.post(WEBHOOK, json={"embeds": [j]})
+    print(f"discord: {r}\n", end="")
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/image-noexif.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+import subprocess as sp
+
+
+_ = r"""
+remove exif tags from uploaded images; the eventhook edition of
+https://github.com/9001/copyparty/blob/hovudstraum/bin/mtag/image-noexif.py
+
+dependencies:
+    exiftool / perl-Image-ExifTool
+
+being an upload hook, this will take effect after upload completion
+    but before copyparty has hashed/indexed the file, which means that
+    copyparty will never index the original file, so deduplication will
+    not work as expected... which is mostly OK but ehhh
+
+note: modifies the file in-place, so don't set the `f` (fork) flag
+
+example usages; either as global config (all volumes) or as volflag:
+    --xau bin/hooks/image-noexif.py
+    -v srv/inc:inc:r:rw,ed:c,xau=bin/hooks/image-noexif.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+explained:
+    share fs-path srv/inc at /inc (readable by all, read-write for user ed)
+    running this xau (execute-after-upload) plugin for all uploaded files
+"""
+
+
+# filetypes to process; ignores everything else
+EXTS = ("jpg", "jpeg", "avif", "heif", "heic")
+
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p.encode("utf-8")
+
+
+def main():
+    fp = sys.argv[1]
+    ext = fp.lower().split(".")[-1]
+    if ext not in EXTS:
+        return
+
+    cwd, fn = os.path.split(fp)
+    os.chdir(cwd)
+    f1 = fsenc(fn)
+    cmd = [
+        b"exiftool",
+        b"-exif:all=",
+        b"-iptc:all=",
+        b"-xmp:all=",
+        b"-P",
+        b"-overwrite_original",
+        b"--",
+        f1,
+    ]
+    sp.check_output(cmd)
+    print("image-noexif: stripped")
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except:
+        pass

bin/hooks/notify.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+import subprocess as sp
+from plyer import notification
+
+
+_ = r"""
+show os notification on upload; works on windows, linux, macos, android
+
+depdencies:
+    windows: python3 -m pip install --user -U plyer
+    linux:   python3 -m pip install --user -U plyer
+    macos:   python3 -m pip install --user -U plyer pyobjus
+    android: just termux and termux-api
+
+example usages; either as global config (all volumes) or as volflag:
+    --xau f,bin/hooks/notify.py
+    -v srv/inc:inc:r:rw,ed:c,xau=f,bin/hooks/notify.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params listed below)
+
+parameters explained,
+    xau = execute after upload
+    f   = fork so it doesn't block uploads
+"""
+
+
+try:
+    from copyparty.util import humansize
+except:
+
+    def humansize(n):
+        return n
+
+
+def main():
+    fp = sys.argv[1]
+    dp, fn = os.path.split(fp)
+    try:
+        sz = humansize(os.path.getsize(fp))
+    except:
+        sz = "?"
+
+    msg = "{} ({})\n📁 {}".format(fn, sz, dp)
+    title = "File received"
+
+    if "com.termux" in sys.executable:
+        sp.run(["termux-notification", "-t", title, "-c", msg])
+        return
+
+    icon = "emblem-documents-symbolic" if sys.platform == "linux" else ""
+    notification.notify(
+        title=title,
+        message=msg,
+        app_icon=icon,
+        timeout=10,
+    )
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/notify2.py

@@ -0,0 +1,72 @@
+#!/usr/bin/env python3
+
+import json
+import os
+import sys
+import subprocess as sp
+from datetime import datetime
+from plyer import notification
+
+
+_ = r"""
+same as notify.py but with additional info (uploader, ...)
+and also supports --xm (notify on 📟 message)
+
+example usages; either as global config (all volumes) or as volflag:
+    --xm  f,j,bin/hooks/notify2.py
+    --xau f,j,bin/hooks/notify2.py
+    -v srv/inc:inc:r:rw,ed:c,xm=f,j,bin/hooks/notify2.py
+    -v srv/inc:inc:r:rw,ed:c,xau=f,j,bin/hooks/notify2.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads / msgs with the params listed below)
+
+parameters explained,
+    xau = execute after upload
+    f   = fork so it doesn't block uploads
+    j   = provide json instead of filepath list
+"""
+
+
+try:
+    from copyparty.util import humansize
+except:
+
+    def humansize(n):
+        return n
+
+
+def main():
+    inf = json.loads(sys.argv[1])
+    fp = inf["ap"]
+    sz = humansize(inf["sz"])
+    dp, fn = os.path.split(fp)
+    mt = datetime.utcfromtimestamp(inf["mt"]).strftime("%Y-%m-%d %H:%M:%S")
+
+    msg = f"{fn} ({sz})\n📁 {dp}"
+    title = "File received"
+    icon = "emblem-documents-symbolic" if sys.platform == "linux" else ""
+
+    if inf.get("txt"):
+        msg = inf["txt"]
+        title = "Message received"
+        icon = "mail-unread-symbolic" if sys.platform == "linux" else ""
+
+    msg += f"\n👤 {inf['user']} ({inf['ip']})\n🕒 {mt}"
+
+    if "com.termux" in sys.executable:
+        sp.run(["termux-notification", "-t", title, "-c", msg])
+        return
+
+    notification.notify(
+        title=title,
+        message=msg,
+        app_icon=icon,
+        timeout=10,
+    )
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/reject-extension.py

@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+
+import sys
+
+
+_ = r"""
+reject file uploads by file extension
+
+example usage as global config:
+    --xbu c,bin/hooks/reject-extension.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xbu=c,bin/hooks/reject-extension.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params listed below)
+
+parameters explained,
+    xbu = execute before upload
+    c   = check result, reject upload if error
+"""
+
+
+def main():
+    bad = "exe scr com pif bat ps1 jar msi"
+
+    ext = sys.argv[1].split(".")[-1]
+
+    sys.exit(1 if ext in bad.split() else 0)
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/reject-mimetype.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+
+import sys
+import magic
+
+
+_ = r"""
+reject file uploads by mimetype
+
+dependencies (linux, macos):
+    python3 -m pip install --user -U python-magic
+
+dependencies (windows):
+    python3 -m pip install --user -U python-magic-bin
+
+example usage as global config:
+    --xau c,bin/hooks/reject-mimetype.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xau=c,bin/hooks/reject-mimetype.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all uploads with the params listed below)
+
+parameters explained,
+    xau = execute after upload
+    c   = check result, reject upload if error
+"""
+
+
+def main():
+    ok = ["image/jpeg", "image/png"]
+
+    mt = magic.from_file(sys.argv[1], mime=True)
+
+    print(mt)
+
+    sys.exit(1 if mt not in ok else 0)
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/wget.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+import json
+import subprocess as sp
+
+
+_ = r"""
+use copyparty as a file downloader by POSTing URLs as
+application/x-www-form-urlencoded (for example using the
+message/pager function on the website)
+
+example usage as global config:
+    --xm f,j,t3600,bin/hooks/wget.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xm=f,j,t3600,bin/hooks/wget.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on all messages with the params listed below)
+
+parameters explained,
+    xm = execute on message-to-server-log
+    f = fork so it doesn't block uploads
+    j = provide message information as json; not just the text
+    c3 = mute all output
+    t3600 = timeout and kill download after 1 hour
+"""
+
+
+def main():
+    inf = json.loads(sys.argv[1])
+    url = inf["txt"]
+    if "://" not in url:
+        url = "https://" + url
+
+    os.chdir(inf["ap"])
+
+    name = url.split("?")[0].split("/")[-1]
+    tfn = "-- DOWNLOADING " + name
+    print(f"{tfn}\n", end="")
+    open(tfn, "wb").close()
+
+    cmd = ["wget", "--trust-server-names", "-nv", "--", url]
+
+    try:
+        sp.check_call(cmd)
+    except:
+        t = "-- FAILED TO DONWLOAD " + name
+        print(f"{t}\n", end="")
+        open(t, "wb").close()
+
+    os.unlink(tfn)
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/xiu-sha.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+
+import hashlib
+import json
+import sys
+from datetime import datetime
+
+
+_ = r"""
+this hook will produce a single sha512 file which
+covers all recent uploads (plus metadata comments)
+
+use this with --xiu, which makes copyparty buffer
+uploads until server is idle, providing file infos
+on stdin (filepaths or json)
+
+example usage as global config:
+    --xiu i5,j,bin/hooks/xiu-sha.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xiu=i5,j,bin/hooks/xiu-sha.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on batches of uploads with the params listed below)
+
+parameters explained,
+    xiu = execute after uploads...
+    i5  = ...after volume has been idle for 5sec
+    j   = provide json instead of filepath list
+
+note the "f" (fork) flag is not set, so this xiu
+will block other xiu hooks while it's running
+"""
+
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p
+
+
+def humantime(ts):
+    return datetime.utcfromtimestamp(ts).strftime("%Y-%m-%d %H:%M:%S")
+
+
+def find_files_root(inf):
+    di = 9000
+    for f1, f2 in zip(inf, inf[1:]):
+        p1 = f1["ap"].replace("\\", "/").rsplit("/", 1)[0]
+        p2 = f2["ap"].replace("\\", "/").rsplit("/", 1)[0]
+        di = min(len(p1), len(p2), di)
+        di = next((i for i in range(di) if p1[i] != p2[i]), di)
+
+    return di + 1
+
+
+def find_vol_root(inf):
+    return len(inf[0]["ap"][: -len(inf[0]["vp"])])
+
+
+def main():
+    zb = sys.stdin.buffer.read()
+    zs = zb.decode("utf-8", "replace")
+    inf = json.loads(zs)
+
+    # root directory (where to put the sha512 file);
+    # di = find_files_root(inf)  # next to the file closest to volume root
+    di = find_vol_root(inf)  # top of the entire volume
+
+    ret = []
+    total_sz = 0
+    for md in inf:
+        ap = md["ap"]
+        rp = ap[di:]
+        total_sz += md["sz"]
+        fsize = "{:,}".format(md["sz"])
+        mtime = humantime(md["mt"])
+        up_ts = humantime(md["at"])
+
+        h = hashlib.sha512()
+        with open(fsenc(md["ap"]), "rb", 512 * 1024) as f:
+            while True:
+                buf = f.read(512 * 1024)
+                if not buf:
+                    break
+
+                h.update(buf)
+
+        cksum = h.hexdigest()
+        meta = " | ".join([md["wark"], up_ts, mtime, fsize, md["ip"]])
+        ret.append("# {}\n{} *{}".format(meta, cksum, rp))
+
+    ret.append("# {} files, {} bytes total".format(len(inf), total_sz))
+    ret.append("")
+    ftime = datetime.utcnow().strftime("%Y-%m%d-%H%M%S.%f")
+    fp = "{}xfer-{}.sha512".format(inf[0]["ap"][:di], ftime)
+    with open(fsenc(fp), "wb") as f:
+        f.write("\n".join(ret).encode("utf-8", "replace"))
+
+    print("wrote checksums to {}".format(fp))
+
+
+if __name__ == "__main__":
+    main()

bin/hooks/xiu.py

@@ -0,0 +1,50 @@
+#!/usr/bin/env python3
+
+import json
+import sys
+
+
+_ = r"""
+this hook prints absolute filepaths + total size
+
+use this with --xiu, which makes copyparty buffer
+uploads until server is idle, providing file infos
+on stdin (filepaths or json)
+
+example usage as global config:
+    --xiu i1,j,bin/hooks/xiu.py
+
+example usage as a volflag (per-volume config):
+    -v srv/inc:inc:r:rw,ed:c,xiu=i1,j,bin/hooks/xiu.py
+                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/inc as volume /inc,
+     readable by everyone, read-write for user 'ed',
+     running this plugin on batches of uploads with the params listed below)
+
+parameters explained,
+    xiu = execute after uploads...
+    i1  = ...after volume has been idle for 1sec
+    j   = provide json instead of filepath list
+
+note the "f" (fork) flag is not set, so this xiu
+will block other xiu hooks while it's running
+"""
+
+
+def main():
+    zb = sys.stdin.buffer.read()
+    zs = zb.decode("utf-8", "replace")
+    inf = json.loads(zs)
+
+    total_sz = 0
+    for upload in inf:
+        sz = upload["sz"]
+        total_sz += sz
+        print("{:9} {}".format(sz, upload["ap"]))
+
+    print("{} files, {} bytes total".format(len(inf), total_sz))
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/README.md

@@ -1,5 +1,9 @@
 standalone programs which take an audio file as argument
 
+you may want to forget about all this fancy complicated stuff and just use [event hooks](../hooks/) instead (which doesn't need `-e2ts` or ffmpeg) 
+
+----
+
 **NOTE:** these all require `-e2ts` to be functional, meaning you need to do at least one of these: `apt install ffmpeg` or `pip3 install mutagen`
 
 some of these rely on libraries which are not MIT-compatible
@@ -17,6 +21,7 @@ these do not have any problematic dependencies at all:
 * [cksum.py](./cksum.py) computes various checksums
 * [exe.py](./exe.py) grabs metadata from .exe and .dll files (example for retrieving multiple tags with one parser)
 * [wget.py](./wget.py) lets you download files by POSTing URLs to copyparty
+  * also available as an [event hook](../hooks/wget.py)
 
 
 # dependencies
@@ -26,7 +31,7 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 *alternatively* (or preferably) use packages from your distro instead, then you'll need at least these:
 
 * from distro: `numpy vamp-plugin-sdk beatroot-vamp mixxx-keyfinder ffmpeg`
-* from pypy: `keyfinder vamp`
+* from pip: `keyfinder vamp`
 
 
 # usage from copyparty

bin/mtag/audio-bpm.py

@@ -16,6 +16,10 @@ dep: ffmpeg
 """
 
 
+# save beat timestamps to ".beats/filename.txt"
+SAVE = False
+
+
 def det(tf):
     # fmt: off
     sp.check_call([
@@ -23,12 +27,11 @@ def det(tf):
         b"-nostdin",
         b"-hide_banner",
         b"-v", b"fatal",
-        b"-ss", b"13",
         b"-y", b"-i", fsenc(sys.argv[1]),
         b"-map", b"0:a:0",
         b"-ac", b"1",
         b"-ar", b"22050",
-        b"-t", b"300",
+        b"-t", b"360",
         b"-f", b"f32le",
         fsenc(tf)
     ])
@@ -47,10 +50,29 @@ def det(tf):
             print(c["list"][0]["label"].split(" ")[0])
             return
 
-        # throws if detection failed:
-        bpm = float(cl[-1]["timestamp"] - cl[1]["timestamp"])
-        bpm = round(60 * ((len(cl) - 1) / bpm), 2)
-        print(f"{bpm:.2f}")
+    # throws if detection failed:
+    beats = [float(x["timestamp"]) for x in cl]
+    bds = [b - a for a, b in zip(beats, beats[1:])]
+    bds.sort()
+    n0 = int(len(bds) * 0.2)
+    n1 = int(len(bds) * 0.75) + 1
+    bds = bds[n0:n1]
+    bpm = sum(bds)
+    bpm = round(60 * (len(bds) / bpm), 2)
+    print(f"{bpm:.2f}")
+
+    if SAVE:
+        fdir, fname = os.path.split(sys.argv[1])
+        bdir = os.path.join(fdir, ".beats")
+        try:
+            os.mkdir(fsenc(bdir))
+        except:
+            pass
+
+        fp = os.path.join(bdir, fname) + ".txt"
+        with open(fsenc(fp), "wb") as f:
+            txt = "\n".join([f"{x:.2f}" for x in beats])
+            f.write(txt.encode("utf-8"))
 
 
 def main():

bin/mtag/image-noexif.py

@@ -61,7 +61,7 @@ def main():
 
     os.chdir(cwd)
     f1 = fsenc(fn)
-    f2 = os.path.join(b"noexif", f1)
+    f2 = fsenc(os.path.join(b"noexif", fn))
     cmd = [
         b"exiftool",
         b"-exif:all=",

bin/mtag/install-deps.sh

@@ -57,6 +57,7 @@ hash -r
 	command -v python3 && pybin=python3 || pybin=python
 }
 
+$pybin -c 'import numpy' ||
 $pybin -m pip install --user numpy
 
 
@@ -224,7 +225,7 @@ install_vamp() {
 	$pybin -m pip install --user vamp
 
 	cd "$td"
-	echo '#include <vamp-sdk/Plugin.h>' | gcc -x c -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
+	echo '#include <vamp-sdk/Plugin.h>' | g++ -x c++ -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
 		printf '\033[33mcould not find the vamp-sdk, building from source\033[0m\n'
 		(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/2588/vamp-plugin-sdk-2.9.0.tar.gz)
 		sha512sum -c <(

bin/mtag/wget.py

@@ -1,6 +1,11 @@
 #!/usr/bin/env python3
 
 """
+DEPRECATED -- replaced by event hooks;
+https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/wget.py
+
+---
+
 use copyparty as a file downloader by POSTing URLs as
 application/x-www-form-urlencoded (for example using the
 message/pager function on the website)

bin/partyfuse.py

@@ -997,7 +997,7 @@ def main():
     ap.add_argument(
         "-cf", metavar="NUM_BLOCKS", type=int, default=nf, help="file cache"
     )
-    ap.add_argument("-a", metavar="PASSWORD", help="password")
+    ap.add_argument("-a", metavar="PASSWORD", help="password or $filepath")
     ap.add_argument("-d", action="store_true", help="enable debug")
     ap.add_argument("-te", metavar="PEM_FILE", help="certificate to expect/verify")
     ap.add_argument("-td", action="store_true", help="disable certificate check")

bin/prisonparty.sh

@@ -4,8 +4,9 @@ set -e
 # runs copyparty (or any other program really) in a chroot
 #
 # assumption: these directories, and everything within, are owned by root
-sysdirs=( /bin /lib /lib32 /lib64 /sbin /usr )
-
+sysdirs=(); for v in /bin /lib /lib32 /lib64 /sbin /usr /etc/alternatives ; do
+	[ -e $v ] && sysdirs+=($v)
+done
 
 # error-handler
 help() { cat <<'EOF'
@@ -38,7 +39,7 @@ while true; do
 	v="$1"; shift
 	[ "$v" = -- ] && break  # end of volumes
 	[ "$#" -eq 0 ] && break  # invalid usage
-	vols+=( "$(realpath "$v")" )
+	vols+=( "$(realpath "$v" || echo "$v")" )
 done
 pybin="$1"; shift
 pybin="$(command -v "$pybin")"
@@ -82,7 +83,7 @@ jail="${jail%/}"
 printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | sed -r 's`/$``' | LC_ALL=C sort | uniq |
 while IFS= read -r v; do
 	[ -e "$v" ] || {
-		# printf '\033[1;31mfolder does not exist:\033[0m %s\n' "/$v"
+		printf '\033[1;31mfolder does not exist:\033[0m %s\n' "$v"
 		continue
 	}
 	i1=$(stat -c%D.%i "$v"      2>/dev/null || echo a)
@@ -97,9 +98,11 @@ done
 
 cln() {
 	rv=$?
-	# cleanup if not in use
-	lsof "$jail" | grep -qF "$jail" &&
-		echo "chroot is in use, will not cleanup" ||
+	wait -f -p rv $p || true
+	cd /
+	echo "stopping chroot..."
+	lsof "$jail" | grep -F "$jail" &&
+		echo "chroot is in use; will not unmount" ||
 	{
 		mount | grep -F " on $jail" |
 		awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' |
@@ -115,6 +118,15 @@ mkdir -p "$jail/tmp"
 chmod 777 "$jail/tmp"
 
 
+# create a dev
+(cd $jail; mkdir -p dev; cd dev
+[ -e null ]    || mknod -m 666 null    c 1 3
+[ -e zero ]    || mknod -m 666 zero    c 1 5
+[ -e random ]  || mknod -m 444 random  c 1 8
+[ -e urandom ] || mknod -m 444 urandom c 1 9
+)
+
+
 # run copyparty
 export HOME=$(getent passwd $uid | cut -d: -f6)
 export USER=$(getent passwd $uid | cut -d: -f1)
@@ -124,5 +136,6 @@ export LOGNAME="$USER"
 #echo "cpp [$cpp]"
 chroot --userspec=$uid:$gid "$jail" "$pybin" $pyarg "$cpp" "$@" &
 p=$!
+trap 'kill -USR1 $p' USR1
 trap 'kill $p' INT TERM
 wait

bin/up2k.py

@@ -1,16 +1,17 @@
 #!/usr/bin/env python3
 from __future__ import print_function, unicode_literals
 
+S_VERSION = "1.6"
+S_BUILD_DT = "2023-04-20"
+
 """
 up2k.py: upload to copyparty
-2022-11-29, v0.22, ed <irc.rizon.net>, MIT-Licensed
+2021, ed <irc.rizon.net>, MIT-Licensed
 https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py
 
 - dependencies: requests
-- supports python 2.6, 2.7, and 3.3 through 3.11
-
-- almost zero error-handling
-- but if something breaks just try again and it'll autoresume
+- supports python 2.6, 2.7, and 3.3 through 3.12
+- if something breaks just try again and it'll autoresume
 """
 
 import os
@@ -26,6 +27,8 @@ import platform
 import threading
 import datetime
 
+EXE = sys.executable.endswith("exe")
+
 try:
     import argparse
 except:
@@ -36,12 +39,15 @@ except:
 try:
     import requests
 except ImportError:
-    if sys.version_info > (2, 7):
+    if EXE:
+        raise
+    elif sys.version_info > (2, 7):
         m = "\nERROR: need 'requests'; please run this command:\n {0} -m pip install --user requests\n"
     else:
         m = "requests/2.18.4 urllib3/1.23 chardet/3.0.4 certifi/2020.4.5.1 idna/2.7"
         m = ["   https://pypi.org/project/" + x + "/#files" for x in m.split()]
         m = "\n  ERROR: need these:\n" + "\n".join(m) + "\n"
+        m += "\n  for f in *.whl; do unzip $f; done; rm -r *.dist-info\n"
 
     print(m.format(sys.executable))
     sys.exit(1)
@@ -246,7 +252,13 @@ def eprint(*a, **ka):
 
 
 def flushing_print(*a, **ka):
-    _print(*a, **ka)
+    try:
+        _print(*a, **ka)
+    except:
+        v = " ".join(str(x) for x in a)
+        v = v.encode("ascii", "replace").decode("ascii")
+        _print(v, **ka)
+
     if "flush" not in ka:
         sys.stdout.flush()
 
@@ -262,10 +274,10 @@ def termsize():
         try:
             import fcntl, termios, struct
 
-            cr = struct.unpack("hh", fcntl.ioctl(fd, termios.TIOCGWINSZ, "1234"))
+            r = struct.unpack(b"hh", fcntl.ioctl(fd, termios.TIOCGWINSZ, b"AAAA"))
+            return r[::-1]
         except:
-            return
-        return cr
+            return None
 
     cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
     if not cr:
@@ -275,12 +287,11 @@ def termsize():
             os.close(fd)
         except:
             pass
-    if not cr:
-        try:
-            cr = (env["LINES"], env["COLUMNS"])
-        except:
-            cr = (25, 80)
-    return int(cr[1]), int(cr[0])
+
+    try:
+        return cr or (int(env["COLUMNS"]), int(env["LINES"]))
+    except:
+        return 80, 25
 
 
 class CTermsize(object):
@@ -362,26 +373,46 @@ def walkdir(err, top, seen):
 
     seen = seen[:] + [atop]
     for ap, inf in sorted(statdir(err, top)):
+        yield ap, inf
         if stat.S_ISDIR(inf.st_mode):
             try:
                 for x in walkdir(err, ap, seen):
                     yield x
             except Exception as ex:
                 err.append((ap, str(ex)))
-        else:
-            yield ap, inf
 
 
 def walkdirs(err, tops):
     """recursive statdir for a list of tops, yields [top, relpath, stat]"""
     sep = "{0}".format(os.sep).encode("ascii")
+    if not VT100:
+        za = []
+        for td in tops:
+            try:
+                ap = os.path.abspath(os.path.realpath(td))
+                if td[-1:] in (b"\\", b"/"):
+                    ap += sep
+            except:
+                # maybe cpython #88013 (ok)
+                ap = td
+
+            za.append(ap)
+
+        za = [x if x.startswith(b"\\\\") else b"\\\\?\\" + x for x in za]
+        za = [x.replace(b"/", b"\\") for x in za]
+        tops = za
+
     for top in tops:
+        isdir = os.path.isdir(top)
         if top[-1:] == sep:
             stop = top.rstrip(sep)
+            yield stop, b"", os.stat(stop)
         else:
-            stop = os.path.dirname(top)
+            stop, dn = os.path.split(top)
+            if isdir:
+                yield stop, dn, os.stat(stop)
 
-        if os.path.isdir(top):
+        if isdir:
             for ap, inf in walkdir(err, top, []):
                 yield stop, ap[len(stop) :].lstrip(sep), inf
         else:
@@ -472,14 +503,17 @@ def get_hashlist(file, pcb, mth):
         file.kchunks[k] = [v1, v2]
 
 
-def handshake(url, file, pw, search):
-    # type: (str, File, Any, bool) -> tuple[list[str], bool]
+def handshake(ar, file, search):
+    # type: (argparse.Namespace, File, bool) -> tuple[list[str], bool]
     """
     performs a handshake with the server; reply is:
       if search, a list of search results
       otherwise, a list of chunks to upload
     """
 
+    url = ar.url
+    pw = ar.a
+
     req = {
         "hash": [x[0] for x in file.cids],
         "name": file.name,
@@ -488,36 +522,49 @@ def handshake(url, file, pw, search):
     }
     if search:
         req["srch"] = 1
+    elif ar.dr:
+        req["replace"] = True
 
-    headers = {"Content-Type": "text/plain"}  # wtf ed
+    headers = {"Content-Type": "text/plain"}  # <=1.5.1 compat
     if pw:
         headers["Cookie"] = "=".join(["cppwd", pw])
 
+    file.recheck = False
     if file.url:
         url = file.url
     elif b"/" in file.rel:
         url += quotep(file.rel.rsplit(b"/", 1)[0]).decode("utf-8", "replace")
 
     while True:
+        sc = 600
+        txt = ""
         try:
             r = req_ses.post(url, headers=headers, json=req)
-            break
+            sc = r.status_code
+            txt = r.text
+            if sc < 400:
+                break
+
+            raise Exception("http {0}: {1}".format(sc, txt))
+
         except Exception as ex:
-            em = str(ex).split("SSLError(")[-1]
+            em = str(ex).split("SSLError(")[-1].split("\nURL: ")[0].strip()
+
+            if (
+                sc == 422
+                or "<pre>partial upload exists at a different" in txt
+                or "<pre>source file busy; please try again" in txt
+            ):
+                file.recheck = True
+                return [], False
+            elif sc == 409 or "<pre>upload rejected, file already exists" in txt:
+                return [], False
+            elif "<pre>you don't have " in txt:
+                raise
+
             eprint("handshake failed, retrying: {0}\n  {1}\n\n".format(file.name, em))
             time.sleep(1)
 
-    sc = r.status_code
-    if sc >= 400:
-        txt = r.text
-        if sc == 422 or "<pre>partial upload exists at a different" in txt:
-            file.recheck = True
-            return [], False
-        elif sc == 409 or "<pre>upload rejected, file already exists" in txt:
-            return [], False
-
-        raise Exception("http {0}: {1}".format(sc, txt))
-
     try:
         r = r.json()
     except:
@@ -539,8 +586,8 @@ def handshake(url, file, pw, search):
     return r["hash"], r["sprs"]
 
 
-def upload(file, cid, pw):
-    # type: (File, str, Any) -> None
+def upload(file, cid, pw, stats):
+    # type: (File, str, str, str) -> None
     """upload one specific chunk, `cid` (a chunk-hash)"""
 
     headers = {
@@ -548,6 +595,10 @@ def upload(file, cid, pw):
         "X-Up2k-Wark": file.wark,
         "Content-Type": "application/octet-stream",
     }
+
+    if stats:
+        headers["X-Up2k-Stat"] = stats
+
     if pw:
         headers["Cookie"] = "=".join(["cppwd", pw])
 
@@ -564,27 +615,20 @@ def upload(file, cid, pw):
 
 class Ctl(object):
     """
-    this will be the coordinator which runs everything in parallel
-    (hashing, handshakes, uploads)  but right now it's p dumb
+    the coordinator which runs everything in parallel
+    (hashing, handshakes, uploads)
     """
 
-    def __init__(self, ar):
-        self.ar = ar
-        ar.files = [
-            os.path.abspath(os.path.realpath(x.encode("utf-8")))
-            + (x[-1:] if x[-1:] == os.sep else "").encode("utf-8")
-            for x in ar.files
-        ]
-        ar.url = ar.url.rstrip("/") + "/"
-        if "://" not in ar.url:
-            ar.url = "http://" + ar.url
-
+    def _scan(self):
+        ar = self.ar
         eprint("\nscanning {0} locations\n".format(len(ar.files)))
-
         nfiles = 0
         nbytes = 0
         err = []
         for _, _, inf in walkdirs(err, ar.files):
+            if stat.S_ISDIR(inf.st_mode):
+                continue
+
             nfiles += 1
             nbytes += inf.st_size
 
@@ -606,8 +650,16 @@ class Ctl(object):
                 return
 
         eprint("found {0} files, {1}\n\n".format(nfiles, humansize(nbytes)))
-        self.nfiles = nfiles
-        self.nbytes = nbytes
+        return nfiles, nbytes
+
+    def __init__(self, ar, stats=None):
+        self.ok = False
+        self.ar = ar
+        self.stats = stats or self._scan()
+        if not self.stats:
+            return
+
+        self.nfiles, self.nbytes = self.stats
 
         if ar.td:
             requests.packages.urllib3.disable_warnings()
@@ -616,6 +668,8 @@ class Ctl(object):
             req_ses.verify = ar.te
 
         self.filegen = walkdirs([], ar.files)
+        self.recheck = []  # type: list[File]
+
         if ar.safe:
             self._safe()
         else:
@@ -634,11 +688,11 @@ class Ctl(object):
             self.t0 = time.time()
             self.t0_up = None
             self.spd = None
+            self.eta = "99:99:99"
 
             self.mutex = threading.Lock()
             self.q_handshake = Queue()  # type: Queue[File]
             self.q_upload = Queue()  # type: Queue[tuple[File, str]]
-            self.recheck = []  # type: list[File]
 
             self.st_hash = [None, "(idle, starting...)"]  # type: tuple[File, int]
             self.st_up = [None, "(idle, starting...)"]  # type: tuple[File, int]
@@ -647,10 +701,15 @@ class Ctl(object):
 
             self._fancy()
 
+        self.ok = True
+
     def _safe(self):
         """minimal basic slow boring fallback codepath"""
         search = self.ar.s
         for nf, (top, rel, inf) in enumerate(self.filegen):
+            if stat.S_ISDIR(inf.st_mode) or not rel:
+                continue
+
             file = File(top, rel, inf.st_size, inf.st_mtime)
             upath = file.abs.decode("utf-8", "replace")
 
@@ -660,7 +719,7 @@ class Ctl(object):
             burl = self.ar.url[:12] + self.ar.url[8:].split("/")[0] + "/"
             while True:
                 print("  hs...")
-                hs, _ = handshake(self.ar.url, file, self.ar.a, search)
+                hs, _ = handshake(self.ar, file, search)
                 if search:
                     if hs:
                         for hit in hs:
@@ -677,7 +736,8 @@ class Ctl(object):
                 ncs = len(hs)
                 for nc, cid in enumerate(hs):
                     print("  {0} up {1}".format(ncs - nc, cid))
-                    upload(file, cid, self.ar.a)
+                    stats = "{0}/0/0/{1}".format(nf, self.nfiles - nf)
+                    upload(file, cid, self.ar.a, stats)
 
             print("  ok!")
             if file.recheck:
@@ -688,10 +748,10 @@ class Ctl(object):
 
         eprint("finalizing {0} duplicate files".format(len(self.recheck)))
         for file in self.recheck:
-            handshake(self.ar.url, file, self.ar.a, search)
+            handshake(self.ar, file, search)
 
     def _fancy(self):
-        if VT100:
+        if VT100 and not self.ar.ns:
             atexit.register(self.cleanup_vt100)
             ss.scroll_region(3)
 
@@ -715,7 +775,7 @@ class Ctl(object):
                 else:
                     idles = 0
 
-            if VT100:
+            if VT100 and not self.ar.ns:
                 maxlen = ss.w - len(str(self.nfiles)) - 14
                 txt = "\033[s\033[{0}H".format(ss.g)
                 for y, k, st, f in [
@@ -752,12 +812,12 @@ class Ctl(object):
                 eta = (self.nbytes - self.up_b) / (spd + 1)
 
             spd = humansize(spd)
-            eta = str(datetime.timedelta(seconds=int(eta)))
+            self.eta = str(datetime.timedelta(seconds=int(eta)))
             sleft = humansize(self.nbytes - self.up_b)
             nleft = self.nfiles - self.up_f
-            tail = "\033[K\033[u" if VT100 else "\r"
+            tail = "\033[K\033[u" if VT100 and not self.ar.ns else "\r"
 
-            t = "{0} eta @ {1}/s, {2}, {3}# left".format(eta, spd, sleft, nleft)
+            t = "{0} eta @ {1}/s, {2}, {3}# left".format(self.eta, spd, sleft, nleft)
             eprint(txt + "\033]0;{0}\033\\\r{0}{1}".format(t, tail))
 
         if not self.recheck:
@@ -765,7 +825,7 @@ class Ctl(object):
 
         eprint("finalizing {0} duplicate files".format(len(self.recheck)))
         for file in self.recheck:
-            handshake(self.ar.url, file, self.ar.a, False)
+            handshake(self.ar, file, False)
 
     def cleanup_vt100(self):
         ss.scroll_region(None)
@@ -778,8 +838,10 @@ class Ctl(object):
         prd = None
         ls = {}
         for top, rel, inf in self.filegen:
-            if self.ar.z:
-                rd = os.path.dirname(rel)
+            isdir = stat.S_ISDIR(inf.st_mode)
+            if self.ar.z or self.ar.drd:
+                rd = rel if isdir else os.path.dirname(rel)
+                srd = rd.decode("utf-8", "replace").replace("\\", "/")
                 if prd != rd:
                     prd = rd
                     headers = {}
@@ -788,19 +850,37 @@ class Ctl(object):
 
                     ls = {}
                     try:
-                        print("      ls ~{0}".format(rd.decode("utf-8", "replace")))
-                        r = req_ses.get(
-                            self.ar.url.encode("utf-8") + quotep(rd) + b"?ls",
-                            headers=headers,
-                        )
-                        for f in r.json()["files"]:
-                            rfn = f["href"].split("?")[0].encode("utf-8", "replace")
-                            ls[unquote(rfn)] = f
-                    except:
-                        print("   mkdir ~{0}".format(rd.decode("utf-8", "replace")))
+                        print("      ls ~{0}".format(srd))
+                        zb = self.ar.url.encode("utf-8")
+                        zb += quotep(rd.replace(b"\\", b"/"))
+                        r = req_ses.get(zb + b"?ls&dots", headers=headers)
+                        if not r:
+                            raise Exception("HTTP {0}".format(r.status_code))
 
+                        j = r.json()
+                        for f in j["dirs"] + j["files"]:
+                            rfn = f["href"].split("?")[0].rstrip("/")
+                            ls[unquote(rfn.encode("utf-8", "replace"))] = f
+                    except Exception as ex:
+                        print("   mkdir ~{0}  ({1})".format(srd, ex))
+
+                    if self.ar.drd:
+                        dp = os.path.join(top, rd)
+                        lnodes = set(os.listdir(dp))
+                        bnames = [x for x in ls if x not in lnodes]
+                        if bnames:
+                            vpath = self.ar.url.split("://")[-1].split("/", 1)[-1]
+                            names = [x.decode("utf-8", "replace") for x in bnames]
+                            locs = [vpath + srd + "/" + x for x in names]
+                            print("DELETING ~{0}/#{1}".format(srd, len(names)))
+                            req_ses.post(self.ar.url + "?delete", json=locs)
+
+            if isdir:
+                continue
+
+            if self.ar.z:
                 rf = ls.get(os.path.basename(rel), None)
-                if rf and rf["sz"] == inf.st_size and abs(rf["ts"] - inf.st_mtime) <= 1:
+                if rf and rf["sz"] == inf.st_size and abs(rf["ts"] - inf.st_mtime) <= 2:
                     self.nfiles -= 1
                     self.nbytes -= inf.st_size
                     continue
@@ -850,7 +930,10 @@ class Ctl(object):
                 self.handshaker_busy += 1
 
             upath = file.abs.decode("utf-8", "replace")
-            hs, sprs = handshake(self.ar.url, file, self.ar.a, search)
+            if not VT100:
+                upath = upath[4:]
+
+            hs, sprs = handshake(self.ar, file, search)
             if search:
                 if hs:
                     for hit in hs:
@@ -883,6 +966,9 @@ class Ctl(object):
                     self.up_c += len(file.cids) - file.up_c
                     self.up_b += file.size - file.up_b
 
+                    if not file.recheck:
+                        self.up_done(file)
+
                 if hs and file.up_c:
                     # some chunks failed
                     self.up_c -= len(hs)
@@ -912,12 +998,24 @@ class Ctl(object):
                 self.uploader_busy += 1
                 self.t0_up = self.t0_up or time.time()
 
+            zs = "{0}/{1}/{2}/{3} {4}/{5} {6}"
+            stats = zs.format(
+                self.up_f,
+                len(self.recheck),
+                self.uploader_busy,
+                self.nfiles - self.up_f,
+                int(self.nbytes / (1024 * 1024)),
+                int((self.nbytes - self.up_b) / (1024 * 1024)),
+                self.eta,
+            )
+
             file, cid = task
             try:
-                upload(file, cid, self.ar.a)
-            except:
-                eprint("upload failed, retrying: {0} #{1}\n".format(file.name, cid[:8]))
-                pass  # handshake will fix it
+                upload(file, cid, self.ar.a, stats)
+            except Exception as ex:
+                t = "upload failed, retrying: {0} #{1} ({2})\n"
+                eprint(t.format(file.name, cid[:8], ex))
+                # handshake will fix it
 
             with self.mutex:
                 sz = file.kchunks[cid][1]
@@ -933,6 +1031,10 @@ class Ctl(object):
                 self.up_c += 1
                 self.uploader_busy -= 1
 
+    def up_done(self, file):
+        if self.ar.dl:
+            os.unlink(file.abs)
+
 
 class APF(argparse.ArgumentDefaultsHelpFormatter, argparse.RawDescriptionHelpFormatter):
     pass
@@ -946,8 +1048,15 @@ def main():
     cores = (os.cpu_count() if hasattr(os, "cpu_count") else 0) or 2
     hcores = min(cores, 3)  # 4% faster than 4+ on py3.9 @ r5-4500U
 
+    ver = "{0}, v{1}".format(S_BUILD_DT, S_VERSION)
+    if "--version" in sys.argv:
+        print(ver)
+        return
+
+    sys.argv = [x for x in sys.argv if x != "--ws"]
+
     # fmt: off
-    ap = app = argparse.ArgumentParser(formatter_class=APF, epilog="""
+    ap = app = argparse.ArgumentParser(formatter_class=APF, description="copyparty up2k uploader / filesearch tool, " + ver, epilog="""
 NOTE:
 source file/folder selection uses rsync syntax, meaning that:
   "foo" uploads the entire folder to URL/foo/
@@ -957,21 +1066,81 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("url", type=unicode, help="server url, including destination folder")
     ap.add_argument("files", type=unicode, nargs="+", help="files and/or folders to process")
     ap.add_argument("-v", action="store_true", help="verbose")
-    ap.add_argument("-a", metavar="PASSWORD", help="password")
+    ap.add_argument("-a", metavar="PASSWORD", help="password or $filepath")
     ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
     ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
+    ap.add_argument("--version", action="store_true", help="show version and exit")
+
+    ap = app.add_argument_group("compatibility")
+    ap.add_argument("--cls", action="store_true", help="clear screen before start")
+
+    ap = app.add_argument_group("folder sync")
+    ap.add_argument("--dl", action="store_true", help="delete local files after uploading")
+    ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally")
+    ap.add_argument("--drd", action="store_true", help="delete remote files during upload instead of afterwards; reduces peak disk space usage, but will reupload instead of detecting renames")
+
     ap = app.add_argument_group("performance tweaks")
     ap.add_argument("-j", type=int, metavar="THREADS", default=4, help="parallel connections")
     ap.add_argument("-J", type=int, metavar="THREADS", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
     ap.add_argument("-nh", action="store_true", help="disable hashing while uploading")
+    ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles)")
     ap.add_argument("--safe", action="store_true", help="use simple fallback approach")
     ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")
+
     ap = app.add_argument_group("tls")
     ap.add_argument("-te", metavar="PEM_FILE", help="certificate to expect/verify")
     ap.add_argument("-td", action="store_true", help="disable certificate check")
     # fmt: on
 
-    Ctl(app.parse_args())
+    try:
+        ar = app.parse_args()
+    finally:
+        if EXE and not sys.argv[1:]:
+            print("*** hit enter to exit ***")
+            try:
+                input()
+            except:
+                pass
+
+    if ar.drd:
+        ar.dr = True
+
+    for k in "dl dr drd".split():
+        errs = []
+        if ar.safe and getattr(ar, k):
+            errs.append(k)
+
+        if errs:
+            raise Exception("--safe is incompatible with " + str(errs))
+
+    ar.files = [
+        os.path.abspath(os.path.realpath(x.encode("utf-8")))
+        + (x[-1:] if x[-1:] in ("\\", "/") else "").encode("utf-8")
+        for x in ar.files
+    ]
+
+    ar.url = ar.url.rstrip("/") + "/"
+    if "://" not in ar.url:
+        ar.url = "http://" + ar.url
+
+    if ar.a and ar.a.startswith("$"):
+        fn = ar.a[1:]
+        print("reading password from file [{0}]".format(fn))
+        with open(fn, "rb") as f:
+            ar.a = f.read().decode("utf-8").strip()
+
+    if ar.cls:
+        print("\x1b\x5b\x48\x1b\x5b\x32\x4a\x1b\x5b\x33\x4a", end="")
+
+    ctl = Ctl(ar)
+
+    if ar.dr and not ar.drd and ctl.ok:
+        print("\npass 2/2: delete")
+        ar.drd = True
+        ar.z = True
+        ctl = Ctl(ar, ctl.stats)
+
+    sys.exit(0 if ctl.ok else 1)
 
 
 if __name__ == "__main__":

contrib/README.md

@@ -29,11 +29,11 @@ however if your copyparty is behind a reverse-proxy, you may want to use [`share
 * disables thumbnails and folder-type detection in windows explorer
 * makes it way faster (especially for slow/networked locations (such as partyfuse))
 
-### [`webdav-basicauth.reg`](webdav-basicauth.reg)
-* enables webdav basic-auth over plaintext http; takes effect after a reboot OR after running `webdav-unlimit.bat`
-
-### [`webdav-unlimit.bat`](webdav-unlimit.bat)
-* removes the 47.6 MiB filesize limit when downloading from webdav
+### [`webdav-cfg.reg`](webdav-cfg.bat)
+* improves the native webdav support in windows;
+  * removes the 47.6 MiB filesize limit when downloading from webdav
+  * optionally enables webdav basic-auth over plaintext http
+  * optionally helps disable wpad, removing the 10sec latency
 
 ### [`cfssl.sh`](cfssl.sh)
 * creates CA and server certificates using cfssl

contrib/apache/copyparty.conf

@@ -0,0 +1,14 @@
+# when running copyparty behind a reverse proxy,
+# the following arguments are recommended:
+#
+#   -i 127.0.0.1    only accept connections from nginx
+#
+# if you are doing location-based proxying (such as `/stuff` below)
+# you must run copyparty with --rp-loc=stuff
+#
+# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
+
+LoadModule proxy_module modules/mod_proxy.so
+ProxyPass "/stuff" "http://127.0.0.1:3923/stuff"
+# do not specify ProxyPassReverse
+RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}

contrib/index.html

@@ -3,7 +3,7 @@
 
 <head>
 	<meta charset="utf-8">
-	<title>⇆🎉 redirect</title>
+	<title>💾🎉 redirect</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<style>
 

contrib/nginx/copyparty.conf

@@ -1,7 +1,6 @@
 # when running copyparty behind a reverse proxy,
 # the following arguments are recommended:
 #
-#   --http-only     lower latency on initial connection
 #   -i 127.0.0.1    only accept connections from nginx
 #
 # -nc must match or exceed the webserver's max number of concurrent clients;
@@ -9,7 +8,9 @@
 # nginx default is 512  (worker_processes 1, worker_connections 512)
 #
 # you may also consider adding -j0 for CPU-intensive configurations
-# (not that i can really think of any good examples)
+# (5'000 requests per second, or 20gbps upload/download in parallel)
+#
+# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
 
 upstream cpp {
 	server 127.0.0.1:3923;
@@ -37,3 +38,9 @@ server {
 		proxy_set_header   Connection        "Keep-Alive";
 	}
 }
+
+# default client_max_body_size (1M) blocks uploads larger than 256 MiB
+client_max_body_size 1024M;
+client_header_timeout 610m;
+client_body_timeout 610m;
+send_timeout 610m;

contrib/nixos/modules/copyparty.nix

@@ -0,0 +1,281 @@
+{ config, pkgs, lib, ... }:
+
+with lib;
+
+let
+  mkKeyValue = key: value:
+    if value == true then
+    # sets with a true boolean value are coerced to just the key name
+      key
+    else if value == false then
+    # or omitted completely when false
+      ""
+    else
+      (generators.mkKeyValueDefault { inherit mkValueString; } ": " key value);
+
+  mkAttrsString = value: (generators.toKeyValue { inherit mkKeyValue; } value);
+
+  mkValueString = value:
+    if isList value then
+      (concatStringsSep ", " (map mkValueString value))
+    else if isAttrs value then
+      "\n" + (mkAttrsString value)
+    else
+      (generators.mkValueStringDefault { } value);
+
+  mkSectionName = value: "[" + (escape [ "[" "]" ] value) + "]";
+
+  mkSection = name: attrs: ''
+    ${mkSectionName name}
+    ${mkAttrsString attrs}
+  '';
+
+  mkVolume = name: attrs: ''
+    ${mkSectionName name}
+    ${attrs.path}
+    ${mkAttrsString {
+      accs = attrs.access;
+      flags = attrs.flags;
+    }}
+  '';
+
+  passwordPlaceholder = name: "{{password-${name}}}";
+
+  accountsWithPlaceholders = mapAttrs (name: attrs: passwordPlaceholder name);
+
+  configStr = ''
+    ${mkSection "global" cfg.settings}
+    ${mkSection "accounts" (accountsWithPlaceholders cfg.accounts)}
+    ${concatStringsSep "\n" (mapAttrsToList mkVolume cfg.volumes)}
+  '';
+
+  name = "copyparty";
+  cfg = config.services.copyparty;
+  configFile = pkgs.writeText "${name}.conf" configStr;
+  runtimeConfigPath = "/run/${name}/${name}.conf";
+  home = "/var/lib/${name}";
+  defaultShareDir = "${home}/data";
+in {
+  options.services.copyparty = {
+    enable = mkEnableOption "web-based file manager";
+
+    package = mkOption {
+      type = types.package;
+      default = pkgs.copyparty;
+      defaultText = "pkgs.copyparty";
+      description = ''
+        Package of the application to run, exposed for overriding purposes.
+      '';
+    };
+
+    openFilesLimit = mkOption {
+      default = 4096;
+      type = types.either types.int types.str;
+      description = "Number of files to allow copyparty to open.";
+    };
+
+    settings = mkOption {
+      type = types.attrs;
+      description = ''
+        Global settings to apply.
+        Directly maps to values in the [global] section of the copyparty config.
+        See `${getExe cfg.package} --help` for more details.
+      '';
+      default = {
+        i = "127.0.0.1";
+        no-reload = true;
+      };
+      example = literalExpression ''
+        {
+          i = "0.0.0.0";
+          no-reload = true;
+        }
+      '';
+    };
+
+    accounts = mkOption {
+      type = types.attrsOf (types.submodule ({ ... }: {
+        options = {
+          passwordFile = mkOption {
+            type = types.str;
+            description = ''
+              Runtime file path to a file containing the user password.
+              Must be readable by the copyparty user.
+            '';
+            example = "/run/keys/copyparty/ed";
+          };
+        };
+      }));
+      description = ''
+        A set of copyparty accounts to create.
+      '';
+      default = { };
+      example = literalExpression ''
+        {
+          ed.passwordFile = "/run/keys/copyparty/ed";
+        };
+      '';
+    };
+
+    volumes = mkOption {
+      type = types.attrsOf (types.submodule ({ ... }: {
+        options = {
+          path = mkOption {
+            type = types.str;
+            description = ''
+              Path of a directory to share.
+            '';
+          };
+          access = mkOption {
+            type = types.attrs;
+            description = ''
+              Attribute list of permissions and the users to apply them to.
+
+              The key must be a string containing any combination of allowed permission:
+                "r" (read):   list folder contents, download files
+                "w" (write):  upload files; need "r" to see the uploads
+                "m" (move):   move files and folders; need "w" at destination
+                "d" (delete): permanently delete files and folders
+                "g" (get):    download files, but cannot see folder contents
+                "G" (upget):  "get", but can see filekeys of their own uploads
+
+              For example: "rwmd"
+
+              The value must be one of:
+                an account name, defined in `accounts`
+                a list of account names
+                "*", which means "any account"
+            '';
+            example = literalExpression ''
+              {
+                # wG = write-upget = see your own uploads only
+                wG = "*";
+                # read-write-modify-delete for users "ed" and "k"
+                rwmd = ["ed" "k"];
+              };
+            '';
+          };
+          flags = mkOption {
+            type = types.attrs;
+            description = ''
+              Attribute list of volume flags to apply.
+              See `${getExe cfg.package} --help-flags` for more details.
+            '';
+            example = literalExpression ''
+              {
+                # "fk" enables filekeys (necessary for upget permission) (4 chars long)
+                fk = 4;
+                # scan for new files every 60sec
+                scan = 60;
+                # volflag "e2d" enables the uploads database
+                e2d = true;
+                # "d2t" disables multimedia parsers (in case the uploads are malicious)
+                d2t = true;
+                # skips hashing file contents if path matches *.iso
+                nohash = "\.iso$";
+              };
+            '';
+            default = { };
+          };
+        };
+      }));
+      description = "A set of copyparty volumes to create";
+      default = {
+        "/" = {
+          path = defaultShareDir;
+          access = { r = "*"; };
+        };
+      };
+      example = literalExpression ''
+        {
+          "/" = {
+            path = ${defaultShareDir};
+            access = {
+              # wG = write-upget = see your own uploads only
+              wG = "*";
+              # read-write-modify-delete for users "ed" and "k"
+              rwmd = ["ed" "k"];
+            };
+          };
+        };
+      '';
+    };
+  };
+
+  config = mkIf cfg.enable {
+    systemd.services.copyparty = {
+      description = "http file sharing hub";
+      wantedBy = [ "multi-user.target" ];
+
+      environment = {
+        PYTHONUNBUFFERED = "true";
+        XDG_CONFIG_HOME = "${home}/.config";
+      };
+
+      preStart = let
+        replaceSecretCommand = name: attrs:
+          "${getExe pkgs.replace-secret} '${
+            passwordPlaceholder name
+          }' '${attrs.passwordFile}' ${runtimeConfigPath}";
+      in ''
+        set -euo pipefail
+        install -m 600 ${configFile} ${runtimeConfigPath}
+        ${concatStringsSep "\n"
+        (mapAttrsToList replaceSecretCommand cfg.accounts)}
+      '';
+
+      serviceConfig = {
+        Type = "simple";
+        ExecStart = "${getExe cfg.package} -c ${runtimeConfigPath}";
+
+        # Hardening options
+        User = "copyparty";
+        Group = "copyparty";
+        RuntimeDirectory = name;
+        RuntimeDirectoryMode = "0700";
+        StateDirectory = [ name "${name}/data" "${name}/.config" ];
+        StateDirectoryMode = "0700";
+        WorkingDirectory = home;
+        TemporaryFileSystem = "/:ro";
+        BindReadOnlyPaths = [
+          "/nix/store"
+          "-/etc/resolv.conf"
+          "-/etc/nsswitch.conf"
+          "-/etc/hosts"
+          "-/etc/localtime"
+        ] ++ (mapAttrsToList (k: v: "-${v.passwordFile}") cfg.accounts);
+        BindPaths = [ home ] ++ (mapAttrsToList (k: v: v.path) cfg.volumes);
+        # Would re-mount paths ignored by temporary root
+        #ProtectSystem = "strict";
+        ProtectHome = true;
+        PrivateTmp = true;
+        PrivateDevices = true;
+        ProtectKernelTunables = true;
+        ProtectControlGroups = true;
+        RestrictSUIDSGID = true;
+        PrivateMounts = true;
+        ProtectKernelModules = true;
+        ProtectKernelLogs = true;
+        ProtectHostname = true;
+        ProtectClock = true;
+        ProtectProc = "invisible";
+        ProcSubset = "pid";
+        RestrictNamespaces = true;
+        RemoveIPC = true;
+        UMask = "0077";
+        LimitNOFILE = cfg.openFilesLimit;
+        NoNewPrivileges = true;
+        LockPersonality = true;
+        RestrictRealtime = true;
+      };
+    };
+
+    users.groups.copyparty = { };
+    users.users.copyparty = {
+      description = "Service user for copyparty";
+      group = "copyparty";
+      home = home;
+      isSystemUser = true;
+    };
+  };
+}

contrib/openrc/copyparty

@@ -14,5 +14,5 @@ name="$SVCNAME"
 command_background=true
 pidfile="/var/run/$SVCNAME.pid"
 
-command="/usr/bin/python /usr/local/bin/copyparty-sfx.py"
+command="/usr/bin/python3 /usr/local/bin/copyparty-sfx.py"
 command_args="-q -v /mnt::rw"

contrib/package/arch/PKGBUILD

@@ -0,0 +1,57 @@
+# Maintainer: icxes <dev.null@need.moe>
+pkgname=copyparty
+pkgver="1.6.13"
+pkgrel=1
+pkgdesc="Portable file sharing hub"
+arch=("any")
+url="https://github.com/9001/${pkgname}"
+license=('MIT')
+depends=("python" "lsof")
+optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags" 
+            "python-jinja: faster html generator" 
+            "python-mutagen: music tags (alternative)" 
+            "python-pillow: thumbnails for images" 
+            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)" 
+            "libkeyfinder-git: detection of musical keys" 
+            "qm-vamp-plugins: BPM detection" 
+            "python-pyopenssl: ftps functionality" 
+            "python-impacket-git: smb support (bad idea)"
+)
+source=("${url}/releases/download/v${pkgver}/${pkgname}-sfx.py" 
+        "${pkgname}.conf"
+        "${pkgname}.service"
+        "prisonparty.service"
+        "index.md"
+        "https://raw.githubusercontent.com/9001/${pkgname}/v${pkgver}/bin/prisonparty.sh"
+        "https://raw.githubusercontent.com/9001/${pkgname}/v${pkgver}/LICENSE"
+)
+backup=("etc/${pkgname}.d/init" )
+sha256sums=("96b47616c6e671eb2f4a563671726824eaea201b38d138543c8ddd0ae71797bc"
+            "b8565eba5e64dedba1cf6c7aac7e31c5a731ed7153d6810288a28f00a36c28b2"
+            "f65c207e0670f9d78ad2e399bda18d5502ff30d2ac79e0e7fc48e7fbdc39afdc"
+            "c4f396b083c9ec02ad50b52412c84d2a82be7f079b2d016e1c9fad22d68285ff"
+            "dba701de9fd584405917e923ea1e59dbb249b96ef23bad479cf4e42740b774c8"
+            "8e89d281483e22d11d111bed540652af35b66af6f14f49faae7b959f6cdc6475"
+            "cb2ce3d6277bf2f5a82ecf336cc44963bc6490bcf496ffbd75fc9e21abaa75f3"
+)
+
+package() {
+    cd "${srcdir}/"
+
+    install -dm755 "${pkgdir}/etc/${pkgname}.d"
+    install -Dm755 "${pkgname}-sfx.py" "${pkgdir}/usr/bin/${pkgname}"
+    install -Dm755 "prisonparty.sh" "${pkgdir}/usr/bin/prisonparty"
+    install -Dm644 "${pkgname}.conf" "${pkgdir}/etc/${pkgname}.d/init"
+    install -Dm644 "${pkgname}.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}.service"
+    install -Dm644 "prisonparty.service" "${pkgdir}/usr/lib/systemd/system/prisonparty.service"
+    install -Dm644 "index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
+    install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
+
+    find /etc/${pkgname}.d -iname '*.conf' 2>/dev/null | grep -qE . && return
+    echo "┏━━━━━━━━━━━━━━━──-"
+    echo "┃ Configure ${pkgname} by adding .conf files into /etc/${pkgname}.d/"
+    echo "┃ and maybe copy+edit one of the following to /etc/systemd/system/:"
+    echo "┣━♦ /usr/lib/systemd/system/${pkgname}.service   (standard)"
+    echo "┣━♦ /usr/lib/systemd/system/prisonparty.service (chroot)"
+    echo "┗━━━━━━━━━━━━━━━──-"
+}

contrib/package/arch/copyparty.conf

@@ -0,0 +1,7 @@
+## import all *.conf files from the current folder (/etc/copyparty.d)
+% ./
+
+# add additional .conf files to this folder;
+# see example config files for reference:
+# https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf
+# https://github.com/9001/copyparty/tree/hovudstraum/docs/copyparty.d

contrib/package/arch/copyparty.service

@@ -0,0 +1,32 @@
+# this will start `/usr/bin/copyparty-sfx.py`
+# and read config from `/etc/copyparty.d/*.conf`
+#
+# you probably want to:
+#   change "User=cpp" and "/home/cpp/" to another user
+#
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
+#   Environment=PYTHONUNBUFFERED=x
+
+[Unit]
+Description=copyparty file server
+
+[Service]
+Type=notify
+SyslogIdentifier=copyparty
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# user to run as + where the TLS certificate is (if any)
+User=cpp
+Environment=XDG_CONFIG_HOME=/home/cpp/.config
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
+ExecStart=/usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init
+
+[Install]
+WantedBy=multi-user.target

contrib/package/arch/index.md

@@ -0,0 +1,3 @@
+this is `/var/lib/copyparty-jail`, the fallback webroot when copyparty has not yet been configured
+
+please add some `*.conf` files to `/etc/copyparty.d/`

contrib/package/arch/prisonparty.service

@@ -0,0 +1,31 @@
+# this will start `/usr/bin/copyparty-sfx.py`
+# in a chroot, preventing accidental access elsewhere
+# and read config from `/etc/copyparty.d/*.conf`
+#
+# expose additional filesystem locations to copyparty
+#   by listing them between the last `1000` and `--`
+#
+# `1000 1000` = what user to run copyparty as
+#
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
+#   Environment=PYTHONUNBUFFERED=x
+
+[Unit]
+Description=copyparty file server
+
+[Service]
+SyslogIdentifier=prisonparty
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
+ExecStart=/bin/bash /usr/bin/prisonparty /var/lib/copyparty-jail 1000 1000 /etc/copyparty.d -- \
+  /usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init
+
+[Install]
+WantedBy=multi-user.target

contrib/package/nix/copyparty/default.nix

@@ -0,0 +1,55 @@
+{ lib, stdenv, makeWrapper, fetchurl, utillinux, python, jinja2, impacket, pyftpdlib, pyopenssl, pillow, pyvips, ffmpeg, mutagen,
+
+# create thumbnails with Pillow; faster than FFmpeg / MediaProcessing
+withThumbnails ? true,
+
+# create thumbnails with PyVIPS; even faster, uses more memory
+# -- can be combined with Pillow to support more filetypes
+withFastThumbnails ? false,
+
+# enable FFmpeg; thumbnails for most filetypes (also video and audio), extract audio metadata, transcode audio to opus
+# -- possibly dangerous if you allow anonymous uploads, since FFmpeg has a huge attack surface
+# -- can be combined with Thumbnails and/or FastThumbnails, since FFmpeg is slower than both
+withMediaProcessing ? true,
+
+# if MediaProcessing is not enabled, you probably want this instead (less accurate, but much safer and faster)
+withBasicAudioMetadata ? false,
+
+# enable FTPS support in the FTP server
+withFTPS ? false,
+
+# samba/cifs server; dangerous and buggy, enable if you really need it
+withSMB ? false,
+
+}:
+
+let
+  pinData = lib.importJSON ./pin.json;
+  pyEnv = python.withPackages (ps:
+    with ps; [
+      jinja2
+    ]
+    ++ lib.optional withSMB impacket
+    ++ lib.optional withFTPS pyopenssl
+    ++ lib.optional withThumbnails pillow
+    ++ lib.optional withFastThumbnails pyvips
+    ++ lib.optional withMediaProcessing ffmpeg
+    ++ lib.optional withBasicAudioMetadata mutagen
+    );
+in stdenv.mkDerivation {
+  pname = "copyparty";
+  version = pinData.version;
+  src = fetchurl {
+    url = pinData.url;
+    hash = pinData.hash;
+  };
+  buildInputs = [ makeWrapper ];
+  dontUnpack = true;
+  dontBuild = true;
+  installPhase = ''
+    install -Dm755 $src $out/share/copyparty-sfx.py
+    makeWrapper ${pyEnv.interpreter} $out/bin/copyparty \
+      --set PATH '${lib.makeBinPath ([ utillinux ] ++ lib.optional withMediaProcessing ffmpeg)}:$PATH' \
+      --add-flags "$out/share/copyparty-sfx.py"
+  '';
+}

contrib/package/nix/copyparty/pin.json

@@ -0,0 +1,5 @@
+{
+    "url": "https://github.com/9001/copyparty/releases/download/v1.6.13/copyparty-sfx.py",
+    "version": "1.6.13",
+    "hash": "sha256-lrR2FsbmcesvSlY2cXJoJOrqIBs40ThUPI3dCucXl7w="
+}

contrib/package/nix/copyparty/update.py

@@ -0,0 +1,77 @@
+#!/usr/bin/env python3
+
+# Update the Nix package pin
+#
+# Usage: ./update.sh [PATH]
+# When the [PATH] is not set, it will fetch the latest release from the repo.
+# With [PATH] set, it will hash the given file and generate the URL,
+# base on the version contained within the file
+
+import base64
+import json
+import hashlib
+import sys
+import re
+from pathlib import Path
+
+OUTPUT_FILE = Path("pin.json")
+TARGET_ASSET = "copyparty-sfx.py"
+HASH_TYPE = "sha256"
+LATEST_RELEASE_URL = "https://api.github.com/repos/9001/copyparty/releases/latest"
+DOWNLOAD_URL = lambda version: f"https://github.com/9001/copyparty/releases/download/v{version}/{TARGET_ASSET}"
+
+
+def get_formatted_hash(binary):
+    hasher = hashlib.new("sha256")
+    hasher.update(binary)
+    asset_hash = hasher.digest()
+    encoded_hash = base64.b64encode(asset_hash).decode("ascii")
+    return f"{HASH_TYPE}-{encoded_hash}"
+
+
+def version_from_sfx(binary):
+    result = re.search(b'^VER = "(.*)"$', binary, re.MULTILINE)
+    if result:
+        return result.groups(1)[0].decode("ascii")
+
+    raise ValueError("version not found in provided file")
+
+
+def remote_release_pin():
+    import requests
+
+    response = requests.get(LATEST_RELEASE_URL).json()
+    version = response["tag_name"].lstrip("v")
+    asset_info = [a for a in response["assets"] if a["name"] == TARGET_ASSET][0]
+    download_url = asset_info["browser_download_url"]
+    asset = requests.get(download_url)
+    formatted_hash = get_formatted_hash(asset.content)
+
+    result = {"url": download_url, "version": version, "hash": formatted_hash}
+    return result
+
+
+def local_release_pin(path):
+    asset = path.read_bytes()
+    version = version_from_sfx(asset)
+    download_url = DOWNLOAD_URL(version)
+    formatted_hash = get_formatted_hash(asset)
+
+    result = {"url": download_url, "version": version, "hash": formatted_hash}
+    return result
+
+
+def main():
+    if len(sys.argv) > 1:
+        asset_path = Path(sys.argv[1])
+        result = local_release_pin(asset_path)
+    else:
+        result = remote_release_pin()
+
+    print(result)
+    json_result = json.dumps(result, indent=4)
+    OUTPUT_FILE.write_text(json_result)
+
+
+if __name__ == "__main__":
+    main()

contrib/plugins/minimal-up2k.html

@@ -1,13 +1,22 @@
 <!--
+  NOTE: DEPRECATED; please use the javascript version instead:
+  https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/minimal-up2k.js
+
+  ----
+
   save this as .epilogue.html inside a write-only folder to declutter the UI,  makes it look like
   https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png
+
+  only works if you disable the prologue/epilogue sandbox with --no-sb-lg
+  which should probably be combined with --no-dot-ren to prevent damage
+  (`no_sb_lg` can also be set per-volume with volflags)
 -->
 
 <style>
 
     /* make the up2k ui REALLY minimal by hiding a bunch of stuff: */
 
-    #ops, #tree, #path, #epi+h2,  /* main tabs and navigators (tree/breadcrumbs) */
+    #ops, #tree, #path, #wfp,  /* main tabs and navigators (tree/breadcrumbs) */
 
     #u2conf tr:first-child>td[rowspan]:not(#u2btn_cw),  /* most of the config options */
 

contrib/plugins/minimal-up2k.js

@@ -17,7 +17,7 @@ almost the same as minimal-up2k.html except this one...:
 var u2min = `
 <style>
 
-#ops, #path, #tree, #files, #epi+div+h2,
+#ops, #path, #tree, #files, #wfp,
 #u2conf td.c+.c, #u2cards, #srch_dz, #srch_zd {
   display: none !important;
 }
@@ -55,5 +55,5 @@ var u2min = `
 if (!has(perms, 'read')) {
   var e2 = mknod('div');
   e2.innerHTML = u2min;
-  ebi('wrap').insertBefore(e2, QS('#epi+h2'));
+  ebi('wrap').insertBefore(e2, QS('#wfp'));
 }

contrib/plugins/rave.js

@@ -0,0 +1,208 @@
+/* untz untz untz untz */
+
+(function () {
+
+    var can, ctx, W, H, fft, buf, bars, barw, pv,
+        hue = 0,
+        ibeat = 0,
+        beats = [9001],
+        beats_url = '',
+        uofs = 0,
+        ops = ebi('ops'),
+        raving = false,
+        recalc = 0,
+        cdown = 0,
+        FC = 0.9,
+        css = `<style>
+
+#fft {
+    position: fixed;
+    top: 0;
+    left: 0;
+    z-index: -1;
+}
+body {
+    box-shadow: inset 0 0 0 white;
+}
+#ops>a,
+#path>a {
+    display: inline-block;
+}
+/*
+body.untz {
+    animation: untz-body 200ms ease-out;
+}
+@keyframes untz-body {
+	0% {inset 0 0 20em white}
+	100% {inset 0 0 0 white}
+}
+*/
+:root, html.a, html.b, html.c, html.d, html.e {
+    --row-alt: rgba(48,52,78,0.2);
+}
+#files td {
+    background: none;
+}
+
+</style>`;
+
+    QS('body').appendChild(mknod('div', null, css));
+
+    function rave_load() {
+        console.log('rave_load');
+        can = mknod('canvas', 'fft');
+        QS('body').appendChild(can);
+        ctx = can.getContext('2d');
+
+        fft = new AnalyserNode(actx, {
+            "fftSize": 2048,
+            "maxDecibels": 0,
+            "smoothingTimeConstant": 0.7,
+        });
+        ibeat = 0;
+        beats = [9001];
+        buf = new Uint8Array(fft.frequencyBinCount);
+        bars = buf.length * FC;
+        afilt.filters.push(fft);
+        if (!raving) {
+            raving = true;
+            raver();
+        }
+        beats_url = mp.au.src.split('?')[0].replace(/(.*\/)(.*)/, '$1.beats/$2.txt');
+        console.log("reading beats from", beats_url);
+        var xhr = new XHR();
+        xhr.open('GET', beats_url, true);
+        xhr.onload = readbeats;
+        xhr.url = beats_url;
+        xhr.send();
+    }
+
+    function rave_unload() {
+        qsr('#fft');
+        can = null;
+    }
+
+    function readbeats() {
+        if (this.url != beats_url)
+            return console.log('old beats??', this.url, beats_url);
+
+        var sbeats = this.responseText.replace(/\r/g, '').split(/\n/g);
+        if (sbeats.length < 3)
+            return;
+
+        beats = [];
+        for (var a = 0; a < sbeats.length; a++)
+            beats.push(parseFloat(sbeats[a]));
+
+        var end = beats.slice(-2),
+            t = end[1],
+            d = t - end[0];
+
+        while (d > 0.1 && t < 1200)
+            beats.push(t += d);
+    }
+
+    function hrand() {
+        return Math.random() - 0.5;
+    }
+
+    function raver() {
+        if (!can) {
+            raving = false;
+            return;
+        }
+
+        requestAnimationFrame(raver);
+        if (!mp || !mp.au || mp.au.paused)
+            return;
+
+        if (--uofs >= 0) {
+            document.body.style.marginLeft = hrand() * uofs + 'px';
+            ebi('tree').style.marginLeft = hrand() * uofs + 'px';
+            for (var a of QSA('#ops>a, #path>a, #pctl>a'))
+                a.style.transform = 'translate(' + hrand() * uofs * 1 + 'px, ' + hrand() * uofs * 0.7 + 'px) rotate(' + Math.random() * uofs * 0.7 + 'deg)'
+        }
+
+        if (--recalc < 0) {
+            recalc = 60;
+            var tree = ebi('tree'),
+                x = tree.style.display == 'none' ? 0 : tree.offsetWidth;
+
+            //W = can.width = window.innerWidth - x;
+            //H = can.height = window.innerHeight;
+            //H = ebi('widget').offsetTop;
+            W = can.width = bars;
+            H = can.height = 512;
+            barw = 1; //parseInt(0.8 + W / bars);
+            can.style.left = x + 'px';
+            can.style.width = (window.innerWidth - x) + 'px';
+            can.style.height = ebi('widget').offsetTop + 'px';
+        }
+
+        //if (--cdown == 1)
+        //    clmod(ops, 'untz');
+
+        fft.getByteFrequencyData(buf);
+
+        var imax = 0, vmax = 0;
+        for (var a = 10; a < 50; a++)
+            if (vmax < buf[a]) {
+                vmax = buf[a];
+                imax = a;
+            }
+
+        hue = hue * 0.93 + imax * 0.07;
+
+        ctx.fillStyle = 'rgba(0,0,0,0)';
+        ctx.fillRect(0, 0, W, H);
+        ctx.clearRect(0, 0, W, H);
+        ctx.fillStyle = 'hsla(' + (hue * 2.5) + ',100%,50%,0.7)';
+
+        var x = 0, mul = (H / 256) * 0.5;
+        for (var a = 0; a < buf.length * FC; a++) {
+            var v = buf[a] * mul * (1 + 0.69 * a / buf.length);
+            ctx.fillRect(x, H - v, barw, v);
+            x += barw;
+        }
+
+        var t = mp.au.currentTime + 0.05;
+
+        if (ibeat >= beats.length || beats[ibeat] > t)
+            return;
+
+        while (ibeat < beats.length && beats[ibeat++] < t)
+            continue;
+
+        return untz();
+
+        var cv = 0;
+        for (var a = 0; a < 128; a++)
+            cv += buf[a];
+
+        if (cv - pv > 1000) {
+            console.log(pv, cv, cv - pv);
+            if (cdown < 0) {
+                clmod(ops, 'untz', 1);
+                cdown = 20;
+            }
+        }
+        pv = cv;
+    }
+
+    function untz() {
+        console.log('untz');
+        uofs = 14;
+        document.body.animate([
+            { boxShadow: 'inset 0 0 1em #f0c' },
+            { boxShadow: 'inset 0 0 20em #f0c', offset: 0.2 },
+            { boxShadow: 'inset 0 0 0 #f0c' },
+        ], { duration: 200, iterations: 1 });
+    }
+
+    afilt.plugs.push({
+        "en": true,
+        "load": rave_load,
+        "unload": rave_unload
+    });
+
+})();

contrib/systemd/copyparty.service

@@ -2,12 +2,16 @@
 # and share '/mnt' with anonymous read+write
 #
 # installation:
-#   cp -pv copyparty.service /etc/systemd/system
-#   restorecon -vr /etc/systemd/system/copyparty.service
+#   wget https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py -O /usr/local/bin/copyparty-sfx.py
+#   cp -pv copyparty.service /etc/systemd/system/
+#   restorecon -vr /etc/systemd/system/copyparty.service  # on fedora/rhel
 #   firewall-cmd --permanent --add-port={80,443,3923}/tcp  # --zone=libvirt
 #   firewall-cmd --reload
 #   systemctl daemon-reload && systemctl enable --now copyparty
 #
+# if it fails to start, first check this: systemctl status copyparty
+# then try starting it while viewing logs: journalctl -fan 100
+#
 # you may want to:
 #   change "User=cpp" and "/home/cpp/" to another user
 #   remove the nft lines to only listen on port 3923
@@ -44,7 +48,7 @@ ExecReload=/bin/kill -s USR1 $MAINPID
 User=cpp
 Environment=XDG_CONFIG_HOME=/home/cpp/.config
 
-# setup forwarding from ports 80 and 443 to port 3923
+# OPTIONAL: setup forwarding from ports 80 and 443 to port 3923
 ExecStartPre=+/bin/bash -c 'nft -n -a list table nat | awk "/ to :3923 /{print\$NF}" | xargs -rL1 nft delete rule nat prerouting handle; true'
 ExecStartPre=+nft add table ip nat
 ExecStartPre=+nft -- add chain ip nat prerouting { type nat hook prerouting priority -100 \; }

contrib/systemd/prisonparty.service

@@ -6,12 +6,17 @@
 #   1) put copyparty-sfx.py and prisonparty.sh in /usr/local/bin
 #   2) cp -pv prisonparty.service /etc/systemd/system && systemctl enable --now prisonparty
 #
+# expose additional filesystem locations to copyparty
+#   by listing them between the last `1000` and `--`
+#
+# `1000 1000` = what user to run copyparty as
+#
 # you may want to:
 #   change '/mnt::rw' to another location or permission-set
 #    (remember to change the '/mnt' chroot arg too)
 #
-# enable line-buffering for realtime logging (slight performance cost):
-#   inside the [Service] block, add the following line:
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
 #   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
@@ -19,7 +24,14 @@ Description=copyparty file server
 
 [Service]
 SyslogIdentifier=prisonparty
-WorkingDirectory=/usr/local/bin
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
 ExecStart=/bin/bash /usr/local/bin/prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt -- \
   /usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
 

contrib/webdav-cfg.bat

@@ -1,9 +1,7 @@
 @echo off
 rem removes the 47.6 MiB filesize limit when downloading from webdav
 rem + optionally allows/enables password-auth over plaintext http
-rem + optionally helps disable wpad
-
-setlocal enabledelayedexpansion
+rem + optionally helps disable wpad, removing the 10sec latency
 
 net session >nul 2>&1
 if %errorlevel% neq 0 (
@@ -20,30 +18,26 @@ echo OK;
 echo allow webdav basic-auth over plaintext http?
 echo Y: login works, but the password will be visible in wireshark etc
 echo N: login will NOT work unless you use https and valid certificates
-set c=.
-set /p "c=(Y/N): "
-echo(
-if /i not "!c!"=="y" goto :g1
-reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\WebClient\Parameters /v BasicAuthLevel /t REG_DWORD /d 0x2 /f
-rem default is 1 (require tls)
+choice
+if %errorlevel% equ 1 (
+    reg add HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\WebClient\Parameters /v BasicAuthLevel /t REG_DWORD /d 0x2 /f
+    rem default is 1 (require tls)
+)
 
-:g1
 echo(
 echo OK;
 echo do you want to disable wpad?
 echo can give a HUGE speed boost depending on network settings
-set c=.
-set /p "c=(Y/N): "
-echo(
-if /i not "!c!"=="y" goto :g2
-echo(
-echo i'm about to open the [Connections] tab in [Internet Properties] for you;
-echo please click [LAN settings] and disable [Automatically detect settings]
-echo(
-pause
-control inetcpl.cpl,,4
+choice
+if %errorlevel% equ 1 (
+    echo(
+    echo i'm about to open the [Connections] tab in [Internet Properties] for you;
+    echo please click [LAN settings] and disable [Automatically detect settings]
+    echo(
+    pause
+    control inetcpl.cpl,,4
+)
 
-:g2
 net stop webclient
 net start webclient
 echo(

copyparty/__init__.py

@@ -34,6 +34,8 @@ ANYWIN = WINDOWS or sys.platform in ["msys", "cygwin"]
 
 MACOS = platform.system() == "Darwin"
 
+EXE = bool(getattr(sys, "frozen", False))
+
 try:
     CORES = len(os.sched_getaffinity(0))
 except:

copyparty/__main__.py

@@ -23,9 +23,10 @@ import traceback
 import uuid
 from textwrap import dedent
 
-from .__init__ import ANYWIN, CORES, PY2, VT100, WINDOWS, E, EnvParams, unicode
+from .__init__ import ANYWIN, CORES, EXE, PY2, VT100, WINDOWS, E, EnvParams, unicode
 from .__version__ import CODENAME, S_BUILD_DT, S_VERSION
-from .authsrv import expand_config_file, re_vol
+from .authsrv import expand_config_file, re_vol, split_cfg_ln, upgrade_cfg_fmt
+from .cfg import flagcats, onedash
 from .svchub import SvcHub
 from .util import (
     IMPLICATIONS,
@@ -37,6 +38,7 @@ from .util import (
     ansi_re,
     min_ex,
     py_desc,
+    pybin,
     termsize,
     wrap,
 )
@@ -53,7 +55,9 @@ try:
 except:
     HAVE_SSL = False
 
+u = unicode
 printed: list[str] = []
+zsid = uuid.uuid4().urn[4:]
 
 
 class RiceFormatter(argparse.HelpFormatter):
@@ -228,9 +232,10 @@ def get_srvname() -> str:
             ret = f.read().decode("utf-8", "replace").strip()
     except:
         ret = ""
-        while len(ret) < 7:
+        namelen = 5
+        while len(ret) < namelen:
             ret += base64.b32encode(os.urandom(4))[:7].decode("utf-8").lower()
-            ret = re.sub("[234567=]", "", ret)[:7]
+            ret = re.sub("[234567=]", "", ret)[:namelen]
         with open(fp, "wb") as f:
             f.write(ret.encode("utf-8") + b"\n")
 
@@ -238,20 +243,25 @@ def get_srvname() -> str:
 
 
 def ensure_locale() -> None:
+    safe = "en_US.UTF-8"
     for x in [
-        "en_US.UTF-8",
+        safe,
         "English_United States.UTF8",
         "English_United States.1252",
     ]:
         try:
             locale.setlocale(locale.LC_ALL, x)
-            lprint("Locale: {}\n".format(x))
-            break
+            if x != safe:
+                lprint("Locale: {}\n".format(x))
+            return
         except:
             continue
 
+    t = "setlocale {} failed,\n  sorting and dates might get funky\n"
+    warn(t.format(safe))
 
-def ensure_cert() -> None:
+
+def ensure_cert(al: argparse.Namespace) -> None:
     """
     the default cert (and the entire TLS support) is only here to enable the
     crypto.subtle javascript API, which is necessary due to the webkit guys
@@ -260,15 +270,30 @@ def ensure_cert() -> None:
     i feel awful about this and so should they
     """
     cert_insec = os.path.join(E.mod, "res/insecure.pem")
-    cert_cfg = os.path.join(E.cfg, "cert.pem")
-    if not os.path.exists(cert_cfg):
-        shutil.copy(cert_insec, cert_cfg)
+    cert_appdata = os.path.join(E.cfg, "cert.pem")
+    if not os.path.isfile(al.cert):
+        if cert_appdata != al.cert:
+            raise Exception("certificate file does not exist: " + al.cert)
+
+        shutil.copy(cert_insec, al.cert)
+
+    with open(al.cert, "rb") as f:
+        buf = f.read()
+        o1 = buf.find(b" PRIVATE KEY-")
+        o2 = buf.find(b" CERTIFICATE-")
+        m = "unsupported certificate format: "
+        if o1 < 0:
+            raise Exception(m + "no private key inside pem")
+        if o2 < 0:
+            raise Exception(m + "no server certificate inside pem")
+        if o1 > o2:
+            raise Exception(m + "private key must appear before server certificate")
 
     try:
-        if filecmp.cmp(cert_cfg, cert_insec):
+        if filecmp.cmp(al.cert, cert_insec):
             lprint(
-                "\033[33m  using default TLS certificate; https will be insecure."
-                + "\033[36m\n  certificate location: {}\033[0m\n".format(cert_cfg)
+                "\033[33musing default TLS certificate; https will be insecure."
+                + "\033[36m\ncertificate location: {}\033[0m\n".format(al.cert)
             )
     except:
         pass
@@ -347,27 +372,28 @@ def configure_ssl_ciphers(al: argparse.Namespace) -> None:
 def args_from_cfg(cfg_path: str) -> list[str]:
     lines: list[str] = []
     expand_config_file(lines, cfg_path, "")
+    lines = upgrade_cfg_fmt(None, argparse.Namespace(vc=False), lines, "")
 
     ret: list[str] = []
-    skip = False
+    skip = True
     for ln in lines:
-        if not ln:
+        sn = ln.split("  #")[0].strip()
+        if sn.startswith("["):
+            skip = True
+        if sn.startswith("[global]"):
             skip = False
             continue
-
-        if ln.startswith("#"):
+        if skip or not sn.split("#")[0].strip():
             continue
-
-        if not ln.startswith("-"):
-            continue
-
-        if skip:
-            continue
-
-        try:
-            ret.extend(ln.split(" ", 1))
-        except:
-            ret.append(ln)
+        for k, v in split_cfg_ln(sn).items():
+            k = k.lstrip("-")
+            if not k:
+                continue
+            prefix = "-" if k in onedash else "--"
+            if v is True:
+                ret.append(prefix + k)
+            else:
+                ret.append(prefix + k + "=" + v)
 
     return ret
 
@@ -441,27 +467,8 @@ def showlic() -> None:
         print(f.read().decode("utf-8", "replace"))
 
 
-def run_argparse(
-    argv: list[str], formatter: Any, retry: bool, nc: int
-) -> argparse.Namespace:
-    ap = argparse.ArgumentParser(
-        formatter_class=formatter,
-        prog="copyparty",
-        description="http file sharing hub v{} ({})".format(S_VERSION, S_BUILD_DT),
-    )
-
-    try:
-        fk_salt = unicode(os.path.getmtime(os.path.join(E.cfg, "cert.pem")))
-    except:
-        fk_salt = "hunter2"
-
-    hcores = min(CORES, 3)  # 4% faster than 4+ on py3.9 @ r5-4500U
-
-    tty = os.environ.get("TERM", "").lower() == "linux"
-
-    srvname = get_srvname()
-
-    sects = [
+def get_sects():
+    return [
         [
             "accounts",
             "accounts and volumes",
@@ -480,7 +487,7 @@ def run_argparse(
               "g" (get):    download files, but cannot see folder contents
               "G" (upget):  "get", but can see filekeys of their own uploads
 
-            too many volflags to list here, see the other sections
+            too many volflags to list here, see --help-flags
 
             example:\033[35m
               -a ed:hunter2 -v .::r:rw,ed -v ../inc:dump:w:rw,ed:c,nodupe  \033[36m
@@ -507,64 +514,67 @@ def run_argparse(
                 """
             volflags are appended to volume definitions, for example,
             to create a write-only volume with the \033[33mnodupe\033[0m and \033[32mnosub\033[0m flags:
-              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub
+              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub\033[0m
 
-            \033[0muploads, general:
-              \033[36mnodupe\033[35m rejects existing files (instead of symlinking them)
-              \033[36mnosub\033[35m forces all uploads into the top folder of the vfs
-              \033[36mmagic$\033[35m enables filetype detection for nameless uploads
-              \033[36mgz\033[35m allows server-side gzip of uploads with ?gz (also c,xz)
-              \033[36mpk\033[35m forces server-side compression, optional arg: xz,9
+            if global config defines a volflag for all volumes,
+            you can unset it for a specific volume with -flag
+            """
+            ).rstrip()
+            + build_flags_desc(),
+        ],
+        [
+            "hooks",
+            "execute commands before/after various events",
+            dedent(
+                """
+            execute a command (a program or script) before or after various events;
+             \033[36mxbu\033[35m executes CMD before a file upload starts
+             \033[36mxau\033[35m executes CMD after  a file upload finishes
+             \033[36mxiu\033[35m executes CMD after  all uploads finish and volume is idle
+             \033[36mxbr\033[35m executes CMD before a file rename/move
+             \033[36mxar\033[35m executes CMD after  a file rename/move
+             \033[36mxbd\033[35m executes CMD before a file delete
+             \033[36mxad\033[35m executes CMD after  a file delete
+             \033[36mxm\033[35m executes CMD on message
+            \033[0m
+            can be defined as --args or volflags; for example \033[36m
+             --xau notify-send
+             -v .::r:c,xau=notify-send
+            \033[0m
+            commands specified as --args are appended to volflags;
+            each --arg and volflag can be specified multiple times,
+            each command will execute in order unless one returns non-zero
 
-            \033[0mupload rules:
-              \033[36mmaxn=250,600\033[35m max 250 uploads over 15min
-              \033[36mmaxb=1g,300\033[35m max 1 GiB over 5min (suffixes: b, k, m, g)
-              \033[36msz=1k-3m\033[35m allow filesizes between 1 KiB and 3MiB
-              \033[36mdf=1g\033[35m ensure 1 GiB free disk space
+            optionally prefix the command with comma-sep. flags similar to -mtp:
 
-            \033[0mupload rotation:
-            (moves all uploads into the specified folder structure)
-              \033[36mrotn=100,3\033[35m 3 levels of subfolders with 100 entries in each
-              \033[36mrotf=%Y-%m/%d-%H\033[35m date-formatted organizing
-              \033[36mlifetime=3600\033[35m uploads are deleted after 1 hour
+             \033[36mf\033[35m forks the process, doesn't wait for completion
+             \033[36mc\033[35m checks return code, blocks the action if non-zero
+             \033[36mj\033[35m provides json with info as 1st arg instead of filepath
+             \033[36mwN\033[35m waits N sec after command has been started before continuing
+             \033[36mtN\033[35m sets an N sec timeout before the command is abandoned
+             \033[36miN\033[35m xiu only: volume must be idle for N sec (default = 5)
 
-            \033[0mdatabase, general:
-              \033[36me2d\033[35m sets -e2d (all -e2* args can be set using ce2* volflags)
-              \033[36md2ts\033[35m disables metadata collection for existing files
-              \033[36md2ds\033[35m disables onboot indexing, overrides -e2ds*
-              \033[36md2t\033[35m disables metadata collection, overrides -e2t*
-              \033[36md2v\033[35m disables file verification, overrides -e2v*
-              \033[36md2d\033[35m disables all database stuff, overrides -e2*
-              \033[36mhist=/tmp/cdb\033[35m puts thumbnails and indexes at that location
-              \033[36mscan=60\033[35m scan for new files every 60sec, same as --re-maxage
-              \033[36mnohash=\\.iso$\033[35m skips hashing file contents if path matches *.iso
-              \033[36mnoidx=\\.iso$\033[35m fully ignores the contents at paths matching *.iso
-              \033[36mnoforget$\033[35m don't forget files when deleted from disk
-              \033[36mxlink$\033[35m cross-volume dupe detection / linking
-              \033[36mxdev\033[35m do not descend into other filesystems
-              \033[36mxvol\033[35m skip symlinks leaving the volume root
+             \033[36mkt\033[35m kills the entire process tree on timeout (default),
+             \033[36mkm\033[35m kills just the main process
+             \033[36mkn\033[35m lets it continue running until copyparty is terminated
 
-            \033[0mdatabase, audio tags:
-            "mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...
-              \033[36mmtp=.bpm=f,audio-bpm.py\033[35m uses the "audio-bpm.py" program to
-                generate ".bpm" tags from uploads (f = overwrite tags)
-              \033[36mmtp=ahash,vhash=media-hash.py\033[35m collects two tags at once
+             \033[36mc0\033[35m show all process output (default)
+             \033[36mc1\033[35m show only stderr
+             \033[36mc2\033[35m show only stdout
+             \033[36mc3\033[35m mute all process otput
+            \033[0m
+            each hook is executed once for each event, except for \033[36mxiu\033[0m
+            which builds up a backlog of uploads, running the hook just once
+            as soon as the volume has been idle for iN seconds (5 by default)
 
-            \033[0mthumbnails:
-              \033[36mdthumb\033[35m disables all thumbnails
-              \033[36mdvthumb\033[35m disables video thumbnails
-              \033[36mdathumb\033[35m disables audio thumbnails (spectrograms)
-              \033[36mdithumb\033[35m disables image thumbnails
+            \033[36mxiu\033[0m is also unique in that it will pass the metadata to the
+            executed program on STDIN instead of as argv arguments, and
+            it also includes the wark (file-id/hash) as a json property
 
-            \033[0mclient and ux:
-              \033[36mhtml_head=TXT\033[35m includes TXT in the <head>
-              \033[36mrobots\033[35m allows indexing by search engines (default)
-              \033[36mnorobots\033[35m kindly asks search engines to leave
-
-            \033[0mothers:
-              \033[36mfk=8\033[35m generates per-file accesskeys,
-                which will then be required at the "g" permission
-            \033[0m"""
+            except for \033[36mxm\033[0m, only one hook / one action can run at a time,
+            so it's recommended to use the \033[36mf\033[0m flag unless you really need
+            to wait for the hook to finish before continuing (without \033[36mf\033[0m
+            the upload speed can easily drop to 10% for small files)"""
             ),
         ],
         [
@@ -600,10 +610,43 @@ def run_argparse(
             """
             ),
         ],
+        [
+            "dbd",
+            "database durability profiles",
+            dedent(
+                """
+            mainly affects uploads of many small files on slow HDDs; speeds measured uploading 520 files on a WD20SPZX (SMR 2.5" 5400rpm 4kb)
+
+            \033[32macid\033[0m = extremely safe but slow; the old default. Should never lose any data no matter what
+
+            \033[32mswal\033[0m = 2.4x faster uploads yet 99.9%% as safe -- theoretical chance of losing metadata for the ~200 most recently uploaded files if there's a power-loss or your OS crashes
+
+            \033[32mwal\033[0m = another 21x faster on HDDs yet 90%% as safe; same pitfall as \033[33mswal\033[0m except more likely
+
+            \033[32myolo\033[0m = another 1.5x faster, and removes the occasional sudden upload-pause while the disk syncs, but now you're at risk of losing the entire database in a powerloss / OS-crash
+
+            profiles can be set globally (--dbd=yolo), or per-volume with volflags: -v ~/Music:music:r:c,dbd=acid
+            """
+            ),
+        ],
     ]
 
-    # fmt: off
-    u = unicode
+
+def build_flags_desc():
+    ret = ""
+    for grp, flags in flagcats.items():
+        ret += "\n\n\033[0m" + grp
+        for k, v in flags.items():
+            v = v.replace("\n", "\n    ")
+            ret += "\n  \033[36m{}\033[35m {}".format(k, v)
+
+    return ret + "\033[0m"
+
+
+# fmt: off
+
+
+def add_general(ap, nc, srvname):
     ap2 = ap.add_argument_group('general options')
     ap2.add_argument("-c", metavar="PATH", type=u, action="append", help="add config file")
     ap2.add_argument("-nc", metavar="NUM", type=int, default=nc, help="max num clients")
@@ -619,6 +662,8 @@ def run_argparse(
     ap2.add_argument("--license", action="store_true", help="show licenses and exit")
     ap2.add_argument("--version", action="store_true", help="show versions and exit")
 
+
+def add_qr(ap, tty):
     ap2 = ap.add_argument_group('qr options')
     ap2.add_argument("--qr", action="store_true", help="show http:// QR-code on startup")
     ap2.add_argument("--qrs", action="store_true", help="show https:// QR-code on startup")
@@ -629,18 +674,23 @@ def run_argparse(
     ap2.add_argument("--qrp", metavar="CELLS", type=int, default=4, help="padding (spec says 4 or more, but 1 is usually fine)")
     ap2.add_argument("--qrz", metavar="N", type=int, default=0, help="[\033[32m1\033[0m]=1x, [\033[32m2\033[0m]=2x, [\033[32m0\033[0m]=auto (try [\033[32m2\033[0m] on broken fonts)")
 
+
+def add_upload(ap):
     ap2 = ap.add_argument_group('upload options')
     ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads, hiding them from clients unless -ed")
     ap2.add_argument("--plain-ip", action="store_true", help="when avoiding filename collisions by appending the uploader's ip to the filename: append the plaintext ip instead of salting and hashing the ip")
     ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled")
+    ap2.add_argument("--blank-wt", metavar="SEC", type=int, default=300, help="file write grace period (any client can write to a blank file last-modified more recently than SEC seconds ago)")
     ap2.add_argument("--reg-cap", metavar="N", type=int, default=38400, help="max number of uploads to keep in memory when running without -e2d; roughly 1 MiB RAM per 600")
     ap2.add_argument("--no-fpool", action="store_true", help="disable file-handle pooling -- instead, repeatedly close and reopen files during upload (very slow on windows)")
     ap2.add_argument("--use-fpool", action="store_true", help="force file-handle pooling, even when it might be dangerous (multiprocessing, filesystems lacking sparse-files support, ...)")
-    ap2.add_argument("--hardlink", action="store_true", help="prefer hardlinks instead of symlinks when possible (within same filesystem)")
-    ap2.add_argument("--never-symlink", action="store_true", help="do not fallback to symlinks when a hardlink cannot be made")
-    ap2.add_argument("--no-dedup", action="store_true", help="disable symlink/hardlink creation; copy file contents instead")
+    ap2.add_argument("--hardlink", action="store_true", help="prefer hardlinks instead of symlinks when possible (within same filesystem) (volflag=hardlink)")
+    ap2.add_argument("--never-symlink", action="store_true", help="do not fallback to symlinks when a hardlink cannot be made (volflag=neversymlink)")
+    ap2.add_argument("--no-dedup", action="store_true", help="disable symlink/hardlink creation; copy file contents instead (volflag=copydupes")
     ap2.add_argument("--no-dupe", action="store_true", help="reject duplicate files during upload; only matches within the same volume (volflag=nodupe)")
     ap2.add_argument("--no-snap", action="store_true", help="disable snapshots -- forget unfinished uploads on shutdown; don't create .hist/up2k.snap files -- abandoned/interrupted uploads must be cleaned up manually")
+    ap2.add_argument("--rand", action="store_true", help="force randomized filenames, --nrand chars long (volflag=rand)")
+    ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
     ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
     ap2.add_argument("--df", metavar="GiB", type=float, default=0, help="ensure GiB free disk space by rejecting upload requests")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
@@ -648,33 +698,50 @@ def run_argparse(
     ap2.add_argument("--u2sort", metavar="TXT", type=u, default="s", help="upload order; [\033[32ms\033[0m]=smallest-first, [\033[32mn\033[0m]=alphabetical, [\033[32mfs\033[0m]=force-s, [\033[32mfn\033[0m]=force-n -- alphabetical is a bit slower on fiber/LAN but makes it easier to eyeball if everything went fine")
     ap2.add_argument("--write-uplog", action="store_true", help="write POST reports to textfiles in working-directory")
 
+
+def add_network(ap):
     ap2 = ap.add_argument_group('network options')
     ap2.add_argument("-i", metavar="IP", type=u, default="::", help="ip to bind (comma-sep.), default: all IPv4 and IPv6")
     ap2.add_argument("-p", metavar="PORT", type=u, default="3923", help="ports to bind (comma/range)")
+    ap2.add_argument("--ll", action="store_true", help="include link-local IPv4/IPv6 even if the NIC has routable IPs (breaks some mdns clients)")
     ap2.add_argument("--rproxy", metavar="DEPTH", type=int, default=1, help="which ip to keep; [\033[32m0\033[0m]=tcp, [\033[32m1\033[0m]=origin (first x-fwd), [\033[32m2\033[0m]=cloudflare, [\033[32m3\033[0m]=nginx, [\033[32m-1\033[0m]=closest proxy")
+    ap2.add_argument("--rp-loc", metavar="PATH", type=u, default="", help="if reverse-proxying on a location instead of a dedicated domain/subdomain, provide the base location here (eg. /foo/bar)")
+    if ANYWIN:
+        ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
+    else:
+        ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
     ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
     ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0, help="debug: socket write delay in seconds")
     ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0, help="debug: response delay in seconds")
+    ap2.add_argument("--rsp-jtr", metavar="SEC", type=float, default=0, help="debug: response delay, random duration 0..SEC")
 
+
+def add_tls(ap, cert_path):
     ap2 = ap.add_argument_group('SSL/TLS options')
     ap2.add_argument("--http-only", action="store_true", help="disable ssl/tls -- force plaintext")
     ap2.add_argument("--https-only", action="store_true", help="disable plaintext -- force tls")
+    ap2.add_argument("--cert", metavar="PATH", type=u, default=cert_path, help="path to TLS certificate")
     ap2.add_argument("--ssl-ver", metavar="LIST", type=u, help="set allowed ssl/tls versions; [\033[32mhelp\033[0m] shows available versions; default is what your python version considers safe")
     ap2.add_argument("--ciphers", metavar="LIST", type=u, help="set allowed ssl/tls ciphers; [\033[32mhelp\033[0m] shows available ciphers")
     ap2.add_argument("--ssl-dbg", action="store_true", help="dump some tls info")
     ap2.add_argument("--ssl-log", metavar="PATH", type=u, help="log master secrets for later decryption in wireshark")
 
+
+def add_zeroconf(ap):
     ap2 = ap.add_argument_group("Zeroconf options")
     ap2.add_argument("-z", action="store_true", help="enable all zeroconf backends (mdns, ssdp)")
-    ap2.add_argument("--z-on", metavar="NICS/NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes\n └─example: \033[32meth0, wlo1, virhost0, 192.168.123.0/24, fd00:fda::/96\033[0m")
-    ap2.add_argument("--z-off", metavar="NICS/NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--z-on", metavar="NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes\n └─example: \033[32meth0, wlo1, virhost0, 192.168.123.0/24, fd00:fda::/96\033[0m")
+    ap2.add_argument("--z-off", metavar="NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--z-chk", metavar="SEC", type=int, default=10, help="check for network changes every SEC seconds (0=disable)")
     ap2.add_argument("-zv", action="store_true", help="verbose all zeroconf backends")
     ap2.add_argument("--mc-hop", metavar="SEC", type=int, default=0, help="rejoin multicast groups every SEC seconds (workaround for some switches/routers which cause mDNS to suddenly stop working after some time); try [\033[32m300\033[0m] or [\033[32m180\033[0m]")
 
-    ap2 = ap.add_argument_group("Zeroconf-mDNS options:")
+
+def add_zc_mdns(ap):
+    ap2 = ap.add_argument_group("Zeroconf-mDNS options")
     ap2.add_argument("--zm", action="store_true", help="announce the enabled protocols over mDNS (multicast DNS-SD) -- compatible with KDE, gnome, macOS, ...")
-    ap2.add_argument("--zm-on", metavar="NICS/NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes")
-    ap2.add_argument("--zm-off", metavar="NICS/NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--zm-on", metavar="NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--zm-off", metavar="NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
     ap2.add_argument("--zm4", action="store_true", help="IPv4 only -- try this if some clients can't connect")
     ap2.add_argument("--zm6", action="store_true", help="IPv6 only")
     ap2.add_argument("--zmv", action="store_true", help="verbose mdns")
@@ -688,14 +755,18 @@ def run_argparse(
     ap2.add_argument("--zm-msub", action="store_true", help="merge subnets on each NIC -- always enabled for ipv6 -- reduces network load, but gnome-gvfs clients may stop working")
     ap2.add_argument("--zm-noneg", action="store_true", help="disable NSEC replies -- try this if some clients don't see copyparty")
 
-    ap2 = ap.add_argument_group("Zeroconf-SSDP options:")
-    ap2.add_argument("--zs", action="store_true", help="announce the enabled protocols over SSDP -- compatible with Windows")
-    ap2.add_argument("--zs-on", metavar="NICS/NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes")
-    ap2.add_argument("--zs-off", metavar="NICS/NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
-    ap2.add_argument("--zsv", action="store_true", help="verbose SSDP")
-    ap2.add_argument("--zsl", metavar="PATH", type=u, default="/?hc", help="location to include in the url (or a complete external URL), for example [\033[32mpriv/?pw=hunter2\033[0m] or [\033[32mpriv/?pw=hunter2\033[0m]")
-    ap2.add_argument("--zsid", metavar="UUID", type=u, default=uuid.uuid4().urn[4:], help="USN (device identifier) to announce")
 
+def add_zc_ssdp(ap):
+    ap2 = ap.add_argument_group("Zeroconf-SSDP options")
+    ap2.add_argument("--zs", action="store_true", help="announce the enabled protocols over SSDP -- compatible with Windows")
+    ap2.add_argument("--zs-on", metavar="NETS", type=u, default="", help="enable zeroconf ONLY on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--zs-off", metavar="NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
+    ap2.add_argument("--zsv", action="store_true", help="verbose SSDP")
+    ap2.add_argument("--zsl", metavar="PATH", type=u, default="/?hc", help="location to include in the url (or a complete external URL), for example [\033[32mpriv/?pw=hunter2\033[0m] (goes directly to /priv/ with password hunter2) or [\033[32m?hc=priv&pw=hunter2\033[0m] (shows mounting options for /priv/ with password)")
+    ap2.add_argument("--zsid", metavar="UUID", type=u, default=zsid, help="USN (device identifier) to announce")
+
+
+def add_ftp(ap):
     ap2 = ap.add_argument_group('FTP options')
     ap2.add_argument("--ftp", metavar="PORT", type=int, help="enable FTP server on PORT, for example \033[32m3921")
     ap2.add_argument("--ftps", metavar="PORT", type=int, help="enable FTPS server on PORT, for example \033[32m3990")
@@ -704,11 +775,15 @@ def run_argparse(
     ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, help="the NAT address to use for passive connections")
     ap2.add_argument("--ftp-pr", metavar="P-P", type=u, help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
 
+
+def add_webdav(ap):
     ap2 = ap.add_argument_group('WebDAV options')
-    ap2.add_argument("--daw", action="store_true", help="enable full write support. \033[1;31mWARNING:\033[0m This has side-effects -- PUT-operations will now \033[1;31mOVERWRITE\033[0m existing files, rather than inventing new filenames to avoid loss of data. You might want to instead set this as a volflag where needed. By not setting this flag, uploaded files can get written to a filename which the client does not expect (which might be okay, depending on client)")
+    ap2.add_argument("--daw", action="store_true", help="enable full write support, even if client may not be webdav. \033[1;31mWARNING:\033[0m This has side-effects -- PUT-operations will now \033[1;31mOVERWRITE\033[0m existing files, rather than inventing new filenames to avoid loss of data. You might want to instead set this as a volflag where needed. By not setting this flag, uploaded files can get written to a filename which the client does not expect (which might be okay, depending on client)")
     ap2.add_argument("--dav-inf", action="store_true", help="allow depth:infinite requests (recursive file listing); extremely server-heavy but required for spec compliance -- luckily few clients rely on this")
     ap2.add_argument("--dav-mac", action="store_true", help="disable apple-garbage filter -- allow macos to create junk files (._* and .DS_Store, .Spotlight-*, .fseventsd, .Trashes, .AppleDouble, __MACOS)")
 
+
+def add_smb(ap):
     ap2 = ap.add_argument_group('SMB/CIFS options')
     ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless --smb-port is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is dangerous! Never expose to the internet. Account permissions are coalesced; if one account has write-access to a volume, then all accounts do.")
     ap2.add_argument("--smbw", action="store_true", help="enable write support (please dont)")
@@ -720,6 +795,26 @@ def run_argparse(
     ap2.add_argument("--smbvv", action="store_true", help="verboser")
     ap2.add_argument("--smbvvv", action="store_true", help="verbosest")
 
+
+def add_hooks(ap):
+    ap2 = ap.add_argument_group('event hooks (see --help-hooks)')
+    ap2.add_argument("--xbu", metavar="CMD", type=u, action="append", help="execute CMD before a file upload starts")
+    ap2.add_argument("--xau", metavar="CMD", type=u, action="append", help="execute CMD after  a file upload finishes")
+    ap2.add_argument("--xiu", metavar="CMD", type=u, action="append", help="execute CMD after  all uploads finish and volume is idle")
+    ap2.add_argument("--xbr", metavar="CMD", type=u, action="append", help="execute CMD before a file move/rename")
+    ap2.add_argument("--xar", metavar="CMD", type=u, action="append", help="execute CMD after  a file move/rename")
+    ap2.add_argument("--xbd", metavar="CMD", type=u, action="append", help="execute CMD before a file delete")
+    ap2.add_argument("--xad", metavar="CMD", type=u, action="append", help="execute CMD after  a file delete")
+    ap2.add_argument("--xm", metavar="CMD", type=u, action="append", help="execute CMD on message")
+
+
+def add_yolo(ap):
+    ap2 = ap.add_argument_group('yolo options')
+    ap2.add_argument("--allow-csrf", action="store_true", help="disable csrf protections; let other domains/sites impersonate you through cross-site requests")
+    ap2.add_argument("--getmod", action="store_true", help="permit ?move=[...] and ?delete as GET")
+
+
+def add_optouts(ap):
     ap2 = ap.add_argument_group('opt-outs')
     ap2.add_argument("-nw", action="store_true", help="never write anything to disk (debug/benchmark)")
     ap2.add_argument("--keep-qem", action="store_true", help="do not disable quick-edit-mode on windows (it is disabled to avoid accidental text selection which will deadlock copyparty)")
@@ -728,15 +823,18 @@ def run_argparse(
     ap2.add_argument("--no-mv", action="store_true", help="disable move/rename operations")
     ap2.add_argument("-nih", action="store_true", help="no info hostname -- don't show in UI")
     ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI")
+    ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
     ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
     ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (as specified by the 'lifetime' volflag)")
 
+
+def add_safety(ap, fk_salt):
     ap2 = ap.add_argument_group('safety options')
     ap2.add_argument("-s", action="count", default=0, help="increase safety: Disable thumbnails / potentially dangerous software (ffmpeg/pillow/vips), hide partial uploads, avoid crawlers.\n └─Alias of\033[32m --dotpart --no-thumb --no-mtag-ff --no-robots --force-js")
-    ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --no-dot-mv --no-dot-ren --unpost=0 --no-del --no-mv --hardlink --vague-403 --ban-404=50,60,1440 -nih")
-    ap2.add_argument("-sss", action="store_true", help="further increase safety: Enable logging to disk, scan for dangerous symlinks.\n └─Alias of\033[32m -ss --no-dav -lo=cpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz --ls=**,*,ln,p,r")
+    ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --hardlink --vague-403 --ban-404=50,60,1440 -nih")
+    ap2.add_argument("-sss", action="store_true", help="further increase safety: Enable logging to disk, scan for dangerous symlinks.\n └─Alias of\033[32m -ss --no-dav --no-logues --no-readme -lo=cpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz --ls=**,*,ln,p,r")
     ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m; example [\033[32m**,*,ln,p,r\033[0m]")
-    ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt; used to generate unpredictable internal identifiers for uploads -- doesn't really matter")
+    ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
     ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files -- this one DOES matter")
     ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
     ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to make something a dotfile")
@@ -750,12 +848,18 @@ def run_argparse(
     ap2.add_argument("--ban-404", metavar="N,W,B", type=u, default="no", help="hitting more than \033[33mN\033[0m 404's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes (disabled by default since turbo-up2k counts as 404s)")
     ap2.add_argument("--aclose", metavar="MIN", type=int, default=10, help="if a client maxes out the server connection limit, downgrade it from connection:keep-alive to connection:close for MIN minutes (and also kill its active connections) -- disable with 0")
     ap2.add_argument("--loris", metavar="B", type=int, default=60, help="if a client maxes out the server connection limit without sending headers, ban it for B minutes; disable with [\033[32m0\033[0m]")
+    ap2.add_argument("--acao", metavar="V[,V]", type=u, default="*", help="Access-Control-Allow-Origin; list of origins (domains/IPs without port) to accept requests from; [\033[32mhttps://1.2.3.4\033[0m]. Default [\033[32m*\033[0m] allows requests from all sites but removes cookies and http-auth; only ?pw=hunter2 survives")
+    ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like described in --acao)")
 
+
+def add_shutdown(ap):
     ap2 = ap.add_argument_group('shutdown options')
     ap2.add_argument("--ign-ebind", action="store_true", help="continue running even if it's impossible to listen on some of the requested endpoints")
     ap2.add_argument("--ign-ebind-all", action="store_true", help="continue running even if it's impossible to receive connections at all")
-    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after WHEN has finished; for example [\033[32midx\033[0m] will do volume indexing + metadata analysis")
+    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after WHEN has finished; [\033[32mcfg\033[0m] config parsing, [\033[32midx\033[0m] volscan + multimedia indexing")
 
+
+def add_logging(ap):
     ap2 = ap.add_argument_group('logging options')
     ap2.add_argument("-q", action="store_true", help="quiet")
     ap2.add_argument("-lo", metavar="PATH", type=u, help="logfile, example: \033[32mcpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz")
@@ -765,11 +869,15 @@ def run_argparse(
     ap2.add_argument("--ihead", metavar="HEADER", type=u, action='append', help="dump incoming header")
     ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|\?th=[wj]$|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching")
 
+
+def add_admin(ap):
     ap2 = ap.add_argument_group('admin panel options')
     ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
     ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
     ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
 
+
+def add_thumbnail(ap):
     ap2 = ap.add_argument_group('thumbnail options')
     ap2.add_argument("--no-thumb", action="store_true", help="disable all thumbnails (volflag=dthumb)")
     ap2.add_argument("--no-vthumb", action="store_true", help="disable video thumbnails (volflag=dvthumb)")
@@ -786,32 +894,38 @@ def run_argparse(
     ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown -- avoids doing keepalive pokes (updating the mtime) on thumbnail folders more often than SEC seconds")
     ap2.add_argument("--th-clean", metavar="SEC", type=int, default=43200, help="cleanup interval; 0=disabled")
     ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age -- folders which haven't been poked for longer than --th-poke seconds will get deleted every --th-clean seconds")
-    ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for")
+    ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat/look for; case-insensitive if -e2d")
     # https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
     # https://github.com/libvips/libvips
     # ffmpeg -hide_banner -demuxers | awk '/^ D  /{print$2}' | while IFS= read -r x; do ffmpeg -hide_banner -h demuxer=$x; done | grep -E '^Demuxer |extensions:'
-    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="bmp,dib,gif,icns,ico,jpg,jpeg,jp2,jpx,pcx,png,pbm,pgm,ppm,pnm,sgi,tga,tif,tiff,webp,xbm,dds,xpm,heif,heifs,heic,heics,avif,avifs", help="image formats to decode using pillow")
-    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="jpg,jpeg,jp2,jpx,jxl,tif,tiff,png,webp,heic,avif,fit,fits,fts,exr,svg,hdr,ppm,pgm,pfm,gif,nii", help="image formats to decode using pyvips")
-    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
-    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="av1,asf,avi,flv,m4v,mkv,mjpeg,mjpg,mpg,mpeg,mpg2,mpeg2,h264,avc,mts,h265,hevc,mov,3gp,mp4,ts,mpegts,nut,ogv,ogm,rm,vob,webm,wmv", help="video formats to decode using ffmpeg")
-    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,m4a,ogg,opus,flac,alac,mp3,mp2,ac3,dts,wma,ra,wav,aif,aiff,au,alaw,ulaw,mulaw,amr,gsm,ape,tak,tta,wv,mpc", help="audio formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,dcx,dds,dib,emf,eps,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,pbm,pcx,pgm,png,pnm,ppm,psd,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
+    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
+    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,m4a,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,tak,tta,ulaw,wav,wma,wv,xm,xpk", help="audio formats to decode using ffmpeg")
 
+
+def add_transcoding(ap):
     ap2 = ap.add_argument_group('transcoding options')
     ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
     ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete cached transcode output after SEC seconds")
 
+
+def add_db_general(ap, hcores):
     ap2 = ap.add_argument_group('general db options')
-    ap2.add_argument("-e2d", action="store_true", help="enable up2k database, making files searchable + enables upload deduplocation")
+    ap2.add_argument("-e2d", action="store_true", help="enable up2k database, making files searchable + enables upload deduplication")
     ap2.add_argument("-e2ds", action="store_true", help="scan writable folders for new files on startup; sets -e2d")
     ap2.add_argument("-e2dsa", action="store_true", help="scans all folders on startup; sets -e2ds")
     ap2.add_argument("-e2v", action="store_true", help="verify file integrity; rehash all files and compare with db")
     ap2.add_argument("-e2vu", action="store_true", help="on hash mismatch: update the database with the new hash")
     ap2.add_argument("-e2vp", action="store_true", help="on hash mismatch: panic and quit copyparty")
     ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs) (volflag=hist)")
-    ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching paths during e2ds folder scans (volflag=nohash)")
-    ap2.add_argument("--no-idx", metavar="PTN", type=u, help="regex: disable indexing of matching paths during e2ds folder scans (volflag=noidx)")
+    ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
+    ap2.add_argument("--no-idx", metavar="PTN", type=u, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
     ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
+    ap2.add_argument("--re-dhash", action="store_true", help="rebuild the cache if it gets out of sync (for example crash on startup during metadata scanning)")
     ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice (volflag=noforget)")
+    ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see --help-dbd (volflag=dbd)")
     ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
     ap2.add_argument("--xdev", action="store_true", help="do not descend into other filesystems (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
     ap2.add_argument("--xvol", action="store_true", help="skip symlinks leaving the volume root (volflag=xvol)")
@@ -820,7 +934,10 @@ def run_argparse(
     ap2.add_argument("--db-act", metavar="SEC", type=float, default=10, help="defer any scheduled volume reindexing until SEC seconds after last db write (uploads, renames, ...)")
     ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than SEC seconds")
     ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
+    ap2.add_argument("--dotsrch", action="store_true", help="show dotfiles in search results (volflags: dotsrch | nodotsrch)")
 
+
+def add_db_metadata(ap):
     ap2 = ap.add_argument_group('metadata db options')
     ap2.add_argument("-e2t", action="store_true", help="enable metadata indexing; makes it possible to search for artist/title/codec/resolution/...")
     ap2.add_argument("-e2ts", action="store_true", help="scan existing files on startup; sets -e2t")
@@ -838,19 +955,33 @@ def run_argparse(
         default=".vq,.aq,vc,ac,fmt,res,.fps")
     ap2.add_argument("-mtp", metavar="M=[f,]BIN", type=u, action="append", help="read tag M using program BIN to parse the file")
 
+
+def add_ui(ap, retry):
     ap2 = ap.add_argument_group('ui options')
     ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language")
     ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use")
     ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
     ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
+    ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
     ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
     ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
+    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI)")
     ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
     ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
     ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty", help="title / service-name to show in html documents")
+    ap2.add_argument("--pb-url", metavar="URL", type=u, default="https://github.com/9001/copyparty", help="powered-by link; disable with -np")
+    ap2.add_argument("--ver", action="store_true", help="show version on the control panel (incompatible by -np)")
+    ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox")
+    ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for prologue/epilogue docs  (volflag=lg_sbf)")
+    ap2.add_argument("--no-sb-md", action="store_true", help="don't sandbox README.md documents (volflags: no_sb_md | sb_md)")
+    ap2.add_argument("--no-sb-lg", action="store_true", help="don't sandbox prologue/epilogue docs (volflags: no_sb_lg | sb_lg); enables non-js support")
 
+
+def add_debug(ap):
     ap2 = ap.add_argument_group('debug options')
+    ap2.add_argument("--vc", action="store_true", help="verbose config file parser (explain config)")
+    ap2.add_argument("--cgen", action="store_true", help="generate config file from current config (best-effort; probably buggy)")
     ap2.add_argument("--no-sendfile", action="store_true", help="disable sendfile; instead using a traditional file read loop")
     ap2.add_argument("--no-scandir", action="store_true", help="disable scandir; instead using listdir + stat on each file")
     ap2.add_argument("--no-fastboot", action="store_true", help="wait for up2k indexing before starting the httpd")
@@ -862,9 +993,60 @@ def run_argparse(
     ap2.add_argument("--bak-flips", action="store_true", help="[up2k] if a client uploads a bitflipped/corrupted chunk, store a copy according to --bf-nc and --bf-dir")
     ap2.add_argument("--bf-nc", metavar="NUM", type=int, default=200, help="bak-flips: stop if there's more than NUM files at --kf-dir already; default: 6.3 GiB max (200*32M)")
     ap2.add_argument("--bf-dir", metavar="PATH", type=u, default="bf", help="bak-flips: store corrupted chunks at PATH; default: folder named 'bf' wherever copyparty was started")
-    # fmt: on
+
+
+# fmt: on
+
+
+def run_argparse(
+    argv: list[str], formatter: Any, retry: bool, nc: int
+) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(
+        formatter_class=formatter,
+        prog="copyparty",
+        description="http file sharing hub v{} ({})".format(S_VERSION, S_BUILD_DT),
+    )
+
+    cert_path = os.path.join(E.cfg, "cert.pem")
+
+    try:
+        fk_salt = unicode(os.path.getmtime(cert_path))
+    except:
+        fk_salt = "hunter2"
+
+    hcores = min(CORES, 4)  # optimal on py3.11 @ r5-4500U
+
+    tty = os.environ.get("TERM", "").lower() == "linux"
+
+    srvname = get_srvname()
+
+    add_general(ap, nc, srvname)
+    add_network(ap)
+    add_tls(ap, cert_path)
+    add_qr(ap, tty)
+    add_zeroconf(ap)
+    add_zc_mdns(ap)
+    add_zc_ssdp(ap)
+    add_upload(ap)
+    add_db_general(ap, hcores)
+    add_db_metadata(ap)
+    add_thumbnail(ap)
+    add_transcoding(ap)
+    add_ftp(ap)
+    add_webdav(ap)
+    add_smb(ap)
+    add_safety(ap, fk_salt)
+    add_optouts(ap)
+    add_shutdown(ap)
+    add_yolo(ap)
+    add_hooks(ap)
+    add_ui(ap, retry)
+    add_admin(ap)
+    add_logging(ap)
+    add_debug(ap)
 
     ap2 = ap.add_argument_group("help sections")
+    sects = get_sects()
     for k, h, _ in sects:
         ap2.add_argument("--help-" + k, action="store_true", help=h)
 
@@ -920,9 +1102,10 @@ def main(argv: Optional[list[str]] = None) -> None:
         showlic()
         sys.exit(0)
 
+    if EXE:
+        print("pybin: {}\n".format(pybin), end="")
+
     ensure_locale()
-    if HAVE_SSL:
-        ensure_cert()
 
     for k, v in zip(argv[1:], argv[2:]):
         if k == "-c" and os.path.isfile(v):
@@ -952,7 +1135,8 @@ def main(argv: Optional[list[str]] = None) -> None:
         if da:
             argv.extend(["--qr"])
             if ANYWIN or not os.geteuid():
-                argv.extend(["-p80,443,3923", "--ign-ebind"])
+                # win10 allows symlinks if admin; can be unexpected
+                argv.extend(["-p80,443,3923", "--ign-ebind", "--no-dedup"])
     except:
         pass
 
@@ -974,6 +1158,7 @@ def main(argv: Optional[list[str]] = None) -> None:
     for fmtr in [RiceFormatter, RiceFormatter, Dodge11874, BasicDodge11874]:
         try:
             al = run_argparse(argv, fmtr, retry, nc)
+            dal = run_argparse([], fmtr, retry, nc)
             break
         except SystemExit:
             raise
@@ -981,8 +1166,15 @@ def main(argv: Optional[list[str]] = None) -> None:
             retry = True
             lprint("\n[ {} ]:\n{}\n".format(fmtr, min_ex()))
 
-    assert al  # type: ignore
-    al.E = E  # __init__ is not shared when oxidized
+    try:
+        assert al  # type: ignore
+        assert dal  # type: ignore
+        al.E = E  # __init__ is not shared when oxidized
+    except:
+        sys.exit(1)
+
+    if HAVE_SSL:
+        ensure_cert(al)
 
     if WINDOWS and not al.keep_qem:
         try:
@@ -1085,7 +1277,7 @@ def main(argv: Optional[list[str]] = None) -> None:
 
     # signal.signal(signal.SIGINT, sighandler)
 
-    SvcHub(al, argv, "".join(printed)).run()
+    SvcHub(al, dal, argv, "".join(printed)).run()
 
 
 if __name__ == "__main__":

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 5, 0)
-CODENAME = "babel"
-BUILD_DT = (2022, 12, 3)
+VERSION = (1, 6, 14)
+CODENAME = "cors k"
+BUILD_DT = (2023, 4, 24)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -14,6 +14,7 @@ from datetime import datetime
 
 from .__init__ import ANYWIN, TYPE_CHECKING, WINDOWS
 from .bos import bos
+from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
 from .util import (
     IMPLICATIONS,
     META_NOBOTS,
@@ -21,7 +22,7 @@ from .util import (
     UNPLICATIONS,
     Pebkac,
     absreal,
-    fsenc,
+    afsenc,
     get_df,
     humansize,
     relchk,
@@ -36,7 +37,7 @@ if True:  # pylint: disable=using-constant-test
 
     from typing import Any, Generator, Optional, Union
 
-    from .util import RootLogger
+    from .util import NamedLogger, RootLogger
 
 if TYPE_CHECKING:
     pass
@@ -355,7 +356,8 @@ class VFS(object):
         flags = {k: v for k, v in self.flags.items()}
         hist = flags.get("hist")
         if hist and hist != "-":
-            flags["hist"] = "{}/{}".format(hist.rstrip("/"), name)
+            zs = "{}/{}".format(hist.rstrip("/"), name)
+            flags["hist"] = os.path.expanduser(zs) if zs.startswith("~") else zs
 
         return flags
 
@@ -370,7 +372,6 @@ class VFS(object):
 
     def _find(self, vpath: str) -> tuple["VFS", str]:
         """return [vfs,remainder]"""
-        vpath = undot(vpath)
         if vpath == "":
             return self, ""
 
@@ -381,7 +382,7 @@ class VFS(object):
             rem = ""
 
         if name in self.nodes:
-            return self.nodes[name]._find(rem)
+            return self.nodes[name]._find(undot(rem))
 
         return self, vpath
 
@@ -389,7 +390,7 @@ class VFS(object):
         self, vpath: str, uname: str
     ) -> tuple[bool, bool, bool, bool, bool, bool]:
         """can Read,Write,Move,Delete,Get,Upget"""
-        vn, _ = self._find(vpath)
+        vn, _ = self._find(undot(vpath))
         c = vn.axs
         return (
             uname in c.uread or "*" in c.uread,
@@ -419,7 +420,7 @@ class VFS(object):
                     self.log("vfs", "invalid relpath [{}]".format(vpath))
                 raise Pebkac(404)
 
-        vn, rem = self._find(vpath)
+        vn, rem = self._find(undot(vpath))
         c: AXS = vn.axs
 
         for req, d, msg in [
@@ -577,6 +578,7 @@ class VFS(object):
 
     def zipgen(
         self,
+        vpath: str,
         vrem: str,
         flt: set[str],
         uname: str,
@@ -588,7 +590,7 @@ class VFS(object):
 
         # if multiselect: add all items to archive root
         # if single folder: the folder itself is the top-level item
-        folder = "" if flt or not wrap else (vrem.split("/")[-1] or "top")
+        folder = "" if flt or not wrap else (vpath.split("/")[-1].lstrip(".") or "top")
 
         g = self.walk(folder, vrem, [], uname, [[True, False]], dots, scandir, False)
         for _, _, vpath, apath, files, rd, vd in g:
@@ -654,11 +656,15 @@ class AuthSrv(object):
         args: argparse.Namespace,
         log_func: Optional["RootLogger"],
         warn_anonwrite: bool = True,
+        dargs: Optional[argparse.Namespace] = None,
     ) -> None:
         self.args = args
+        self.dargs = dargs or args
         self.log_func = log_func
         self.warn_anonwrite = warn_anonwrite
         self.line_ctr = 0
+        self.indent = ""
+        self.desc = []
 
         self.mutex = threading.Lock()
         self.reload()
@@ -691,17 +697,47 @@ class AuthSrv(object):
             raise Exception("invalid config")
 
         if src in mount.values():
-            t = "warning: filesystem-path [{}] mounted in multiple locations:"
+            t = "filesystem-path [{}] mounted in multiple locations:"
             t = t.format(src)
             for v in [k for k, v in mount.items() if v == src] + [dst]:
                 t += "\n  /{}".format(v)
 
             self.log(t, c=3)
+            raise Exception("invalid config")
+
+        if not bos.path.isdir(src):
+            self.log("warning: filesystem-path does not exist: {}".format(src), 3)
 
         mount[dst] = src
         daxs[dst] = AXS()
         mflags[dst] = {}
 
+    def _e(self, desc: Optional[str] = None) -> None:
+        if not self.args.vc or not self.line_ctr:
+            return
+
+        if not desc and not self.indent:
+            self.log("")
+            return
+
+        desc = desc or ""
+        desc = desc.replace("[", "[\033[0m").replace("]", "\033[90m]")
+        self.log(" >>> {}{}".format(self.indent, desc), "90")
+
+    def _l(self, ln: str, c: int, desc: str) -> None:
+        if not self.args.vc or not self.line_ctr:
+            return
+
+        if c < 10:
+            c += 30
+
+        t = "\033[97m{:4} \033[{}m{}{}"
+        if desc:
+            t += "  \033[0;90m# {}\033[0m"
+            desc = desc.replace("[", "[\033[0m").replace("]", "\033[90m]")
+
+        self.log(t.format(self.line_ctr, c, self.indent, ln, desc))
+
     def _parse_config_file(
         self,
         fp: str,
@@ -711,61 +747,140 @@ class AuthSrv(object):
         mflags: dict[str, dict[str, Any]],
         mount: dict[str, str],
     ) -> None:
-        skip = False
-        vol_src = None
-        vol_dst = None
+        self.desc = []
         self.line_ctr = 0
+
         expand_config_file(cfg_lines, fp, "")
+        if self.args.vc:
+            lns = ["{:4}: {}".format(n, s) for n, s in enumerate(cfg_lines, 1)]
+            self.log("expanded config file (unprocessed):\n" + "\n".join(lns))
+
+        cfg_lines = upgrade_cfg_fmt(self.log, self.args, cfg_lines, fp)
+
+        cat = ""
+        catg = "[global]"
+        cata = "[accounts]"
+        catx = "accs:"
+        catf = "flags:"
+        ap: Optional[str] = None
+        vp: Optional[str] = None
         for ln in cfg_lines:
             self.line_ctr += 1
-            if not ln and vol_src is not None:
-                vol_src = None
-                vol_dst = None
-
-            if skip:
-                if not ln:
-                    skip = False
+            ln = ln.split("  #")[0].strip()
+            if not ln.split("#")[0].strip():
                 continue
 
-            if not ln or ln.startswith("#"):
-                continue
+            subsection = ln in (catx, catf)
+            if ln.startswith("[") or subsection:
+                self._e()
+                if ap is None and vp is not None:
+                    t = "the first line after [/{}] must be a filesystem path to share on that volume"
+                    raise Exception(t.format(vp))
 
-            if vol_src is None:
-                if ln.startswith("u "):
-                    u, p = ln[2:].split(":", 1)
-                    acct[u] = p
-                elif ln.startswith("-"):
-                    skip = True  # argv
+                cat = ln
+                if not subsection:
+                    ap = vp = None
+                    self.indent = ""
                 else:
-                    vol_src = ln
+                    self.indent = "  "
+
+                if ln == catg:
+                    t = "begin commandline-arguments (anything from --help; dashes are optional)"
+                    self._l(ln, 6, t)
+                elif ln == cata:
+                    self._l(ln, 5, "begin user-accounts section")
+                elif ln.startswith("[/"):
+                    vp = ln[1:-1].strip("/")
+                    self._l(ln, 2, "define volume at URL [/{}]".format(vp))
+                elif subsection:
+                    if ln == catx:
+                        self._l(ln, 5, "volume access config:")
+                    else:
+                        t = "volume-specific config (anything from --help-flags)"
+                        self._l(ln, 6, t)
+                else:
+                    raise Exception("invalid section header")
+
+                self.indent = "    " if subsection else "  "
                 continue
 
-            if vol_src and vol_dst is None:
-                vol_dst = ln
-                if not vol_dst.startswith("/"):
-                    raise Exception('invalid mountpoint "{}"'.format(vol_dst))
-
-                if vol_src.startswith("~"):
-                    vol_src = os.path.expanduser(vol_src)
-
-                # cfg files override arguments and previous files
-                vol_src = absreal(vol_src)
-                vol_dst = vol_dst.strip("/")
-                self._map_volume(vol_src, vol_dst, mount, daxs, mflags)
+            if cat == catg:
+                self._l(ln, 6, "")
+                zt = split_cfg_ln(ln)
+                for zs, za in zt.items():
+                    zs = zs.lstrip("-")
+                    if za is True:
+                        self._e("└─argument [{}]".format(zs))
+                    else:
+                        self._e("└─argument [{}] with value [{}]".format(zs, za))
                 continue
 
-            try:
-                lvl, uname = ln.split(" ", 1)
-            except:
-                lvl = ln
-                uname = "*"
+            if cat == cata:
+                try:
+                    u, p = [zs.strip() for zs in ln.split(":", 1)]
+                    self._l(ln, 5, "account [{}], password [{}]".format(u, p))
+                    acct[u] = p
+                except:
+                    t = 'lines inside the [accounts] section must be "username: password"'
+                    raise Exception(t)
+                continue
 
-            if lvl == "a":
-                t = "WARNING (config-file): permission flag 'a' is deprecated; please use 'rw' instead"
-                self.log(t, 1)
+            if vp is not None and ap is None:
+                ap = ln
+                if ap.startswith("~"):
+                    ap = os.path.expanduser(ap)
 
-            assert vol_dst is not None
-            self._read_vol_str(lvl, uname, daxs[vol_dst], mflags[vol_dst])
+                ap = absreal(ap)
+                self._l(ln, 2, "bound to filesystem-path [{}]".format(ap))
+                self._map_volume(ap, vp, mount, daxs, mflags)
+                continue
+
+            if cat == catx:
+                err = ""
+                try:
+                    self._l(ln, 5, "volume access config:")
+                    sk, sv = ln.split(":")
+                    if re.sub("[rwmdgG]", "", sk) or not sk:
+                        err = "invalid accs permissions list; "
+                        raise Exception(err)
+                    if " " in re.sub(", *", "", sv).strip():
+                        err = "list of users is not comma-separated; "
+                        raise Exception(err)
+                    self._read_vol_str(sk, sv.replace(" ", ""), daxs[vp], mflags[vp])
+                    continue
+                except:
+                    err += "accs entries must be 'rwmdgG: user1, user2, ...'"
+                    raise Exception(err)
+
+            if cat == catf:
+                err = ""
+                try:
+                    self._l(ln, 6, "volume-specific config:")
+                    zd = split_cfg_ln(ln)
+                    fstr = ""
+                    for sk, sv in zd.items():
+                        bad = re.sub(r"[a-z0-9_-]", "", sk).lstrip("-")
+                        if bad:
+                            err = "bad characters [{}] in volflag name [{}]; "
+                            err = err.format(bad, sk)
+                            raise Exception(err)
+                        if sv is True:
+                            fstr += "," + sk
+                        else:
+                            fstr += ",{}={}".format(sk, sv)
+                            self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
+                            fstr = ""
+                    if fstr:
+                        self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
+                    continue
+                except:
+                    err += "flags entries (volflags) must be one of the following:\n  'flag1, flag2, ...'\n  'key: value'\n  'flag1, flag2, key: value'"
+                    raise Exception(err)
+
+            raise Exception("unprocessable line in config")
+
+        self._e()
+        self.line_ctr = 0
 
     def _read_vol_str(
         self, lvl: str, uname: str, axs: AXS, flags: dict[str, Any]
@@ -804,6 +919,13 @@ class AuthSrv(object):
                 ("G", axs.upget),
             ]:  # b bb bbb
                 if ch in lvl:
+                    if un == "*":
+                        t = "└─add permission [{0}] for [everyone] -- {2}"
+                    else:
+                        t = "└─add permission [{0}] for user [{1}] -- {2}"
+
+                    desc = permdescs.get(ch, "?")
+                    self._e(t.format(ch, un, desc))
                     al.add(un)
 
     def _read_volflag(
@@ -813,7 +935,20 @@ class AuthSrv(object):
         value: Union[str, bool, list[str]],
         is_list: bool,
     ) -> None:
-        if name not in ["mtp"]:
+        desc = flagdescs.get(name.lstrip("-"), "?").replace("\n", " ")
+
+        if re.match("^-[^-]+$", name):
+            t = "└─unset volflag [{}]  ({})"
+            self._e(t.format(name[1:], desc))
+            flags[name] = True
+            return
+
+        if name not in "mtp xbu xau xiu xbr xar xbd xad xm".split():
+            if value is True:
+                t = "└─add volflag [{}] = {}  ({})"
+            else:
+                t = "└─add volflag [{}] = [{}]  ({})"
+            self._e(t.format(name, value, desc))
             flags[name] = value
             return
 
@@ -826,6 +961,7 @@ class AuthSrv(object):
             vals += [value]
 
         flags[name] = vals
+        self._e("volflag [{}] += {}  ({})".format(name, vals, desc))
 
     def reload(self) -> None:
         """
@@ -876,6 +1012,18 @@ class AuthSrv(object):
                 lns: list[str] = []
                 try:
                     self._parse_config_file(cfg_fn, lns, acct, daxs, mflags, mount)
+
+                    zs = "#\033[36m cfg files in "
+                    zst = [x[len(zs) :] for x in lns if x.startswith(zs)]
+                    for zs in list(set(zst)):
+                        self.log("discovered config files in " + zs, 6)
+
+                    zs = "#\033[36m opening cfg file"
+                    zstt = [x.split(" -> ") for x in lns if x.startswith(zs)]
+                    zst = [(max(0, len(x) - 2) * " ") + "└" + x[-1] for x in zstt]
+                    t = "loaded {} config files:\n{}"
+                    self.log(t.format(len(zst), "\n".join(zst)))
+
                 except:
                     lns = lns[: self.line_ctr]
                     slns = ["{:4}: {}".format(n, s) for n, s in enumerate(lns, 1)]
@@ -956,12 +1104,15 @@ class AuthSrv(object):
         promote = []
         demote = []
         for vol in vfs.all_vols.values():
-            zb = hashlib.sha512(fsenc(vol.realpath)).digest()
+            zb = hashlib.sha512(afsenc(vol.realpath)).digest()
             hid = base64.b32encode(zb).decode("ascii").lower()
             vflag = vol.flags.get("hist")
             if vflag == "-":
                 pass
             elif vflag:
+                if vflag.startswith("~"):
+                    vflag = os.path.expanduser(vflag)
+
                 vol.histpath = uncyg(vflag) if WINDOWS else vflag
             elif self.args.hist:
                 for nch in range(len(hid)):
@@ -975,7 +1126,7 @@ class AuthSrv(object):
                     except:
                         owner = None
 
-                    me = fsenc(vol.realpath).rstrip()
+                    me = afsenc(vol.realpath).rstrip()
                     if owner not in [None, me]:
                         continue
 
@@ -1115,19 +1266,30 @@ class AuthSrv(object):
                 if ptn:
                     vol.flags[vf] = re.compile(ptn)
 
-            for k in ["e2t", "e2ts", "e2tsr", "e2v", "e2vu", "e2vp", "xdev", "xvol"]:
-                if getattr(self.args, k):
-                    vol.flags[k] = True
-
-            for ga, vf in (
-                ("no_forget", "noforget"),
-                ("no_dupe", "nodupe"),
-                ("magic", "magic"),
-                ("xlink", "xlink"),
-            ):
+            for ga, vf in vf_bmap().items():
                 if getattr(self.args, ga):
                     vol.flags[vf] = True
 
+            for ve, vd in (
+                ("nodotsrch", "dotsrch"),
+                ("sb_lg", "no_sb_lg"),
+                ("sb_md", "no_sb_md"),
+            ):
+                if ve in vol.flags:
+                    vol.flags.pop(vd, None)
+
+            for ga, vf in vf_vmap().items():
+                if vf not in vol.flags:
+                    vol.flags[vf] = getattr(self.args, ga)
+
+            for k in ("nrand",):
+                if k not in vol.flags:
+                    vol.flags[k] = getattr(self.args, k)
+
+            for k in ("nrand",):
+                if k in vol.flags:
+                    vol.flags[k] = int(vol.flags[k])
+
             for k1, k2 in IMPLICATIONS:
                 if k1 in vol.flags:
                     vol.flags[k2] = True
@@ -1136,6 +1298,12 @@ class AuthSrv(object):
                 if k1 in vol.flags:
                     vol.flags[k2] = False
 
+            dbds = "acid|swal|wal|yolo"
+            vol.flags["dbd"] = dbd = vol.flags.get("dbd") or self.args.dbd
+            if dbd not in dbds.split("|"):
+                t = "invalid dbd [{}]; must be one of [{}]"
+                raise Exception(t.format(dbd, dbds))
+
             # default tag cfgs if unset
             if "mte" not in vol.flags:
                 vol.flags["mte"] = self.args.mte
@@ -1146,8 +1314,32 @@ class AuthSrv(object):
             if "mth" not in vol.flags:
                 vol.flags["mth"] = self.args.mth
 
-            # append parsers from argv to volflags
-            self._read_volflag(vol.flags, "mtp", self.args.mtp, True)
+            # append additive args from argv to volflags
+            hooks = "xbu xau xiu xbr xar xbd xad xm".split()
+            for name in ["mtp"] + hooks:
+                self._read_volflag(vol.flags, name, getattr(self.args, name), True)
+
+            for hn in hooks:
+                cmds = vol.flags.get(hn)
+                if not cmds:
+                    continue
+
+                ncmds = []
+                for cmd in cmds:
+                    hfs = []
+                    ocmd = cmd
+                    while "," in cmd[:6]:
+                        zs, cmd = cmd.split(",", 1)
+                        hfs.append(zs)
+
+                    if "c" in hfs and "f" in hfs:
+                        t = "cannot combine flags c and f; removing f from eventhook [{}]"
+                        self.log(t.format(ocmd), 1)
+                        hfs = [x for x in hfs if x != "f"]
+                        ocmd = ",".join(hfs + [cmd])
+
+                    ncmds.append(ocmd)
+                vol.flags[hn] = ncmds
 
             # d2d drops all database features for a volume
             for grp, rm in [["d2d", "e2d"], ["d2t", "e2t"], ["d2d", "e2v"]]:
@@ -1183,10 +1375,22 @@ class AuthSrv(object):
                 if k in ints:
                     vol.flags[k] = int(vol.flags[k])
 
-            if "lifetime" in vol.flags and "e2d" not in vol.flags:
-                t = 'removing lifetime config from volume "/{}" because e2d is disabled'
-                self.log(t.format(vol.vpath), 1)
-                del vol.flags["lifetime"]
+            if "e2d" not in vol.flags:
+                if "lifetime" in vol.flags:
+                    t = 'removing lifetime config from volume "/{}" because e2d is disabled'
+                    self.log(t.format(vol.vpath), 1)
+                    del vol.flags["lifetime"]
+
+                needs_e2d = [x for x in hooks if x != "xm"]
+                drop = [x for x in needs_e2d if vol.flags.get(x)]
+                if drop:
+                    t = 'removing [{}] from volume "/{}" because e2d is disabled'
+                    self.log(t.format(", ".join(drop), vol.vpath), 1)
+                    for x in drop:
+                        vol.flags.pop(x)
+
+            if vol.flags.get("neversymlink") and not vol.flags.get("hardlink"):
+                vol.flags["copydupes"] = True
 
             # verify tags mentioned by -mt[mp] are used by -mte
             local_mtp = {}
@@ -1244,6 +1448,12 @@ class AuthSrv(object):
             self.log(t, 1)
             errors = True
 
+        for vol in vfs.all_vols.values():
+            for k in list(vol.flags.keys()):
+                if re.match("^-[^-]+$", k):
+                    vol.flags.pop(k[1:], None)
+                    vol.flags.pop(k)
+
         if errors:
             sys.exit(1)
 
@@ -1432,32 +1642,298 @@ class AuthSrv(object):
         if not flag_r:
             sys.exit(0)
 
+    def cgen(self) -> None:
+        ret = [
+            "## WARNING:",
+            "##  there will probably be mistakes in",
+            "##  commandline-args (and maybe volflags)",
+            "",
+        ]
+
+        csv = set("i p".split())
+        lst = set("c ihead mtm mtp xad xar xau xiu xbd xbr xbu xm".split())
+        askip = set("a v c vc cgen theme".split())
+
+        # keymap from argv to vflag
+        amap = vf_bmap()
+        amap.update(vf_vmap())
+        amap.update(vf_cmap())
+        vmap = {v: k for k, v in amap.items()}
+
+        args = {k: v for k, v in vars(self.args).items()}
+        pops = []
+        for k1, k2 in IMPLICATIONS:
+            if args.get(k1):
+                pops.append(k2)
+        for pop in pops:
+            args.pop(pop, None)
+
+        if args:
+            ret.append("[global]")
+            for k, v in args.items():
+                if k in askip:
+                    continue
+                if k in csv:
+                    v = ", ".join([str(za) for za in v])
+                try:
+                    v2 = getattr(self.dargs, k)
+                    if v == v2:
+                        continue
+                except:
+                    continue
+
+                dk = "  " + k.replace("_", "-")
+                if k in lst:
+                    for ve in v:
+                        ret.append("{}: {}".format(dk, ve))
+                else:
+                    if v is True:
+                        ret.append(dk)
+                    elif v not in (False, None, ""):
+                        ret.append("{}: {}".format(dk, v))
+            ret.append("")
+
+        if self.acct:
+            ret.append("[accounts]")
+            for u, p in self.acct.items():
+                ret.append("  {}: {}".format(u, p))
+            ret.append("")
+
+        for vol in self.vfs.all_vols.values():
+            ret.append("[/{}]".format(vol.vpath))
+            ret.append("  " + vol.realpath)
+            ret.append("  accs:")
+            perms = {
+                "r": "uread",
+                "w": "uwrite",
+                "m": "umove",
+                "d": "udel",
+                "g": "uget",
+                "G": "upget",
+            }
+            users = {}
+            for pkey in perms.values():
+                for uname in getattr(vol.axs, pkey):
+                    try:
+                        users[uname] += 1
+                    except:
+                        users[uname] = 1
+            lusers = [(v, k) for k, v in users.items()]
+            vperms = {}
+            for _, uname in sorted(lusers):
+                pstr = ""
+                for pchar, pkey in perms.items():
+                    if uname in getattr(vol.axs, pkey):
+                        pstr += pchar
+                if "g" in pstr and "G" in pstr:
+                    pstr = pstr.replace("g", "")
+                try:
+                    vperms[pstr].append(uname)
+                except:
+                    vperms[pstr] = [uname]
+            for pstr, uname in vperms.items():
+                ret.append("    {}: {}".format(pstr, ", ".join(uname)))
+            trues = []
+            vals = []
+            for k, v in sorted(vol.flags.items()):
+                try:
+                    ak = vmap[k]
+                    if getattr(self.args, ak) is v:
+                        continue
+                except:
+                    pass
+
+                if k in lst:
+                    for ve in v:
+                        vals.append("{}: {}".format(k, ve))
+                elif v is True:
+                    trues.append(k)
+                elif v is not False:
+                    try:
+                        v = v.pattern
+                    except:
+                        pass
+
+                    vals.append("{}: {}".format(k, v))
+            pops = []
+            for k1, k2 in IMPLICATIONS:
+                if k1 in trues:
+                    pops.append(k2)
+            trues = [x for x in trues if x not in pops]
+            if trues:
+                vals.append(", ".join(trues))
+            if vals:
+                ret.append("  flags:")
+                for zs in vals:
+                    ret.append("    " + zs)
+            ret.append("")
+
+        self.log("generated config:\n\n" + "\n".join(ret))
+
+
+def split_cfg_ln(ln: str) -> dict[str, Any]:
+    # "a, b, c: 3" => {a:true, b:true, c:3}
+    ret = {}
+    while True:
+        ln = ln.strip()
+        if not ln:
+            break
+        ofs_sep = ln.find(",") + 1
+        ofs_var = ln.find(":") + 1
+        if not ofs_sep and not ofs_var:
+            ret[ln] = True
+            break
+        if ofs_sep and (ofs_sep < ofs_var or not ofs_var):
+            k, ln = ln.split(",", 1)
+            ret[k.strip()] = True
+        else:
+            k, ln = ln.split(":", 1)
+            ret[k.strip()] = ln.strip()
+            break
+    return ret
+
 
 def expand_config_file(ret: list[str], fp: str, ipath: str) -> None:
     """expand all % file includes"""
     fp = absreal(fp)
-    ipath += " -> " + fp
-    ret.append("#\033[36m opening cfg file{}\033[0m".format(ipath))
     if len(ipath.split(" -> ")) > 64:
         raise Exception("hit max depth of 64 includes")
 
     if os.path.isdir(fp):
-        for fn in sorted(os.listdir(fp)):
+        names = os.listdir(fp)
+        ret.append("#\033[36m cfg files in {} => {}\033[0m".format(fp, names))
+        for fn in sorted(names):
             fp2 = os.path.join(fp, fn)
-            if not os.path.isfile(fp2):
-                continue  # dont recurse
+            if not fp2.endswith(".conf") or fp2 in ipath:
+                continue
 
             expand_config_file(ret, fp2, ipath)
         return
 
+    ipath += " -> " + fp
+    ret.append("#\033[36m opening cfg file{}\033[0m".format(ipath))
+
     with open(fp, "rb") as f:
-        for ln in [x.decode("utf-8").strip() for x in f]:
+        for oln in [x.decode("utf-8").rstrip() for x in f]:
+            ln = oln.split("  #")[0].strip()
             if ln.startswith("% "):
+                pad = " " * len(oln.split("%")[0])
                 fp2 = ln[1:].strip()
                 fp2 = os.path.join(os.path.dirname(fp), fp2)
+                ofs = len(ret)
                 expand_config_file(ret, fp2, ipath)
+                for n in range(ofs, len(ret)):
+                    ret[n] = pad + ret[n]
                 continue
 
-            ret.append(ln)
+            ret.append(oln)
 
     ret.append("#\033[36m closed{}\033[0m".format(ipath))
+
+
+def upgrade_cfg_fmt(
+    log: Optional["NamedLogger"], args: argparse.Namespace, orig: list[str], cfg_fp: str
+) -> list[str]:
+    """convert from v1 to v2 format"""
+    zst = [x.split("#")[0].strip() for x in orig]
+    zst = [x for x in zst if x]
+    if (
+        "[global]" in zst
+        or "[accounts]" in zst
+        or "accs:" in zst
+        or "flags:" in zst
+        or [x for x in zst if x.startswith("[/")]
+        or len(zst) == len([x for x in zst if x.startswith("%")])
+    ):
+        return orig
+
+    zst = [x for x in orig if "#\033[36m opening cfg file" not in x]
+    incl = len(zst) != len(orig) - 1
+
+    t = "upgrading config file [{}] from v1 to v2"
+    if not args.vc:
+        t += ". Run with argument '--vc' to see the converted config if you want to upgrade"
+    if incl:
+        t += ". Please don't include v1 configs from v2 files or vice versa! Upgrade all of them at the same time."
+    if log:
+        log(t.format(cfg_fp), 3)
+
+    ret = []
+    vp = ""
+    ap = ""
+    cat = ""
+    catg = "[global]"
+    cata = "[accounts]"
+    catx = "  accs:"
+    catf = "  flags:"
+    for ln in orig:
+        sn = ln.strip()
+        if not sn:
+            cat = vp = ap = ""
+        if not sn.split("#")[0]:
+            ret.append(ln)
+        elif sn.startswith("-") and cat in ("", catg):
+            if cat != catg:
+                cat = catg
+                ret.append(cat)
+            sn = sn.lstrip("-")
+            zst = sn.split(" ", 1)
+            if len(zst) > 1:
+                sn = "{}: {}".format(zst[0], zst[1].strip())
+            ret.append("  " + sn)
+        elif sn.startswith("u ") and cat in ("", catg, cata):
+            if cat != cata:
+                cat = cata
+                ret.append(cat)
+            s1, s2 = sn[1:].split(":", 1)
+            ret.append("  {}: {}".format(s1.strip(), s2.strip()))
+        elif not ap:
+            ap = sn
+        elif not vp:
+            vp = "/" + sn.strip("/")
+            cat = "[{}]".format(vp)
+            ret.append(cat)
+            ret.append("  " + ap)
+        elif sn.startswith("c "):
+            if cat != catf:
+                cat = catf
+                ret.append(cat)
+            sn = sn[1:].strip()
+            if "=" in sn:
+                zst = sn.split("=", 1)
+                sn = zst[0].replace(",", ", ")
+                sn += ": " + zst[1]
+            else:
+                sn = sn.replace(",", ", ")
+            ret.append("    " + sn)
+        elif sn[:1] in "rwmdgG":
+            if cat != catx:
+                cat = catx
+                ret.append(cat)
+            zst = sn.split(" ")
+            zst = [x for x in zst if x]
+            if len(zst) == 1:
+                zst.append("*")
+            ret.append("    {}: {}".format(zst[0], ", ".join(zst[1:])))
+        else:
+            t = "did not understand line {} in the config"
+            t1 = t
+            n = 0
+            for ln in orig:
+                n += 1
+                t += "\n{:4} {}".format(n, ln)
+            if log:
+                log(t, 1)
+            else:
+                print("\033[31m" + t)
+            raise Exception(t1)
+
+    if args.vc and log:
+        t = "new config syntax (copy/paste this to upgrade your config):\n"
+        t += "\n# ======================[ begin upgraded config ]======================\n\n"
+        for ln in ret:
+            t += ln + "\n"
+        t += "\n# ======================[ end of upgraded config ]======================\n"
+        log(t)
+
+    return ret

copyparty/bos/bos.py

@@ -24,13 +24,15 @@ def listdir(p: str = ".") -> list[str]:
     return [fsdec(x) for x in os.listdir(fsenc(p))]
 
 
-def makedirs(name: str, mode: int = 0o755, exist_ok: bool = True) -> None:
+def makedirs(name: str, mode: int = 0o755, exist_ok: bool = True) -> bool:
     bname = fsenc(name)
     try:
         os.makedirs(bname, mode)
+        return True
     except:
         if not exist_ok or not os.path.isdir(bname):
             raise
+        return False
 
 
 def mkdir(p: str, mode: int = 0o755) -> None:

copyparty/broker_mp.py

@@ -3,6 +3,7 @@ from __future__ import print_function, unicode_literals
 
 import threading
 import time
+import traceback
 
 import queue
 
@@ -93,12 +94,15 @@ class BrokerMp(object):
 
             else:
                 # new ipc invoking managed service in hub
-                obj = self.hub
-                for node in dest.split("."):
-                    obj = getattr(obj, node)
+                try:
+                    obj = self.hub
+                    for node in dest.split("."):
+                        obj = getattr(obj, node)
 
-                # TODO will deadlock if dest performs another ipc
-                rv = try_exec(retq_id, obj, *args)
+                    # TODO will deadlock if dest performs another ipc
+                    rv = try_exec(retq_id, obj, *args)
+                except:
+                    rv = ["exception", "stack", traceback.format_exc()]
 
                 if retq_id:
                     proc.q_pend.put((retq_id, "retq", rv))

copyparty/cfg.py

@@ -0,0 +1,150 @@
+# coding: utf-8
+from __future__ import print_function, unicode_literals
+
+# awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
+zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nw p q s ss sss v z zv"
+onedash = set(zs.split())
+
+
+def vf_bmap() -> dict[str, str]:
+    """argv-to-volflag: simple bools"""
+    ret = {
+        "never_symlink": "neversymlink",
+        "no_dedup": "copydupes",
+        "no_dupe": "nodupe",
+        "no_forget": "noforget",
+    }
+    for k in (
+        "dotsrch",
+        "e2t",
+        "e2ts",
+        "e2tsr",
+        "e2v",
+        "e2vu",
+        "e2vp",
+        "hardlink",
+        "magic",
+        "no_sb_md",
+        "no_sb_lg",
+        "rand",
+        "xdev",
+        "xlink",
+        "xvol",
+    ):
+        ret[k] = k
+    return ret
+
+
+def vf_vmap() -> dict[str, str]:
+    """argv-to-volflag: simple values"""
+    ret = {}
+    for k in ("lg_sbf", "md_sbf"):
+        ret[k] = k
+    return ret
+
+
+def vf_cmap() -> dict[str, str]:
+    """argv-to-volflag: complex/lists"""
+    ret = {}
+    for k in ("dbd", "html_head", "mte", "mth", "nrand"):
+        ret[k] = k
+    return ret
+
+
+permdescs = {
+    "r": "read; list folder contents, download files",
+    "w": 'write; upload files; need "r" to see the uploads',
+    "m": 'move; move files and folders; need "w" at destination',
+    "d": "delete; permanently delete files and folders",
+    "g": "get; download files, but cannot see folder contents",
+    "G": 'upget; same as "g" but can see filekeys of their own uploads',
+}
+
+
+flagcats = {
+    "uploads, general": {
+        "nodupe": "rejects existing files (instead of symlinking them)",
+        "hardlink": "does dedup with hardlinks instead of symlinks",
+        "neversymlink": "disables symlink fallback; full copy instead",
+        "copydupes": "disables dedup, always saves full copies of dupes",
+        "daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
+        "nosub": "forces all uploads into the top folder of the vfs",
+        "magic": "enables filetype detection for nameless uploads",
+        "gz": "allows server-side gzip of uploads with ?gz (also c,xz)",
+        "pk": "forces server-side compression, optional arg: xz,9",
+    },
+    "upload rules": {
+        "maxn=250,600": "max 250 uploads over 15min",
+        "maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g)",
+        "rand": "force randomized filenames, 9 chars long by default",
+        "nrand=N": "randomized filenames are N chars long",
+        "sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
+        "df=1g": "ensure 1 GiB free disk space",
+    },
+    "upload rotation\n(moves all uploads into the specified folder structure)": {
+        "rotn=100,3": "3 levels of subfolders with 100 entries in each",
+        "rotf=%Y-%m/%d-%H": "date-formatted organizing",
+        "lifetime=3600": "uploads are deleted after 1 hour",
+    },
+    "database, general": {
+        "e2d": "enable database; makes files searchable + enables upload dedup",
+        "e2ds": "scan writable folders for new files on startup; also sets -e2d",
+        "e2dsa": "scans all folders for new files on startup; also sets -e2d",
+        "e2t": "enable multimedia indexing; makes it possible to search for tags",
+        "e2ts": "scan existing files for tags on startup; also sets -e2t",
+        "e2tsa": "delete all metadata from DB (full rescan); also sets -e2ts",
+        "d2ts": "disables metadata collection for existing files",
+        "d2ds": "disables onboot indexing, overrides -e2ds*",
+        "d2t": "disables metadata collection, overrides -e2t*",
+        "d2v": "disables file verification, overrides -e2v*",
+        "d2d": "disables all database stuff, overrides -e2*",
+        "hist=/tmp/cdb": "puts thumbnails and indexes at that location",
+        "scan=60": "scan for new files every 60sec, same as --re-maxage",
+        "nohash=\\.iso$": "skips hashing file contents if path matches *.iso",
+        "noidx=\\.iso$": "fully ignores the contents at paths matching *.iso",
+        "noforget": "don't forget files when deleted from disk",
+        "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
+        "xlink": "cross-volume dupe detection / linking",
+        "xdev": "do not descend into other filesystems",
+        "xvol": "skip symlinks leaving the volume root",
+        "dotsrch": "show dotfiles in search results",
+        "nodotsrch": "hide dotfiles in search results (default)",
+    },
+    'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
+        "mtp=.bpm=f,audio-bpm.py": 'uses the "audio-bpm.py" program to\ngenerate ".bpm" tags from uploads (f = overwrite tags)',
+        "mtp=ahash,vhash=media-hash.py": "collects two tags at once",
+    },
+    "thumbnails": {
+        "dthumb": "disables all thumbnails",
+        "dvthumb": "disables video thumbnails",
+        "dathumb": "disables audio thumbnails (spectrograms)",
+        "dithumb": "disables image thumbnails",
+    },
+    "event hooks\n(better explained in --help-hooks)": {
+        "xbu=CMD": "execute CMD before a file upload starts",
+        "xau=CMD": "execute CMD after  a file upload finishes",
+        "xiu=CMD": "execute CMD after  all uploads finish and volume is idle",
+        "xbr=CMD": "execute CMD before a file rename/move",
+        "xar=CMD": "execute CMD after  a file rename/move",
+        "xbd=CMD": "execute CMD before a file delete",
+        "xad=CMD": "execute CMD after  a file delete",
+        "xm=CMD": "execute CMD on message",
+    },
+    "client and ux": {
+        "html_head=TXT": "includes TXT in the <head>",
+        "robots": "allows indexing by search engines (default)",
+        "norobots": "kindly asks search engines to leave",
+        "no_sb_md": "disable js sandbox for markdown files",
+        "no_sb_lg": "disable js sandbox for prologue/epilogue",
+        "sb_md": "enable js sandbox for markdown files (default)",
+        "sb_lg": "enable js sandbox for prologue/epilogue (default)",
+        "md_sbf": "list of markdown-sandbox safeguards to disable",
+        "lg_sbf": "list of *logue-sandbox safeguards to disable",
+    },
+    "others": {
+        "fk=8": 'generates per-file accesskeys,\nwhich will then be required at the "g" permission'
+    },
+}
+
+
+flagdescs = {k.split("=")[0]: v for tab in flagcats.values() for k, v in tab.items()}

copyparty/ftpd.py

@@ -13,9 +13,21 @@ from pyftpdlib.filesystems import AbstractedFS, FilesystemError
 from pyftpdlib.handlers import FTPHandler
 from pyftpdlib.servers import FTPServer
 
-from .__init__ import PY2, TYPE_CHECKING, E
+from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
+from .authsrv import VFS
 from .bos import bos
-from .util import Daemon, Pebkac, exclude_dotfiles, fsenc, ipnorm
+from .util import (
+    Daemon,
+    Pebkac,
+    exclude_dotfiles,
+    fsenc,
+    ipnorm,
+    pybin,
+    relchk,
+    runhook,
+    sanitize_fn,
+    vjoin,
+)
 
 try:
     from pyftpdlib.ioloop import IOLoop
@@ -43,6 +55,7 @@ class FtpAuth(DummyAuthorizer):
         self, username: str, password: str, handler: Any
     ) -> None:
         handler.username = "{}:{}".format(username, password)
+        handler.uname = "*"
 
         ip = handler.addr[0]
         if ip.startswith("::ffff:"):
@@ -74,14 +87,14 @@ class FtpAuth(DummyAuthorizer):
 
             raise AuthenticationFailed("Authentication failed.")
 
-        handler.username = uname
+        handler.uname = uname
 
     def get_home_dir(self, username: str) -> str:
         return "/"
 
     def has_user(self, username: str) -> bool:
         asrv = self.hub.asrv
-        return username in asrv.acct
+        return username in asrv.acct or username in asrv.iacct
 
     def has_perm(self, username: str, perm: int, path: Optional[str] = None) -> bool:
         return True  # handled at filesystem layer
@@ -103,8 +116,7 @@ class FtpFs(AbstractedFS):
         self.h = self.cmd_channel = cmd_channel  # type: FTPHandler
         self.hub: "SvcHub" = cmd_channel.hub
         self.args = cmd_channel.args
-
-        self.uname = self.hub.asrv.iacct.get(cmd_channel.password, "*")
+        self.uname = cmd_channel.uname
 
         self.cwd = "/"  # pyftpdlib convention of leading slash
         self.root = "/var/lib/empty"
@@ -115,6 +127,10 @@ class FtpFs(AbstractedFS):
         self.listdirinfo = self.listdir
         self.chdir(".")
 
+    def die(self, msg):
+        self.h.die(msg)
+        raise Exception()
+
     def v2a(
         self,
         vpath: str,
@@ -122,16 +138,23 @@ class FtpFs(AbstractedFS):
         w: bool = False,
         m: bool = False,
         d: bool = False,
-    ) -> str:
+    ) -> tuple[str, VFS, str]:
         try:
             vpath = vpath.replace("\\", "/").lstrip("/")
+            rd, fn = os.path.split(vpath)
+            if ANYWIN and relchk(rd):
+                logging.warning("malicious vpath: %s", vpath)
+                self.die("Unsupported characters in filepath")
+
+            fn = sanitize_fn(fn or "", "", [".prologue.html", ".epilogue.html"])
+            vpath = vjoin(rd, fn)
             vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
             if not vfs.realpath:
-                raise FilesystemError("no filesystem mounted at this path")
+                self.die("No filesystem mounted at this path")
 
-            return os.path.join(vfs.realpath, rem)
+            return os.path.join(vfs.realpath, rem), vfs, rem
         except Pebkac as ex:
-            raise FilesystemError(str(ex))
+            self.die(str(ex))
 
     def rv2a(
         self,
@@ -140,7 +163,7 @@ class FtpFs(AbstractedFS):
         w: bool = False,
         m: bool = False,
         d: bool = False,
-    ) -> str:
+    ) -> tuple[str, VFS, str]:
         return self.v2a(os.path.join(self.cwd, vpath), r, w, m, d)
 
     def ftp2fs(self, ftppath: str) -> str:
@@ -154,7 +177,7 @@ class FtpFs(AbstractedFS):
     def validpath(self, path: str) -> bool:
         if "/.hist/" in path:
             if "/up2k." in path or path.endswith("/dir.txt"):
-                raise FilesystemError("access to this file is forbidden")
+                self.die("Access to this file is forbidden")
 
         return True
 
@@ -162,7 +185,7 @@ class FtpFs(AbstractedFS):
         r = "r" in mode
         w = "w" in mode or "a" in mode or "+" in mode
 
-        ap = self.rv2a(filename, r, w)
+        ap = self.rv2a(filename, r, w)[0]
         if w:
             try:
                 st = bos.stat(ap)
@@ -171,7 +194,7 @@ class FtpFs(AbstractedFS):
                 td = 0
 
             if td < -1 or td > self.args.ftp_wt:
-                raise FilesystemError("cannot open existing file for writing")
+                self.die("Cannot open existing file for writing")
 
         self.validpath(ap)
         return open(fsenc(ap), mode)
@@ -182,7 +205,7 @@ class FtpFs(AbstractedFS):
         ap = vfs.canonical(rem)
         if not bos.path.isdir(ap):
             # returning 550 is library-default and suitable
-            raise FilesystemError("Failed to change directory")
+            self.die("Failed to change directory")
 
         self.cwd = nwd
         (
@@ -192,11 +215,11 @@ class FtpFs(AbstractedFS):
             self.can_delete,
             self.can_get,
             self.can_upget,
-        ) = self.hub.asrv.vfs.can_access(self.cwd.lstrip("/"), self.h.username)
+        ) = self.hub.asrv.vfs.can_access(self.cwd.lstrip("/"), self.h.uname)
 
     def mkdir(self, path: str) -> None:
-        ap = self.rv2a(path, w=True)
-        bos.mkdir(ap)
+        ap = self.rv2a(path, w=True)[0]
+        bos.makedirs(ap)  # filezilla expects this
 
     def listdir(self, path: str) -> list[str]:
         vpath = join(self.cwd, path).lstrip("/")
@@ -227,43 +250,42 @@ class FtpFs(AbstractedFS):
             return list(sorted(list(r.keys())))
 
     def rmdir(self, path: str) -> None:
-        ap = self.rv2a(path, d=True)
+        ap = self.rv2a(path, d=True)[0]
         bos.rmdir(ap)
 
     def remove(self, path: str) -> None:
         if self.args.no_del:
-            raise FilesystemError("the delete feature is disabled in server config")
+            self.die("The delete feature is disabled in server config")
 
         vp = join(self.cwd, path).lstrip("/")
         try:
-            self.hub.up2k.handle_rm(self.uname, self.h.remote_ip, [vp], [])
+            self.hub.up2k.handle_rm(self.uname, self.h.cli_ip, [vp], [])
         except Exception as ex:
-            raise FilesystemError(str(ex))
+            self.die(str(ex))
 
     def rename(self, src: str, dst: str) -> None:
         if not self.can_move:
-            raise FilesystemError("not allowed for user " + self.h.username)
+            self.die("Not allowed for user " + self.h.uname)
 
         if self.args.no_mv:
-            t = "the rename/move feature is disabled in server config"
-            raise FilesystemError(t)
+            self.die("The rename/move feature is disabled in server config")
 
         svp = join(self.cwd, src).lstrip("/")
         dvp = join(self.cwd, dst).lstrip("/")
         try:
             self.hub.up2k.handle_mv(self.uname, svp, dvp)
         except Exception as ex:
-            raise FilesystemError(str(ex))
+            self.die(str(ex))
 
     def chmod(self, path: str, mode: str) -> None:
         pass
 
     def stat(self, path: str) -> os.stat_result:
         try:
-            ap = self.rv2a(path, r=True)
+            ap = self.rv2a(path, r=True)[0]
             return bos.stat(ap)
         except:
-            ap = self.rv2a(path)
+            ap = self.rv2a(path)[0]
             st = bos.stat(ap)
             if not stat.S_ISDIR(st.st_mode):
                 raise
@@ -271,11 +293,11 @@ class FtpFs(AbstractedFS):
             return st
 
     def utime(self, path: str, timeval: float) -> None:
-        ap = self.rv2a(path, w=True)
+        ap = self.rv2a(path, w=True)[0]
         return bos.utime(ap, (timeval, timeval))
 
     def lstat(self, path: str) -> os.stat_result:
-        ap = self.rv2a(path)
+        ap = self.rv2a(path)[0]
         return bos.stat(ap)
 
     def isfile(self, path: str) -> bool:
@@ -286,7 +308,7 @@ class FtpFs(AbstractedFS):
             return False  # expected for mojibake in ftp_SIZE()
 
     def islink(self, path: str) -> bool:
-        ap = self.rv2a(path)
+        ap = self.rv2a(path)[0]
         return bos.path.islink(ap)
 
     def isdir(self, path: str) -> bool:
@@ -297,18 +319,18 @@ class FtpFs(AbstractedFS):
             return True
 
     def getsize(self, path: str) -> int:
-        ap = self.rv2a(path)
+        ap = self.rv2a(path)[0]
         return bos.path.getsize(ap)
 
     def getmtime(self, path: str) -> float:
-        ap = self.rv2a(path)
+        ap = self.rv2a(path)[0]
         return bos.path.getmtime(ap)
 
     def realpath(self, path: str) -> str:
         return path
 
     def lexists(self, path: str) -> bool:
-        ap = self.rv2a(path)
+        ap = self.rv2a(path)[0]
         return bos.path.lexists(ap)
 
     def get_user_by_uid(self, uid: int) -> str:
@@ -322,27 +344,52 @@ class FtpHandler(FTPHandler):
     abstracted_fs = FtpFs
     hub: "SvcHub"
     args: argparse.Namespace
+    uname: str
 
     def __init__(self, conn: Any, server: Any, ioloop: Any = None) -> None:
         self.hub: "SvcHub" = FtpHandler.hub
         self.args: argparse.Namespace = FtpHandler.args
+        self.uname = "*"
 
         if PY2:
             FTPHandler.__init__(self, conn, server, ioloop)
         else:
             super(FtpHandler, self).__init__(conn, server, ioloop)
 
+        cip = self.remote_ip
+        self.cli_ip = cip[7:] if cip.startswith("::ffff:") else cip
+
         # abspath->vpath mapping to resolve log_transfer paths
         self.vfs_map: dict[str, str] = {}
 
         # reduce non-debug logging
         self.log_cmds_list = [x for x in self.log_cmds_list if x not in ("CWD", "XCWD")]
 
+    def die(self, msg):
+        self.respond("550 {}".format(msg))
+        raise FilesystemError(msg)
+
     def ftp_STOR(self, file: str, mode: str = "w") -> Any:
         # Optional[str]
         vp = join(self.fs.cwd, file).lstrip("/")
-        ap = self.fs.v2a(vp)
+        ap, vfs, rem = self.fs.v2a(vp)
         self.vfs_map[ap] = vp
+        xbu = vfs.flags.get("xbu")
+        if xbu and not runhook(
+            None,
+            xbu,
+            ap,
+            vfs.canonical(rem),
+            "",
+            self.uname,
+            0,
+            0,
+            self.cli_ip,
+            0,
+            "",
+        ):
+            self.die("Upload blocked by xbu server config")
+
         # print("ftp_STOR: {} {} => {}".format(vp, mode, ap))
         ret = FTPHandler.ftp_STOR(self, file, mode)
         # print("ftp_STOR: {} {} OK".format(vp, mode))
@@ -363,15 +410,17 @@ class FtpHandler(FTPHandler):
         # print("xfer_end: {} => {}".format(ap, vp))
         if vp:
             vp, fn = os.path.split(vp)
-            vfs, rem = self.hub.asrv.vfs.get(vp, self.username, False, True)
+            vfs, rem = self.hub.asrv.vfs.get(vp, self.uname, False, True)
             vfs, rem = vfs.get_dbv(rem)
             self.hub.up2k.hash_file(
                 vfs.realpath,
+                vfs.vpath,
                 vfs.flags,
                 rem,
                 fn,
-                self.remote_ip,
+                self.cli_ip,
                 time.time(),
+                self.uname,
             )
 
         return FTPHandler.log_transfer(
@@ -402,10 +451,10 @@ class Ftpd(object):
                 h1 = SftpHandler
             except:
                 t = "\nftps requires pyopenssl;\nplease run the following:\n\n  {} -m pip install --user pyopenssl\n"
-                print(t.format(sys.executable))
+                print(t.format(pybin))
                 sys.exit(1)
 
-            h1.certfile = os.path.join(self.args.E.cfg, "cert.pem")
+            h1.certfile = self.args.cert
             h1.tls_control_required = True
             h1.tls_data_required = True
 
@@ -435,10 +484,18 @@ class Ftpd(object):
         lgr = logging.getLogger("pyftpdlib")
         lgr.setLevel(logging.DEBUG if self.args.ftpv else logging.INFO)
 
+        ips = self.args.i
+        if "::" in ips:
+            ips.append("0.0.0.0")
+
         ioloop = IOLoop()
-        for ip in self.args.i:
+        for ip in ips:
             for h, lp in hs:
-                FTPServer((ip, int(lp)), h, ioloop)
+                try:
+                    FTPServer((ip, int(lp)), h, ioloop)
+                except:
+                    if ip != "0.0.0.0" or "::" not in ips:
+                        raise
 
         Daemon(ioloop.loop, "ftp")
 

copyparty/httpconn.py

@@ -65,6 +65,7 @@ class HttpConn(object):
         self.ico: Ico = Ico(self.args)  # mypy404
 
         self.t0: float = time.time()  # mypy404
+        self.freshen_pwd: float = 0.0
         self.stopping = False
         self.nreq: int = -1  # mypy404
         self.nbyte: int = 0  # mypy404
@@ -102,11 +103,12 @@ class HttpConn(object):
     def log(self, msg: str, c: Union[int, str] = 0) -> None:
         self.log_func(self.log_src, msg, c)
 
-    def get_u2idx(self) -> U2idx:
-        # one u2idx per tcp connection;
+    def get_u2idx(self) -> Optional[U2idx]:
+        # grab from a pool of u2idx instances;
         # sqlite3 fully parallelizes under python threads
+        # but avoid running out of FDs by creating too many
         if not self.u2idx:
-            self.u2idx = U2idx(self)
+            self.u2idx = self.hsrv.get_u2idx(str(self.addr))
 
         return self.u2idx
 
@@ -214,3 +216,7 @@ class HttpConn(object):
             self.cli = HttpCli(self)
             if not self.cli.run():
                 return
+
+            if self.u2idx:
+                self.hsrv.put_u2idx(str(self.addr), self.u2idx)
+                self.u2idx = None

copyparty/httpsrv.py

@@ -11,9 +11,19 @@ import time
 
 import queue
 
+from .__init__ import ANYWIN, CORES, EXE, MACOS, TYPE_CHECKING, EnvParams
+
+try:
+    MNFE = ModuleNotFoundError
+except:
+    MNFE = ImportError
+
 try:
     import jinja2
-except ImportError:
+except MNFE:
+    if EXE:
+        raise
+
     print(
         """\033[1;31m
   you do not have jinja2 installed,\033[33m
@@ -28,9 +38,9 @@ except ImportError:
     )
     sys.exit(1)
 
-from .__init__ import MACOS, TYPE_CHECKING, EnvParams
 from .bos import bos
 from .httpconn import HttpConn
+from .u2idx import U2idx
 from .util import (
     E_SCK,
     FHC,
@@ -81,8 +91,7 @@ class HttpSrv(object):
         self.bans: dict[str, int] = {}
         self.aclose: dict[str, int] = {}
 
-        self.ip = ""
-        self.port = 0
+        self.bound: set[tuple[str, int]] = set()
         self.name = "hsrv" + nsuf
         self.mutex = threading.Lock()
         self.stopping = False
@@ -103,6 +112,9 @@ class HttpSrv(object):
         self.cb_ts = 0.0
         self.cb_v = ""
 
+        self.u2idx_free: dict[str, U2idx] = {}
+        self.u2idx_n = 0
+
         env = jinja2.Environment()
         env.loader = jinja2.FileSystemLoader(os.path.join(self.E.mod, "web"))
         jn = ["splash", "svcs", "browser", "browser2", "msg", "md", "mde", "cf"]
@@ -110,12 +122,17 @@ class HttpSrv(object):
         zs = os.path.join(self.E.mod, "web", "deps", "prism.js.gz")
         self.prism = os.path.exists(zs)
 
+        self.mallow = "GET HEAD POST PUT DELETE OPTIONS".split()
+        if not self.args.no_dav:
+            zs = "PROPFIND PROPPATCH LOCK UNLOCK MKCOL COPY MOVE"
+            self.mallow += zs.split()
+
         if self.args.zs:
             from .ssdp import SSDPr
 
             self.ssdp = SSDPr(broker)
 
-        cert_path = os.path.join(self.E.cfg, "cert.pem")
+        cert_path = self.args.cert
         if bos.path.exists(cert_path):
             self.cert_path = cert_path
         else:
@@ -142,7 +159,11 @@ class HttpSrv(object):
             pass
 
     def set_netdevs(self, netdevs: dict[str, Netdev]) -> None:
-        self.nm = NetMap([self.ip], netdevs)
+        ips = set()
+        for ip, _ in self.bound:
+            ips.add(ip)
+
+        self.nm = NetMap(list(ips), netdevs)
 
     def start_threads(self, n: int) -> None:
         self.tp_nthr += n
@@ -178,20 +199,19 @@ class HttpSrv(object):
     def listen(self, sck: socket.socket, nlisteners: int) -> None:
         if self.args.j != 1:
             # lost in the pickle; redefine
-            try:
-                sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
-            except:
-                pass
-            sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-            sck.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
-            sck.settimeout(None)  # < does not inherit, ^ does
+            if not ANYWIN or self.args.reuseaddr:
+                sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
 
-        self.ip, self.port = sck.getsockname()[:2]
+            sck.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
+            sck.settimeout(None)  # < does not inherit, ^ opts above do
+
+        ip, port = sck.getsockname()[:2]
         self.srvs.append(sck)
+        self.bound.add((ip, port))
         self.nclimax = math.ceil(self.args.nc * 1.0 / nlisteners)
         Daemon(
             self.thr_listen,
-            "httpsrv-n{}-listen-{}-{}".format(self.nid or "0", self.ip, self.port),
+            "httpsrv-n{}-listen-{}-{}".format(self.nid or "0", ip, port),
             (sck,),
         )
 
@@ -429,6 +449,9 @@ class HttpSrv(object):
                     self.clients.remove(cli)
                     self.ncli -= 1
 
+                if cli.u2idx:
+                    self.put_u2idx(str(addr), cli.u2idx)
+
     def cachebuster(self) -> str:
         if time.time() - self.cb_ts < 1:
             return self.cb_v
@@ -450,3 +473,31 @@ class HttpSrv(object):
             self.cb_v = v.decode("ascii")[-4:]
             self.cb_ts = time.time()
             return self.cb_v
+
+    def get_u2idx(self, ident: str) -> Optional[U2idx]:
+        utab = self.u2idx_free
+        for _ in range(100):  # 5/0.05 = 5sec
+            with self.mutex:
+                if utab:
+                    if ident in utab:
+                        return utab.pop(ident)
+
+                    return utab.pop(list(utab.keys())[0])
+
+                if self.u2idx_n < CORES:
+                    self.u2idx_n += 1
+                    return U2idx(self)
+
+            time.sleep(0.05)
+            # not using conditional waits, on a hunch that
+            # average performance will be faster like this
+            # since most servers won't be fully saturated
+
+        return None
+
+    def put_u2idx(self, ident: str, u2idx: U2idx) -> None:
+        with self.mutex:
+            while ident in self.u2idx_free:
+                ident += "a"
+
+            self.u2idx_free[ident] = u2idx

copyparty/mdns.py

@@ -11,6 +11,7 @@ from ipaddress import IPv4Network, IPv6Network
 from .__init__ import TYPE_CHECKING
 from .__init__ import unicode as U
 from .multicast import MC_Sck, MCast
+from .stolen.dnslib import AAAA
 from .stolen.dnslib import CLASS as DC
 from .stolen.dnslib import (
     NSEC,
@@ -20,12 +21,11 @@ from .stolen.dnslib import (
     SRV,
     TXT,
     A,
-    AAAA,
     DNSHeader,
     DNSQuestion,
     DNSRecord,
 )
-from .util import CachedSet, Daemon, Netdev, min_ex
+from .util import CachedSet, Daemon, Netdev, list_ips, min_ex
 
 if TYPE_CHECKING:
     from .svchub import SvcHub
@@ -55,10 +55,11 @@ class MDNS_Sck(MC_Sck):
         self.bp_bye = b""
 
         self.last_tx = 0.0
+        self.tx_ex = False
 
 
 class MDNS(MCast):
-    def __init__(self, hub: "SvcHub") -> None:
+    def __init__(self, hub: "SvcHub", ngen: int) -> None:
         al = hub.args
         grp4 = "" if al.zm6 else MDNS4
         grp6 = "" if al.zm4 else MDNS6
@@ -66,7 +67,8 @@ class MDNS(MCast):
             hub, MDNS_Sck, al.zm_on, al.zm_off, grp4, grp6, 5353, hub.args.zmv
         )
         self.srv: dict[socket.socket, MDNS_Sck] = {}
-
+        self.logsrc = "mDNS-{}".format(ngen)
+        self.ngen = ngen
         self.ttl = 300
 
         zs = self.args.name + ".local."
@@ -89,7 +91,7 @@ class MDNS(MCast):
         self.defend: dict[MDNS_Sck, float] = {}  # server -> deadline
 
     def log(self, msg: str, c: Union[int, str] = 0) -> None:
-        self.log_func("mDNS", msg, c)
+        self.log_func(self.logsrc, msg, c)
 
     def build_svcs(self) -> tuple[dict[str, dict[str, Any]], set[str]]:
         zms = self.args.zms
@@ -287,12 +289,15 @@ class MDNS(MCast):
             rx: list[socket.socket] = rdy[0]  # type: ignore
             self.rx4.cln()
             self.rx6.cln()
+            buf = b""
+            addr = ("0", 0)
             for sck in rx:
-                buf, addr = sck.recvfrom(4096)
                 try:
+                    buf, addr = sck.recvfrom(4096)
                     self.eat(buf, addr, sck)
                 except:
                     if not self.running:
+                        self.log("stopped", 2)
                         return
 
                     t = "{} {} \033[33m|{}| {}\n{}".format(
@@ -309,29 +314,38 @@ class MDNS(MCast):
                 self.log(t.format(self.hn[:-1]), 2)
                 self.probing = 0
 
+        self.log("stopped", 2)
+
     def stop(self, panic=False) -> None:
         self.running = False
-        if not panic:
-            for srv in self.srv.values():
-                srv.sck.sendto(srv.bp_bye, (srv.grp, 5353))
+        for srv in self.srv.values():
+            try:
+                if panic:
+                    srv.sck.close()
+                else:
+                    srv.sck.sendto(srv.bp_bye, (srv.grp, 5353))
+            except:
+                pass
 
         self.srv = {}
 
     def eat(self, buf: bytes, addr: tuple[str, int], sck: socket.socket) -> None:
         cip = addr[0]
         v6 = ":" in cip
-        if cip.startswith("169.254") or v6 and not cip.startswith("fe80"):
+        if (cip.startswith("169.254") and not self.ll_ok) or (
+            v6 and not cip.startswith("fe80")
+        ):
             return
 
         cache = self.rx6 if v6 else self.rx4
         if buf in cache.c:
             return
 
-        cache.add(buf)
         srv: Optional[MDNS_Sck] = self.srv[sck] if v6 else self.map_client(cip)  # type: ignore
         if not srv:
             return
 
+        cache.add(buf)
         now = time.time()
 
         if self.args.zmv and cip != srv.ip and cip not in srv.ips:
@@ -369,6 +383,14 @@ class MDNS(MCast):
                 # avahi broadcasting 127.0.0.1-only packets
                 return
 
+            # check if we've been given additional IPs
+            for ip in list_ips():
+                if ip in cips:
+                    self.sips.add(ip)
+
+            if not self.sips.isdisjoint(cips):
+                return
+
             t = "mdns zeroconf: "
             if self.probing:
                 t += "Cannot start; hostname '{}' is occupied"
@@ -502,6 +524,15 @@ class MDNS(MCast):
         if now < srv.last_tx + cooldown:
             return False
 
-        srv.sck.sendto(msg, (srv.grp, 5353))
-        srv.last_tx = now
+        try:
+            srv.sck.sendto(msg, (srv.grp, 5353))
+            srv.last_tx = now
+        except Exception as ex:
+            if srv.tx_ex:
+                return True
+
+            srv.tx_ex = True
+            t = "tx({},|{}|,{}): {}"
+            self.log(t.format(srv.ip, len(msg), cooldown, ex), 3)
+
         return True

copyparty/mtag.py

@@ -8,9 +8,19 @@ import shutil
 import subprocess as sp
 import sys
 
-from .__init__ import PY2, WINDOWS, E, unicode
+from .__init__ import EXE, PY2, WINDOWS, E, unicode
 from .bos import bos
-from .util import REKOBO_LKEY, fsenc, min_ex, retchk, runcmd, uncyg
+from .util import (
+    FFMPEG_URL,
+    REKOBO_LKEY,
+    fsenc,
+    min_ex,
+    pybin,
+    retchk,
+    runcmd,
+    sfsenc,
+    uncyg,
+)
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Union
@@ -259,7 +269,9 @@ class MTag(object):
         self.args = args
         self.usable = True
         self.prefer_mt = not args.no_mtag_ff
-        self.backend = "ffprobe" if args.no_mutagen else "mutagen"
+        self.backend = (
+            "ffprobe" if args.no_mutagen or (HAVE_FFPROBE and EXE) else "mutagen"
+        )
         self.can_ffprobe = HAVE_FFPROBE and not args.no_mtag_ff
         mappings = args.mtm
         or_ffprobe = " or FFprobe"
@@ -285,9 +297,14 @@ class MTag(object):
                 self.log(msg, c=3)
 
         if not self.usable:
+            if EXE:
+                t = "copyparty.exe cannot use mutagen; need ffprobe.exe to read media tags: "
+                self.log(t + FFMPEG_URL)
+                return
+
             msg = "need Mutagen{} to read media tags so please run this:\n{}{} -m pip install --user mutagen\n"
-            pybin = os.path.basename(sys.executable)
-            self.log(msg.format(or_ffprobe, " " * 37, pybin), c=1)
+            pyname = os.path.basename(pybin)
+            self.log(msg.format(or_ffprobe, " " * 37, pyname), c=1)
             return
 
         # https://picard-docs.musicbrainz.org/downloads/MusicBrainz_Picard_Tag_Map.html
@@ -456,7 +473,10 @@ class MTag(object):
                     self.log("mutagen: {}\033[0m".format(" ".join(zl)), "90")
             if not md.info.length and not md.info.codec:
                 raise Exception()
-        except:
+        except Exception as ex:
+            if self.args.mtag_v:
+                self.log("mutagen-err [{}] @ [{}]".format(ex, abspath), "90")
+
             return self.get_ffprobe(abspath) if self.can_ffprobe else {}
 
         sz = bos.path.getsize(abspath)
@@ -519,12 +539,15 @@ class MTag(object):
 
         env = os.environ.copy()
         try:
+            if EXE:
+                raise Exception()
+
             pypath = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
             zsl = [str(pypath)] + [str(x) for x in sys.path if x]
             pypath = str(os.pathsep.join(zsl))
             env["PYTHONPATH"] = pypath
         except:
-            if not E.ox:
+            if not E.ox and not EXE:
                 raise
 
         ret: dict[str, Any] = {}
@@ -532,7 +555,7 @@ class MTag(object):
             try:
                 cmd = [parser.bin, abspath]
                 if parser.bin.endswith(".py"):
-                    cmd = [sys.executable] + cmd
+                    cmd = [pybin] + cmd
 
                 args = {
                     "env": env,
@@ -551,7 +574,7 @@ class MTag(object):
                 else:
                     cmd = ["nice"] + cmd
 
-                bcmd = [fsenc(x) for x in cmd]
+                bcmd = [sfsenc(x) for x in cmd[:-1]] + [fsenc(cmd[-1])]
                 rc, v, err = runcmd(bcmd, **args)  # type: ignore
                 retchk(rc, bcmd, err, self.log, 5, self.args.mtag_v)
                 v = v.strip()

copyparty/multicast.py

@@ -5,10 +5,17 @@ import socket
 import time
 
 import ipaddress
-from ipaddress import IPv4Address, IPv4Network, IPv6Address, IPv6Network
+from ipaddress import (
+    IPv4Address,
+    IPv4Network,
+    IPv6Address,
+    IPv6Network,
+    ip_address,
+    ip_network,
+)
 
-from .__init__ import TYPE_CHECKING
-from .util import MACOS, Netdev, min_ex, spack
+from .__init__ import MACOS, TYPE_CHECKING
+from .util import Netdev, find_prefix, min_ex, spack
 
 if TYPE_CHECKING:
     from .svchub import SvcHub
@@ -75,6 +82,7 @@ class MCast(object):
 
         self.srv: dict[socket.socket, MC_Sck] = {}  # listening sockets
         self.sips: set[str] = set()  # all listening ips (including failed attempts)
+        self.ll_ok: set[str] = set()  # fallback linklocal IPv4 and IPv6 addresses
         self.b2srv: dict[bytes, MC_Sck] = {}  # binary-ip -> server socket
         self.b4: list[bytes] = []  # sorted list of binary-ips
         self.b6: list[bytes] = []  # sorted list of binary-ips
@@ -102,15 +110,23 @@ class MCast(object):
             )
 
         ips = [x for x in ips if x not in ("::1", "127.0.0.1")]
-
-        # ip -> ip/prefix
-        ips = [[x for x in netdevs if x.startswith(y + "/")][0] for y in ips]
+        ips = find_prefix(ips, netdevs)
 
         on = self.on[:]
         off = self.off[:]
         for lst in (on, off):
             for av in list(lst):
+                try:
+                    arg_net = ip_network(av, False)
+                except:
+                    arg_net = None
+
                 for sk, sv in netdevs.items():
+                    if arg_net:
+                        net_ip = ip_address(sk.split("/")[0])
+                        if net_ip in arg_net and sk not in lst:
+                            lst.append(sk)
+
                     if (av == str(sv.idx) or av == sv.name) and sk not in lst:
                         lst.append(sk)
 
@@ -166,9 +182,21 @@ class MCast(object):
                 srv.ips[oth_ip.split("/")[0]] = ipaddress.ip_network(oth_ip, False)
 
             # gvfs breaks if a linklocal ip appears in a dns reply
-            srv.ips = {k: v for k, v in srv.ips.items() if not k.startswith("fe80")}
+            ll = {
+                k: v
+                for k, v in srv.ips.items()
+                if k.startswith("169.254") or k.startswith("fe80")
+            }
+            rt = {k: v for k, v in srv.ips.items() if k not in ll}
+
+            if self.args.ll or not rt:
+                self.ll_ok.update(list(ll))
+
+            if not self.args.ll:
+                srv.ips = rt or ll
+
             if not srv.ips:
-                self.log("no routable IPs on {}; skipping [{}]".format(netdev, ip), 3)
+                self.log("no IPs on {}; skipping [{}]".format(netdev, ip), 3)
                 continue
 
             try:
@@ -318,6 +346,16 @@ class MCast(object):
             # just give it something
             ret = list(self.srv.values())[0]
 
+        if not ret and cip.startswith("169.254"):
+            # idk how to map LL IPv4 msgs to nics;
+            # just pick one and hope for the best
+            lls = (
+                x
+                for x in self.srv.values()
+                if next((y for y in x.ips if y in self.ll_ok), None)
+            )
+            ret = next(lls, None)
+
         if ret:
             t = "new client on {} ({}): {}"
             self.log(t.format(ret.name, ret.net, cip), 6)

copyparty/smbd.py

@@ -9,13 +9,13 @@ import sys
 import time
 from types import SimpleNamespace
 
-from .__init__ import ANYWIN, TYPE_CHECKING
+from .__init__ import ANYWIN, EXE, TYPE_CHECKING
 from .authsrv import LEELOO_DALLAS, VFS
 from .bos import bos
-from .util import Daemon, min_ex
+from .util import Daemon, min_ex, pybin, runhook
 
 if True:  # pylint: disable=using-constant-test
-    from typing import Any
+    from typing import Any, Union
 
 if TYPE_CHECKING:
     from .svchub import SvcHub
@@ -42,8 +42,12 @@ class SMB(object):
             from impacket import smbserver
             from impacket.ntlm import compute_lmhash, compute_nthash
         except ImportError:
+            if EXE:
+                print("copyparty.exe cannot do SMB")
+                sys.exit(1)
+
             m = "\033[36m\n{}\033[31m\n\nERROR: need 'impacket'; please run this command:\033[33m\n {} -m pip install --user impacket\n\033[0m"
-            print(m.format(min_ex(), sys.executable))
+            print(m.format(min_ex(), pybin))
             sys.exit(1)
 
         # patch vfs into smbserver.os
@@ -109,6 +113,9 @@ class SMB(object):
         self.stop = srv.stop
         self.log("smb", "listening @ {}:{}".format(ip, port))
 
+    def nlog(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.log("smb", msg, c)
+
     def start(self) -> None:
         Daemon(self.srv.start)
 
@@ -165,8 +172,15 @@ class SMB(object):
             yeet("blocked write (no --smbw): " + vpath)
 
         vfs, ap = self._v2a("open", vpath, *a)
-        if wr and not vfs.axs.uwrite:
-            yeet("blocked write (no-write-acc): " + vpath)
+        if wr:
+            if not vfs.axs.uwrite:
+                yeet("blocked write (no-write-acc): " + vpath)
+
+            xbu = vfs.flags.get("xbu")
+            if xbu and not runhook(
+                self.nlog, xbu, ap, vpath, "", "", 0, 0, "1.7.6.2", 0, ""
+            ):
+                yeet("blocked by xbu server config: " + vpath)
 
         ret = bos.open(ap, flags, *a, mode=chmod, **ka)
         if wr:
@@ -194,11 +208,13 @@ class SMB(object):
         vfs, rem = vfs.get_dbv(rem)
         self.hub.up2k.hash_file(
             vfs.realpath,
+            vfs.vpath,
             vfs.flags,
             rem,
             fn,
             "1.7.6.2",
             time.time(),
+            "",
         )
 
     def _rename(self, vp1: str, vp2: str) -> None:

copyparty/ssdp.py

@@ -8,7 +8,7 @@ from email.utils import formatdate
 
 from .__init__ import TYPE_CHECKING
 from .multicast import MC_Sck, MCast
-from .util import CachedSet, min_ex
+from .util import CachedSet, html_escape, min_ex
 
 if TYPE_CHECKING:
     from .broker_util import BrokerCli
@@ -73,13 +73,15 @@ class SSDPr(object):
     </device>
 </root>"""
 
+        c = html_escape
         sip, sport = hc.s.getsockname()[:2]
+        sip = sip.replace("::ffff:", "")
         proto = "https" if self.args.https_only else "http"
         ubase = "{}://{}:{}".format(proto, sip, sport)
         zsl = self.args.zsl
         url = zsl if "://" in zsl else ubase + "/" + zsl.lstrip("/")
         name = "{} @ {}".format(self.args.doctitle, self.args.name)
-        zs = zs.strip().format(ubase, url, name, self.args.zsid)
+        zs = zs.strip().format(c(ubase), c(url), c(name), c(self.args.zsid))
         hc.reply(zs.encode("utf-8", "replace"))
         return False  # close connectino
 
@@ -87,19 +89,22 @@ class SSDPr(object):
 class SSDPd(MCast):
     """communicates with ssdp clients over multicast"""
 
-    def __init__(self, hub: "SvcHub") -> None:
+    def __init__(self, hub: "SvcHub", ngen: int) -> None:
         al = hub.args
         vinit = al.zsv and not al.zmv
         super(SSDPd, self).__init__(
             hub, SSDP_Sck, al.zs_on, al.zs_off, GRP, "", 1900, vinit
         )
         self.srv: dict[socket.socket, SSDP_Sck] = {}
+        self.logsrc = "SSDP-{}".format(ngen)
+        self.ngen = ngen
+
         self.rxc = CachedSet(0.7)
         self.txc = CachedSet(5)  # win10: every 3 sec
         self.ptn_st = re.compile(b"\nst: *upnp:rootdevice", re.I)
 
     def log(self, msg: str, c: Union[int, str] = 0) -> None:
-        self.log_func("SSDP", msg, c)
+        self.log_func(self.logsrc, msg, c)
 
     def run(self) -> None:
         try:
@@ -125,41 +130,51 @@ class SSDPd(MCast):
 
         self.log("listening")
         while self.running:
-            rdy = select.select(self.srv, [], [], 180)
+            rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
             rx: list[socket.socket] = rdy[0]  # type: ignore
             self.rxc.cln()
+            buf = b""
+            addr = ("0", 0)
             for sck in rx:
-                buf, addr = sck.recvfrom(4096)
                 try:
+                    buf, addr = sck.recvfrom(4096)
                     self.eat(buf, addr)
                 except:
                     if not self.running:
-                        return
+                        break
 
                     t = "{} {} \033[33m|{}| {}\n{}".format(
                         self.srv[sck].name, addr, len(buf), repr(buf)[2:-1], min_ex()
                     )
                     self.log(t, 6)
 
+        self.log("stopped", 2)
+
     def stop(self) -> None:
         self.running = False
+        for srv in self.srv.values():
+            try:
+                srv.sck.close()
+            except:
+                pass
+
         self.srv = {}
 
     def eat(self, buf: bytes, addr: tuple[str, int]) -> None:
         cip = addr[0]
-        if cip.startswith("169.254"):
+        if cip.startswith("169.254") and not self.ll_ok:
             return
 
         if buf in self.rxc.c:
             return
 
-        self.rxc.add(buf)
         srv: Optional[SSDP_Sck] = self.map_client(cip)  # type: ignore
         if not srv:
             return
 
+        self.rxc.add(buf)
         if not buf.startswith(b"M-SEARCH * HTTP/1."):
-            raise Exception("not an ssdp message")
+            return
 
         if not self.ptn_st.search(buf):
             return
@@ -183,7 +198,8 @@ BOOTID.UPNP.ORG: 0
 CONFIGID.UPNP.ORG: 1
 
 """
-        zs = zs.format(formatdate(usegmt=True), srv.ip, srv.hport, self.args.zsid)
+        v4 = srv.ip.replace("::ffff:", "")
+        zs = zs.format(formatdate(usegmt=True), v4, srv.hport, self.args.zsid)
         zb = zs[1:].replace("\n", "\r\n").encode("utf-8", "replace")
         srv.sck.sendto(zb, addr[:2])
 

copyparty/star.py

@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
+import stat
 import tarfile
 
 from queue import Queue
@@ -79,6 +80,9 @@ class StreamTar(StreamArc):
         src = f["ap"]
         fsi = f["st"]
 
+        if stat.S_ISDIR(fsi.st_mode):
+            return
+
         inf = tarfile.TarInfo(name=name)
         inf.mode = fsi.st_mode
         inf.size = fsi.st_size

copyparty/svchub.py

@@ -4,6 +4,7 @@ from __future__ import print_function, unicode_literals
 import argparse
 import base64
 import calendar
+import errno
 import gzip
 import logging
 import os
@@ -27,13 +28,14 @@ if True:  # pylint: disable=using-constant-test
     import typing
     from typing import Any, Optional, Union
 
-from .__init__ import ANYWIN, MACOS, TYPE_CHECKING, VT100, EnvParams, unicode
+from .__init__ import ANYWIN, EXE, MACOS, TYPE_CHECKING, VT100, EnvParams, unicode
 from .authsrv import AuthSrv
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
 from .tcpsrv import TcpSrv
 from .th_srv import HAVE_PIL, HAVE_VIPS, HAVE_WEBP, ThumbSrv
 from .up2k import Up2k
 from .util import (
+    FFMPEG_URL,
     VERSIONS,
     Daemon,
     Garda,
@@ -43,6 +45,7 @@ from .util import (
     ansi_re,
     min_ex,
     mp,
+    pybin,
     start_log_thrs,
     start_stackmon,
 )
@@ -66,8 +69,15 @@ class SvcHub(object):
     put() can return a queue (if want_reply=True) which has a blocking get() with the response.
     """
 
-    def __init__(self, args: argparse.Namespace, argv: list[str], printed: str) -> None:
+    def __init__(
+        self,
+        args: argparse.Namespace,
+        dargs: argparse.Namespace,
+        argv: list[str],
+        printed: str,
+    ) -> None:
         self.args = args
+        self.dargs = dargs
         self.argv = argv
         self.E: EnvParams = args.E
         self.logf: Optional[typing.TextIO] = None
@@ -96,13 +106,13 @@ class SvcHub(object):
         if args.sss or args.s >= 3:
             args.ss = True
             args.no_dav = True
+            args.no_logues = True
+            args.no_readme = True
             args.lo = args.lo or "cpp-%Y-%m%d-%H%M%S.txt.xz"
             args.ls = args.ls or "**,*,ln,p,r"
 
         if args.ss or args.s >= 2:
             args.s = True
-            args.no_logues = True
-            args.no_readme = True
             args.unpost = 0
             args.no_del = True
             args.no_mv = True
@@ -118,6 +128,9 @@ class SvcHub(object):
             args.no_robots = True
             args.force_js = True
 
+        if not self._process_config():
+            raise Exception("bad config")
+
         self.log = self._log_disabled if args.q else self._log_enabled
         if args.lo:
             self._setup_logfile(printed)
@@ -139,25 +152,26 @@ class SvcHub(object):
             self.log("root", t.format(args.j))
 
         if not args.no_fpool and args.j != 1:
-            t = "WARNING: --use-fpool combined with multithreading is untested and can probably cause undefined behavior"
-            if ANYWIN:
-                t = 'windows cannot do multithreading without --no-fpool, so enabling that -- note that upload performance will suffer if you have microsoft defender "real-time protection" enabled, so you probably want to use -j 1 instead'
-                args.no_fpool = True
-
-            self.log("root", t, c=3)
+            t = "WARNING: ignoring --use-fpool because multithreading (-j{}) is enabled"
+            self.log("root", t.format(args.j), c=3)
+            args.no_fpool = True
 
         bri = "zy"[args.theme % 2 :][:1]
         ch = "abcdefghijklmnopqrstuvwx"[int(args.theme / 2)]
         args.theme = "{0}{1} {0} {1}".format(ch, bri)
 
-        if not args.hardlink and args.never_symlink:
-            args.no_dedup = True
-
         if args.log_fk:
             args.log_fk = re.compile(args.log_fk)
 
         # initiate all services to manage
-        self.asrv = AuthSrv(self.args, self.log)
+        self.asrv = AuthSrv(self.args, self.log, dargs=self.dargs)
+
+        if args.cgen:
+            self.asrv.cgen()
+
+        if args.exit == "cfg":
+            sys.exit(0)
+
         if args.ls:
             self.asrv.dbg_ls()
 
@@ -166,9 +180,6 @@ class SvcHub(object):
 
         self.log("root", "max clients: {}".format(self.args.nc))
 
-        if not self._process_config():
-            raise Exception("bad config")
-
         self.tcpsrv = TcpSrv(self)
         self.up2k = Up2k(self)
 
@@ -182,6 +193,7 @@ class SvcHub(object):
 
         self.args.th_dec = list(decs.keys())
         self.thumbsrv = None
+        want_ff = False
         if not args.no_thumb:
             t = ", ".join(self.args.th_dec) or "(None available)"
             self.log("thumb", "decoder preference: {}".format(t))
@@ -193,8 +205,12 @@ class SvcHub(object):
             if self.args.th_dec:
                 self.thumbsrv = ThumbSrv(self)
             else:
+                want_ff = True
                 msg = "need either Pillow, pyvips, or FFmpeg to create thumbnails; for example:\n{0}{1} -m pip install --user Pillow\n{0}{1} -m pip install --user pyvips\n{0}apt install ffmpeg"
-                msg = msg.format(" " * 37, os.path.basename(sys.executable))
+                msg = msg.format(" " * 37, os.path.basename(pybin))
+                if EXE:
+                    msg = "copyparty.exe cannot use Pillow or pyvips; need ffprobe.exe and ffmpeg.exe to create thumbnails"
+
                 self.log("thumb", msg, c=3)
 
         if not args.no_acode and args.no_thumb:
@@ -206,6 +222,10 @@ class SvcHub(object):
             msg = "setting --no-acode because either FFmpeg or FFprobe is not available"
             self.log("thumb", msg, c=6)
             args.no_acode = True
+            want_ff = True
+
+        if want_ff and ANYWIN:
+            self.log("thumb", "download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
 
         args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
 
@@ -236,6 +256,7 @@ class SvcHub(object):
         if not args.zms:
             args.zms = zms
 
+        self.zc_ngen = 0
         self.mdns: Optional["MDNS"] = None
         self.ssdp: Optional["SSDPd"] = None
 
@@ -295,11 +316,53 @@ class SvcHub(object):
         al.zs_on = al.zs_on or al.z_on
         al.zm_off = al.zm_off or al.z_off
         al.zs_off = al.zs_off or al.z_off
-        for n in ("zm_on", "zm_off", "zs_on", "zs_off"):
-            vs = getattr(al, n).replace(" ", ",").split(",")
+        ns = "zm_on zm_off zs_on zs_off acao acam"
+        for n in ns.split(" "):
+            vs = getattr(al, n).split(",")
+            vs = [x.strip() for x in vs]
             vs = [x for x in vs if x]
             setattr(al, n, vs)
 
+        ns = "acao acam"
+        for n in ns.split(" "):
+            vs = getattr(al, n)
+            vd = {zs: 1 for zs in vs}
+            setattr(al, n, vd)
+
+        ns = "acao"
+        for n in ns.split(" "):
+            vs = getattr(al, n)
+            vs = [x.lower() for x in vs]
+            setattr(al, n, vs)
+
+        R = al.rp_loc
+        if "//" in R or ":" in R:
+            t = "found URL in --rp-loc; it should be just the location, for example /foo/bar"
+            raise Exception(t)
+
+        al.R = R = R.strip("/")
+        al.SR = "/" + R if R else ""
+        al.RS = R + "/" if R else ""
+        al.SRS = "/" + R + "/" if R else "/"
+
+        if al.rsp_jtr:
+            al.rsp_slp = 0.000001
+
+        al.th_covers = set(al.th_covers.split(","))
+
+        for k in "c".split(" "):
+            vl = getattr(al, k)
+            if not vl:
+                continue
+
+            vl = [os.path.expanduser(x) if x.startswith("~") else x for x in vl]
+            setattr(al, k, vl)
+
+        for k in "lo hist ssl_log".split(" "):
+            vs = getattr(al, k)
+            if vs and vs.startswith("~"):
+                setattr(al, k, os.path.expanduser(vs))
+
         return True
 
     def _setlimits(self) -> None:
@@ -354,6 +417,7 @@ class SvcHub(object):
 
     def _setup_logfile(self, printed: str) -> None:
         base_fn = fn = sel_fn = self._logname()
+        do_xz = fn.lower().endswith(".xz")
         if fn != self.args.lo:
             ctr = 0
             # yup this is a race; if started sufficiently concurrently, two
@@ -365,7 +429,7 @@ class SvcHub(object):
         fn = sel_fn
 
         try:
-            if fn.lower().endswith(".xz"):
+            if do_xz:
                 import lzma
 
                 lh = lzma.open(fn, "wt", encoding="utf-8", errors="replace", preset=0)
@@ -376,7 +440,7 @@ class SvcHub(object):
 
             lh = codecs.open(fn, "w", encoding="utf-8", errors="replace")
 
-        argv = [sys.executable] + self.argv
+        argv = [pybin] + self.argv
         if hasattr(shlex, "quote"):
             argv = [shlex.quote(x) for x in argv]
         else:
@@ -392,24 +456,10 @@ class SvcHub(object):
 
     def run(self) -> None:
         self.tcpsrv.run()
-
-        if getattr(self.args, "zm", False):
-            try:
-                from .mdns import MDNS
-
-                self.mdns = MDNS(self)
-                Daemon(self.mdns.run, "mdns")
-            except:
-                self.log("root", "mdns startup failed;\n" + min_ex(), 3)
-
-        if getattr(self.args, "zs", False):
-            try:
-                from .ssdp import SSDPd
-
-                self.ssdp = SSDPd(self)
-                Daemon(self.ssdp.run, "ssdp")
-            except:
-                self.log("root", "ssdp startup failed;\n" + min_ex(), 3)
+        if getattr(self.args, "z_chk", 0) and (
+            getattr(self.args, "zm", False) or getattr(self.args, "zs", False)
+        ):
+            Daemon(self.tcpsrv.netmon, "netmon")
 
         Daemon(self.thr_httpsrv_up, "sig-hsrv-up2")
 
@@ -441,6 +491,33 @@ class SvcHub(object):
         else:
             self.stop_thr()
 
+    def start_zeroconf(self) -> None:
+        self.zc_ngen += 1
+
+        if getattr(self.args, "zm", False):
+            try:
+                from .mdns import MDNS
+
+                if self.mdns:
+                    self.mdns.stop(True)
+
+                self.mdns = MDNS(self, self.zc_ngen)
+                Daemon(self.mdns.run, "mdns")
+            except:
+                self.log("root", "mdns startup failed;\n" + min_ex(), 3)
+
+        if getattr(self.args, "zs", False):
+            try:
+                from .ssdp import SSDPd
+
+                if self.ssdp:
+                    self.ssdp.stop()
+
+                self.ssdp = SSDPd(self, self.zc_ngen)
+                Daemon(self.ssdp.run, "ssdp")
+            except:
+                self.log("root", "ssdp startup failed;\n" + min_ex(), 3)
+
     def reload(self) -> str:
         if self.reloading:
             return "cannot reload; already in progress"
@@ -625,13 +702,20 @@ class SvcHub(object):
                     print(msg.encode("utf-8", "replace").decode(), end="")
                 except:
                     print(msg.encode("ascii", "replace").decode(), end="")
+            except OSError as ex:
+                if ex.errno != errno.EPIPE:
+                    raise
 
             if self.logf:
                 self.logf.write(msg)
 
     def pr(self, *a: Any, **ka: Any) -> None:
-        with self.log_mutex:
-            print(*a, **ka)
+        try:
+            with self.log_mutex:
+                print(*a, **ka)
+        except OSError as ex:
+            if ex.errno != errno.EPIPE:
+                raise
 
     def check_mp_support(self) -> str:
         if MACOS:

copyparty/szip.py

@@ -2,6 +2,7 @@
 from __future__ import print_function, unicode_literals
 
 import calendar
+import stat
 import time
 import zlib
 
@@ -238,6 +239,9 @@ class StreamZip(StreamArc):
         src = f["ap"]
         st = f["st"]
 
+        if stat.S_ISDIR(st.st_mode):
+            return
+
         sz = st.st_size
         ts = st.st_mtime
 

copyparty/tcpsrv.py

@@ -5,6 +5,7 @@ import os
 import re
 import socket
 import sys
+import time
 
 from .__init__ import ANYWIN, PY2, TYPE_CHECKING, VT100, unicode
 from .stolen.qrcodegen import QrCode
@@ -28,6 +29,9 @@ if TYPE_CHECKING:
 if not hasattr(socket, "IPPROTO_IPV6"):
     setattr(socket, "IPPROTO_IPV6", 41)
 
+if not hasattr(socket, "IP_FREEBIND"):
+    setattr(socket, "IP_FREEBIND", 15)
+
 
 class TcpSrv(object):
     """
@@ -46,6 +50,8 @@ class TcpSrv(object):
         self.stopping = False
         self.srv: list[socket.socket] = []
         self.bound: list[tuple[str, int]] = []
+        self.netdevs: dict[str, Netdev] = {}
+        self.netlist = ""
         self.nsrv = 0
         self.qr = ""
         pad = False
@@ -121,6 +127,20 @@ class TcpSrv(object):
         else:
             self.netdevs = {}
 
+        # keep IPv6 LL-only nics
+        ll_ok: set[str] = set()
+        for ip, nd in self.netdevs.items():
+            if not ip.startswith("fe80"):
+                continue
+
+            just_ll = True
+            for ip2, nd2 in self.netdevs.items():
+                if nd == nd2 and ":" in ip2 and not ip2.startswith("fe80"):
+                    just_ll = False
+
+            if just_ll or self.args.ll:
+                ll_ok.add(ip.split("/")[0])
+
         qr1: dict[str, list[int]] = {}
         qr2: dict[str, list[int]] = {}
         msgs = []
@@ -128,7 +148,7 @@ class TcpSrv(object):
         title_vars = [x[1:] for x in self.args.wintitle.split(" ") if x.startswith("$")]
         t = "available @ {}://{}:{}/  (\033[33m{}\033[0m)"
         for ip, desc in sorted(eps.items(), key=lambda x: x[1]):
-            if ip.startswith("fe80"):
+            if ip.startswith("fe80") and ip not in ll_ok:
                 continue
 
             for port in sorted(self.args.p):
@@ -195,21 +215,28 @@ class TcpSrv(object):
     def _listen(self, ip: str, port: int) -> None:
         ipv = socket.AF_INET6 if ":" in ip else socket.AF_INET
         srv = socket.socket(ipv, socket.SOCK_STREAM)
-        try:
-            srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
-        except:
-            pass
-        srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+        if not ANYWIN or self.args.reuseaddr:
+            srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
         srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
-        srv.settimeout(None)  # < does not inherit, ^ does
+        srv.settimeout(None)  # < does not inherit, ^ opts above do
 
         try:
             srv.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, False)
         except:
             pass  # will create another ipv4 socket instead
 
+        if not ANYWIN and self.args.freebind:
+            srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
+
         try:
             srv.bind((ip, port))
+            sport = srv.getsockname()[1]
+            if port != sport:
+                # linux 6.0.16 lets you bind a port which is in use
+                # except it just gives you a random port instead
+                raise OSError(E_ADDR_IN_USE[0], "")
             self.srv.append(srv)
         except (OSError, socket.error) as ex:
             if ex.errno in E_ADDR_IN_USE:
@@ -228,6 +255,14 @@ class TcpSrv(object):
             ip, port = srv.getsockname()[:2]
             try:
                 srv.listen(self.args.nc)
+                try:
+                    ok = srv.getsockopt(socket.SOL_SOCKET, socket.SO_ACCEPTCONN)
+                except:
+                    ok = 1  # macos
+
+                if not ok:
+                    # some linux don't throw on listen(0.0.0.0) after listen(::)
+                    raise Exception("failed to listen on {}".format(srv.getsockname()))
             except:
                 if ip == "0.0.0.0" and ("::", port) in bound:
                     # dualstack
@@ -255,7 +290,11 @@ class TcpSrv(object):
         self.srv = srvs
         self.bound = bound
         self.nsrv = len(srvs)
+        self._distribute_netdevs()
+
+    def _distribute_netdevs(self):
         self.hub.broker.say("set_netdevs", self.netdevs)
+        self.hub.start_zeroconf()
 
     def shutdown(self) -> None:
         self.stopping = True
@@ -267,6 +306,27 @@ class TcpSrv(object):
 
         self.log("tcpsrv", "ok bye")
 
+    def netmon(self):
+        while not self.stopping:
+            time.sleep(self.args.z_chk)
+            netdevs = self.detect_interfaces(self.args.i)
+            if not netdevs:
+                continue
+
+            added = "nothing"
+            removed = "nothing"
+            for k, v in netdevs.items():
+                if k not in self.netdevs:
+                    added = "{} = {}".format(k, v)
+            for k, v in self.netdevs.items():
+                if k not in netdevs:
+                    removed = "{} = {}".format(k, v)
+
+            t = "network change detected:\n  added {}\033[0;33m\nremoved {}"
+            self.log("tcpsrv", t.format(added, removed), 3)
+            self.netdevs = netdevs
+            self._distribute_netdevs()
+
     def detect_interfaces(self, listen_ips: list[str]) -> dict[str, Netdev]:
         from .stolen.ifaddr import get_adapters
 
@@ -276,10 +336,6 @@ class TcpSrv(object):
             for nip in nic.ips:
                 ipa = nip.ip[0] if ":" in str(nip.ip) else nip.ip
                 sip = "{}/{}".format(ipa, nip.network_prefix)
-                if sip.startswith("169.254"):
-                    # browsers dont impl linklocal
-                    continue
-
                 nd = Netdev(sip, nic.index or 0, nic.nice_name, "")
                 eps[sip] = nd
                 try:
@@ -291,6 +347,12 @@ class TcpSrv(object):
                 except:
                     pass
 
+        netlist = str(sorted(eps.items()))
+        if netlist == self.netlist and self.netdevs:
+            return {}
+
+        self.netlist = netlist
+
         if "0.0.0.0" not in listen_ips and "::" not in listen_ips:
             eps = {k: v for k, v in eps.items() if k.split("/")[0] in listen_ips}
 

copyparty/th_srv.py

@@ -3,6 +3,7 @@ from __future__ import print_function, unicode_literals
 
 import base64
 import hashlib
+import logging
 import os
 import shutil
 import subprocess as sp
@@ -11,14 +12,16 @@ import time
 
 from queue import Queue
 
-from .__init__ import TYPE_CHECKING
+from .__init__ import ANYWIN, TYPE_CHECKING
 from .bos import bos
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
 from .util import (
+    FFMPEG_URL,
     BytesIO,
     Cooldown,
     Daemon,
     Pebkac,
+    afsenc,
     fsenc,
     min_ex,
     runcmd,
@@ -61,12 +64,16 @@ try:
         HAVE_AVIF = True
     except:
         pass
+
+    logging.getLogger("PIL").setLevel(logging.WARNING)
 except:
     pass
 
 try:
     HAVE_VIPS = True
     import pyvips
+
+    logging.getLogger("pyvips").setLevel(logging.WARNING)
 except:
     HAVE_VIPS = False
 
@@ -77,14 +84,14 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str) -> str:
     # base64 = 64 = 4096
     rd, fn = vsplit(rem)
     if rd:
-        h = hashlib.sha512(fsenc(rd)).digest()
+        h = hashlib.sha512(afsenc(rd)).digest()
         b64 = base64.urlsafe_b64encode(h).decode("ascii")[:24]
         rd = "{}/{}/".format(b64[:2], b64[2:4]).lower() + b64
     else:
         rd = "top"
 
     # could keep original filenames but this is safer re pathlen
-    h = hashlib.sha512(fsenc(fn)).digest()
+    h = hashlib.sha512(afsenc(fn)).digest()
     fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
 
     if fmt in ("opus", "caf"):
@@ -128,6 +135,8 @@ class ThumbSrv(object):
             msg = "cannot create audio/video thumbnails because some of the required programs are not available: "
             msg += ", ".join(missing)
             self.log(msg, c=3)
+            if ANYWIN and self.args.no_acode:
+                self.log("download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
 
         if self.args.th_clean:
             Daemon(self.cleaner, "thumb.cln")
@@ -191,12 +200,12 @@ class ThumbSrv(object):
                 self.log("wait {}".format(tpath))
             except:
                 thdir = os.path.dirname(tpath)
-                bos.makedirs(thdir)
+                bos.makedirs(os.path.join(thdir, "w"))
 
                 inf_path = os.path.join(thdir, "dir.txt")
                 if not bos.path.exists(inf_path):
                     with open(inf_path, "wb") as f:
-                        f.write(fsenc(os.path.dirname(abspath)))
+                        f.write(afsenc(os.path.dirname(abspath)))
 
                 self.busy[tpath] = [cond]
                 do_conv = True
@@ -242,47 +251,55 @@ class ThumbSrv(object):
             abspath, tpath = task
             ext = abspath.split(".")[-1].lower()
             png_ok = False
-            fun = None
+            funs = []
             if not bos.path.exists(tpath):
                 for lib in self.args.th_dec:
-                    if fun:
-                        break
-                    elif lib == "pil" and ext in self.fmt_pil:
-                        fun = self.conv_pil
+                    if lib == "pil" and ext in self.fmt_pil:
+                        funs.append(self.conv_pil)
                     elif lib == "vips" and ext in self.fmt_vips:
-                        fun = self.conv_vips
+                        funs.append(self.conv_vips)
                     elif lib == "ff" and ext in self.fmt_ffi or ext in self.fmt_ffv:
-                        fun = self.conv_ffmpeg
+                        funs.append(self.conv_ffmpeg)
                     elif lib == "ff" and ext in self.fmt_ffa:
                         if tpath.endswith(".opus") or tpath.endswith(".caf"):
-                            fun = self.conv_opus
+                            funs.append(self.conv_opus)
                         elif tpath.endswith(".png"):
-                            fun = self.conv_waves
+                            funs.append(self.conv_waves)
                             png_ok = True
                         else:
-                            fun = self.conv_spec
+                            funs.append(self.conv_spec)
 
             if not png_ok and tpath.endswith(".png"):
                 raise Pebkac(400, "png only allowed for waveforms")
 
-            if fun:
+            tdir, tfn = os.path.split(tpath)
+            ttpath = os.path.join(tdir, "w", tfn)
+
+            for fun in funs:
                 try:
-                    fun(abspath, tpath)
+                    fun(abspath, ttpath)
+                    break
                 except Exception as ex:
                     msg = "{} could not create thumbnail of {}\n{}"
                     msg = msg.format(fun.__name__, abspath, min_ex())
                     c: Union[str, int] = 1 if "<Signals.SIG" in msg else "90"
                     self.log(msg, c)
                     if getattr(ex, "returncode", 0) != 321:
-                        with open(tpath, "wb") as _:
-                            pass
+                        if fun == funs[-1]:
+                            with open(ttpath, "wb") as _:
+                                pass
                     else:
                         # ffmpeg may spawn empty files on windows
                         try:
-                            os.unlink(tpath)
+                            os.unlink(ttpath)
                         except:
                             pass
 
+            try:
+                bos.rename(ttpath, tpath)
+            except:
+                pass
+
             with self.mutex:
                 subs = self.busy[tpath]
                 del self.busy[tpath]
@@ -363,7 +380,8 @@ class ThumbSrv(object):
                 img = pyvips.Image.thumbnail(abspath, w, **kw)
                 break
             except:
-                pass
+                if c == crops[-1]:
+                    raise
 
         img.write_to_file(tpath, Q=40)
 
@@ -543,12 +561,19 @@ class ThumbSrv(object):
         if "ac" not in ret:
             raise Exception("not audio")
 
+        try:
+            dur = ret[".dur"][1]
+        except:
+            dur = 0
+
         src_opus = abspath.lower().endswith(".opus") or ret["ac"][1] == "opus"
         want_caf = tpath.endswith(".caf")
         tmp_opus = tpath
         if want_caf:
             tmp_opus = tpath.rsplit(".", 1)[0] + ".opus"
 
+        caf_src = abspath if src_opus else tmp_opus
+
         if not want_caf or (not src_opus and not bos.path.isfile(tmp_opus)):
             # fmt: off
             cmd = [
@@ -566,7 +591,32 @@ class ThumbSrv(object):
             # fmt: on
             self._run_ff(cmd)
 
-        if want_caf:
+        # iOS fails to play some "insufficiently complex" files
+        # (average file shorter than 8 seconds), so of course we
+        # fix that by mixing in some inaudible pink noise :^)
+        # 6.3 sec seems like the cutoff so lets do 7, and
+        # 7 sec of psyqui-musou.opus @ 3:50 is 174 KiB
+        if want_caf and (dur < 20 or bos.path.getsize(caf_src) < 256 * 1024):
+            # fmt: off
+            cmd = [
+                b"ffmpeg",
+                b"-nostdin",
+                b"-v", b"error",
+                b"-hide_banner",
+                b"-i", fsenc(abspath),
+                b"-filter_complex", b"anoisesrc=a=0.001:d=7:c=pink,asplit[l][r]; [l][r]amerge[s]; [0:a:0][s]amix",
+                b"-map_metadata", b"-1",
+                b"-ac", b"2",
+                b"-c:a", b"libopus",
+                b"-b:a", b"128k",
+                b"-f", b"caf",
+                fsenc(tpath)
+            ]
+            # fmt: on
+            self._run_ff(cmd)
+
+        elif want_caf:
+            # simple remux should be safe
             # fmt: off
             cmd = [
                 b"ffmpeg",

copyparty/u2idx.py

@@ -34,14 +34,14 @@ if True:  # pylint: disable=using-constant-test
     from typing import Any, Optional, Union
 
 if TYPE_CHECKING:
-    from .httpconn import HttpConn
+    from .httpsrv import HttpSrv
 
 
 class U2idx(object):
-    def __init__(self, conn: "HttpConn") -> None:
-        self.log_func = conn.log_func
-        self.asrv = conn.asrv
-        self.args = conn.args
+    def __init__(self, hsrv: "HttpSrv") -> None:
+        self.log_func = hsrv.log
+        self.asrv = hsrv.asrv
+        self.args = hsrv.args
         self.timeout = self.args.srch_time
 
         if not HAVE_SQLITE3:
@@ -51,7 +51,7 @@ class U2idx(object):
         self.active_id = ""
         self.active_cur: Optional["sqlite3.Cursor"] = None
         self.cur: dict[str, "sqlite3.Cursor"] = {}
-        self.mem_cur = sqlite3.connect(":memory:").cursor()
+        self.mem_cur = sqlite3.connect(":memory:", check_same_thread=False).cursor()
         self.mem_cur.execute(r"create table a (b text)")
 
         self.p_end = 0.0
@@ -97,19 +97,23 @@ class U2idx(object):
             return None
 
         cur = None
-        if ANYWIN:
+        if ANYWIN and not bos.path.exists(db_path + "-wal"):
             uri = ""
             try:
                 uri = "{}?mode=ro&nolock=1".format(Path(db_path).as_uri())
-                cur = sqlite3.connect(uri, 2, uri=True).cursor()
+                db = sqlite3.connect(uri, 2, uri=True, check_same_thread=False)
+                cur = db.cursor()
+                cur.execute('pragma table_info("up")').fetchone()
                 self.log("ro: {}".format(db_path))
             except:
                 self.log("could not open read-only: {}\n{}".format(uri, min_ex()))
+                # may not fail until the pragma so unset it
+                cur = None
 
         if not cur:
             # on windows, this steals the write-lock from up2k.deferred_init --
             # seen on win 10.0.17763.2686, py 3.10.4, sqlite 3.37.2
-            cur = sqlite3.connect(db_path, 2).cursor()
+            cur = sqlite3.connect(db_path, 2, check_same_thread=False).cursor()
             self.log("opened {}".format(db_path))
 
         self.cur[ptop] = cur
@@ -117,10 +121,10 @@ class U2idx(object):
 
     def search(
         self, vols: list[tuple[str, str, dict[str, Any]]], uq: str, lim: int
-    ) -> tuple[list[dict[str, Any]], list[str]]:
+    ) -> tuple[list[dict[str, Any]], list[str], bool]:
         """search by query params"""
         if not HAVE_SQLITE3:
-            return [], []
+            return [], [], False
 
         q = ""
         v: Union[str, int] = ""
@@ -272,7 +276,7 @@ class U2idx(object):
         have_up: bool,
         have_mt: bool,
         lim: int,
-    ) -> tuple[list[dict[str, Any]], list[str]]:
+    ) -> tuple[list[dict[str, Any]], list[str], bool]:
         done_flag: list[bool] = []
         self.active_id = "{:.6f}_{}".format(
             time.time(), threading.current_thread().ident
@@ -290,6 +294,7 @@ class U2idx(object):
         self.log("qs: {!r} {!r}".format(uq, uv))
 
         ret = []
+        seen_rps: set[str] = set()
         lim = min(lim, int(self.args.srch_hits))
         taglist = {}
         for (vtop, ptop, flags) in vols:
@@ -308,16 +313,21 @@ class U2idx(object):
 
             sret = []
             fk = flags.get("fk")
+            dots = flags.get("dotsrch")
             c = cur.execute(uq, tuple(vuv))
             for hit in c:
                 w, ts, sz, rd, fn, ip, at = hit[:7]
-                lim -= 1
-                if lim < 0:
-                    break
 
                 if rd.startswith("//") or fn.startswith("//"):
                     rd, fn = s3dec(rd, fn)
 
+                rp = quotep("/".join([x for x in [vtop, rd, fn] if x]))
+                if not dots and "/." in ("/" + rp):
+                    continue
+
+                if rp in seen_rps:
+                    continue
+
                 if not fk:
                     suf = ""
                 else:
@@ -334,8 +344,12 @@ class U2idx(object):
                         )[:fk]
                     )
 
-                rp = quotep("/".join([x for x in [vtop, rd, fn] if x])) + suf
-                sret.append({"ts": int(ts), "sz": sz, "rp": rp, "w": w[:16]})
+                lim -= 1
+                if lim < 0:
+                    break
+
+                seen_rps.add(rp)
+                sret.append({"ts": int(ts), "sz": sz, "rp": rp + suf, "w": w[:16]})
 
             for hit in sret:
                 w = hit["w"]
@@ -354,17 +368,9 @@ class U2idx(object):
         done_flag.append(True)
         self.active_id = ""
 
-        # undupe hits from multiple metadata keys
-        if len(ret) > 1:
-            ret = [ret[0]] + [
-                y
-                for x, y in zip(ret[:-1], ret[1:])
-                if x["rp"].split("?")[0] != y["rp"].split("?")[0]
-            ]
-
         ret.sort(key=itemgetter("rp"))
 
-        return ret, list(taglist.keys())
+        return ret, list(taglist.keys()), lim < 0
 
     def terminator(self, identifier: str, done_flag: list[bool]) -> None:
         for _ in range(self.timeout):

copyparty/util.py

@@ -6,6 +6,7 @@ import contextlib
 import errno
 import hashlib
 import hmac
+import json
 import logging
 import math
 import mimetypes
@@ -13,6 +14,7 @@ import os
 import platform
 import re
 import select
+import shutil
 import signal
 import socket
 import stat
@@ -29,7 +31,7 @@ from email.utils import formatdate
 from ipaddress import IPv4Address, IPv4Network, IPv6Address, IPv6Network
 from queue import Queue
 
-from .__init__ import ANYWIN, MACOS, PY2, TYPE_CHECKING, VT100, WINDOWS
+from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, VT100, WINDOWS
 from .__version__ import S_BUILD_DT, S_VERSION
 from .stolen import surrogateescape
 
@@ -142,12 +144,15 @@ SYMTIME = sys.version_info > (3, 6) and os.utime in os.supports_follow_symlinks
 
 META_NOBOTS = '<meta name="robots" content="noindex, nofollow">'
 
+FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"
+
 HTTPCODE = {
     200: "OK",
     201: "Created",
     204: "No Content",
     206: "Partial Content",
     207: "Multi-Status",
+    301: "Moved Permanently",
     302: "Found",
     304: "Not Modified",
     400: "Bad Request",
@@ -227,6 +232,7 @@ application msi=x-ms-installer cab=vnd.ms-cab-compressed rpm=x-rpm crx=x-chrome-
 application epub=epub+zip mobi=x-mobipocket-ebook lit=x-ms-reader rss=rss+xml atom=atom+xml torrent=x-bittorrent
 application p7s=pkcs7-signature dcm=dicom shx=vnd.shx shp=vnd.shp dbf=x-dbf gml=gml+xml gpx=gpx+xml amf=x-amf
 application swf=x-shockwave-flash m3u=vnd.apple.mpegurl db3=vnd.sqlite3 sqlite=vnd.sqlite3
+text ass=plain ssa=plain
 image jpg=jpeg xpm=x-xpixmap psd=vnd.adobe.photoshop jpf=jpx tif=tiff ico=x-icon djvu=vnd.djvu
 image heic=heic-sequence heif=heif-sequence hdr=vnd.radiance svg=svg+xml
 audio caf=x-caf mp3=mpeg m4a=mp4 mid=midi mpc=musepack aif=aiff au=basic qcp=qcelp
@@ -287,6 +293,19 @@ REKOBO_KEY = {
 REKOBO_LKEY = {k.lower(): v for k, v in REKOBO_KEY.items()}
 
 
+pybin = sys.executable or ""
+if EXE:
+    pybin = ""
+    for p in "python3 python".split():
+        try:
+            p = shutil.which(p)
+            if p:
+                pybin = p
+                break
+        except:
+            pass
+
+
 def py_desc() -> str:
     interp = platform.python_implementation()
     py_ver = ".".join([str(x) for x in sys.version_info])
@@ -360,8 +379,11 @@ class Daemon(threading.Thread):
         name: Optional[str] = None,
         a: Optional[Iterable[Any]] = None,
         r: bool = True,
+        ka: Optional[dict[Any, Any]] = None,
     ) -> None:
-        threading.Thread.__init__(self, target=target, name=name, args=a or ())
+        threading.Thread.__init__(
+            self, target=target, name=name, args=a or (), kwargs=ka
+        )
         self.daemon = True
         if r:
             self.start()
@@ -377,6 +399,9 @@ class Netdev(object):
     def __str__(self):
         return "{}-{}{}".format(self.idx, self.name, self.desc)
 
+    def __repr__(self):
+        return "'{}-{}'".format(self.idx, self.name)
+
     def __lt__(self, rhs):
         return str(self) < str(rhs)
 
@@ -436,9 +461,7 @@ class HLog(logging.Handler):
         else:
             c = 1
 
-        if record.name.startswith("PIL") and lv < logging.WARNING:
-            return
-        elif record.name == "pyftpdlib":
+        if record.name == "pyftpdlib":
             m = self.ptn_ftp.match(msg)
             if m:
                 ip = m.group(1)
@@ -468,7 +491,7 @@ class NetMap(object):
             )
 
         ips = [x for x in ips if x not in ("::1", "127.0.0.1")]
-        ips = [[x for x in netdevs if x.startswith(y + "/")][0] for y in ips]
+        ips = find_prefix(ips, netdevs)
 
         self.cache: dict[str, str] = {}
         self.b2sip: dict[bytes, str] = {}
@@ -645,6 +668,7 @@ class FHC(object):
 
     def __init__(self) -> None:
         self.cache: dict[str, FHC.CE] = {}
+        self.aps: set[str] = set()
 
     def close(self, path: str) -> None:
         try:
@@ -656,6 +680,7 @@ class FHC(object):
             fh.close()
 
         del self.cache[path]
+        self.aps.remove(path)
 
     def clean(self) -> None:
         if not self.cache:
@@ -676,6 +701,7 @@ class FHC(object):
         return self.cache[path].fhs.pop()
 
     def put(self, path: str, fh: typing.BinaryIO) -> None:
+        self.aps.add(path)
         try:
             ce = self.cache[path]
             ce.fhs.append(fh)
@@ -1146,20 +1172,12 @@ def ren_open(
     fun = kwargs.pop("fun", open)
     fdir = kwargs.pop("fdir", None)
     suffix = kwargs.pop("suffix", None)
-    overwrite = kwargs.pop("overwrite", None)
 
     if fname == os.devnull:
         with fun(fname, *args, **kwargs) as f:
             yield {"orz": (f, fname)}
             return
 
-    if overwrite:
-        assert fdir
-        fpath = os.path.join(fdir, fname)
-        with fun(fsenc(fpath), *args, **kwargs) as f:
-            yield {"orz": (f, fname)}
-            return
-
     if suffix:
         ext = fname.split(".")[-1]
         if len(ext) < 7:
@@ -1186,7 +1204,7 @@ def ren_open(
             else:
                 fpath = fname
 
-            if suffix and os.path.exists(fsenc(fpath)):
+            if suffix and os.path.lexists(fsenc(fpath)):
                 fpath += suffix
                 fname += suffix
                 ext += suffix
@@ -1506,6 +1524,28 @@ def read_header(sr: Unrecv) -> list[str]:
         return ret[:ofs].decode("utf-8", "surrogateescape").lstrip("\r\n").split("\r\n")
 
 
+def rand_name(fdir: str, fn: str, rnd: int) -> str:
+    ok = False
+    try:
+        ext = "." + fn.rsplit(".", 1)[1]
+    except:
+        ext = ""
+
+    for extra in range(16):
+        for _ in range(16):
+            if ok:
+                break
+
+            nc = rnd + extra
+            nb = int((6 + 6 * nc) / 8)
+            zb = os.urandom(nb)
+            zb = base64.urlsafe_b64encode(zb)
+            fn = zb[:nc].decode("utf-8") + ext
+            ok = not os.path.exists(fsenc(os.path.join(fdir, fn)))
+
+    return fn
+
+
 def gen_filekey(salt: str, fspath: str, fsize: int, inode: int) -> str:
     return base64.urlsafe_b64encode(
         hashlib.sha512(
@@ -1548,14 +1588,16 @@ def gen_filekey_dbg(
     return ret
 
 
-def gencookie(k: str, v: str, dur: Optional[int]) -> str:
+def gencookie(k: str, v: str, r: str, tls: bool, dur: Optional[int]) -> str:
     v = v.replace(";", "")
     if dur:
         exp = formatdate(time.time() + dur, usegmt=True)
     else:
         exp = "Fri, 15 Aug 1997 01:00:00 GMT"
 
-    return "{}={}; Path=/; Expires={}; SameSite=Lax".format(k, v, exp)
+    return "{}={}; Path=/{}; Expires={}{}; SameSite=Lax".format(
+        k, v, r, exp, "; Secure" if tls else ""
+    )
 
 
 def humansize(sz: float, terse: bool = False) -> str:
@@ -1681,7 +1723,7 @@ def relchk(rp: str) -> str:
 
 def absreal(fpath: str) -> str:
     try:
-        return fsdec(os.path.abspath(os.path.realpath(fsenc(fpath))))
+        return fsdec(os.path.abspath(os.path.realpath(afsenc(fpath))))
     except:
         if not WINDOWS:
             raise
@@ -1711,6 +1753,15 @@ def ipnorm(ip: str) -> str:
     return ip
 
 
+def find_prefix(ips: list[str], netdevs: dict[str, Netdev]) -> list[str]:
+    ret = []
+    for ip in ips:
+        hit = next((x for x in netdevs if x.startswith(ip + "/")), None)
+        if hit:
+            ret.append(hit)
+    return ret
+
+
 def html_escape(s: str, quot: bool = False, crlf: bool = False) -> str:
     """html.escape but also newlines"""
     s = s.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
@@ -1792,6 +1843,32 @@ def _w8enc3(txt: str) -> bytes:
     return txt.encode(FS_ENCODING, "surrogateescape")
 
 
+def _msdec(txt: bytes) -> str:
+    ret = txt.decode(FS_ENCODING, "surrogateescape")
+    return ret[4:] if ret.startswith("\\\\?\\") else ret
+
+
+def _msaenc(txt: str) -> bytes:
+    return txt.replace("/", "\\").encode(FS_ENCODING, "surrogateescape")
+
+
+def _uncify(txt: str) -> str:
+    txt = txt.replace("/", "\\")
+    if ":" not in txt and not txt.startswith("\\\\"):
+        txt = absreal(txt)
+
+    return txt if txt.startswith("\\\\") else "\\\\?\\" + txt
+
+
+def _msenc(txt: str) -> bytes:
+    txt = txt.replace("/", "\\")
+    if ":" not in txt and not txt.startswith("\\\\"):
+        txt = absreal(txt)
+
+    ret = txt.encode(FS_ENCODING, "surrogateescape")
+    return ret if ret.startswith(b"\\\\") else b"\\\\?\\" + ret
+
+
 w8dec = _w8dec3 if not PY2 else _w8dec2
 w8enc = _w8enc3 if not PY2 else _w8enc2
 
@@ -1806,9 +1883,16 @@ def w8b64enc(txt: str) -> str:
     return base64.urlsafe_b64encode(w8enc(txt)).decode("ascii")
 
 
-if not PY2 or not WINDOWS:
-    fsenc = w8enc
+if not PY2 and WINDOWS:
+    sfsenc = w8enc
+    afsenc = _msaenc
+    fsenc = _msenc
+    fsdec = _msdec
+    uncify = _uncify
+elif not PY2 or not WINDOWS:
+    fsenc = afsenc = sfsenc = w8enc
     fsdec = w8dec
+    uncify = str
 else:
     # moonrunes become \x3f with bytestrings,
     # losing mojibake support is worth
@@ -1818,8 +1902,9 @@ else:
     def _not_actually_mbcs_dec(txt: bytes) -> str:
         return txt
 
-    fsenc = _not_actually_mbcs_enc
+    fsenc = afsenc = sfsenc = _not_actually_mbcs_enc
     fsdec = _not_actually_mbcs_dec
+    uncify = str
 
 
 def s3enc(mem_cur: "sqlite3.Cursor", rd: str, fn: str) -> tuple[str, str]:
@@ -2007,6 +2092,20 @@ def read_socket_chunked(
             raise Pebkac(400, t.format(x))
 
 
+def list_ips() -> list[str]:
+    from .stolen.ifaddr import get_adapters
+
+    ret: set[str] = set()
+    for nic in get_adapters():
+        for ipo in nic.ips:
+            if len(ipo.ip) < 7:
+                ret.add(ipo.ip[0])  # ipv6 is (ip,0,0)
+            else:
+                ret.add(ipo.ip)
+
+    return list(ret)
+
+
 def yieldfile(fn: str) -> Generator[bytes, None, None]:
     with open(fsenc(fn), "rb", 512 * 1024) as f:
         while True:
@@ -2182,18 +2281,21 @@ def rmdirs(
     return ok, ng
 
 
-def rmdirs_up(top: str) -> tuple[list[str], list[str]]:
+def rmdirs_up(top: str, stop: str) -> tuple[list[str], list[str]]:
     """rmdir on self, then all parents"""
+    if top == stop:
+        return [], [top]
+
     try:
         os.rmdir(fsenc(top))
     except:
         return [], [top]
 
     par = os.path.dirname(top)
-    if not par:
+    if not par or par == stop:
         return [top], []
 
-    ok, ng = rmdirs_up(par)
+    ok, ng = rmdirs_up(par, stop)
     return [top] + ok, ng
 
 
@@ -2427,6 +2529,193 @@ def retchk(
         raise Exception(t)
 
 
+def _parsehook(
+    log: Optional["NamedLogger"], cmd: str
+) -> tuple[bool, bool, bool, float, dict[str, Any], str]:
+    chk = False
+    fork = False
+    jtxt = False
+    wait = 0.0
+    tout = 0.0
+    kill = "t"
+    cap = 0
+    ocmd = cmd
+    while "," in cmd[:6]:
+        arg, cmd = cmd.split(",", 1)
+        if arg == "c":
+            chk = True
+        elif arg == "f":
+            fork = True
+        elif arg == "j":
+            jtxt = True
+        elif arg.startswith("w"):
+            wait = float(arg[1:])
+        elif arg.startswith("t"):
+            tout = float(arg[1:])
+        elif arg.startswith("c"):
+            cap = int(arg[1:])  # 0=none 1=stdout 2=stderr 3=both
+        elif arg.startswith("k"):
+            kill = arg[1:]  # [t]ree [m]ain [n]one
+        elif arg.startswith("i"):
+            pass
+        else:
+            t = "hook: invalid flag {} in {}"
+            (log or print)(t.format(arg, ocmd))
+
+    env = os.environ.copy()
+    try:
+        if EXE:
+            raise Exception()
+
+        pypath = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
+        zsl = [str(pypath)] + [str(x) for x in sys.path if x]
+        pypath = str(os.pathsep.join(zsl))
+        env["PYTHONPATH"] = pypath
+    except:
+        if not EXE:
+            raise
+
+    sp_ka = {
+        "env": env,
+        "timeout": tout,
+        "kill": kill,
+        "capture": cap,
+    }
+
+    if cmd.startswith("~"):
+        cmd = os.path.expanduser(cmd)
+
+    return chk, fork, jtxt, wait, sp_ka, cmd
+
+
+def runihook(
+    log: Optional["NamedLogger"],
+    cmd: str,
+    vol: "VFS",
+    ups: list[tuple[str, int, int, str, str, str, int]],
+) -> bool:
+    ocmd = cmd
+    chk, fork, jtxt, wait, sp_ka, cmd = _parsehook(log, cmd)
+    bcmd = [sfsenc(cmd)]
+    if cmd.endswith(".py"):
+        bcmd = [sfsenc(pybin)] + bcmd
+
+    vps = [vjoin(*list(s3dec(x[3], x[4]))) for x in ups]
+    aps = [djoin(vol.realpath, x) for x in vps]
+    if jtxt:
+        # 0w 1mt 2sz 3rd 4fn 5ip 6at
+        ja = [
+            {
+                "ap": uncify(ap),  # utf8 for json
+                "vp": vp,
+                "wark": x[0][:16],
+                "mt": x[1],
+                "sz": x[2],
+                "ip": x[5],
+                "at": x[6],
+            }
+            for x, vp, ap in zip(ups, vps, aps)
+        ]
+        sp_ka["sin"] = json.dumps(ja).encode("utf-8", "replace")
+    else:
+        sp_ka["sin"] = b"\n".join(fsenc(x) for x in aps)
+
+    t0 = time.time()
+    if fork:
+        Daemon(runcmd, ocmd, [bcmd], ka=sp_ka)
+    else:
+        rc, v, err = runcmd(bcmd, **sp_ka)  # type: ignore
+        if chk and rc:
+            retchk(rc, bcmd, err, log, 5)
+            return False
+
+    wait -= time.time() - t0
+    if wait > 0:
+        time.sleep(wait)
+
+    return True
+
+
+def _runhook(
+    log: Optional["NamedLogger"],
+    cmd: str,
+    ap: str,
+    vp: str,
+    host: str,
+    uname: str,
+    mt: float,
+    sz: int,
+    ip: str,
+    at: float,
+    txt: str,
+) -> bool:
+    ocmd = cmd
+    chk, fork, jtxt, wait, sp_ka, cmd = _parsehook(log, cmd)
+    if jtxt:
+        ja = {
+            "ap": ap,
+            "vp": vp,
+            "mt": mt,
+            "sz": sz,
+            "ip": ip,
+            "at": at or time.time(),
+            "host": host,
+            "user": uname,
+            "txt": txt,
+        }
+        arg = json.dumps(ja)
+    else:
+        arg = txt or ap
+
+    acmd = [cmd, arg]
+    if cmd.endswith(".py"):
+        acmd = [pybin] + acmd
+
+    bcmd = [fsenc(x) if x == ap else sfsenc(x) for x in acmd]
+
+    t0 = time.time()
+    if fork:
+        Daemon(runcmd, ocmd, [bcmd], ka=sp_ka)
+    else:
+        rc, v, err = runcmd(bcmd, **sp_ka)  # type: ignore
+        if chk and rc:
+            retchk(rc, bcmd, err, log, 5)
+            return False
+
+    wait -= time.time() - t0
+    if wait > 0:
+        time.sleep(wait)
+
+    return True
+
+
+def runhook(
+    log: Optional["NamedLogger"],
+    cmds: list[str],
+    ap: str,
+    vp: str,
+    host: str,
+    uname: str,
+    mt: float,
+    sz: int,
+    ip: str,
+    at: float,
+    txt: str,
+) -> bool:
+    vp = vp.replace("\\", "/")
+    for cmd in cmds:
+        try:
+            if not _runhook(log, cmd, ap, vp, host, uname, mt, sz, ip, at, txt):
+                return False
+        except Exception as ex:
+            (log or print)("hook: {}".format(ex))
+            if ",c," in "," + cmd:
+                return False
+            break
+
+    return True
+
+
 def gzip_orig_sz(fn: str) -> int:
     with open(fsenc(fn), "rb") as f:
         f.seek(-4, 2)
@@ -2536,7 +2825,7 @@ def termsize() -> tuple[int, int]:
     def ioctl_GWINSZ(fd: int) -> Optional[tuple[int, int]]:
         try:
             cr = sunpack(b"hh", fcntl.ioctl(fd, termios.TIOCGWINSZ, b"AAAA"))
-            return int(cr[1]), int(cr[0])
+            return cr[::-1]
         except:
             return None
 
@@ -2549,15 +2838,23 @@ def termsize() -> tuple[int, int]:
         except:
             pass
 
-    if cr:
-        return cr
-
     try:
-        return int(env["COLUMNS"]), int(env["LINES"])
+        return cr or (int(env["COLUMNS"]), int(env["LINES"]))
     except:
         return 80, 25
 
 
+def hidedir(dp) -> None:
+    if ANYWIN:
+        try:
+            k32 = ctypes.WinDLL("kernel32")
+            attrs = k32.GetFileAttributesW(dp)
+            if attrs >= 0:
+                k32.SetFileAttributesW(dp, attrs | 2)
+        except:
+            pass
+
+
 class Pebkac(Exception):
     def __init__(self, code: int, msg: Optional[str] = None) -> None:
         super(Pebkac, self).__init__(msg or HTTPCODE[code])

copyparty/web/baguettebox.js

@@ -27,8 +27,8 @@ window.baguetteBox = (function () {
         isOverlayVisible = false,
         touch = {},  // start-pos
         touchFlag = false,  // busy
-        re_i = /.+\.(gif|jpe?g|png|webp)(\?|$)/i,
-        re_v = /.+\.(webm|mkv|mp4)(\?|$)/i,
+        re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
+        re_v = /^[^?]+\.(webm|mkv|mp4)(\?|$)/i,
         anims = ['slideIn', 'fadeIn', 'none'],
         data = {},  // all galleries
         imagesElements = [],
@@ -277,8 +277,8 @@ window.baguetteBox = (function () {
             playpause();
         else if (k == "KeyU" || k == "KeyO")
             relseek(k == "KeyU" ? -10 : 10);
-        else if (k.indexOf('Digit') === 0)
-            vid().currentTime = vid().duration * parseInt(k.slice(-1)) * 0.1;
+        else if (k.indexOf('Digit') === 0 && v)
+            v.currentTime = v.duration * parseInt(k.slice(-1)) * 0.1;
         else if (k == "KeyM" && v) {
             v.muted = vmute = !vmute;
             mp_ctl();
@@ -580,6 +580,7 @@ window.baguetteBox = (function () {
     function hideOverlay(e) {
         ev(e);
         playvid(false);
+        removeFromCache('#files');
         if (options.noScrollbars) {
             document.documentElement.style.overflowY = 'auto';
             document.body.style.overflowY = 'auto';
@@ -812,10 +813,16 @@ window.baguetteBox = (function () {
     }
 
     function vid() {
+        if (currentIndex >= imagesElements.length)
+            return;
+
         return imagesElements[currentIndex].querySelector('video');
     }
 
     function vidimg() {
+        if (currentIndex >= imagesElements.length)
+            return;
+
         return imagesElements[currentIndex].querySelector('img, video');
     }
 

copyparty/web/browser.css

@@ -93,6 +93,7 @@
 	--g-fsel-bg: #d39;
 	--g-fsel-b1: #f4a;
 	--g-fsel-ts: #804;
+	--g-dfg: var(--srv-3);
 	--g-fg: var(--a-hil);
 	--g-bg: var(--bg-u2);
 	--g-b1: var(--bg-u4);
@@ -327,6 +328,7 @@ html.c {
 }
 html.cz {
 	--bgg: var(--bg-u2);
+	--srv-3: #fff;
 }
 html.cy {
 	--fg: #fff;
@@ -354,6 +356,7 @@ html.cy {
 	--chk-fg: #fd0;
 
 	--srv-1: #f00;
+	--srv-3: #fff;
 	--op-aa-bg: #fff;
 
 	--u2-b1-bg: #f00;
@@ -572,6 +575,11 @@ html.dy {
 * {
 	line-height: 1.2em;
 }
+::selection {
+	color: var(--bg-d1);
+	background: var(--fg);
+	text-shadow: none;
+}
 html,body,tr,th,td,#files,a {
 	color: inherit;
 	background: none;
@@ -754,8 +762,9 @@ html.y #files thead th {
 	display: inline;
 }
 #path a {
-	margin: 0 0 0 -.2em;
-	padding: 0 0 0 .4em;
+	padding: 0 .35em;
+	position: relative;
+	z-index: 1;
 	/* ie: */
 	border-bottom: .1em solid #777\9;
 	margin-right: 1em\9;
@@ -763,18 +772,17 @@ html.y #files thead th {
 #path a:first-child {
 	padding-left: .8em;
 }
-#path a:not(:last-child):after {
-	content: '';
+#path i {
 	width: 1.05em;
 	height: 1.05em;
-	margin: -.2em .3em -.2em -.4em;
+	margin: -.5em .15em -.15em -.7em;
 	display: inline-block;
 	border: 1px solid rgba(255,224,192,0.3);
 	border-width: .05em .05em 0 0;
 	transform: rotate(45deg);
 	background: linear-gradient(45deg, rgba(0,0,0,0) 40%, rgba(0,0,0,0.25) 75%, rgba(0,0,0,0.35));
 }
-html.y #path a:not(:last-child)::after {
+html.y #path i {
 	background: none;
 	border-color: rgba(0,0,0,0.2);
 	border-width: .1em .1em 0 0;
@@ -788,11 +796,31 @@ html.y #path a:hover {
 }
 .logue {
 	padding: .2em 0;
+	position: relative;
+	z-index: 1;
 }
 .logue.hidden,
 .logue:empty {
 	display: none;
 }
+#doc>iframe,
+.logue>iframe {
+	background: var(--bgg);
+	border: 1px solid var(--bgg);
+	border-width: 0 .3em 0 .3em;
+	border-radius: .5em;
+	visibility: hidden;
+	margin: 0 -.3em;
+	width: 100%;
+	height: 0;
+}
+#doc>iframe.focus,
+.logue>iframe.focus {
+	box-shadow: 0 0 .1em .1em var(--a);
+}
+#pro.logue>iframe {
+	height: 100vh;
+}
 #pro.logue {
 	margin-bottom: .8em;
 }
@@ -817,6 +845,10 @@ html.y #path a:hover {
 .mdo {
 	max-width: 52em;
 }
+.mdo.sb,
+#epi.logue.mdo>iframe {
+	max-width: 54em;
+}
 .mdo,
 .mdo * {
 	line-height: 1.4em;
@@ -937,6 +969,9 @@ html.y #path a:hover {
 #ggrid>a.dir:before {
 	content: '📂';
 }
+#ggrid>a.dir>span {
+	color: var(--g-dfg);
+}
 #ggrid>a.au:before {
 	content: '💾';
 }
@@ -983,6 +1018,9 @@ html.np_open #ggrid>a.au:before {
 	background: var(--g-sel-bg);
 	border-color: var(--g-sel-b1);
 }
+#ggrid>a.sel>span {
+	color: var(--g-sel-fg);
+}
 #ggrid>a.sel,
 #ggrid>a[tt].sel {
 	border-top: 1px solid var(--g-fsel-b1);
@@ -1036,6 +1074,9 @@ html.np_open #ggrid>a.au:before {
 	background: var(--bg-d3);
 	box-shadow: -.2em .2em 0 var(--f-sel-sh), -.2em -.2em 0 var(--f-sel-sh);
 }
+#player {
+	display: none;
+}
 #widget {
 	position: fixed;
 	font-size: 1.4em;
@@ -1075,18 +1116,18 @@ html.y #widget.open {
 	top: -.12em;
 }
 #wtico {
-	cursor: url(/.cpr/dd/4.png), pointer;
+	cursor: url(dd/4.png), pointer;
 	animation: cursor 500ms;
 }
 #wtico:hover {
 	animation: cursor 500ms infinite;
 }
 @keyframes cursor {
-	 0% {cursor: url(/.cpr/dd/2.png), pointer}
-	30% {cursor: url(/.cpr/dd/3.png), pointer}
-	50% {cursor: url(/.cpr/dd/4.png), pointer}
-	75% {cursor: url(/.cpr/dd/5.png), pointer}
-	85% {cursor: url(/.cpr/dd/4.png), pointer}
+	 0% {cursor: url(dd/2.png), pointer}
+	30% {cursor: url(dd/3.png), pointer}
+	50% {cursor: url(dd/4.png), pointer}
+	75% {cursor: url(dd/5.png), pointer}
+	85% {cursor: url(dd/4.png), pointer}
 }
 @keyframes spin {
 	100% {transform: rotate(360deg)}
@@ -1294,6 +1335,10 @@ html.y #ops svg circle {
 	padding: .3em .6em;
 	white-space: nowrap;
 }
+#noie {
+	color: #b60;
+	margin: 0 0 0 .5em;
+}
 .opbox {
 	padding: .5em;
 	border-radius: 0 .3em .3em 0;
@@ -2557,7 +2602,6 @@ html.b #u2conf a.b:hover {
 #u2conf input[type="checkbox"]:checked+label:hover {
 	background: var(--u2-o-1h-bg);
 }
-#op_up2k.srch #u2conf td:nth-child(1)>*,
 #op_up2k.srch #u2conf td:nth-child(2)>*,
 #op_up2k.srch #u2conf td:nth-child(3)>* {
 	background: #777;

copyparty/web/browser.html

@@ -8,8 +8,8 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="/.cpr/browser.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
 	{%- if css %}
 	<link rel="stylesheet" media="screen" href="{{ css }}?_={{ ts }}">
 	{%- endif %}
@@ -36,7 +36,7 @@
 			<input type="file" name="f" multiple /><br />
 			<input type="submit" value="start upload">
 		</form>
-		<a id="bbsw" href="?b=u"><br />switch to basic browser</a>
+		<a id="bbsw" href="?b=u" rel="nofollow"><br />switch to basic browser</a>
 	</div>
 
 	<div id="op_mkdir" class="opview opbox act">
@@ -71,7 +71,7 @@
 	<h1 id="path">
 		<a href="#" id="entree">🌲</a>
 		{%- for n in vpnodes %}
-		<a href="/{{ n[0] }}">{{ n[1] }}</a>
+		<a href="{{ r }}/{{ n[0] }}">{{ n[1] }}</a>
 		{%- endfor %}
 	</h1>
 	
@@ -85,7 +85,7 @@
 	<div id="bdoc"></div>
 	{%- endif %}
 
-	<div id="pro" class="logue">{{ logues[0] }}</div>
+	<div id="pro" class="logue">{{ "" if sb_lg else logues[0] }}</div>
 
 	<table id="files">
 		<thead>
@@ -119,9 +119,9 @@
 		</tbody>
 	</table>
 	
-	<div id="epi" class="logue">{{ logues[1] }}</div>
+	<div id="epi" class="logue">{{ "" if sb_lg else logues[1] }}</div>
 
-	<h2><a href="/?h" id="goh">control-panel</a></h2>
+	<h2 id="wfp"><a href="{{ r }}/?h" id="goh">control-panel</a></h2>
 	
 	<a href="#" id="repl">π</a>
 
@@ -134,7 +134,9 @@
 	<div id="widget"></div>
 
 	<script>
-		var acct = "{{ acct }}",
+		var SR = {{ r|tojson }},
+			TS = "{{ ts }}",
+			acct = "{{ acct }}",
 			perms = {{ perms }},
 			themes = {{ themes }},
 			dtheme = "{{ dtheme }}",
@@ -149,21 +151,25 @@
 			have_del = {{ have_del|tojson }},
 			have_unpost = {{ have_unpost }},
 			have_zip = {{ have_zip|tojson }},
+			sb_md = "{{ sb_md }}",
+			sb_lg = "{{ sb_lg }}",
 			lifetime = {{ lifetime }},
 			turbolvl = {{ turbolvl }},
+			idxh = {{ idxh }},
+			frand = {{ frand|tojson }},
 			u2sort = "{{ u2sort }}",
 			have_emp = {{ have_emp|tojson }},
 			txt_ext = "{{ txt_ext }}",
-			{% if no_prism %}no_prism = 1,{% endif %}
+			logues = {{ logues|tojson if sb_lg else "[]" }},
 			readme = {{ readme|tojson }},
 			ls0 = {{ ls0|tojson }};
 
 		document.documentElement.className = localStorage.theme || dtheme;
 	</script>
-	<script src="/.cpr/util.js?_={{ ts }}"></script>
-	<script src="/.cpr/baguettebox.js?_={{ ts }}"></script>
-	<script src="/.cpr/browser.js?_={{ ts }}"></script>
-	<script src="/.cpr/up2k.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/baguettebox.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/browser.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/up2k.js?_={{ ts }}"></script>
 	{%- if js %}
 	<script src="{{ js }}?_={{ ts }}"></script>
 	{%- endif %}

copyparty/web/browser2.html

@@ -57,7 +57,7 @@
 	<div>{{ logues[1] }}</div><br />
 	{%- endif %}
 	
-	<h2><a href="/{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
+	<h2><a href="{{ r }}/{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
 
 </body>
 </html>

copyparty/web/md.html

@@ -5,10 +5,10 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" href="/.cpr/md.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
 	{%- if edit %}
-	<link rel="stylesheet" href="/.cpr/md2.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/md2.css?_={{ ts }}">
 	{%- endif %}
 </head>
 <body>
@@ -128,7 +128,8 @@ write markdown (most html is 🙆 too)
 	
 	<script>
 
-var last_modified = {{ lastmod }},
+var SR = {{ r|tojson }},
+	last_modified = {{ lastmod }},
 	have_emp = {{ have_emp|tojson }},
 	dfavico = "{{ favico }}";
 
@@ -153,10 +154,10 @@ l.light = drk? 0:1;
 })();
 
 	</script>
-	<script src="/.cpr/util.js?_={{ ts }}"></script>
-	<script src="/.cpr/deps/marked.js?_={{ ts }}"></script>
-	<script src="/.cpr/md.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/deps/marked.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/md.js?_={{ ts }}"></script>
 	{%- if edit %}
-	<script src="/.cpr/md2.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/md2.js?_={{ ts }}"></script>
 	{%- endif %}
 </body></html>

copyparty/web/md.js

@@ -231,11 +231,11 @@ function convert_markdown(md_text, dest_dom) {
     var nodes = md_dom.getElementsByTagName('a');
     for (var a = nodes.length - 1; a >= 0; a--) {
         var href = nodes[a].getAttribute('href');
-        var txt = nodes[a].textContent;
+        var txt = nodes[a].innerHTML;
 
         if (!txt)
             nodes[a].textContent = href;
-        else if (href !== txt)
+        else if (href !== txt && !nodes[a].className)
             nodes[a].className = 'vis';
     }
 

copyparty/web/md2.js

@@ -930,7 +930,9 @@ var set_lno = (function () {
 (function () {
     function keydown(ev) {
         ev = ev || window.event;
-        var kc = ev.code || ev.keyCode || ev.which;
+        var kc = ev.code || ev.keyCode || ev.which,
+            editing = document.activeElement == dom_src;
+
         //console.log(ev.key, ev.code, ev.keyCode, ev.which);
         if (ctrl(ev) && (ev.code == "KeyS" || kc == 83)) {
             save();
@@ -941,12 +943,17 @@ var set_lno = (function () {
             if (d)
                 d.click();
         }
-        if (document.activeElement != dom_src)
-            return true;
-
-        set_lno();
+        if (editing)
+            set_lno();
 
         if (ctrl(ev)) {
+            if (ev.code == "KeyE") {
+                dom_nsbs.click();
+                return false;
+            }
+            if (!editing)
+                return true;
+
             if (ev.code == "KeyH" || kc == 72) {
                 md_header(ev.shiftKey);
                 return false;
@@ -971,10 +978,6 @@ var set_lno = (function () {
                 iter_uni();
                 return false;
             }
-            if (ev.code == "KeyE") {
-                dom_nsbs.click();
-                return false;
-            }
             var up = ev.code == "ArrowUp" || kc == 38;
             var dn = ev.code == "ArrowDown" || kc == 40;
             if (up || dn) {
@@ -987,6 +990,9 @@ var set_lno = (function () {
             }
         }
         else {
+            if (!editing)
+                return true;
+
             if (ev.code == "Tab" || kc == 9) {
                 md_indent(ev.shiftKey);
                 return false;

copyparty/web/mde.html

@@ -5,10 +5,10 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" href="/.cpr/mde.css?_={{ ts }}">
-	<link rel="stylesheet" href="/.cpr/deps/mini-fa.css?_={{ ts }}">
-	<link rel="stylesheet" href="/.cpr/deps/easymde.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/mde.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/deps/mini-fa.css?_={{ ts }}">
+	<link rel="stylesheet" href="{{ r }}/.cpr/deps/easymde.css?_={{ ts }}">
 </head>
 <body>
 	<div id="mw">
@@ -26,7 +26,8 @@
 	<a href="#" id="repl">π</a>
 	<script>
 
-var last_modified = {{ lastmod }},
+var SR = {{ r|tojson }},
+	last_modified = {{ lastmod }},
 	have_emp = {{ have_emp|tojson }},
 	dfavico = "{{ favico }}";
 
@@ -48,8 +49,8 @@ l.light = drk? 0:1;
 })();
 
 	</script>
-	<script src="/.cpr/util.js?_={{ ts }}"></script>
-	<script src="/.cpr/deps/marked.js?_={{ ts }}"></script>
-	<script src="/.cpr/deps/easymde.js?_={{ ts }}"></script>
-	<script src="/.cpr/mde.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/deps/marked.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/deps/easymde.js?_={{ ts }}"></script>
+	<script src="{{ r }}/.cpr/mde.js?_={{ ts }}"></script>
 </body></html>

copyparty/web/msg.html

@@ -8,7 +8,7 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" media="screen" href="/.cpr/msg.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/splash.css

@@ -16,7 +16,8 @@ html {
 h1 {
 	border-bottom: 1px solid #ccc;
 	margin: 2em 0 .4em 0;
-	padding: 0 0 .2em 0;
+	padding: 0;
+	line-height: 1em;
 	font-weight: normal;
 }
 li {
@@ -26,6 +27,7 @@ a {
 	color: #047;
 	background: #fff;
 	text-decoration: none;
+	white-space: nowrap;
 	border-bottom: 1px solid #8ab;
 	border-radius: .2em;
 	padding: .2em .6em;
@@ -34,6 +36,11 @@ a {
 td a {
 	margin: 0;
 }
+#w {
+	color: #fff;
+	background: #940;
+	border-color: #b70;
+}
 .af,
 .logout {
 	float: right;
@@ -49,12 +56,30 @@ a.g {
 	border-color: #3a0;
 	box-shadow: 0 .3em 1em #4c0;
 }
-#repl {
+#repl,
+#pb a {
 	border: none;
 	background: none;
 	color: inherit;
 	padding: 0;
 }
+#repl {
+	position: fixed;
+	bottom: .25em;
+	left: .2em;
+}
+#pb {
+	opacity: .5;
+	position: fixed;
+	bottom: .25em;
+	right: .3em;
+}
+#pb span {
+	opacity: .6;
+}
+#pb a {
+	margin: 0;
+}
 table {
 	border-collapse: collapse;
 }
@@ -155,15 +180,19 @@ html.z a.g {
 	border-color: #af4;
 	box-shadow: 0 .3em 1em #7d0;
 }
-html.z input {
-	color: #fff;
-	background: #626;
-	border: 1px solid #c2c;
-	border-width: 1px 0 0 0;
+input {
+	color: #a50;
+	background: #fff;
+	border: 1px solid #a50;
 	border-radius: .5em;
 	padding: .5em .7em;
 	margin: 0 .5em 0 0;
 }
+html.z input {
+	color: #fff;
+	background: #626;
+	border-color: #c2c;
+}
 html.z .num {
 	border-color: #777;
 }

copyparty/web/splash.html

@@ -8,19 +8,19 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" media="screen" href="/.cpr/splash.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 </head>
 
 <body>
 	<div id="wrap">
-		<a id="a" href="/?h" class="af">refresh</a>
-		<a id="v" href="/?hc" class="af">connect</a>
+		<a id="a" href="{{ r }}/?h" class="af">refresh</a>
+		<a id="v" href="{{ r }}/?hc" class="af">connect</a>
 
 		{%- if this.uname == '*' %}
 			<p id="b">howdy stranger &nbsp; <small>(you're not logged in)</small></p>
 		{%- else %}
-			<a id="c" href="/?pw=x" class="logout">logout</a>
+			<a id="c" href="{{ r }}/?pw=x" class="logout">logout</a>
 			<p><span id="m">welcome back,</span> <strong>{{ this.uname }}</strong></p>
 		{%- endif %}
 
@@ -46,15 +46,15 @@
 				<tbody>
 					{% for mp in avol %}
 					{%- if mp in vstate and vstate[mp] %}
-					<tr><td><a href="{{ mp }}{{ url_suf }}">{{ mp }}</a></td><td><a class="s" href="{{ mp }}?scan">rescan</a></td><td>{{ vstate[mp] }}</td></tr>
+					<tr><td><a href="{{ r }}{{ mp }}{{ url_suf }}">{{ mp }}</a></td><td><a class="s" href="{{ r }}{{ mp }}?scan">rescan</a></td><td>{{ vstate[mp] }}</td></tr>
 					{%- endif %}
 					{% endfor %}
 				</tbody>
 			</table>
 		</td></tr></table>
 		<div class="btns">
-			<a id="d" href="/?stack">dump stack</a>
-			<a id="e" href="/?reload=cfg">reload cfg</a>
+			<a id="d" href="{{ r }}/?stack">dump stack</a>
+			<a id="e" href="{{ r }}/?reload=cfg">reload cfg</a>
 		</div>
 		{%- endif %}
 
@@ -62,7 +62,7 @@
 		<h1 id="f">you can browse:</h1>
 		<ul>
 			{% for mp in rvol %}
-			<li><a href="{{ mp }}{{ url_suf }}">{{ mp }}</a></li>
+			<li><a href="{{ r }}{{ mp }}{{ url_suf }}">{{ mp }}</a></li>
 			{% endfor %}
 		</ul>
 		{%- endif %}
@@ -71,7 +71,7 @@
 		<h1 id="g">you can upload to:</h1>
 		<ul>
 			{% for mp in wvol %}
-			<li><a href="{{ mp }}{{ url_suf }}">{{ mp }}</a></li>
+			<li><a href="{{ r }}{{ mp }}{{ url_suf }}">{{ mp }}</a></li>
 			{% endfor %}
 		</ul>
 		{%- endif %}
@@ -79,34 +79,41 @@
 		<h1 id="cc">client config:</h1>
 		<ul>
 			{% if k304 %}
-			<li><a id="h" href="/?k304=n">disable k304</a> (currently enabled)
+			<li><a id="h" href="{{ r }}/?k304=n">disable k304</a> (currently enabled)
 			{%- else %}
-			<li><a id="i" href="/?k304=y" class="r">enable k304</a> (currently disabled)
+			<li><a id="i" href="{{ r }}/?k304=y" class="r">enable k304</a> (currently disabled)
 			{% endif %}
 			<blockquote id="j">enabling this will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
 			
-			<li><a id="k" href="/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
+			<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
 		</ul>
 
 		<h1 id="l">login for more:</h1>
-		<ul>
-			<form method="post" enctype="multipart/form-data" action="/{{ qvpath }}">
+		<div>
+			<form method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
 				<input type="hidden" name="act" value="login" />
 				<input type="password" name="cppwd" />
 				<input type="submit" value="Login" />
+				{% if ahttps %}
+				<a id="w" href="{{ ahttps }}">switch to https</a>
+				{% endif %}
 			</form>
-		</ul>
+		</div>
 	</div>
 	<a href="#" id="repl">π</a>
+	{%- if not this.args.nb %}
+	<span id="pb"><span>powered by</span> <a href="{{ this.args.pb_url }}">copyparty {{ver}}</a></span>
+	{%- endif %}
 	<script>
 
-var lang="{{ lang }}",
+var SR = {{ r|tojson }},
+	lang="{{ lang }}",
 	dfavico="{{ favico }}";
 
 document.documentElement.className=localStorage.theme||"{{ this.args.theme }}";
 
 </script>
-<script src="/.cpr/util.js?_={{ ts }}"></script>
-<script src="/.cpr/splash.js?_={{ ts }}"></script>
+<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+<script src="{{ r }}/.cpr/splash.js?_={{ ts }}"></script>
 </body>
 </html>

copyparty/web/splash.js

@@ -17,15 +17,16 @@ var Ls = {
 		"l1": "logg inn:",
 		"m1": "velkommen tilbake,",
 		"n1": "404: filen finnes ikke  ┐( ´ -`)┌",
-		"o1": 'eller kanskje du ikke har tilgang? prøv å logge inn eller <a href="/?h">gå hjem</a>',
+		"o1": 'eller kanskje du ikke har tilgang? prøv å logge inn eller <a href="' + SR + '/?h">gå hjem</a>',
 		"p1": "403: tilgang nektet &nbsp;~┻━┻",
-		"q1": 'du må logge inn eller <a href="/?h">gå hjem</a>',
+		"q1": 'du må logge inn eller <a href="' + SR + '/?h">gå hjem</a>',
 		"r1": "gå hjem",
 		".s1": "kartlegg",
 		"t1": "handling",
 		"u2": "tid siden noen sist skrev til serveren$N( opplastning / navneendring / ... )$N$N17d = 17 dager$N1h23 = 1 time 23 minutter$N4m56 = 4 minuter 56 sekunder",
 		"v1": "koble til",
-		"v2": "bruk denne serveren som en lokal harddisk$N$NADVARSEL: kommer til å vise passordet ditt!"
+		"v2": "bruk denne serveren som en lokal harddisk$N$NADVARSEL: kommer til å vise passordet ditt!",
+		"w1": "bytt til https",
 	},
 	"eng": {
 		"d2": "shows the state of all active threads",

copyparty/web/svcs.html

@@ -8,14 +8,14 @@
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
 	<meta name="theme-color" content="#333">
 {{ html_head }}
-	<link rel="stylesheet" media="screen" href="/.cpr/splash.css?_={{ ts }}">
-	<link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 </head>
 
 <body>
 	<div id="wrap" class="w">
         <div class="cn">
-            <p class="btns"><a href="/">browse files</a> // <a href="/?h">control panel</a></p>
+            <p class="btns"><a href="/{{ rvp }}">browse files</a> // <a href="{{ r }}/?h">control panel</a></p>
             <p>or choose your OS for cooler alternatives:</p>
             <div class="ossel">
                 <a id="swin" href="#">Windows</a>
@@ -28,7 +28,7 @@
             make this server appear on your computer as a regular HDD!<br />
             pick your favorite below (sorted by performance, best first) and lets 🎉<br />
             <br />
-            placeholders:
+            <span class="os win lin mac">placeholders:</span>
             <span class="os win">
                 {% if accs %}<code><b>{{ pw }}</b></code>=password, {% endif %}<code><b>W:</b></code>=mountpoint
             </span>
@@ -43,31 +43,35 @@
         <h1>WebDAV</h1>
 
         <div class="os win">
-            <p><em>note: rclone-FTP is a bit faster, so {% if args.ftp or args.ftps %}try that first{% else %}consider enabling FTP in server settings{% endif %}</em></p>
             <p>if you can, install <a href="https://winfsp.dev/rel/">winfsp</a>+<a href="https://downloads.rclone.org/rclone-current-windows-amd64.zip">rclone</a> and then paste this in cmd:</p>
             <pre>
-                rclone config create {{ aname }}-dav webdav url=http{{ s }}://{{ rip }}{{ hport }} vendor=other{% if accs %} user=k pass=<b>{{ pw }}</b>{% endif %}
-                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-dav:{{ vp }} <b>W:</b>
+                rclone config create {{ aname }}-dav webdav url=http{{ s }}://{{ rip }}{{ hport }} vendor=owncloud pacer_min_sleep=0.01ms{% if accs %} user=k pass=<b>{{ pw }}</b>{% endif %}
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-dav:{{ rvp }} <b>W:</b>
             </pre>
             {% if s %}
             <p><em>note: if you are on LAN (or just dont have valid certificates), add <code>--no-check-certificate</code> to the mount command</em><br />---</p>
             {% endif %}
             
-            <p>if you want to use the native WebDAV client in windows instead (slow and buggy), first run <a href="/.cpr/a/webdav-cfg.bat">webdav-cfg.bat</a> to remove the 47 MiB filesize limit (also fixes latency and password login), then connect:</p>
+            <p>if you want to use the native WebDAV client in windows instead (slow and buggy), first run <a href="{{ r }}/.cpr/a/webdav-cfg.bat">webdav-cfg.bat</a> to remove the 47 MiB filesize limit (also fixes latency and password login), then connect:</p>
             <pre>
-                net use <b>w:</b> http{{ s }}://{{ ep }}/{{ vp }}{% if accs %} k /user:<b>{{ pw }}</b>{% endif %}
+                net use <b>w:</b> http{{ s }}://{{ ep }}/{{ rvp }}{% if accs %} k /user:<b>{{ pw }}</b>{% endif %}
             </pre>
         </div>
 
         <div class="os lin">
             <pre>
                 yum install davfs2
-                {% if accs %}printf '%s\n' <b>{{ pw }}</b> k | {% endif %}mount -t davfs -ouid=1000 http{{ s }}://{{ ep }}/{{ vp }} <b>mp</b>
+                {% if accs %}printf '%s\n' <b>{{ pw }}</b> k | {% endif %}mount -t davfs -ouid=1000 http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b>
             </pre>
-            <p>or you can use rclone instead, which is much slower but doesn't require root:</p>
+            <p>make it automount on boot:</p>
             <pre>
-                rclone config create {{ aname }}-dav webdav url=http{{ s }}://{{ rip }}{{ hport }} vendor=other{% if accs %} user=k pass=<b>{{ pw }}</b>{% endif %}
-                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-dav:{{ vp }} <b>mp</b>
+                printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>{{ pw }}</b> k" >> /etc/davfs2/secrets
+                printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b> davfs rw,user,uid=1000,noauto 0 0" >> /etc/fstab
+            </pre>
+            <p>or you can use rclone instead, which is much slower but doesn't require root (plus it keeps lastmodified on upload):</p>
+            <pre>
+                rclone config create {{ aname }}-dav webdav url=http{{ s }}://{{ rip }}{{ hport }} vendor=owncloud pacer_min_sleep=0.01ms{% if accs %} user=k pass=<b>{{ pw }}</b>{% endif %}
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-dav:{{ rvp }} <b>mp</b>
             </pre>
             {% if s %}
             <p><em>note: if you are on LAN (or just dont have valid certificates), add <code>--no-check-certificate</code> to the mount command</em><br />---</p>
@@ -77,20 +81,20 @@
             <!-- gnome-bug: ignores vp -->
             <pre>
                 {%- if accs %}
-                echo <b>{{ pw }}</b> | gio mount dav{{ s }}://k@{{ ep }}/{{ vp }}
+                echo <b>{{ pw }}</b> | gio mount dav{{ s }}://k@{{ ep }}/{{ rvp }}
                 {%- else %}
-                gio mount -a dav{{ s }}://{{ ep }}/{{ vp }}
+                gio mount -a dav{{ s }}://{{ ep }}/{{ rvp }}
                 {%- endif %}
             </pre>
         </div>
 
         <div class="os mac">
             <pre>
-                osascript -e ' mount volume "http{{ s }}://k:<b>{{ pw }}</b>@{{ ep }}/{{ vp }}" '
+                osascript -e ' mount volume "http{{ s }}://k:<b>{{ pw }}</b>@{{ ep }}/{{ rvp }}" '
             </pre>
             <p>or you can open up a Finder, press command-K and paste this instead:</p>
             <pre>
-                http{{ s }}://k:<b>{{ pw }}</b>@{{ ep }}/{{ vp }}
+                http{{ s }}://k:<b>{{ pw }}</b>@{{ ep }}/{{ rvp }}
             </pre>
             
             {% if s %}
@@ -106,28 +110,50 @@
 
         <div class="os win">
             <p>if you can, install <a href="https://winfsp.dev/rel/">winfsp</a>+<a href="https://downloads.rclone.org/rclone-current-windows-amd64.zip">rclone</a> and then paste this in cmd:</p>
+            {% if args.ftp %}
+            <p>connect with plaintext FTP:</p>
             <pre>
-                rclone config create {{ aname }}-ftp ftp host={{ rip }} port={{ args.ftp or args.ftps }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls={{ "false" if args.ftp else "true" }}
-                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftp:{{ vp }} <b>W:</b>
+                rclone config create {{ aname }}-ftp ftp host={{ rip }} port={{ args.ftp }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls=false
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftp:{{ rvp }} <b>W:</b>
             </pre>
+            {% endif %}
+            {% if args.ftps %}
+            <p>connect with TLS-encrypted FTPS:</p>
+            <pre>
+                rclone config create {{ aname }}-ftps ftp host={{ rip }} port={{ args.ftps }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls=false explicit_tls=true
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftps:{{ rvp }} <b>W:</b>
+            </pre>
+            <p><em>note: if you are on LAN (or just dont have valid certificates), add <code>no_check_certificate=true</code> to the config command</em><br />---</p>
+            {% endif %}
             <p>if you want to use the native FTP client in windows instead (please dont), press <code>win+R</code> and run this command:</p>
             <pre>
-                explorer {{ "ftp" if args.ftp else "ftps" }}://{% if accs %}<b>{{ pw }}</b>:k@{% endif %}{{ host }}:{{ args.ftp or args.ftps }}/{{ vp }}
+                explorer {{ "ftp" if args.ftp else "ftps" }}://{% if accs %}<b>{{ pw }}</b>:k@{% endif %}{{ host }}:{{ args.ftp or args.ftps }}/{{ rvp }}
             </pre>
         </div>
 
         <div class="os lin">
+            {% if args.ftp %}
+            <p>connect with plaintext FTP:</p>
             <pre>
-                rclone config create {{ aname }}-ftp ftp host={{ rip }} port={{ args.ftp or args.ftps }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls={{ "false" if args.ftp else "true" }}
-                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftp:{{ vp }} <b>mp</b>
+                rclone config create {{ aname }}-ftp ftp host={{ rip }} port={{ args.ftp }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls=false
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftp:{{ rvp }} <b>mp</b>
             </pre>
+            {% endif %}
+            {% if args.ftps %}
+            <p>connect with TLS-encrypted FTPS:</p>
+            <pre>
+                rclone config create {{ aname }}-ftps ftp host={{ rip }} port={{ args.ftps }} pass=k user={% if accs %}<b>{{ pw }}</b>{% else %}anonymous{% endif %} tls=false explicit_tls=true
+                rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-ftps:{{ rvp }} <b>mp</b>
+            </pre>
+            <p><em>note: if you are on LAN (or just dont have valid certificates), add <code>no_check_certificate=true</code> to the config command</em><br />---</p>
+            {% endif %}
             <p>emergency alternative (gnome/gui-only):</p>
             <!-- gnome-bug: ignores vp -->
             <pre>
                 {%- if accs %}
-                echo <b>{{ pw }}</b> | gio mount ftp{{ "" if args.ftp else "s" }}://k@{{ host }}:{{ args.ftp or args.ftps }}/{{ vp }}
+                echo <b>{{ pw }}</b> | gio mount ftp{{ "" if args.ftp else "s" }}://k@{{ host }}:{{ args.ftp or args.ftps }}/{{ rvp }}
                 {%- else %}
-                gio mount -a ftp{{ "" if args.ftp else "s" }}://{{ host }}:{{ args.ftp or args.ftps }}/{{ vp }}
+                gio mount -a ftp{{ "" if args.ftp else "s" }}://{{ host }}:{{ args.ftp or args.ftps }}/{{ rvp }}
                 {%- endif %}
             </pre>
         </div>
@@ -135,7 +161,7 @@
         <div class="os mac">
             <p>note: FTP is read-only on macos; please use WebDAV instead</p>
             <pre>
-                open {{ "ftp" if args.ftp else "ftps" }}://{% if accs %}k:<b>{{ pw }}</b>@{% else %}anonymous:@{% endif %}{{ host }}:{{ args.ftp or args.ftps }}/{{ vp }}
+                open {{ "ftp" if args.ftp else "ftps" }}://{% if accs %}k:<b>{{ pw }}</b>@{% else %}anonymous:@{% endif %}{{ host }}:{{ args.ftp or args.ftps }}/{{ rvp }}
             </pre>
         </div>
         {% endif %}
@@ -144,18 +170,18 @@
 
         <h1>partyfuse</h1>
         <p>
-            <a href="/.cpr/a/partyfuse.py">partyfuse.py</a> -- fast, read-only,
+            <a href="{{ r }}/.cpr/a/partyfuse.py">partyfuse.py</a> -- fast, read-only,
             <span class="os win">needs <a href="https://winfsp.dev/rel/">winfsp</a></span>
             <span class="os lin">doesn't need root</span>
         </p>
         <pre>
-            partyfuse.py{% if accs %} -a <b>{{ pw }}</b>{% endif %} http{{ s }}://{{ ep }}/{{ vp }} <b><span class="os win">W:</span><span class="os lin mac">mp</span></b>
+            partyfuse.py{% if accs %} -a <b>{{ pw }}</b>{% endif %} http{{ s }}://{{ ep }}/{{ rvp }} <b><span class="os win">W:</span><span class="os lin mac">mp</span></b>
         </pre>
         {% if s %}
         <p><em>note: if you are on LAN (or just dont have valid certificates), add <code>-td</code></em></p>
         {% endif %}
         <p>
-            you can use <a href="/.cpr/a/up2k.py">up2k.py</a> to upload (sometimes faster than web-browsers)
+            you can use <a href="{{ r }}/.cpr/a/up2k.py">up2k.py</a> to upload (sometimes faster than web-browsers)
         </p>
 
 
@@ -188,13 +214,14 @@
 	<a href="#" id="repl">π</a>
 	<script>
 
-var lang="{{ lang }}",
+var SR = {{ r|tojson }},
+    lang="{{ lang }}",
 	dfavico="{{ favico }}";
 
 document.documentElement.className=localStorage.theme||"{{ args.theme }}";
 
 </script>
-<script src="/.cpr/util.js?_={{ ts }}"></script>
-<script src="/.cpr/svcs.js?_={{ ts }}"></script>
+<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
+<script src="{{ r }}/.cpr/svcs.js?_={{ ts }}"></script>
 </body>
 </html>

copyparty/web/svcs.js

@@ -39,4 +39,4 @@ function setos(os) {
         clmod(oa[a], 'g', oa[a].id.slice(1) == os);
 }
 
-setos(WINDOWS ? 'win' : LINUX ? 'lin' : MACOS ? 'mac' : '');
+setos(WINDOWS ? 'win' : LINUX ? 'lin' : MACOS ? 'mac' : 'idk');

copyparty/web/ui.css

@@ -1,7 +1,7 @@
 @font-face {
 	font-family: 'scp';
 	font-display: swap;
-	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
+	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(deps/scp.woff2) format('woff2');
 }
 html {
 	touch-action: manipulation;
@@ -42,6 +42,10 @@ html {
 	text-shadow: 1px 1px 0 #000;
 	color: #fff;
 }
+#toast.top {
+	top: 2em;
+	bottom: unset;
+}
 #toast a {
 	color: inherit;
 	text-shadow: inherit;
@@ -69,6 +73,7 @@ html {
 #toastb {
 	max-height: 70vh;
 	overflow-y: auto;
+	padding: 1px;
 }
 #toast.scroll #toastb {
 	overflow-y: scroll;
@@ -446,6 +451,20 @@ html.y textarea:focus {
 	padding: .2em .5em;
 	border: .12em solid #aaa;
 }
+.mdo .mdth,
+.mdo .mdthl,
+.mdo .mdthr {
+	margin: .5em .5em .5em 0;
+}
+.mdthl {
+	float: left;
+}
+.mdthr {
+	float: right;
+}
+hr {
+	clear: both;
+}
 
 @media screen {
 	.mdo {

copyparty/web/up2k.js

@@ -114,10 +114,10 @@ function up2k_flagbus() {
             do_take(now);
             return;
         }
-        if (flag.owner && now - flag.owner[1] > 5000) {
+        if (flag.owner && now - flag.owner[1] > 12000) {
             flag.owner = null;
         }
-        if (flag.wants && now - flag.wants[1] > 5000) {
+        if (flag.wants && now - flag.wants[1] > 12000) {
             flag.wants = null;
         }
         if (!flag.owner && !flag.wants) {
@@ -672,7 +672,7 @@ function Donut(uc, st) {
             favico.upd();
             wintitle();
             if (document.visibilityState == 'hidden')
-                tenstrobe = setTimeout(enstrobe, 500); //debounce
+                tenstrobe = setTimeout(r.enstrobe, 500); //debounce
         }
     };
 
@@ -709,7 +709,7 @@ function Donut(uc, st) {
         }
     };
 
-    function enstrobe() {
+    r.enstrobe = function () {
         strobes = ['████████████████', '________________', '████████████████'];
         tstrober = setInterval(strobe, 300);
 
@@ -772,6 +772,7 @@ function fsearch_explain(n) {
 
 function up2k_init(subtle) {
     var r = {
+        "tact": Date.now(),
         "init_deps": init_deps,
         "set_fsearch": set_fsearch,
         "gotallfiles": [gotallfiles]  // hooks
@@ -779,7 +780,7 @@ function up2k_init(subtle) {
 
     setTimeout(function () {
         if (window.WebAssembly && !hws.length)
-            fetch('/.cpr/w.hash.js' + CB);
+            fetch(SR + '/.cpr/w.hash.js' + CB);
     }, 1000);
 
     function showmodal(msg) {
@@ -809,7 +810,7 @@ function up2k_init(subtle) {
                 m = L.u_https1 + ' <a href="' + (window.location + '').replace(':', 's:') + '">' + L.u_https2 + '</a> ' + L.u_https3;
 
             showmodal('<h1>loading ' + fn + '</h1>');
-            import_js('/.cpr/deps/' + fn, unmodal);
+            import_js(SR + '/.cpr/deps/' + fn, unmodal);
 
             if (HTTPS) {
                 // chrome<37 firefox<34 edge<12 opera<24 safari<7
@@ -856,6 +857,7 @@ function up2k_init(subtle) {
         fdom_ctr = 0,
         biggest_file = 0;
 
+    bcfg_bind(uc, 'rand', 'u2rand', false, null, false);
     bcfg_bind(uc, 'multitask', 'multitask', true, null, false);
     bcfg_bind(uc, 'potato', 'potato', false, set_potato, false);
     bcfg_bind(uc, 'ask_up', 'ask_up', true, null, false);
@@ -867,7 +869,7 @@ function up2k_init(subtle) {
     bcfg_bind(uc, 'az', 'u2sort', u2sort.indexOf('n') + 1, set_u2sort);
     bcfg_bind(uc, 'hashw', 'hashw', !!window.WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
     bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
-    bcfg_bind(uc, 'upsfx', 'upsfx', false);
+    bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);
 
     var st = {
         "files": [],
@@ -895,15 +897,25 @@ function up2k_init(subtle) {
             "finished": 0
         },
         "time": {
-            "hashing": 0,
-            "uploading": 0,
-            "busy": 0
+            "hashing": 0.01,
+            "uploading": 0.01,
+            "busy": 0.01
         },
         "eta": {
             "h": "",
             "u": "",
             "t": ""
         },
+        "etaw": {
+            "h": [['', 0, 0, 0]],
+            "u": [['', 0, 0, 0]],
+            "t": [['', 0, 0, 0]]
+        },
+        "etac": {
+            "h": 0,
+            "u": 0,
+            "t": 0
+        },
         "car": 0,
         "slow_io": null,
         "oserr": false,
@@ -1312,7 +1324,7 @@ function up2k_init(subtle) {
 
         if (window.WebAssembly && !hws.length) {
             for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
-                hws.push(new Worker('/.cpr/w.hash.js' + CB));
+                hws.push(new Worker(SR + '/.cpr/w.hash.js' + CB));
 
             console.log(hws.length + " hashers");
         }
@@ -1353,6 +1365,10 @@ function up2k_init(subtle) {
 
             if (uc.fsearch)
                 entry.srch = 1;
+            else if (uc.rand) {
+                entry.rand = true;
+                entry.name = 'a\n' + entry.name;
+            }
 
             if (biggest_file < entry.size)
                 biggest_file = entry.size;
@@ -1388,7 +1404,7 @@ function up2k_init(subtle) {
         ebi('u2tabw').className = 'ye';
 
         setTimeout(function () {
-            if (!actx || actx.state != 'suspended' || toast.tag == L.u_unpt)
+            if (!actx || actx.state != 'suspended' || toast.visible)
                 return;
 
             toast.warn(30, "<div onclick=\"start_actx();toast.inf(3,'thanks!')\">please click this text to<br />unlock full upload speed</div>");
@@ -1408,7 +1424,36 @@ function up2k_init(subtle) {
     }
     more_one_file();
 
-    var etaref = 0, etaskip = 0, utw_minh = 0, utw_read = 0;
+    function linklist() {
+        var ret = [],
+            base = document.location.origin.replace(/\/$/, '');
+
+        for (var a = 0; a < st.files.length; a++) {
+            var t = st.files[a],
+                url = t.purl + uricom_enc(t.name);
+
+            if (t.fk)
+                url += '?k=' + t.fk;
+
+            ret.push(base + url);
+        }
+        return ret.join('\r\n');
+    }
+
+    ebi('luplinks').onclick = function (e) {
+        ev(e);
+        modal.alert(linklist());
+    };
+
+    ebi('cuplinks').onclick = function (e) {
+        ev(e);
+        var txt = linklist();
+        cliptxt(txt + '\n', function () {
+            toast.inf(5, txt.split('\n').length + ' links copied to clipboard');
+        });
+    };
+
+    var etaref = 0, etaskip = 0, utw_minh = 0, utw_read = 0, utw_card = 0;
     function etafun() {
         var nhash = st.busy.head.length + st.busy.hash.length + st.todo.head.length + st.todo.hash.length,
             nsend = st.busy.upload.length + st.todo.upload.length,
@@ -1421,6 +1466,12 @@ function up2k_init(subtle) {
 
         //ebi('acc_info').innerHTML = humantime(st.time.busy) + ' ' + f2f(now / 1000, 1);
 
+        if (utw_card != pvis.act) {
+            utw_card = pvis.act;
+            utw_read = 9001;
+            ebi('u2tabw').style.minHeight = '0px';
+        }
+
         if (++utw_read >= 20) {
             utw_read = 0;
             utw_minh = parseInt(ebi('u2tabw').style.minHeight || '0');
@@ -1480,10 +1531,20 @@ function up2k_init(subtle) {
             }
         }
         for (var a = 0; a < t.length; a++) {
-            var rem = st.bytes.total - t[a][2],
-                bps = t[a][1] / t[a][3],
-                hid = t[a][0],
+            var hid = t[a][0],
                 eid = hid.slice(-1),
+                etaw = st.etaw[eid];
+
+            if (st.etac[eid] > 100) {  // num chunks
+                st.etac[eid] = 0;
+                etaw.push(jcp(t[a]));
+                if (etaw.length > 5)
+                    etaw.shift();
+            }
+
+            var h = etaw[0],
+                rem = st.bytes.total - t[a][2],
+                bps = (t[a][1] - h[1]) / Math.max(0.1, t[a][3] - h[3]),
                 eta = Math.floor(rem / bps);
 
             if (t[a][1] < 1024 || t[a][3] < 0.1) {
@@ -1535,11 +1596,11 @@ function up2k_init(subtle) {
             st.busy.handshake.length)
             return false;
 
-        if (t.n - st.car > 8)
+        if (t.n - st.car > Math.max(8, parallel_uploads))
             // prevent runahead from a stuck upload (slow server hdd)
             return false;
 
-        if ((uc.multitask ? 1 : 0) <
+        if ((uc.multitask ? parallel_uploads : 0) <
             st.todo.upload.length +
             st.busy.upload.length)
             return false;
@@ -1551,21 +1612,22 @@ function up2k_init(subtle) {
         if (!parallel_uploads)
             return false;
 
+        var nhs = st.todo.handshake.length + st.busy.handshake.length,
+            nup = st.todo.upload.length + st.busy.upload.length;
+
         if (uc.multitask) {
+            if (nhs + nup < parallel_uploads)
+                return true;
+
             if (!uc.az)
-                return st.todo.handshake.length + st.busy.handshake.length < 2;
+                return nhs < 2;
 
             var ahead = st.bytes.hashed - st.bytes.finished,
                 nmax = ahead < biggest_file / 8 ? 32 : 16;
 
-            return ahead < biggest_file &&
-                st.todo.handshake.length + st.busy.handshake.length < nmax;
+            return ahead < biggest_file && nhs < nmax;
         }
-        return handshakes_permitted() && 0 ==
-            st.todo.handshake.length +
-            st.busy.handshake.length +
-            st.todo.upload.length +
-            st.busy.upload.length;
+        return handshakes_permitted() && 0 == nhs + nup;
     }
 
     var tasker = (function () {
@@ -1586,8 +1648,14 @@ function up2k_init(subtle) {
             running = true;
             while (true) {
                 var now = Date.now(),
+                    blocktime = now - r.tact,
                     is_busy = st.car < st.files.length;
 
+                if (blocktime > 2500)
+                    console.log('main thread blocked for ' + blocktime);
+
+                r.tact = now;
+
                 if (was_busy && !is_busy) {
                     for (var a = 0; a < st.files.length; a++) {
                         var t = st.files[a];
@@ -1727,23 +1795,34 @@ function up2k_init(subtle) {
     })();
 
     function uptoast() {
+        if (st.busy.handshake.length)
+            return;
+
+        for (var a = 0; a < st.files.length; a++) {
+            var t = st.files[a];
+            if (t.want_recheck && !t.rechecks)
+                return;
+        }
+
         var sr = uc.fsearch,
             ok = pvis.ctr.ok,
             ng = pvis.ctr.ng,
+            spd = Math.floor(st.bytes.finished / st.time.busy),
+            suf = '\n\n{0} @ {1}/s'.format(shumantime(st.time.busy), humansize(spd)),
             t = uc.ask_up ? 0 : 10;
 
         console.log('toast', ok, ng);
 
         if (ok && ng)
-            toast.warn(t, uc.nagtxt = (sr ? L.ur_sm : L.ur_um).format(ok, ng));
+            toast.warn(t, uc.nagtxt = (sr ? L.ur_sm : L.ur_um).format(ok, ng) + suf);
         else if (ok > 1)
-            toast.ok(t, uc.nagtxt = (sr ? L.ur_aso : L.ur_auo).format(ok));
+            toast.ok(t, uc.nagtxt = (sr ? L.ur_aso : L.ur_auo).format(ok) + suf);
         else if (ok)
-            toast.ok(t, uc.nagtxt = sr ? L.ur_1so : L.ur_1uo);
+            toast.ok(t, uc.nagtxt = (sr ? L.ur_1so : L.ur_1uo) + suf);
         else if (ng > 1)
-            toast.err(t, uc.nagtxt = (sr ? L.ur_asn : L.ur_aun).format(ng));
+            toast.err(t, uc.nagtxt = (sr ? L.ur_asn : L.ur_aun).format(ng) + suf);
         else if (ng)
-            toast.err(t, uc.nagtxt = sr ? L.ur_1sn : L.ur_1un);
+            toast.err(t, uc.nagtxt = (sr ? L.ur_1sn : L.ur_1un) + suf);
 
         timer.rm(etafun);
         timer.rm(donut.do);
@@ -1854,6 +1933,7 @@ function up2k_init(subtle) {
                 cdr = Math.min(chunksize + car, t.size);
 
             st.bytes.hashed += cdr - car;
+            st.etac.h++;
 
             function orz(e) {
                 bpend--;
@@ -1979,6 +2059,8 @@ function up2k_init(subtle) {
             nbusy++;
             reading++;
             nchunk++;
+            if (Date.now() - up2k.tact > 1500)
+                tasker();
         }
 
         function onmsg(d) {
@@ -2189,13 +2271,24 @@ function up2k_init(subtle) {
 
                 t.sprs = response.sprs;
 
-                var rsp_purl = url_enc(response.purl);
-                if (rsp_purl !== t.purl || response.name !== t.name) {
-                    // server renamed us (file exists / path restrictions)
-                    console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + rsp_purl + "] [" + response.name + "]");
+                var fk = response.fk,
+                    rsp_purl = url_enc(response.purl),
+                    rename = rsp_purl !== t.purl || response.name !== t.name;
+
+                if (rename || fk) {
+                    if (rename)
+                        console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + rsp_purl + "] [" + response.name + "]");
+
                     t.purl = rsp_purl;
                     t.name = response.name;
-                    pvis.seth(t.n, 0, linksplit(t.purl + uricom_enc(t.name)).join(' '));
+
+                    var url = t.purl + uricom_enc(t.name);
+                    if (fk) {
+                        t.fk = fk;
+                        url += '?k=' + fk;
+                    }
+
+                    pvis.seth(t.n, 0, linksplit(url).join(' '));
                 }
 
                 var chunksize = get_chunksize(t.size),
@@ -2298,15 +2391,17 @@ function up2k_init(subtle) {
                 }
 
                 var err_pend = rsp.indexOf('partial upload exists at a different') + 1,
+                    err_srcb = rsp.indexOf('source file busy; please try again') + 1,
+                    err_plug = rsp.indexOf('upload blocked by x') + 1,
                     err_dupe = rsp.indexOf('upload rejected, file already exists') + 1;
 
-                if (err_pend || err_dupe) {
+                if (err_pend || err_srcb || err_plug || err_dupe) {
                     err = rsp;
                     ofs = err.indexOf('\n/');
                     if (ofs !== -1) {
                         err = err.slice(0, ofs + 1) + linksplit(err.slice(ofs + 2).trimEnd()).join(' ');
                     }
-                    if (!t.rechecks && err_pend) {
+                    if (!t.rechecks && (err_pend || err_srcb)) {
                         t.rechecks = 0;
                         t.want_recheck = true;
                     }
@@ -2343,6 +2438,8 @@ function up2k_init(subtle) {
         };
         if (t.srch)
             req.srch = 1;
+        else if (t.rand)
+            req.rand = true;
 
         xhr.open('POST', t.purl, true);
         xhr.responseType = 'text';
@@ -2357,8 +2454,17 @@ function up2k_init(subtle) {
     function can_upload_next() {
         var upt = st.todo.upload[0],
             upf = st.files[upt.nfile],
+            nhs = st.busy.handshake.length,
+            hs = nhs && st.busy.handshake[0],
             now = Date.now();
 
+        if (nhs >= 16)
+            return false;
+
+        if (hs && hs.t_uploaded && Date.now() - hs.t_busied > 10000)
+            // verification HS possibly held back by uploads
+            return false;
+
         for (var a = 0, aa = st.busy.handshake.length; a < aa; a++) {
             var hs = st.busy.handshake[a];
             if (hs.n < upt.nfile && hs.t_busied > now - 10 * 1000 && !st.files[hs.n].bytes_uploaded)
@@ -2398,11 +2504,21 @@ function up2k_init(subtle) {
 
         function orz(xhr) {
             var txt = ((xhr.response && xhr.response.err) || xhr.responseText) + '';
+            if (txt.indexOf('upload blocked by x') + 1) {
+                apop(st.busy.upload, upt);
+                apop(t.postlist, npart);
+                pvis.seth(t.n, 1, "ERROR");
+                pvis.seth(t.n, 2, txt.split(/\n/)[0]);
+                pvis.move(t.n, 'ng');
+                return;
+            }
             if (xhr.status == 200) {
                 pvis.prog(t, npart, cdr - car);
                 st.bytes.finished += cdr - car;
                 st.bytes.uploaded += cdr - car;
                 t.bytes_uploaded += cdr - car;
+                st.etac.u++;
+                st.etac.t++;
             }
             else if (txt.indexOf('already got that') + 1 ||
                 txt.indexOf('already being written') + 1) {
@@ -2530,9 +2646,15 @@ function up2k_init(subtle) {
         if (dir.target) {
             clmod(obj, 'err', 1);
             var v = Math.floor(parseInt(obj.value));
-            if (v < 0 || v > 64 || v !== v)
+            if (v < 0 || v !== v)
                 return;
 
+            if (v > 64) {
+                var p = obj.selectionStart;
+                v = obj.value = 64;
+                obj.selectionStart = obj.selectionEnd = p;
+            }
+
             parallel_uploads = v;
             swrite('nthread', v);
             clmod(obj, 'err');
@@ -2749,6 +2871,21 @@ function up2k_init(subtle) {
 
         if (en && Notification.permission == 'default')
             Notification.requestPermission().then(chknag, chknag);
+
+        set_upsfx(en);
+    }
+
+    function set_upsfx(en) {
+        if (!en)
+            return;
+
+        toast.inf(10, 'OK -- <a href="#" id="nagtest">test it!</a>')
+
+        ebi('nagtest').onclick = function () {
+            start_actx();
+            uc.nagtxt = ':^)';
+            setTimeout(donut.enstrobe, 200);
+        };
     }
 
     if (uc.upnag && (!window.Notification || Notification.permission != 'granted'))
@@ -2806,7 +2943,7 @@ ebi('ico1').onclick = function () {
 if (QS('#op_up2k.act'))
     goto_up2k();
 
-apply_perms(perms);
+apply_perms({ "perms": perms, "frand": frand });
 
 
 (function () {

copyparty/web/util.js

@@ -9,12 +9,15 @@ if (!window.console || !console.log)
 var wah = '',
     L, tt, treectl, thegrid, up2k, asmCrypto, hashwasm, vbar, marked,
     CB = '?_=' + Date.now(),
+    R = SR.slice(1),
+    RS = R ? "/" + R : "",
     HALFMAX = 8192 * 8192 * 8192 * 8192,
     HTTPS = (window.location + '').indexOf('https:') === 0,
     TOUCH = 'ontouchstart' in window,
     MOBILE = TOUCH,
     CHROME = !!window.chrome,
     VCHROME = CHROME ? 1 : 0,
+    IE = /Trident\//.test(navigator.userAgent),
     FIREFOX = ('netscape' in window) && / rv:/.test(navigator.userAgent),
     IPHONE = TOUCH && /iPhone|iPad|iPod/i.test(navigator.userAgent),
     LINUX = /Linux/.test(navigator.userAgent),
@@ -109,12 +112,13 @@ if ((document.location + '').indexOf(',rej,') + 1)
 
 try {
     console.hist = [];
+    var CMAXHIST = 100;
     var hook = function (t) {
         var orig = console[t].bind(console),
             cfun = function () {
                 console.hist.push(Date.now() + ' ' + t + ': ' + Array.from(arguments).join(', '));
-                if (console.hist.length > 100)
-                    console.hist = console.hist.slice(50);
+                if (console.hist.length > CMAXHIST)
+                    console.hist = console.hist.slice(CMAXHIST / 2);
 
                 orig.apply(console, arguments);
             };
@@ -193,8 +197,12 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
         var lsk = Object.keys(ls);
         lsk.sort();
         html.push('<p class="b">');
-        for (var a = 0; a < lsk.length; a++)
+        for (var a = 0; a < lsk.length; a++) {
+            if (ls[lsk[a]].length > 9000)
+                continue;
+
             html.push(' <b>' + esc(lsk[a]) + '</b> <code>' + esc(ls[lsk[a]]) + '</code> ');
+        }
         html.push('</p>');
     }
     catch (e) { }
@@ -325,6 +333,25 @@ if (!String.prototype.format)
         });
     };
 
+try {
+    new URL('/a/', 'https://a.com/');
+}
+catch (ex) {
+    console.log('ie11 shim URL()');
+    window.URL = function (url, base) {
+        if (url.indexOf('//') < 0)
+            url = base + '/' + url.replace(/^\/?/, '');
+        else if (url.indexOf('//') == 0)
+            url = 'https:' + url;
+
+        var x = url.split('?');
+        return {
+            "pathname": '/' + x[0].split('://')[1].replace(/[^/]+\//, ''),
+            "search": x.length > 1 ? x[1] : ''
+        };
+    }
+}
+
 // https://stackoverflow.com/a/950146
 function import_js(url, cb) {
     var head = document.head || document.getElementsByTagName('head')[0];
@@ -605,6 +632,29 @@ function vsplit(vp) {
 }
 
 
+function vjoin(p1, p2) {
+    if (!p1)
+        p1 = '';
+
+    if (!p2)
+        p2 = '';
+
+    if (p1.endsWith('/'))
+        p1 = p1.slice(0, -1);
+
+    if (p2.startsWith('/'))
+        p2 = p2.slice(1);
+
+    if (!p1)
+        return p2;
+
+    if (!p2)
+        return p1;
+
+    return p1 + '/' + p2;
+}
+
+
 function uricom_enc(txt, do_fb_enc) {
     try {
         return encodeURIComponent(txt);
@@ -686,7 +736,9 @@ function noq_href(el) {
 
 
 function get_pwd() {
-    var pwd = ('; ' + document.cookie).split('; cppwd=');
+    var k = HTTPS ? 's=' : 'd=',
+        pwd = ('; ' + document.cookie).split('; cppw' + k);
+
     if (pwd.length < 2)
         return null;
 
@@ -970,6 +1022,7 @@ function sethash(hv) {
     }
 }
 
+
 function dl_file(url) {
     console.log('DL [%s]', url);
     var o = mknod('a');
@@ -979,6 +1032,25 @@ function dl_file(url) {
 }
 
 
+function cliptxt(txt, ok) {
+    var fb = function () {
+        console.log('fb');
+        var o = mknod('input');
+        o.value = txt;
+        document.body.appendChild(o);
+        o.focus();
+        o.select();
+        document.execCommand("copy");
+        document.body.removeChild(o);
+        ok();
+    };
+    try {
+        navigator.clipboard.writeText(txt).then(ok, fb);
+    }
+    catch (ex) { fb(); }
+}
+
+
 var timer = (function () {
     var r = {};
     r.q = [];
@@ -1134,13 +1206,13 @@ var tt = (function () {
         r.th.style.top = (e.pageY + 12 * sy) + 'px';
     };
 
-    if (IPHONE) {
+    if (TOUCH) {
         var f1 = r.show,
             f2 = r.hide,
             q = [];
 
         // if an onclick-handler creates a new timer,
-        // iOS 13.1.2 delays the entire handler by up to 401ms,
+        // webkits delay the entire handler by up to 401ms,
         // win by using a shared timer instead
 
         timer.add(function () {
@@ -1252,17 +1324,17 @@ var toast = (function () {
         r.tag = tag;
     };
 
-    r.ok = function (sec, txt, tag) {
-        r.show('ok', sec, txt, tag);
+    r.ok = function (sec, txt, tag, cls) {
+        r.show('ok ' + (cls || ''), sec, txt, tag);
     };
-    r.inf = function (sec, txt, tag) {
-        r.show('inf', sec, txt, tag);
+    r.inf = function (sec, txt, tag, cls) {
+        r.show('inf ' + (cls || ''), sec, txt, tag);
     };
-    r.warn = function (sec, txt, tag) {
-        r.show('warn', sec, txt, tag);
+    r.warn = function (sec, txt, tag, cls) {
+        r.show('warn ' + (cls || ''), sec, txt, tag);
     };
-    r.err = function (sec, txt, tag) {
-        r.show('err', sec, txt, tag);
+    r.err = function (sec, txt, tag, cls) {
+        r.show('err ' + (cls || ''), sec, txt, tag);
     };
 
     return r;
@@ -1526,25 +1598,41 @@ var md_plug_err = function (ex, js) {
     if (ex)
         console.log(ex, js);
 };
-function load_md_plug(md_text, plug_type) {
+function load_md_plug(md_text, plug_type, defer) {
+    if (defer)
+        md_plug[plug_type] = null;
+
+    if (plug_type == 'pre')
+        try {
+            md_text = md_thumbs(md_text);
+        }
+        catch (ex) {
+            toast.warn(30, '' + ex);
+        }
+
     if (!have_emp)
         return md_text;
 
-    var find = '\n```copyparty_' + plug_type + '\n';
-    var ofs = md_text.indexOf(find);
-    if (ofs === -1)
+    var find = '\n```copyparty_' + plug_type + '\n',
+        md = md_text.replace(/\r/g, ''),
+        ofs = md.indexOf(find),
+        ofs2 = md.indexOf('\n```', ofs + 1);
+
+    if (ofs < 0 || ofs2 < 0)
         return md_text;
 
-    var ofs2 = md_text.indexOf('\n```', ofs + 1);
-    if (ofs2 == -1)
-        return md_text;
+    var js = md.slice(ofs + find.length, ofs2 + 1);
+    md = md.slice(0, ofs + 1) + md.slice(ofs2 + 4);
+    md = md.replace(/$/g, '\r');
 
-    var js = md_text.slice(ofs + find.length, ofs2 + 1);
-    var md = md_text.slice(0, ofs + 1) + md_text.slice(ofs2 + 4);
+    if (defer) { // insert into sandbox
+        md_plug[plug_type] = js;
+        return md;
+    }
 
     var old_plug = md_plug[plug_type];
     if (!old_plug || old_plug[1] != js) {
-        js = 'const x = { ' + js + ' }; x;';
+        js = 'const loc = new URL("' + location.href + '"), x = { ' + js + ' }; x;';
         try {
             var x = eval(js);
             if (x['ctor']) {
@@ -1562,6 +1650,47 @@ function load_md_plug(md_text, plug_type) {
 
     return md;
 }
+function md_thumbs(md) {
+    if (!/(^|\n)<!-- th -->/.exec(md))
+        return md;
+
+    // `!th[flags](some.jpg)`
+    // flags: nothing or "l" or "r"
+
+    md = md.split(/!th\[/g);
+    for (var a = 1; a < md.length; a++) {
+        if (!/^[^\]!()]*\]\([^\][!()]+\)/.exec(md[a])) {
+            md[a] = '!th[' + md[a];
+            continue;
+        }
+
+        var o1 = md[a].indexOf(']('),
+            o2 = md[a].indexOf(')', o1),
+            alt = md[a].slice(0, o1),
+            flags = alt.split(','),
+            url = md[a].slice(o1 + 2, o2),
+            float = has(flags, 'l') ? 'left' : has(flags, 'r') ? 'right' : '';
+
+        if (!/[?&]cache/.exec(url))
+            url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=i';
+
+        md[a] = '<a href="' + url + '" class="mdth mdth' + float.slice(0, 1) + '"><img src="' + url + '&th=w" alt="' + alt + '" /></a>' + md[a].slice(o2 + 1);
+    }
+    return md.join('');
+}
+function md_th_set() {
+    var els = QSA('.mdth');
+    for (var a = 0, aa = els.length; a < aa; a++)
+        els[a].onclick = md_th_click;
+}
+function md_th_click(e) {
+    ev(e);
+    var url = this.getAttribute('href').split('?')[0];
+    if (window.sb_md)
+        window.parent.postMessage("imshow " + url, "*");
+    else
+        thegrid.imshow(url);
+}
 
 
 var svg_decl = '<?xml version="1.0" encoding="UTF-8"?>\n';
@@ -1678,7 +1807,7 @@ function xhrchk(xhr, prefix, e404, lvl, tag) {
 
         qsr('#cf_frame');
         var fr = mknod('iframe', 'cf_frame');
-        fr.src = '/?cf_challenge';
+        fr.src = SR + '/?cf_challenge';
         document.body.appendChild(fr);
     }
 

copyparty/web/w.hash.js

@@ -19,7 +19,7 @@ catch (ex) {
 }
 function load_fb() {
     subtle = null;
-    importScripts('/.cpr/deps/sha512.hw.js');
+    importScripts('deps/sha512.hw.js');
 }
 
 

docs/README.md

@@ -13,15 +13,21 @@
 
 # other stuff
 
+## [`example.conf`](example.conf)
+* example config file for `-c`
+
+## [`versus.md`](versus.md)
+* similar software / alternatives (with pros/cons)
+
 ## [`changelog.md`](changelog.md)
 * occasionally grabbed from github release notes
 
+## [`devnotes.md`](devnotes.md)
+* technical stuff
+
 ## [`rclone.md`](rclone.md)
 * notes on using rclone as a fuse client/server
 
-## [`example.conf`](example.conf)
-* example config file for `-c`
-
 
 
 # junk

docs/changelog.md

@@ -1,3 +1,522 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0420-2141  `v1.6.12`  as seen on nixos
+
+## new features
+* @chinponya [made](https://github.com/9001/copyparty/pull/22) a copyparty [Nix package](https://github.com/9001/copyparty#nix-package) and a [NixOS module](https://github.com/9001/copyparty#nixos-module)! nice 🎉
+  * with [systemd-based hardening](https://github.com/9001/copyparty/blob/hovudstraum/contrib/nixos/modules/copyparty.nix#L230-L270) instead of [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh)
+  * complements the [arch package](https://github.com/9001/copyparty/tree/hovudstraum/contrib/package/arch) very well w
+
+## bugfixes
+* fix an sqlite fd leak
+  * with enough simultaneous traffic, copyparty could run out of file descriptors since it relied on the gc to close sqlite cursors
+  * now there's a pool of cursors shared between the tcp connections instead, limited to the number of CPU cores
+  * performance mostly unaffected (or slightly improved) compared to before, except for a 20% reduction only during max server load caused by directory-listings or searches
+  * ~~somehow explicitly closing the cursors didn't always work... maybe this was actually a python bug :\\/~~
+    * yes, it does incomplete cleanup if opening a WAL database fails
+* multirange requests would fail with an error; now they get a 200 as expected (since they're kinda useless and not worth the overhead)
+  * [the only software i've ever seen do that](https://apps.kde.org/discover/) now works as intended
+* expand `~/` filesystem paths in all remaining args: `-c`, `-lo`, `--hist`, `--ssl-log`, and the `hist` volflag
+* never use IPv6-format IPv4 (`::ffff:127.0.0.1`) in responses
+* [u2cli](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py): don't enter delete stage if some of the uploads failed
+* audio player in safari on touchbar macbooks
+  * songs would play backwards because the touchbar keeps spamming play/pause
+  * playback would stop when the preloader kicks in because safari sees the new audio object and freaks out
+
+## other changes
+* added [windows quickstart / service example](https://github.com/9001/copyparty/blob/hovudstraum/docs/examples/windows.md)
+* updated pyinstaller (it makes smaller exe files now)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0401-2112  `v1.6.11`  not joke
+
+## new features
+* new event-hook: [exif stripper](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/image-noexif.py)
+* [markdown thumbnails](https://a.ocv.me/pub/demo/pics-vids/README.md?v) -- see [readme](https://github.com/9001/copyparty#markdown-viewer)
+* soon: support for [web-scrobbler](https://github.com/web-scrobbler/web-scrobbler/) - the [Last.fm](https://www.last.fm/user/tripflag) browser extension
+  * will update here + readme with more info when [the v3](https://github.com/web-scrobbler/web-scrobbler/projects/5) is out
+
+## bugfixes
+* more sqlite query-planner twiddling
+  * deleting files is MUCH faster now, and uploads / bootup might be a bit better too
+* webdav optimizations / compliance
+  * should make some webdav clients run faster than before
+  * in very related news, the webdav-client in [rclone](https://github.com/rclone/rclone/) v1.63 ([currently beta](https://beta.rclone.org/?filter=latest)) will be ***FAST!***
+    * does cool stuff such as [bidirectional sync](https://github.com/9001/copyparty#folder-sync) between copyparty and a local folder
+* [bpm detector](https://github.com/9001/copyparty/blob/hovudstraum/bin/mtag/audio-bpm.py) is a bit more accurate
+* [u2cli](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) / commandline uploader: better error messages if something goes wrong
+* readme rendering could fail in firefox if certain addons were installed (not sure which)
+* event-hooks: more accurate usage examples
+
+## other changes
+* @chinponya automated the prismjs build step (thx!)
+* updated some js deps (markedjs, codemirror)
+* copyparty.exe: updated Pillow to 9.5.0
+* and finally [the joke](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/rave.js) (looks [like this](https://cd.ocv.me/b/d2/d21/#af-9b927c42))
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0320-2156  `v1.6.10`  rclone sync
+
+## new features
+* [iPhone "app"](https://github.com/9001/copyparty#ios-shortcuts) (upload shortcut) -- thanks @Daedren !
+  * can strip exif, upload files, pics, vids, links, clipboard
+  * can download links and rehost the target file on your server
+* support `rclone sync` to [sync folders](https://github.com/9001/copyparty#folder-sync) to/from copyparty
+  * let webdav clients set lastmodified times during upload
+  * let webdav clients replace files during upload
+
+## bugfixes
+* [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh): FFmpeg transcoding was slow because there was no `/dev/urandom`
+* iphones would fail to play *some* songs (low-bitrate and/or shorter than ~7 seconds)
+  * due to either an iOS bug or an FFmpeg bug in the caf remuxing idk
+  * fixed by mixing in white noise into songs if an iPhone asks for them
+* small correction in the docker readme regarding rootless podman
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0316-2106  `v1.6.9`  index.html
+
+## new features
+* option to show `index.html` instead of the folder listing
+  * arg `--ih` makes it default-enabled
+  * clients can enable/disable it in the `[⚙️]` settings tab
+  * url-param `?v` skips it for a particular folder
+* faster folder-thumbnail validation on startup (mostly on conventional HDDs) 
+
+## bugfixes
+* "load more" button didn't always show up when search results got truncated
+* ux: tooltips could block buttons on android
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0312-1610  `v1.6.8`  folder thumbs
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+## new features
+* folder thumbnails are indexed in the db
+  * now supports non-lowercase names (`Cover.jpg`, `Folder.JPG`)
+  * folders without a specific cover/folder image will show the first pic inside
+* when audio playback continues into an empty folder, keep trying for a bit
+* add no-index hints (google etc) in basic-browser HTML (`?b`, `?b=u`)
+* [commandline uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) supports long filenames on win7
+
+## bugfixes
+* rotated logfiles didn't get xz compressed
+* image-gallery links pointing to a deleted image shows an error instead of a crashpage
+
+## other changes
+* folder thumbnails have purple text to differentiate from files
+* `copyparty32.exe` starts 30% faster (but is 6% larger)
+
+----
+
+# what to download?
+| download link | is it good? | description |
+| -- | -- | -- |
+| **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** | ✅ the best 👍 | runs anywhere! only needs python |
+| [a docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker/README.md) | it's ok | good if you prefer docker 🐋 |
+| [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) |  ⚠️ [acceptable](https://github.com/9001/copyparty#copypartyexe) | for [win8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png) or later; built-in thumbnailer |
+| [up2k.exe](https://github.com/9001/copyparty/releases/latest/download/up2k.exe) | ⚠️ acceptable | [CLI uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) as a win7+ exe ([video](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm)) |
+| [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) | ⛔️ [dangerous](https://github.com/9001/copyparty#copypartyexe) | for [win7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png) -- never expose to the internet! |
+| [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.6.8/copyparty-winpe64.exe) | ⛔️ dangerous | runs on [64bit WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png), otherwise useless |
+
+* except for [up2k.exe](https://github.com/9001/copyparty/releases/latest/download/up2k.exe), all of the options above are equivalent
+* the zip and tar.gz files below are just source code
+* python packages are available at [PyPI](https://pypi.org/project/copyparty/#files)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0305-2018  `v1.6.7`  fix no-dedup + add up2k.exe
+
+## new features
+* controlpanel-connect: add example for webdav automount
+
+## bugfixes
+* fix a race which, in worst case (but unlikely on linux), **could cause data loss**
+  * could only happen if `--no-dedup` or volflag `copydupes` was set (**not** default)
+  * if two identical files were uploaded at the same time, there was a small chance that one of the files would become empty
+  * check if you were affected by doing a search for zero-byte files using either of the following:
+    * https://127.0.0.1:3923/#q=size%20%3D%200
+    * `find -type f -size 0`
+  * let me know if you lost something important and had logging enabled!
+* ftp: mkdir can do multiple levels at once (support filezilla)
+* fix flickering toast on upload finish
+* `[💤]` (upload-baton) could disengage if chrome decides to pause the background tab for 10sec (which it sometimes does)
+
+----
+
+## introducing [up2k.exe](https://github.com/9001/copyparty/releases/latest/download/up2k.exe)
+
+the commandline up2k upload / filesearch client, now as a standalone windows exe
+* based on python 3.7 so it runs on 32bit windows7 or anything newer
+* *no https support* (saves space + the python3.7 openssl is getting old)
+* built from b39ff92f34e3fca389c78109d20d5454af761f8e so it can do long filepaths and mojibake
+
+----
+
+⭐️ **you probably want [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) below;**
+the exe is [not recommended](https://github.com/9001/copyparty#copypartyexe) for longterm use
+and the zip and tar.gz files are source code
+(python packages are available at [PyPI](https://pypi.org/project/copyparty/#files))
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0226-2030  `v1.6.6`  r 2 0 0
+
+two hundred releases wow
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+* currently fighting a ground fault so the demo server will be unreliable for a while
+
+## new features
+* more docker containers! now runs on x64, x32, aarch64, armhf, ppc64, s390x
+  * pls let me know if you actually run copyparty on an IBM mainframe 👍
+* new [event hook](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks) type `xiu` runs just once for all recent uploads
+  * example hook [xiu-sha.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/xiu-sha.py) generates sha512 checksum files
+* new arg `--rsp-jtr` simulates connection jitter
+* copyparty.exe integrity selftest
+* ux:
+  * return to previous page after logging in
+  * show a warning on the login page if you're not using https
+  * freebsd: detect `fetch` and return the [colorful sortable plaintext](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png) listing
+
+## bugfixes
+* permit replacing empty files only during a `--blank-wt` grace period
+* lifetimes: keep upload-time when a size/mtime change triggers a reindex
+* during cleanup after an unlink, never rmdir the entire volume
+* rescan button in the controlpanel required volumes to be e2ds
+* dupes could get indexed with the wrong mtime
+  * only affected the search index; the filesystem got the right one
+* ux: search results could include the same hit twice in case of overlapping volumes
+* ux: upload UI would remain expanded permanently after visiting a huge tab
+* ftp: return proper error messages when client does something illegal
+* ie11: support the back button
+
+## other changes
+* [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) replaces copyparty64.exe -- now built for 64-bit windows 10
+  * **on win10 it just works** -- on win8 it needs [vc redist 2015](https://www.microsoft.com/en-us/download/details.aspx?id=48145) -- no win7 support
+  * has the latest security patches, but sfx.py is still better for long-term use
+  * has pillow and mutagen; can make thumbnails and parse/index media
+* [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) is the old win7-compatible, dangerously-insecure edition
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0212-1411  `v1.6.5`  windows smb fix + win10.exe
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+## bugfixes
+* **windows-only:** smb locations (network drives) could not be accessed
+  * appeared in [v1.6.4](https://github.com/9001/copyparty/releases/tag/v1.6.4) while adding support for long filepaths (260chars+)
+
+## other changes
+* removed tentative support for compressed chiptunes (xmgz, xmz, xmj, ...) since FFmpeg usually doesn't
+
+----
+
+# introducing [copyparty640.exe](https://github.com/9001/copyparty/releases/download/v1.6.5/copyparty640.exe)
+* built for win10, comes with the latest python and deps (supports win8 with [vc redist 2015](https://www.microsoft.com/en-us/download/details.aspx?id=48145))
+* __*much* safer__ than the old win7-compatible `copyparty.exe` and `copyparty64.exe`
+  * but only `copyparty-sfx.py` takes advantage of the operating system security patches
+* includes pillow for thumbnails and mutagen for media indexing
+* around 10% slower (trying to figure out what's up with that)
+
+starting from the next release,
+* `copyparty.exe` (win7 x32) will become `copyparty32.exe`
+* `copyparty640.exe` (win10) will be the new `copyparty.exe`
+* `copyparty64.exe` (win7 x64) will graduate
+
+so the [copyparty64.exe](https://github.com/9001/copyparty/releases/download/v1.6.5/copyparty64.exe) in this release will be the "final" version able to run inside a [64bit Win7-era winPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png) (all regular 32/64-bit win7 editions can just use `copyparty32.exe` instead)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0211-1802  `v1.6.4`  🔧🎲🔗🐳🇦🎶
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [1.6 theme song](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c) // [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md)
+
+## new features
+* 🔧 new [config syntax](https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf) (#20)
+  * the new syntax is still kinda esoteric and funky but it's an improvement
+  * old config files are still supported
+    * `--vc` prints the autoconverted config which you can copy back into the config file to upgrade
+  * `--vc` will also [annotate and explain](https://user-images.githubusercontent.com/241032/217356028-eb3e141f-80a6-4bc6-8d04-d8d1d874c3e9.png) the config files
+  * new argument `--cgen` to generate config from commandline arguments
+    * kinda buggy, especially the `[global]` section, so give it a lookover before saving it
+* 🎲 randomize filenames on upload
+  * either optionally, using the 🎲 button in the up2k ui
+  * or force-enabled; globally with `--rand` or per-volume with volflag `rand`
+  * specify filename length with `nrand` (globally or volflag), default 9
+* 🔗 export a list of links to your recent uploads
+  * `copy links` in the up2k tab (🚀) will copy links to all uploads since last page refresh,
+  * `copy` in the unpost tab (🧯) will copy links to all your recent uploads (max 2000 files / 12 hours by default)
+  * filekeys are included if that's enabled and you have access to view those (permissions `G` or `r`)
+* 🇦 [arch package](https://github.com/9001/copyparty/tree/hovudstraum/contrib/package/arch) -- added in #18, thx @icxes 
+  * maybe in aur soon!
+* 🐳 [docker containers](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) -- 5 editions,
+  * [min](https://hub.docker.com/r/copyparty/min) (57 MiB), just copyparty without thumbnails or audio transcoding
+  * [im](https://hub.docker.com/r/copyparty/im) (70 MiB), thumbnails of popular image formats + media tags with mutagen
+  * [ac (163 MiB)](https://hub.docker.com/r/copyparty/ac) 🥇 adds audio/video thumbnails + audio transcoding + better tags
+  * [iv](https://hub.docker.com/r/copyparty/iv) (211 MiB), makes heif/avic/jxl faster to thumbnail
+  * [dj](https://hub.docker.com/r/copyparty/dj) (309 MiB), adds optional detection of musical key / bpm
+* 🎶 [chiptune player](https://a.ocv.me/pub/demo/music/chiptunes/#af-f6fb2e5f)
+  * transcodes mod/xm/s3m/it/mo3/mptm/mt2/okt to opus
+  * uses FFmpeg (libopenmpt) so the accuracy is not perfect, but most files play OK enough
+  * not **yet** supported in the docker container since Alpine's FFmpeg was built without libopenmpt
+* windows: support long filepaths (over 260 chars)
+  * uses the `//?/` winapi syntax to also support windows 7
+* `--ver` shows the server version on the control panel
+
+## bugfixes
+* markdown files didn't scale properly in the document browser
+* detect and refuse multiple volume definitions sharing the same filesystem path
+* don't return incomplete transcodes if multiple clients try to play the same flac file
+* [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh): more reliable chroot cleanup, sigusr1 for config reload
+* pypi packaging: compress web resources, include webdav.bat
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0131-2103  `v1.6.3`  sandbox k
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* and since [1.6.0](https://github.com/9001/copyparty/releases/tag/v1.6.2) only got 2 days of prime time,
+  * [1.6 theme song](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c) (hosted on the demo server)
+  * [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) / feature comparison
+
+## new features
+* dotfiles are hidden from search results by default
+  * use `--dotsrch` or volflags `dotsrch` / `nodotsrch` to specify otherwise
+  * they were already being excluded from tar/zip-files if `-ed` is not set, so this makes more sense -- dotfiles *should* now be undiscoverable unless `-ed` or `--smb` is set, but please use [volumes](https://github.com/9001/copyparty#accounts-and-volumes) for isolation / access-control instead, much safer
+
+## bugfixes
+* lots of cosmetic fixes for the new readme/prologue/epilogue sandbox
+  * rushed it into the previous release when someone suggested it, bad idea
+  * still flickers a bit (especially prologues), and hotkeys are blocked while the sandboxed document has focus
+  * can be disabled with `--no-sb-md --no-sb-lg` (not recommended)
+* support webdav uploads from davfs2 (fix LOCK response)
+* always unlink files before overwriting them, in case they are hardlinks
+  * was primarily an issue with `--daw` and webdav clients
+* on windows, replace characters in PUT filenames as necessary
+* [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh): support opus transcoding on debian
+  * `rm -rf .hist/ac` to clear the transcode cache if the old version broke some songs
+
+## other changes
+* add `rel="nofollow"` to zip download links, basic-browser link
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0129-1842  `v1.6.2`  cors k
+
+[Ellie Goulding - Stay Awake (kors k Hardcore Bootleg).mp3](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c)
+* 👆 the read-only demo server at https://a.ocv.me/pub/demo/
+
+## breaking changes
+but nothing is affected (that i know of):
+* all requests must pass [cors validation](https://github.com/9001/copyparty#cors)
+  * but they almost definitely did already
+  * sharex and others are OK since they don't supply an `Origin` header
+* [API calls](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#http-api) `?delete` and `?move` are now POST instead of GET
+  * not aware of any clients using these
+
+## known issues
+* the document sandbox is a bit laggy and sometimes eats hotkeys
+  * disable it with `--no-sb-md --no-sb-lg` if you trust everyone who has write and/or move access
+
+## new features
+* [event hooks](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks) -- run programs on new [uploads](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png), renames, deletes
+* [configurable cors](https://github.com/9001/copyparty#cors) (cross-origin resource sharing) behavior; defaults are mostly same as before
+  * `--allow-csrf` disables all csrf protections and makes it intentionally trivial to send authenticated requests from other domains
+* sandboxed readme.md / prologues / epilogues
+  * documents can still run scripts like before, but can no longer tamper with the web-ui / read the login session, so the old advice of `--no-readme` and `--no-logues` is mostly deprecated
+  * unfortunately disables hotkeys while the text has focus + blocks dragdropping files onto that area, oh well
+* password can be provided through http header `PW:` (instead of cookie `cppwd` or or url-param `?pw`)
+* detect network changes (new NICs, IPs) and reconfigure / reannoucne zeroconf
+  * fixes mdns when running as a systemd service and copyparty is started before networking is up
+* add `--freebind` to start listening on IPs before the NIC is up yet (linux-only)
+* per-volume deduplication-control with volflags `hardlink`, `neversymlink`, `copydupes`
+* detect curl and return a [colorful, sortable plaintext](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png) directory listing instead
+* add optional [powered-by-copyparty](https://user-images.githubusercontent.com/241032/215322626-11d1f02b-25f4-45df-a3d9-f8c51354a8eb.png) footnode on the controlpanel
+  * can be disabled with `-nb` or redirected with `--pb-url`
+
+## bugfixes
+* change some API calls (`?delete`, `?move`) from `GET` to `POST`
+  * don't panic! this was safe against authenticated csrf thanks to [SameSite=Lax](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite#lax)
+  * `--getmod` restores the GETs if you need the convenience and accept the risks
+* [u2cli](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) (command-line uploader):
+  * recover from network hiccups
+  * add `-ns` for slow uefi TTYs
+* separate login cookies for http / https
+  * avoids an https login from getting accidentally sent over plaintext
+  * sadly no longer possible to login with internet explorer 4.0 / windows 3.11
+* tar/zip-download of hidden folders
+* unpost filtering was buggy for non-ascii characters
+* moving a deduplicated file on a volume where deduplication was since disabled
+* improved the [linux 6.0.16](https://utcc.utoronto.ca/~cks/space/blog/linux/KernelBindBugIn6016) kernel bug [workaround](https://github.com/9001/copyparty/commit/9065226c3d634a9fc15b14a768116158bc1761ad) because there is similar funk in 5.x
+* add custom text selection colors because chrome is currently broken on fedora
+* blockdevs (`/dev/nvme0n1`) couldn't be downloaded as files
+* misc fixes for location-based reverse-proxying
+* macos dualstack thing
+
+## other changes
+* added a collection of [cursed usecases](https://github.com/9001/copyparty/tree/hovudstraum/docs/cursed-usecases)
+* and [comparisons to similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) in case you ever wanna jump ship
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0112-0515  `v1.5.6`  many hands
+
+hello from warsaw airport (goodbye japan ;_;)
+* read-only demo server at https://a.ocv.me/pub/demo/
+
+## new features
+* multiple upload handshakes in parallel
+  * around **5x faster** when uploading small files
+  * or **50x faster** if the server is on the other side of the planet
+    * just crank up the `parallel uploads` like crazy (max is 64)
+* upload ui: total time and average speed is shown on completion
+
+## bugfixes
+* browser ui didn't allow specifying number of threads for file search
+* dont panic if a digit key is pressed while viewing an image
+* workaround [linux kernel bug](https://utcc.utoronto.ca/~cks/space/blog/linux/KernelBindBugIn6016) causing log spam on dualstack
+  * ~~related issue (also mostly harmless) will be fixed next relese 010770684db95bece206943768621f2c7c27bace~~ 
+    * they fixed it in linux 6.1 so these workarounds will be gone too
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2022-1230-0754  `v1.5.5`  made in japan
+
+hello from tokyo
+* read-only demo server at https://a.ocv.me/pub/demo/
+
+## new features
+* image viewer now supports heif, avif, apng, svg
+* [partyfuse and up2k.py](https://github.com/9001/copyparty/tree/hovudstraum/bin): option to read password from textfile
+
+## bugfixes
+* thumbnailing could fail if a primitive build of libvips is installed
+* ssdp was wonky on dualstack ipv6
+* mdns could crash on networks with invalid routes
+* support fat32 timestamp precisions
+  * fixes spurious file reindexing in volumes located on SD cards on android tablets which lie about timestamps until the next device reboot or filesystem remount
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2022-1213-1956  `v1.5.3`  folder-sync + turbo-rust
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+
+## new features
+* one-way folder sync (client to server) using [up2k.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/README.md#up2kpy) `-z --dr`
+  * great rsync alternative when combined with `-e2ds --hardlink` deduplication on the server
+* **50x faster** when uploading small files to HDD, especially SMR
+  * by switching sqlite to WAL which carries a small chance of temporarily forgetting the ~200 most recent uploads if you have a power outage or your OS crashes; see `--help-dbd` if you have `-mtp` plugins which produces metadata you can't afford to lose
+* location-based [reverse-proxying](https://github.com/9001/copyparty/#reverse-proxy) (but it's still recommended to use a dedicated domain/subdomain instead)
+* IPv6 link-local automatically enabled for TCP and zeroconf on NICs without a routable IPv6
+* zeroconf network filters now accept subnets too, for example `--z-on 192.168.0.0/16`
+* `.hist` folders are hidden on windows
+* ux:
+  * more accurate total ETA on upload
+  * sorting of batch-unpost links was unintuitive / dangerous
+  * hotkey `Y` turns files into download links if nothing's selected
+  * option to replace or disable the mediaplayer-toggle mouse cursor with `--mpmc`
+
+## bugfixes
+* WAL probably/hopefully fixes #10 (we'll know in 6 months roughly)
+* repair db inconsistencies (which can happen if terminated during startup)
+* [davfs2](https://wiki.archlinux.org/title/Davfs2) did not approve of the authentication prompt
+* the `connect` button on the control-panel didn't work on phones
+* couldn't specify windows NICs in arguments `--z-on` / `--z-off` and friends
+* ssdp xml escaping for `--zsl` URL
+* no longer possible to accidentally launch multiple copyparty instances on the same port on windows
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2022-1203-2048  `v1.5.1`  babel
+
+named after [that other thing](https://en.wikipedia.org/wiki/Tower_of_Babel), not [the song](https://soundcloud.com/kanaze/babel-dimension-0-remix)
+* read-only demo server at https://a.ocv.me/pub/demo/
+
+## new features
+* new protocols!
+  * native IPv6 support, no longer requiring a reverse-proxy for that
+  * [webdav server](https://github.com/9001/copyparty#webdav-server) -- read/write-access to copyparty straight from windows explorer, macos finder, kde/gnome
+  * [smb/cifs server](https://github.com/9001/copyparty#smb-server) -- extremely buggy and unsafe, for when there is no other choice
+  * [zeroconf](https://github.com/9001/copyparty#zeroconf) -- copyparty announces itself on the LAN, showing up in various file managers
+    * [mdns](https://github.com/9001/copyparty#mdns) -- macos/kde/gnome + makes copyparty available at http://hostname.local/
+    * [ssdp](https://github.com/9001/copyparty#ssdp) -- windows
+  * commands to mount copyparty as a local disk are in the web-UI at control-panel --> `connect`
+* detect buggy / malicious clients spamming the server with idle connections
+  * first tries to be nice with `Connection: close` (enough to fix windows-webdav)
+  * eventually bans the IP for `--loris` minutes (default: 1 hour)
+* new arg `--xlink` for cross-volume detection of duplicate files on upload
+* new arg `--no-snap` to disable upload tracking on restart
+  * will not create `.hist` folders unless required for thumbnails or markdown backups
+* [config includes](https://github.com/9001/copyparty/blob/hovudstraum/docs/example2.conf) -- split your config across multiple config files
+* ux improvements
+  * hotkey `?` shows a summary of all the hotkeys
+  * hotkey `Y` to download selected files
+  * position indicator when hovering over the audio scrubber
+  * textlabel on the volume slider
+  * placeholder values in textboxes
+  * options to hide scrollbars, compact media player, follow playing song
+  * phone-specific
+    * buttons for prev/next folder
+    * much better ui for hiding folder columns
+
+## bugfixes
+* now possible to upload files larger than 697 GiB
+  * technically a [breaking change](https://github.com/9001/copyparty#breaking-changes) if you wrote your own up2k client
+    * please let me know if you did because that's awesome
+* several macos issues due to hardcoded syscall numbers
+* sfx: fix python 3.12 support (forbids nullbytes in source code)
+* use ctypes to discover network config -- fixes grapheneos, non-english windows
+* detect firefox showing stale markdown documents in the editor
+* detect+ban password bruteforcing on ftp too
+* http 206 failing on empty files
+* incorrect header timestamps on non-english locales
+* remind ftp clients that you cannot cd into an image file -- fixes kde dolphin
+* ux fixes
+  * uploader survives running into inaccessible folders
+  * middleclick documents in the textviewer sidebar to open in a new tab
+  * playing really long audio files (1 week or more) would spinlock the browser
+
+## other changes
+* autodetect max number of clients based on OS limits
+  * `-nc` is probably no longer necessary when running behind a reverse-proxy
+* allow/try playing mkv files in chrome
+* markdown documents returned as plaintext unless `?v`
+* only compress `-lo` logfiles if filename ends with `.xz`
+* changed sfx compression from bz2 to gz
+  * startup is slightly faster
+  * better compatibility with embedded linux
+* copyparty64.exe -- 64bit edition for [running inside WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png)
+  * which was an actual feature request, believe it or not!
+* more attempts at avoiding the [firefox fd leak](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500)
+  * if you are uploading many small files and the browser keeps crashing, use chrome instead
+    * or the commandline client, which is now available for download straight from copyparty
+      * control-panel --> `connect` --> `up2k.py`
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2022-1013-1937  `v1.4.6`  wav2opus
 

docs/copyparty.d/foo/another.conf

@@ -1,5 +1,6 @@
 # this file gets included twice from ../some.conf,
 # setting user permissions for a volume
-rw usr1
-r usr2
-% sibling.conf
+accs:
+  rw: usr1
+  r: usr2
+  % sibling.conf

docs/copyparty.d/foo/sibling.conf

@@ -1,3 +1,3 @@
 # and this config file gets included from ./another.conf,
 # adding a final permission for each of the two volumes in ../some.conf
-m usr1 usr2
+m: usr1, usr2

docs/copyparty.d/some.conf

@@ -1,22 +1,29 @@
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
 # lets make two volumes with the same accounts/permissions for both;
 # first declare the accounts just once:
-u usr1:passw0rd
-u usr2:letmein
+[accounts]
+  usr1: passw0rd
+  usr2: letmein
 
-# and listen on 127.0.0.1 only, port 2434
--i 127.0.0.1
--p 2434
+[global]
+  i: 127.0.0.1  # listen on 127.0.0.1 only,
+  p: 2434       # port 2434
+  e2ds  # enable file indexing+scanning
+  e2ts  # and multimedia indexing+scanning
+  # (inline comments are OK if there is 2 spaces before the #)
 
 # share /usr/share/games from the server filesystem
-/usr/share/games
-/vidya
-# include config file with volume permissions
-% foo/another.conf
+[/vidya]
+  /usr/share/games
+  % foo/another.conf  # include config file with volume permissions
 
 # and share your ~/Music folder too
-~/Music
-/bangers
-% foo/another.conf
+[/bangers]
+  ~/Music
+  % foo/another.conf
 
 # which should result in each of the volumes getting the following permissions:
 # usr1 read/write/move

docs/cursed-usecases/README.md

@@ -0,0 +1,22 @@
+insane ways to use copyparty
+
+
+## wireless keyboard
+
+problem: you wanna control mpv or whatever software from the couch but you don't have a wireless keyboard
+
+"solution": load some custom javascript which renders a virtual keyboard on the upload UI and each keystroke is actually an upload which gets picked up by a dummy metadata parser which forwards the keystrokes into xdotool
+
+[no joke, this actually exists and it wasn't even my idea or handiwork (thx steen)](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/meadup.js)
+
+
+## appxsvc tarpit
+
+problem: `svchost.exe` is using 100% of a cpu core, and upon further inspection (`procmon`) it is `wsappx` desperately trying to install something, repeatedly reading a file named `AppxManifest.xml` and messing with an sqlite3 database
+
+"solution": create a virtual filesystem which is intentionally slow and trick windows into reading it from there instead
+
+* create a file called `AppxManifest.xml` and put something dumb in it
+* serve the file from a copyparty instance with `--rsp-slp=1` so every request will hang for 1 sec
+* `net use m: http://127.0.0.1:3993/`  (mount copyparty using the windows-native webdav client)
+* `mklink /d c:\windows\systemapps\microsoftwindows.client.cbs_cw5n1h2txyewy\AppxManifest.xml m:\AppxManifest.xml`

docs/devnotes.md

@@ -3,7 +3,14 @@
 * top
 * [future plans](#future-plans) - some improvement ideas
 * [design](#design)
+    * [up2k](#up2k) - quick outline of the up2k protocol
+    * [why not tus](#why-not-tus) - I didn't know about [tus](https://tus.io/)
     * [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
+* [http api](#http-api)
+    * [read](#read)
+    * [write](#write)
+    * [admin](#admin)
+    * [general](#general)
 * [assumptions](#assumptions)
     * [mdns](#mdns)
 * [sfx repack](#sfx-repack) - reduce the size of an sfx by removing features
@@ -19,13 +26,15 @@
 
 some improvement ideas
 
-* the JS is a mess -- a preact rewrite would be nice
+* the JS is a mess -- a ~~preact~~ rewrite would be nice
   * preferably without build dependencies like webpack/babel/node.js, maybe a python thing to assemble js files into main.js
   * good excuse to look at using virtual lists (browsers start to struggle when folders contain over 5000 files)
+  * maybe preact / vdom isn't the best choice, could just wait for the Next Big Thing
 * the UX is a mess -- a proper design would be nice
   * very organic (much like the python/js), everything was an afterthought
   * true for both the layout and the visual flair
   * something like the tron board-room ui (or most other hollywood ones, like ironman) would be :100:
+    * would preferably keep the information density, just more organized yet [not too boring](https://blog.rachelbinx.com/2023/02/unbearable-sameness/)
 * some of the python files are way too big
   * `up2k.py` ended up doing all the file indexing / db management
   * `httpcli.py` should be separated into modules in general
@@ -35,7 +44,7 @@ some improvement ideas
 
 ## up2k
 
-quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
+quick outline of the up2k protocol,  see [uploading](https://github.com/9001/copyparty#uploading) for the web-client
 * the up2k client splits a file into an "optimal" number of chunks
   * 1 MiB each, unless that becomes more than 256 chunks
   * tries 1.5M, 2M, 3, 4, 6, ... until <= 256 chunks or size >= 32M
@@ -58,6 +67,13 @@ regarding the frequent server log message during uploads;
 * on this http connection, `2.77 GiB` transferred, `102.9 MiB/s` average, `948` chunks handled
 * client says `4` uploads OK, `0` failed, `3` busy, `1` queued, `10042 MiB` total size, `7198 MiB` and `00:01:09` left
 
+## why not tus
+
+I didn't know about [tus](https://tus.io/)  when I made this, but:
+* up2k has the advantage that it supports parallel uploading of non-contiguous chunks straight into the final file -- [tus does a merge at the end](https://tus.io/protocols/resumable-upload.html#concatenation) which is slow and taxing on the server HDD / filesystem (unless i'm misunderstanding)
+* up2k has the slight disadvantage of requiring the client to hash the entire file before an upload can begin, but this has the benefit of immediately skipping duplicate files
+  * and the hashing happens in a separate thread anyways so it's usually not a bottleneck
+
 ## why chunk-hashes
 
 a single sha512 would be better, right?
@@ -121,7 +137,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 
 | method | params | result |
 |--|--|--|
-| GET | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
+| POST | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
 
 | method | params | body | result |
 |--|--|--|--|
@@ -131,7 +147,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | mPOST | | `act=bput`, `f=FILE` | upload `FILE` into the folder at URL |
 | mPOST | `?j` | `act=bput`, `f=FILE` | ...and reply with json |
 | mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
-| GET | `?delete` | | delete URL recursively |
+| POST | `?delete` | | delete URL recursively |
 | jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
 | uPOST | | `msg=foo` | send message `foo` into server log |
 | mPOST | | `act=tput`, `body=TEXT` | overwrite markdown document at URL |
@@ -223,7 +239,12 @@ rm -rf copyparty/web/deps
 curl -L https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py >x.py
 python3 x.py --version
 rm x.py
-mv /tmp/pe-copyparty/copyparty/web/deps/ copyparty/web/deps/
+cp -R /tmp/pe-copyparty.$(id -u)/copyparty/web/deps copyparty/web/
+```
+
+or you could build the web-dependencies from source instead (NB: does not include prismjs, need to grab that manually):
+```sh
+make -C scripts/deps-docker
 ```
 
 then build the sfx using any of the following examples:

docs/example.conf

@@ -1,59 +1,69 @@
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
 # append some arguments to the commandline;
-# the first space in a line counts as a separator,
-# any additional spaces are part of the value
--e2dsa
--e2ts
--i 127.0.0.1
+# accepts anything listed in --help (leading dashes are optional)
+# and inline comments are OK if there is 2 spaces before the '#'
+[global]
+  p: 8086, 3939  # listen on ports 8086 and 3939
+  e2dsa  # enable file indexing and filesystem scanning
+  e2ts   # and enable multimedia indexing
+  z, qr  # and zeroconf and qrcode (you can comma-separate arguments)
 
 # create users:
-# u username:password
-u ed:123
-u k:k
+[accounts]
+  ed: 123   # username: password
+  k: k
 
-# leave a blank line between volumes
-# (and also between users and volumes)
+# create volumes:
+[/]         # create a volume at "/" (the webroot), which will
+  .         # share the contents of "." (the current directory)
+  accs:
+    r: *    # everyone gets read-access, but
+    rw: ed  # the user "ed" gets read-write
 
-# create a volume:
-# share "." (the current directory)
-# as "/" (the webroot) for the following users:
-# "r" grants read-access for anyone
-# "rw ed" grants read-write to ed
-.
-/
-r
-rw ed
-
-# custom permissions for the "priv" folder:
-# user "k" can only see/read the contents
-# user "ed" gets read-write access
-./priv
-/priv
-r k
-rw ed
-
-# this does the same thing,
-# and will cause an error on startup since /priv is already taken:
-./priv
-/priv
-r ed k
-w ed
+# let's specify different permissions for the "priv" subfolder
+# by creating another volume at that location:
+[/priv]
+  ./priv
+  accs:
+    r: k    # the user "k" can see the contents,
+    rw: ed  # while "ed" gets read-write
 
 # share /home/ed/Music/ as /music and let anyone read it
 # (this will replace any folder called "music" in the webroot)
-/home/ed/Music
-/music
-r
+[/music]
+  /home/ed/Music
+  accs:
+    r: *
+
+# and a folder where anyone can upload, but nobody can see the contents
+[/dump]
+  /home/ed/inc
+  accs:
+    w: *
+  flags:
+    e2d     # the e2d volflag enables the uploads database
+    nodupe  # the nodupe volflag rejects duplicate uploads
+    # (see --help-flags for all available volflags to use)
 
 # and a folder where anyone can upload
-# but nobody can see the contents
-# and set the e2d flag to enable the uploads database
-# and set the nodupe flag to reject duplicate uploads
-/home/ed/inc
-/dump
-w
-c e2d
-c nodupe
+# and anyone can access their own uploads, but nothing else
+[/sharex]
+  /home/ed/inc/sharex
+  accs:
+    wG: *        # wG = write-upget = see your own uploads only
+    rwmd: ed, k  # read-write-modify-delete for users "ed" and "k"
+  flags:
+    e2d, d2t, fk: 4
+    # volflag "e2d" enables the uploads database,
+    # "d2t" disables multimedia parsers (in case the uploads are malicious),
+    # "dthumb" disables thumbnails (same reason),
+    # "fk" enables filekeys (necessary for upget permission) (4 chars long)
+    # -- note that its fine to combine all the volflags on
+    #    one line because only the last volflag has an argument
 
 # this entire config file can be replaced with these arguments:
-# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d,nodupe
+# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d,nodupe -v /home/ed/inc/sharex:sharex:wG:c,e2d,d2t,fk=4
 # but note that the config file always wins in case of conflicts

docs/examples/README.md

@@ -0,0 +1,4 @@
+copyparty server config examples
+
+[windows.md](windows.md) -- running copyparty as a service on windows
+

docs/examples/windows.md

@@ -0,0 +1,115 @@
+# running copyparty on windows
+
+this is a complete example / quickstart for running copyparty on windows, optionally as a service (autostart on boot)
+
+you will definitely need either [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (comfy, portable, more features) or [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) (smaller, safer)
+
+* if you decided to grab `copyparty-sfx.py` instead of the exe you will also need to install the ["Latest Python 3 Release"](https://www.python.org/downloads/windows/)
+
+then you probably want to download [FFmpeg](https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip) and put `ffmpeg.exe` and `ffprobe.exe` in your PATH (so for example `C:\Windows\System32\`) -- this enables thumbnails, audio transcoding, and making music metadata searchable
+
+
+## the config file
+
+open up notepad and save the following as `c:\users\you\documents\party.conf` (for example)
+
+```yaml
+[global]
+  lo: ~/logs/cpp-%Y-%m%d.xz  # log to c:\users\you\logs\
+  e2dsa, e2ts, no-dedup, z   # sets 4 flags; see expl.
+  p: 80, 443  # listen on ports 80 and 443, not 3923
+  theme: 2    # default theme: protonmail-monokai
+  lang: nor   # default language: viking
+
+[accounts]                  # usernames and passwords
+  kevin: shangalabangala    # kevin's password
+
+[/]               # create a volume available at /
+  c:\pub          # sharing this filesystem location
+  accs:           # and set permissions:
+    r: *          # everyone can read/download files,
+    rwmd: kevin   # kevin can read/write/move/delete
+
+[/inc]            # create another volume at /inc
+  c:\pub\inc      # sharing this filesystem location
+  accs:           # permissions:
+    w: *          # everyone can upload, but not browse
+    rwmd: kevin   # kevin is admin here too
+
+[/music]          # and a third volume at /music
+  ~/music         # which shares c:\users\you\music
+  accs:
+    r: *
+    rwmd: kevin
+```
+
+
+### config explained: [global]
+
+the `[global]` section accepts any config parameters you can see when running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
+* `lo: ~/logs/cpp-%Y-%m%d.xz` writes compressed logs (the compression will make them delayed)
+* `e2dsa` enables the upload deduplicator and file indexer, which enables searching
+* `e2ts` enables music metadata indexing, making albums / titles etc. searchable too
+* `no-dedup` writes full dupes to disk instead of symlinking, since lots of windows software doesn't handle symlinks well
+  * but the improved upload speed from `e2dsa` is not affected
+* `z` enables zeroconf, making the server available at `http://HOSTNAME.local/` from any other machine in the LAN
+* `p: 80,443` listens on the ports `80` and `443` instead of the default `3923`
+* `lang: nor` sets default language to viking
+
+
+### config explained: [accounts]
+
+the `[accounts]` section defines all the user accounts, which can then be referenced when granting people access to the different volumes
+
+
+### config explained: volumes
+
+then we create three volumes, one at `/`, one at `/inc`, and one at `/music`
+* `/` and `/music` are readable without requiring people to login (`r: *`) but you need to login as kevin to write/move/delete files (`rwmd: kevin`)
+* anyone can upload to `/inc` but you must be logged in as kevin to see the files inside
+
+
+## run copyparty
+
+to test your config it's best to just run copyparty in a console to watch the output:
+
+```batch
+copyparty.exe -c party.conf
+```
+
+or if you wanna use `copyparty-sfx.py` instead of the exe (understandable),
+
+```batch
+%localappdata%\programs\python\python311\python.exe copyparty-sfx.py -c party.conf
+```
+
+(please adjust `python311` to match the python version you installed, i'm not good enough at windows to make that bit generic)
+
+
+## run it as a service
+
+to run this as a service you need [NSSM](https://nssm.cc/ci/nssm-2.24-101-g897c7ad.zip), so put the exe somewhere in your PATH
+
+then either do this for `copyparty.exe`:
+```batch
+nssm install cpp %homedrive%%homepath%\downloads\copyparty.exe -c %homedrive%%homepath%\documents\party.conf
+```
+
+or do this for `copyparty-sfx.py`:
+```batch
+nssm install cpp %localappdata%\programs\python\python311\python.exe %homedrive%%homepath%\downloads\copyparty-sfx.py -c %homedrive%%homepath%\documents\party.conf
+```
+
+then after creating the service, modify it so it runs with your own windows account (so file permissions don't get wonky and paths expand as expected):
+```batch
+nssm set cpp ObjectName .\yourAccoutName yourWindowsPassword
+nssm start cpp
+```
+
+and that's it, all good
+
+if it doesn't start, enable stderr logging so you can see what went wrong:
+```batch
+nssm set cpp AppStderr %homedrive%%homepath%\logs\cppsvc.err
+nssm set cpp AppStderrCreationDisposition 2
+```

docs/protocol-reference.sh

@@ -0,0 +1,2 @@
+vsftpd a.conf -olisten=YES -olisten_port=3921 -orun_as_launching_user=YES -obackground=NO -olog_ftp_protocol=YES
+