v1.16.21

show warning on ctrl-a in lazyloaded folders
docker: hide healthcheck from logs
2025-04-20 18:36:32 +00:00 · 2025-04-20 13:33:01 +00:00 · 2025-04-20 12:26:56 +00:00 · 2025-04-20 11:45:48 +00:00 · 2025-04-17 18:52:47 +00:00 · 2025-04-17 18:45:50 +00:00
72 changed files with 2426 additions and 530 deletions
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -8,6 +8,7 @@ assignees: '9001'
 ---

 NOTE:
+**please use english, or include an english translation.** aside from that,  
 all of the below are optional, consider them as inspiration, delete and rewrite at will, thx md


--- a/.github/ISSUE_TEMPLATE/feature_request.md
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -7,6 +7,8 @@ assignees: '9001'

 ---

+NOTE:
+**please use english, or include an english translation.** aside from that,  
 all of the below are optional, consider them as inspiration, delete and rewrite at will

 **is your feature request related to a problem? Please describe.**
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1,8 +1,21 @@
-* do something cool
+* **found a bug?** [create an issue!](https://github.com/9001/copyparty/issues) or let me know in the [discord](https://discord.gg/25J8CdTT6G) :>
+* **fixed a bug?** create a PR or post a patch! big thx in advance :>
+* **have a cool idea?** let's discuss it! anywhere's fine, you choose.

-really tho, send a PR or an issue or whatever, all appreciated, anything goes, just behave aight 👍👍
+but please:
+
+
+
+# do not use AI / LMM when writing code
+
+copyparty is 100% organic, free-range, human-written software!
+
+> ⚠ you are now entering a no-copilot zone
+
+the *only* place where LMM/AI *may* be accepted is for [localization](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#translations) if you are fluent and have confirmed that the translation is accurate.
+
+sorry for the harsh tone, but this is important to me 🙏

-but to be more specific,


 # contribution ideas
--- a/README.md
+++ b/README.md
@@ -94,10 +94,13 @@ turn almost any device into a file server with resumable uploads/downloads using
    * [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
        * [real-ip](#real-ip) - teaching copyparty how to see client IPs
        * [reverse-proxy performance](#reverse-proxy-performance)
+    * [permanent cloudflare tunnel](#permanent-cloudflare-tunnel) - if you have a domain and want to get your copyparty online real quick
    * [prometheus](#prometheus) - metrics/stats can be enabled
    * [other extremely specific features](#other-extremely-specific-features) - you'll never find a use for these
        * [custom mimetypes](#custom-mimetypes) - change the association of a file extension
+        * [GDPR compliance](#GDPR-compliance) - imagine using copyparty professionally...
        * [feature chickenbits](#feature-chickenbits) - buggy feature? rip it out
+        * [feature beefybits](#feature-beefybits) - force-enable features with known issues on your OS/env
 * [packages](#packages) - the party might be closer than you think
    * [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
    * [fedora package](#fedora-package) - does not exist yet
@@ -144,6 +147,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
 * or if you are on android, [install copyparty in termux](#install-on-android)
 * or maybe you have a [synology nas / dsm](./docs/synology-dsm.md)
 * or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
+* or if your OS is dead, give the [bootable flashdrive / cd-rom](https://a.ocv.me/pub/stuff/edcd001/enterprise-edition/) a spin
 * or if you don't trust copyparty yet and want to isolate it a little, then...
  * ...maybe [prisonparty](./bin/prisonparty.sh) to create a tiny [chroot](https://wiki.archlinux.org/title/Chroot) (very portable),
  * ...or [bubbleparty](./bin/bubbleparty.sh) to wrap it in [bubblewrap](https://github.com/containers/bubblewrap) (much better)
@@ -159,8 +163,8 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
 * **MacOS:** `port install py-Pillow ffmpeg`
 * **MacOS** (alternative): `brew install pillow ffmpeg`
 * **Windows:** `python -m pip install --user -U Pillow`
-  * install python and ffmpeg manually; do not use `winget` or `Microsoft Store` (it breaks $PATH)
-  * copyparty.exe comes with `Pillow` and only needs `ffmpeg`
+  * install [python](https://www.python.org/downloads/windows/) and [ffmpeg](#optional-dependencies) manually; do not use `winget` or `Microsoft Store` (it breaks $PATH)
+  * copyparty.exe comes with `Pillow` and only needs [ffmpeg](#optional-dependencies) for mediatags/videothumbs
 * see [optional dependencies](#optional-dependencies) to enable even more features

 running copyparty without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; you may want [accounts and volumes](#accounts-and-volumes)
@@ -183,6 +187,8 @@ first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/co

 as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you

+but if you have a domain, then you probably want to skip the random autogenerated URL and instead make a [permanent cloudflare tunnel](#permanent-cloudflare-tunnel)
+
 since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly


@@ -224,6 +230,7 @@ also see [comparison to similar software](./docs/versus.md)
  * ☑ [upnp / zeroconf / mdns / ssdp](#zeroconf)
  * ☑ [event hooks](#event-hooks) / script runner
  * ☑ [reverse-proxy support](https://github.com/9001/copyparty#reverse-proxy)
+  * ☑ cross-platform (Windows, Linux, Macos, Android, FreeBSD, arm32/arm64, ppc64le, s390x, risc-v/riscv64)
 * upload
  * ☑ basic: plain multipart, ie6 support
  * ☑ [up2k](#uploading): js, resumable, multithreaded
@@ -275,6 +282,8 @@ small collection of user feedback

 `good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`, `wow this is better than nextcloud`

+* UI просто ужасно. Если буду описывать детально не смогу удержаться в рамках приличий
+

 # motivations

@@ -294,6 +303,8 @@ project goals / philosophy
 * adaptable, malleable, hackable
  * no build steps; modify the js/python without needing node.js or anything like that

+becoming rich is specifically *not* a motivation, but if you wanna donate then see my [github profile](https://github.com/9001) regarding donations for my FOSS stuff in general (also THANKS!)
+

 ## notes

@@ -323,7 +334,8 @@ roughly sorted by chance of encounter
  * `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
  * `--th-ff-swr` may fix audio thumbnails on some FFmpeg versions
  * if the `up2k.db` (filesystem index) is on a samba-share or network disk, you'll get unpredictable behavior if the share is disconnected for a bit
-    * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db on a local disk instead
+    * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db and thumbnails on a local disk instead
+    * or, if you only want to move the db (and not the thumbnails), then use `--dbpath` or the `dbpath` volflag
  * all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
  * probably more, pls let me know

@@ -376,7 +388,8 @@ same order here too
  * this is an msys2 bug, the regular windows edition of python is fine

 * VirtualBox: sqlite throws `Disk I/O Error` when running in a VM and the up2k database is in a vboxsf
-  * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db inside the vm instead
+  * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db and thumbnails inside the vm instead
+    * or, if you only want to move the db (and not the thumbnails), then use `--dbpath` or the `dbpath` volflag
  * also happens on mergerfs, so put the db elsewhere

 * Ubuntu: dragging files from certain folders into firefox or chrome is impossible
@@ -401,6 +414,9 @@ upgrade notes

 "frequently" asked questions

+* can I change the 🌲 spinning pine-tree loading animation?
+  * [yeah...](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner) :-(
+
 * is it possible to block read-access to folders unless you know the exact URL for a particular file inside?
  * yes, using the [`g` permission](#accounts-and-volumes), see the examples there
  * you can also do this with linux filesystem permissions; `chmod 111 music` will make it possible to access files and folders inside the `music` folder but not list the immediate contents -- also works with other software, not just copyparty
@@ -423,6 +439,14 @@ upgrade notes
 * copyparty seems to think I am using http, even though the URL is https
  * your reverse-proxy is not sending the `X-Forwarded-Proto: https` header; this could be because your reverse-proxy itself is confused. Ensure that none of the intermediates (such as cloudflare) are terminating https before the traffic hits your entrypoint

+* thumbnails are broken (you get a colorful square which says the filetype instead)
+  * you need to install `FFmpeg` or `Pillow`; see [thumbnails](#thumbnails)
+
+* thumbnails are broken (some images appear, but other files just get a blank box, and/or the broken-image placeholder)
+  * probably due to a reverse-proxy messing with the request URLs and stripping the query parameters (`?th=w`), so check your URL rewrite rules
+  * could also be due to incorrect caching settings in reverse-proxies and/or CDNs, so make sure that nothing is set to ignore the query string
+  * could also be due to misbehaving privacy-related browser extensions, so try to disable those
+
 * i want to learn python and/or programming and am considering looking at the copyparty source code in that occasion
  * ```bash
     _|  _      __   _  _|_
@@ -653,6 +677,7 @@ press `g` or `田` to toggle grid-view instead of the file listing  and `t` togg
 it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
 * pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
 * disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
+* for installing FFmpeg on windows, see [optional dependencies](#optional-dependencies)

 audio files are converted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)

@@ -664,6 +689,8 @@ enabling `multiselect` lets you click files to select them, and then shift-click
 * `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
  * the `sel` option can be made default globally with `--gsel` or per-volume with volflag `gsel`

+to show `/icons/exe.png` as the thumbnail for all .exe files, `--ext-th=exe=/icons/exe.png` (optionally as a volflag)
+
 config file example:

 ```yaml
@@ -678,6 +705,7 @@ config file example:
  flags:
    dthumb   # disable ALL thumbnails and audio transcoding
    dvthumb  # only disable video thumbnails
+    ext-th:  exe=/ico/exe.png  # /ico/exe.png is the thumbnail of *.exe
    th-covers:  folder.png,folder.jpg,cover.png,cover.jpg  # the default
 ```

@@ -705,6 +733,7 @@ select which type of archive you want in the `[⚙️] config` tab:
  * `up2k.db` and `dir.txt` is always excluded
 * bsdtar supports streaming unzipping: `curl foo?zip | bsdtar -xv`
  * good, because copyparty's zip is faster than tar on small files
+    * but `?tar` is better for large files, especially if the total exceeds 4 GiB
 * `zip_crc` will take longer to download since the server has to read each file twice
  * this is only to support MS-DOS PKZIP v2.04g (october 1993) and older
    * how are you accessing copyparty actually
@@ -761,8 +790,11 @@ the up2k UI is the epitome of polished intuitive experiences:
 * "parallel uploads" specifies how many chunks to upload at the same time
 * `[🏃]` analysis of other files should continue while one is uploading
 * `[🥔]` shows a simpler UI for faster uploads from slow devices
+* `[🛡️]` decides when to overwrite existing files on the server
+  * `🛡️` = never (generate a new filename instead)
+  * `🕒` = overwrite if the server-file is older
+  * `♻️` = always overwrite if the files are different
 * `[🎲]` generate random filenames during upload
-* `[📅]` preserve last-modified timestamps; server times will match yours
 * `[🔎]` switch between upload and [file-search](#file-search) mode
  * ignore `[🔎]` if you add files by dragging them into the browser

@@ -780,6 +812,8 @@ if you are resuming a massive upload and want to skip hashing the files which al

 if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option `--u2sz` (the default is 96 MiB to support Cloudflare)

+if you want to replace existing files on the server with new uploads by default, run with `--u2ow 2` (only works if users have the delete-permission, and can still be disabled with `🛡️` in the UI)
+

 ### file-search

@@ -1004,6 +1038,7 @@ click the `play` link next to an audio file, or copy the link target to [share i

 open the `[🎺]` media-player-settings tab to configure it,
 * "switches":
+  * `[🔁]` repeats one single song forever
  * `[🔀]` shuffles the files inside each folder
  * `[preload]` starts loading the next track when it's about to end, reduces the silence between songs
  * `[full]` does a full preload by downloading the entire next file; good for unreliable connections, bad for slow connections
@@ -1568,6 +1603,8 @@ copyparty creates a subfolder named `.hist` inside each volume where it stores t
 this can instead be kept in a single place using the `--hist` argument, or the `hist=` volflag, or a mix of both:
 * `--hist ~/.cache/copyparty -v ~/music::r:c,hist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)

+by default, the per-volume `up2k.db` sqlite3-database for `-e2d` and `-e2t` is stored next to the thumbnails according to the `--hist` option, but the global-option `--dbpath` and/or volflag `dbpath` can be used to put the database somewhere else
+
 note:
 * putting the hist-folders on an SSD is strongly recommended for performance
 * markdown edits are always stored in a local `.hist` subdirectory
@@ -1813,7 +1850,7 @@ tell search engines you don't wanna be indexed,  either using the good old [robo
 * volflag `[...]:c,norobots` does the same thing for that single volume
 * volflag `[...]:c,robots` ALLOWS search-engine crawling for that volume, even if `--no-robots` is set globally

-also, `--force-js` disables the plain HTML folder listing, making things harder to parse for search engines
+also, `--force-js` disables the plain HTML folder listing, making things harder to parse for *some* search engines -- note that crawlers which understand javascript (such as google) will not be affected


 ## themes
@@ -1979,6 +2016,26 @@ in summary, `haproxy > caddy > traefik > nginx > apache > lighttpd`, and use uds
 * if these results are bullshit because my config exampels are bad, please submit corrections!


+## permanent cloudflare tunnel
+
+if you have a domain and want to get your copyparty online real quick,  either from your home-PC behind a CGNAT or from a server without an existing [reverse-proxy](#reverse-proxy) setup, one approach is to create a [Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/) (formerly "Argo Tunnel")
+
+I'd recommend making a `Locally-managed tunnel` for more control, but if you prefer to make a `Remotely-managed tunnel` then this is currently how:
+
+* `cloudflare dashboard` » `zero trust` » `networks` » `tunnels` » `create a tunnel` » `cloudflared` » choose a cool `subdomain` and leave the `path` blank, and use `service type` = `http` and `URL` = `127.0.0.1:3923`
+
+* and if you want to just run the tunnel without installing it, skip the `cloudflared service install BASE64` step and instead do `cloudflared --no-autoupdate tunnel run --token BASE64`
+
+NOTE: since people will be connecting through cloudflare, as mentioned in [real-ip](#real-ip) you should run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
+
+config file example:
+
+```yaml
+[global]
+  xff-hdr: cf-connecting-ip
+```
+
+
 ## prometheus

 metrics/stats can be enabled  at URL `/.cpr/metrics` for grafana / prometheus / etc (openmetrics 1.0.0)
@@ -2065,13 +2122,27 @@ in a config file, this is the same as:
 run copyparty with `--mimes` to list all the default mappings


+### GDPR compliance
+
+imagine using copyparty professionally...  **TINLA/IANAL; EU laws are hella confusing**
+
+* remember to disable logging, or configure logrotation to an acceptable timeframe with `-lo cpp-%Y-%m%d.txt.xz` or similar
+
+* if running with the database enabled (recommended), then have it forget uploader-IPs after some time using `--forget-ip 43200`
+  * don't set it too low; [unposting](#unpost) a file is no longer possible after this takes effect
+
+* if you actually *are* a lawyer then I'm open for feedback, would be fun
+
+
 ### feature chickenbits

 buggy feature? rip it out  by setting any of the following environment variables to disable its associated bell or whistle,

 | env-var              | what it does |
 | -------------------- | ------------ |
+| `PRTY_NO_DB_LOCK`    | do not lock session/shares-databases for exclusive access |
 | `PRTY_NO_IFADDR`     | disable ip/nic discovery by poking into your OS with ctypes |
+| `PRTY_NO_IMPRESO`    | do not try to load js/css files using `importlib.resources` |
 | `PRTY_NO_IPV6`       | disable some ipv6 support (should not be necessary since windows 2000) |
 | `PRTY_NO_LZMA`       | disable streaming xz compression of incoming uploads |
 | `PRTY_NO_MP`         | disable all use of the python `multiprocessing` module (actual multithreading, cpu-count for parsers/thumbnailers) |
@@ -2082,6 +2153,15 @@ buggy feature? rip it out  by setting any of the following environment variables
 example: `PRTY_NO_IFADDR=1 python3 copyparty-sfx.py`


+### feature beefybits
+
+force-enable features with known issues on your OS/env  by setting any of the following environment variables, also affectionately known as `fuckitbits` or `hail-mary-bits`
+
+| env-var                  | what it does |
+| ------------------------ | ------------ |
+| `PRTY_FORCE_MP`          | force-enable multiprocessing (real multithreading) on MacOS and other broken platforms |
+
+
 # packages

 the party might be closer than you think
@@ -2252,6 +2332,7 @@ quick summary of more eccentric web-browsers trying to view a directory index:
 | **ie4** and **netscape** 4.0  | can browse, upload with `?b=u`, auth with `&pw=wark` |
 | **ncsa mosaic** 2.7       | does not get a pass, [pic1](https://user-images.githubusercontent.com/241032/174189227-ae816026-cf6f-4be5-a26e-1b3b072c1b2f.png) - [pic2](https://user-images.githubusercontent.com/241032/174189225-5651c059-5152-46e9-ac26-7e98e497901b.png) |
 | **SerenityOS** (7e98457)  | hits a page fault, works with `?b=u`, file upload not-impl |
+| **sony psp** 5.50         | can browse, upload/mkdir/msg (thx dwarf) [screenshot](https://github.com/user-attachments/assets/9d21f020-1110-4652-abeb-6fc09c533d4f) |
 | **nintendo 3ds**          | can browse, upload, view thumbnails (thx bnjmn) |

 <p align="center"><img src="https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853" /></p>
@@ -2379,6 +2460,8 @@ below are some tweaks roughly ordered by usefulness:
 * `--no-hash .` when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable
 * if your volumes are on a network-disk such as NFS / SMB / s3, specifying larger values for `--iobuf` and/or `--s-rd-sz` and/or `--s-wr-sz` may help; try setting all of them to `524288` or `1048576` or `4194304`
 * `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
+* when running on AlpineLinux or other musl-based distro, try mimalloc for higher performance (and twice as much RAM usage); `apk add mimalloc2` and run copyparty with env-var `LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2`
+  * note that mimalloc requires special care when combined with prisonparty and/or bubbleparty/bubblewrap; you must give it access to `/proc` and `/sys` otherwise you'll encounter issues with FFmpeg (audio transcoding, thumbnails)
 * `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
  * lots of connections (many users or heavy clients)
  * simultaneous downloads and uploads saturating a 20gbps connection
@@ -2582,6 +2665,8 @@ enable [smb](#smb-server) support (**not** recommended): `impacket==0.12.0`

 `pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`

+to install FFmpeg on Windows, grab [a recent build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) -- you need `ffmpeg.exe` and `ffprobe.exe` from inside the `bin` folder; copy them into `C:\Windows\System32` or any other folder that's in your `%PATH%`
+

 ### dependency chickenbits

--- a/bin/handlers/README.md
+++ b/bin/handlers/README.md
@@ -21,6 +21,7 @@ each plugin must define a `main()` which takes 3 arguments;
 ## on404

 * [redirect.py](redirect.py) sends an HTTP 301 or 302, redirecting the client to another page/file
+* [randpic.py](randpic.py) redirects `/foo/bar/randpic.jpg` to a random pic in `/foo/bar/`
 * [sorry.py](answer.py) replies with a custom message instead of the usual 404
 * [nooo.py](nooo.py) replies with an endless noooooooooooooo
 * [never404.py](never404.py) 100% guarantee that 404 will never be a thing again as it automatically creates dummy files whenever necessary
--- a/bin/handlers/randpic.py
+++ b/bin/handlers/randpic.py
@@ -0,0 +1,35 @@
+import os
+import random
+from urllib.parse import quote
+
+
+# assuming /foo/bar/ is a valid URL but /foo/bar/randpic.png does not exist,
+# hijack the 404 with a redirect to a random pic in that folder
+#
+# thx to lia & kipu for the idea
+
+
+def main(cli, vn, rem):
+    req_fn = rem.split("/")[-1]
+    if not cli.can_read or not req_fn.startswith("randpic"):
+        return
+
+    req_abspath = vn.canonical(rem)
+    req_ap_dir = os.path.dirname(req_abspath)
+    files_in_dir = os.listdir(req_ap_dir)
+
+    if "." in req_fn:
+        file_ext = "." + req_fn.split(".")[-1]
+        files_in_dir = [x for x in files_in_dir if x.lower().endswith(file_ext)]
+
+    if not files_in_dir:
+        return
+
+    selected_file = random.choice(files_in_dir)
+
+    req_url = "/".join([vn.vpath, rem]).strip("/")
+    req_dir = req_url.rsplit("/", 1)[0]
+    new_url = "/".join([req_dir, quote(selected_file)]).strip("/")
+
+    cli.reply(b"redirecting...", 302, headers={"Location": "/" + new_url})
+    return "true"
--- a/bin/hooks/README.md
+++ b/bin/hooks/README.md
@@ -14,6 +14,8 @@ run copyparty with `--help-hooks` for usage details / hook type explanations (xm
 * [discord-announce.py](discord-announce.py) announces new uploads on discord using webhooks ([example](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png))
 * [reject-mimetype.py](reject-mimetype.py) rejects uploads unless the mimetype is acceptable
 * [into-the-cache-it-goes.py](into-the-cache-it-goes.py) avoids bugs in caching proxies by immediately downloading each file that is uploaded
+* [podcast-normalizer.py](podcast-normalizer.py) creates a second file with dynamic-range-compression whenever an audio file is uploaded
+  * good example of the `idx` [hook effect](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#hook-effects) to tell copyparty about additional files to scan/index


 # upload batches
@@ -25,6 +27,7 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
 # before upload
 * [reject-extension.py](reject-extension.py) rejects uploads if they match a list of file extensions
 * [reloc-by-ext.py](reloc-by-ext.py) redirects an upload to another destination based on the file extension
+  * good example of the `reloc` [hook effect](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#hook-effects)


 # on message
--- a/bin/hooks/podcast-normalizer.py
+++ b/bin/hooks/podcast-normalizer.py
@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+
+import json
+import os
+import sys
+import subprocess as sp
+
+
+_ = r"""
+sends all uploaded audio files through an aggressive
+dynamic-range-compressor to even out the volume levels
+
+dependencies:
+    ffmpeg
+
+being an xau hook, this gets eXecuted After Upload completion
+    but before copyparty has started hashing/indexing the file, so
+    we'll create a second normalized copy in a subfolder and tell
+    copyparty to hash/index that additional file as well
+
+example usage as global config:
+    -e2d -e2t --xau j,c1,bin/hooks/podcast-normalizer.py
+
+parameters explained,
+    e2d/e2t = enable database and metadata indexing
+    xau = execute after upload
+    j   = this hook needs upload information as json (not just the filename)
+    c1  = this hook returns json on stdout, so tell copyparty to read that
+
+example usage as a volflag (per-volume config):
+    -v srv/inc/pods:inc/pods:r:rw,ed:c,xau=j,c1,bin/hooks/podcast-normalizer.py
+                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share fs-path srv/inc/pods at URL /inc/pods,
+     readable by all, read-write for user ed,
+     running this xau (exec-after-upload) plugin for all uploaded files)
+
+example usage as a volflag in a copyparty config file:
+    [/inc/pods]
+      srv/inc/pods
+      accs:
+        r: *
+        rw: ed
+      flags:
+        e2d  # enables file indexing
+        e2t  # metadata tags too
+        xau: j,c1,bin/hooks/podcast-normalizer.py
+
+"""
+
+########################################################################
+### CONFIG
+
+# filetypes to process; ignores everything else
+EXTS = "mp3 flac ogg opus m4a aac wav wma"
+
+# the name of the subdir to put the normalized files in
+SUBDIR = "normalized"
+
+########################################################################
+
+
+# try to enable support for crazy filenames
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p.encode("utf-8")
+
+
+def main():
+    # read info from copyparty
+    inf = json.loads(sys.argv[1])
+    vpath = inf["vp"]
+    abspath = inf["ap"]
+
+    # check if the file-extension is on the to-be-processed list
+    ext = abspath.lower().split(".")[-1]
+    if ext not in EXTS.split():
+        return
+
+    # jump into the folder where the file was uploaded
+    # and create the subfolder to place the normalized copy inside
+    dirpath, filename = os.path.split(abspath)
+    os.chdir(fsenc(dirpath))
+    os.makedirs(SUBDIR, exist_ok=True)
+
+    # the input and output filenames to give ffmpeg
+    fname_in = fsenc(f"./{filename}")
+    fname_out = fsenc(f"{SUBDIR}/{filename}.opus")
+
+    # fmt: off
+    # create and run the ffmpeg command
+    cmd = [
+        b"ffmpeg",
+        b"-nostdin",
+        b"-hide_banner",
+        b"-i", fname_in,
+        b"-af", b"dynaudnorm=f=100:g=9",  # the normalizer config
+        b"-c:a", b"libopus",
+        b"-b:a", b"128k",
+        fname_out,
+    ]
+    # fmt: on
+    sp.check_output(cmd)
+
+    # and finally, tell copyparty about the new file
+    # so it appears in the database and rss-feed:
+    vpath = f"{SUBDIR}/{filename}.opus"
+    print(json.dumps({"idx": {"vp": [vpath]}}))
+
+    # (it's fine to give it a relative path like that; it gets
+    #  resolved relative to the folder the file was uploaded into)
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except Exception as ex:
+        print("podcast-normalizer failed; %r" % (ex,))
--- a/bin/hooks/usb-eject.js
+++ b/bin/hooks/usb-eject.js
@@ -1,15 +1,17 @@
 // see usb-eject.py for usage

 function usbclick() {
-    QS('#treeul a[href="/usb/"]').click();
+    var o = QS('#treeul a[dst="/usb/"]') || QS('#treepar a[dst="/usb/"]');
+    if (o)
+        o.click();
 }

 function eject_cb() {
-    var t = this.responseText;
+    var t = ('' + this.responseText).trim();
    if (t.indexOf('can be safely unplugged') < 0 && t.indexOf('Device can be removed') < 0)
        return toast.err(30, 'usb eject failed:\n\n' + t);

-    toast.ok(5, esc(t.replace(/ - /g, '\n\n')));
+    toast.ok(5, esc(t.replace(/ - /g, '\n\n')).trim());
    usbclick(); setTimeout(usbclick, 10);
 };

@@ -21,10 +23,15 @@ function add_eject_2(a) {
    var v = aw[2],
        k = 'umount_' + v;

-    qsr('#' + k);
-    a.appendChild(mknod('span', k, '⏏'), a);
+    for (var b = 0; b < 9; b++) {
+        var o = ebi(k);
+        if (!o)
+            break;
+        o.parentNode.removeChild(o);
+    }

-    var o = ebi(k);
+    a.appendChild(mknod('span', k, '⏏'), a);
+    o = ebi(k);
    o.style.cssText = 'position:absolute; right:1em; margin-top:-.2em; font-size:1.3em';
    o.onclick = function (e) {
        ev(e);
@@ -38,8 +45,9 @@ function add_eject_2(a) {
 };

 function add_eject() {
-    for (var a of QSA('#treeul a[href^="/usb/"]'))
-        add_eject_2(a);
+    var o = QSA('#treeul a[href^="/usb/"]') || QSA('#treepar a[href^="/usb/"]');
+    for (var a = o.length - 1; a > 0; a--)
+        add_eject_2(o[a]);
 };

 (function() {
--- a/bin/hooks/usb-eject.py
+++ b/bin/hooks/usb-eject.py
@@ -14,13 +14,13 @@ remove those flashdrives, then boy howdy are you in the right place :D
 put usb-eject.js in the webroot (or somewhere else http-accessible)
 then run copyparty with these args:

-   -v /run/media/ed:/usb:A:c,hist=/tmp/junk
+   -v /run/media/egon:/usb:A:c,hist=/tmp/junk
   --xm=c1,bin/hooks/usb-eject.py
   --js-browser=/usb-eject.js

 which does the following respectively,

-  * share all of /run/media/ed as /usb with admin for everyone
+  * share all of /run/media/egon as /usb with admin for everyone
     and put the histpath somewhere it won't cause trouble
  * run the usb-eject hook with stdout redirect to the web-ui
  * add the complementary usb-eject.js to the browser
@@ -31,15 +31,24 @@ which does the following respectively,
 def main():
    try:
        label = sys.argv[1].split(":usb-eject:")[1].split(":")[0]
-        mp = "/run/media/ed/" + label
+        mp = "/run/media/egon/" + label
        # print("ejecting [%s]... " % (mp,), end="")
        mp = os.path.abspath(os.path.realpath(mp.encode("utf-8")))
        st = os.lstat(mp)
        if not stat.S_ISDIR(st.st_mode):
            raise Exception("not a regular directory")

-        cmd = [b"gio", b"mount", b"-e", mp]
-        print(sp.check_output(cmd).decode("utf-8", "replace").strip())
+        # if you're running copyparty as root (thx for the faith)
+        # you'll need something like this to make dbus talkative
+        cmd = b"sudo -u egon DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus gio mount -e"
+
+        # but if copyparty and the ui-session is running
+        # as the same user (good) then this is plenty
+        cmd = b"gio mount -e"
+
+        cmd = cmd.split(b" ") + [mp]
+        ret = sp.check_output(cmd).decode("utf-8", "replace")
+        print(ret.strip() or (label + " can be safely unplugged"))

    except Exception as ex:
        print("unmount failed: %r" % (ex,))
--- a/bin/mtag/cksum.py
+++ b/bin/mtag/cksum.py
@@ -2,11 +2,15 @@

 import sys
 import json
-import zlib
 import struct
 import base64
 import hashlib

+try:
+    from zlib_ng import zlib_ng as zlib
+except:
+    import zlib
+
 try:
    from copyparty.util import fsenc
 except:
--- a/bin/u2c.py
+++ b/bin/u2c.py
@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
 from __future__ import print_function, unicode_literals

-S_VERSION = "2.9"
-S_BUILD_DT = "2025-01-27"
+S_VERSION = "2.10"
+S_BUILD_DT = "2025-02-19"

 """
 u2c.py: upload to copyparty
@@ -807,7 +807,9 @@ def handshake(ar, file, search):
    else:
        if ar.touch:
            req["umod"] = True
-        if ar.ow:
+        if ar.owo:
+            req["replace"] = "mt"
+        elif ar.ow:
            req["replace"] = True

    file.recheck = False
@@ -1538,6 +1540,7 @@ source file/folder selection uses rsync syntax, meaning that:
    ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
    ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
    ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
+    ap.add_argument("--owo", action="store_true", help="overwrite existing files if server-file is older")
    ap.add_argument("--spd", action="store_true", help="print speeds for each file")
    ap.add_argument("--version", action="store_true", help="show version and exit")

--- a/contrib/README.md
+++ b/contrib/README.md
@@ -50,6 +50,9 @@
 * give a 3rd argument to install it to your copyparty config
 * systemd service at [`systemd/cfssl.service`](systemd/cfssl.service)

+### [`zfs-tune.py`](zfs-tune.py)
+* optimizes databases for optimal performance when stored on a zfs filesystem; also see [openzfs docs](https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#database-workloads) and specifically the SQLite subsection
+
 # OS integration
 init-scripts to start copyparty as a service
 * [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
--- a/contrib/package/arch/PKGBUILD
+++ b/contrib/package/arch/PKGBUILD
@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.16.11"
+pkgver="1.16.20"
 pkgrel=1
 pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
@@ -22,7 +22,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("c8b594e9ca22d81a316ee8b10a9f5f4ebb11f0e4767738e0cb1b1704b13b2d70")
+sha256sums=("eec5c16bca8251467b19e1d1baff5524b03479439604c0568eacedf57dbe2a78")

 build() {
    cd "${srcdir}/${pkgname}-${pkgver}"
--- a/contrib/package/nix/copyparty/default.nix
+++ b/contrib/package/nix/copyparty/default.nix
@@ -64,4 +64,5 @@ in stdenv.mkDerivation {
      --set PATH '${lib.makeBinPath ([ utillinux ] ++ lib.optional withMediaProcessing ffmpeg)}:$PATH' \
      --add-flags "$out/share/copyparty-sfx.py"
  '';
+  meta.mainProgram = "copyparty";
 }
--- a/contrib/package/nix/copyparty/pin.json
+++ b/contrib/package/nix/copyparty/pin.json
@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.16.11/copyparty-sfx.py",
-    "version": "1.16.11",
-    "hash": "sha256-Zl8f6mzWamwDeTnL5T5i7NdgIqYHy601oEWw8JPrPrI="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.16.20/copyparty-sfx.py",
+    "version": "1.16.20",
+    "hash": "sha256-4qeJO9P7VjchSMcShRADJBIYgmCnPV7hqDhIqmvrweU="
 }
--- a/contrib/zfs-tune.py
+++ b/contrib/zfs-tune.py
@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+
+import os
+import sqlite3
+import sys
+import traceback
+
+
+"""
+when the up2k-database is stored on a zfs volume, this may give
+slightly higher performance (actual gains not measured yet)
+
+NOTE: must be applied in combination with the related advice in the openzfs documentation;
+https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Workload%20Tuning.html#database-workloads
+and see specifically the SQLite subsection
+
+it is assumed that all databases are stored in a single location,
+for example with `--hist /var/store/hists`
+
+three alternatives for running this script:
+
+1. copy it into /var/store/hists and run "python3 zfs-tune.py s"
+    (s = modify all databases below folder containing script)
+
+2. cd into /var/store/hists and run "python3 ~/zfs-tune.py w"
+    (w = modify all databases below current working directory)
+
+3. python3 ~/zfs-tune.py /var/store/hists
+
+if you use docker, run copyparty with `--hist /cfg/hists`, copy this script into /cfg, and run this:
+podman run --rm -it --entrypoint /usr/bin/python3 ghcr.io/9001/copyparty-ac /cfg/zfs-tune.py s
+
+"""
+
+
+PAGESIZE = 65536
+
+
+# borrowed from copyparty; short efficient stacktrace for errors
+def min_ex(max_lines: int = 8, reverse: bool = False) -> str:
+    et, ev, tb = sys.exc_info()
+    stb = traceback.extract_tb(tb) if tb else traceback.extract_stack()[:-1]
+    fmt = "%s:%d <%s>: %s"
+    ex = [fmt % (fp.split(os.sep)[-1], ln, fun, txt) for fp, ln, fun, txt in stb]
+    if et or ev or tb:
+        ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
+    return "\n".join(ex[-max_lines:][:: -1 if reverse else 1])
+
+
+def set_pagesize(db_path):
+    try:
+        # check current page_size
+        with sqlite3.connect(db_path) as db:
+            v = db.execute("pragma page_size").fetchone()[0]
+            if v == PAGESIZE:
+                print(" `-- OK")
+                return
+
+        # https://www.sqlite.org/pragma.html#pragma_page_size
+        #  `- disable wal; set pagesize; vacuum
+        #      (copyparty will reenable wal if necessary)
+
+        with sqlite3.connect(db_path) as db:
+            db.execute("pragma journal_mode=delete")
+            db.commit()
+
+        with sqlite3.connect(db_path) as db:
+            db.execute(f"pragma page_size = {PAGESIZE}")
+            db.execute("vacuum")
+
+        print(" `-- new pagesize OK")
+
+    except Exception:
+        err = min_ex().replace("\n", "\n -- ")
+        print(f"FAILED: {db_path}\n -- {err}")
+
+
+def main():
+    top = os.path.dirname(os.path.abspath(__file__))
+    cwd = os.path.abspath(os.getcwd())
+    try:
+        x = sys.argv[1]
+    except:
+        print(f"""
+this script takes one mandatory argument:
+specify 's' to start recursing from folder containing this script file ({top})
+specify 'w' to start recursing from the current working directory ({cwd})
+specify a path to start recursing from there
+""")
+        sys.exit(1)
+
+    if x.lower() == "w":
+        top = cwd
+    elif x.lower() != "s":
+        top = x
+
+    for dirpath, dirs, files in os.walk(top):
+        for fname in files:
+            if not fname.endswith(".db"):
+                continue
+            db_path = os.path.join(dirpath, fname)
+            print(db_path)
+            set_pagesize(db_path)
+
+
+if __name__ == "__main__":
+    main()
--- a/copyparty/main.py
+++ b/copyparty/main.py
@@ -40,6 +40,7 @@ from .cfg import flagcats, onedash
 from .svchub import SvcHub
 from .util import (
    APPLESAN_TXT,
+    BAD_BOTS,
    DEF_EXP,
    DEF_MTE,
    DEF_MTH,
@@ -65,6 +66,7 @@ from .util import (
    load_resource,
    min_ex,
    pybin,
+    read_utf8,
    termsize,
    wrap,
 )
@@ -226,7 +228,23 @@ def init_E(EE: EnvParams) -> None:
    if E.mod.endswith("__init__"):
        E.mod = os.path.dirname(E.mod)

-    if sys.platform == "win32":
+    try:
+        p = os.environ.get("XDG_CONFIG_HOME")
+        if not p:
+            raise Exception()
+        if p.startswith("~"):
+            p = os.path.expanduser(p)
+        p = os.path.abspath(os.path.realpath(p))
+        p = os.path.join(p, "copyparty")
+        if not os.path.isdir(p):
+            os.mkdir(p)
+        os.listdir(p)
+    except:
+        p = ""
+
+    if p:
+        E.cfg = p
+    elif sys.platform == "win32":
        bdir = os.environ.get("APPDATA") or os.environ.get("TEMP") or "."
        E.cfg = os.path.normpath(bdir + "/copyparty")
    elif sys.platform == "darwin":
@@ -255,8 +273,7 @@ def get_srvname(verbose) -> str:
    if verbose:
        lprint("using hostname from {}\n".format(fp))
    try:
-        with open(fp, "rb") as f:
-            ret = f.read().decode("utf-8", "replace").strip()
+        return read_utf8(None, fp, True).strip()
    except:
        ret = ""
        namelen = 5
@@ -265,47 +282,18 @@ def get_srvname(verbose) -> str:
            ret = re.sub("[234567=]", "", ret)[:namelen]
        with open(fp, "wb") as f:
            f.write(ret.encode("utf-8") + b"\n")
-
-    return ret
+        return ret


-def get_fk_salt() -> str:
-    fp = os.path.join(E.cfg, "fk-salt.txt")
+def get_salt(name: str, nbytes: int) -> str:
+    fp = os.path.join(E.cfg, "%s-salt.txt" % (name,))
    try:
-        with open(fp, "rb") as f:
-            ret = f.read().strip()
+        return read_utf8(None, fp, True).strip()
    except:
-        ret = b64enc(os.urandom(18))
+        ret = b64enc(os.urandom(nbytes))
        with open(fp, "wb") as f:
            f.write(ret + b"\n")
-
-    return ret.decode("utf-8")
-
-
-def get_dk_salt() -> str:
-    fp = os.path.join(E.cfg, "dk-salt.txt")
-    try:
-        with open(fp, "rb") as f:
-            ret = f.read().strip()
-    except:
-        ret = b64enc(os.urandom(30))
-        with open(fp, "wb") as f:
-            f.write(ret + b"\n")
-
-    return ret.decode("utf-8")
-
-
-def get_ah_salt() -> str:
-    fp = os.path.join(E.cfg, "ah-salt.txt")
-    try:
-        with open(fp, "rb") as f:
-            ret = f.read().strip()
-    except:
-        ret = b64enc(os.urandom(18))
-        with open(fp, "wb") as f:
-            f.write(ret + b"\n")
-
-    return ret.decode("utf-8")
+        return ret.decode("utf-8")


 def ensure_locale() -> None:
@@ -1039,6 +1027,7 @@ def add_upload(ap):
    ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
    ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16 for cross-atlantic (max=64)")
    ap2.add_argument("--u2sz", metavar="N,N,N", type=u, default="1,64,96", help="web-client: default upload chunksize (MiB); sets \033[33mmin,default,max\033[0m in the settings gui. Each HTTP POST will aim for \033[33mdefault\033[0m, and never exceed \033[33mmax\033[0m. Cloudflare max is 96. Big values are good for cross-atlantic but may increase HDD fragmentation on some FS. Disable this optimization with [\033[32m1,1,1\033[0m]")
+    ap2.add_argument("--u2ow", metavar="NUM", type=int, default=0, help="web-client: default setting for when to replace/overwrite existing files; [\033[32m0\033[0m]=never, [\033[32m1\033[0m]=if-client-newer, [\033[32m2\033[0m]=always (volflag=u2ow)")
    ap2.add_argument("--u2sort", metavar="TXT", type=u, default="s", help="upload order; [\033[32ms\033[0m]=smallest-first, [\033[32mn\033[0m]=alphabetical, [\033[32mfs\033[0m]=force-s, [\033[32mfn\033[0m]=force-n -- alphabetical is a bit slower on fiber/LAN but makes it easier to eyeball if everything went fine")
    ap2.add_argument("--write-uplog", action="store_true", help="write POST reports to textfiles in working-directory")

@@ -1057,6 +1046,8 @@ def add_network(ap):
        ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
    else:
        ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
+    ap2.add_argument("--wr-h-eps", metavar="PATH", type=u, default="", help="write list of listening-on ip:port to textfile at \033[33mPATH\033[0m when http-servers have started")
+    ap2.add_argument("--wr-h-aon", metavar="PATH", type=u, default="", help="write list of accessible-on ip:port to textfile at \033[33mPATH\033[0m when http-servers have started")
    ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
    ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=128.0, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
    ap2.add_argument("--s-rd-sz", metavar="B", type=int, default=256*1024, help="socket read size in bytes (indirectly affects filesystem writes; recommendation: keep equal-to or lower-than \033[33m--iobuf\033[0m)")
@@ -1183,6 +1174,7 @@ def add_webdav(ap):
    ap2.add_argument("--dav-mac", action="store_true", help="disable apple-garbage filter -- allow macos to create junk files (._* and .DS_Store, .Spotlight-*, .fseventsd, .Trashes, .AppleDouble, __MACOS)")
    ap2.add_argument("--dav-rt", action="store_true", help="show symlink-destination's lastmodified instead of the link itself; always enabled for recursive listings (volflag=davrt)")
    ap2.add_argument("--dav-auth", action="store_true", help="force auth for all folders (required by davfs2 when only some folders are world-readable) (volflag=davauth)")
+    ap2.add_argument("--dav-ua1", metavar="PTN", type=u, default=r" kioworker/", help="regex of tricky user-agents which expect 401 from GET requests; disable with [\033[32mno\033[0m] or blank")


 def add_tftp(ap):
@@ -1249,6 +1241,7 @@ def add_yolo(ap):
    ap2 = ap.add_argument_group('yolo options')
    ap2.add_argument("--allow-csrf", action="store_true", help="disable csrf protections; let other domains/sites impersonate you through cross-site requests")
    ap2.add_argument("--getmod", action="store_true", help="permit ?move=[...] and ?delete as GET")
+    ap2.add_argument("--wo-up-readme", action="store_true", help="allow users with write-only access to upload logues and readmes without adding the _wo_ filename prefix (volflag=wo_up_readme)")


 def add_optouts(ap):
@@ -1263,12 +1256,17 @@ def add_optouts(ap):
    ap2.add_argument("-nih", action="store_true", help="no info hostname -- don't show in UI")
    ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI")
    ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
+    ap2.add_argument("--zipmaxn", metavar="N", type=u, default="0", help="reject download-as-zip if more than \033[33mN\033[0m files in total; optionally takes a unit suffix: [\033[32m256\033[0m], [\033[32m9K\033[0m], [\033[32m4G\033[0m] (volflag=zipmaxn)")
+    ap2.add_argument("--zipmaxs", metavar="SZ", type=u, default="0", help="reject download-as-zip if total download size exceeds \033[33mSZ\033[0m bytes; optionally takes a unit suffix: [\033[32m256M\033[0m], [\033[32m4G\033[0m], [\033[32m2T\033[0m] (volflag=zipmaxs)")
+    ap2.add_argument("--zipmaxt", metavar="TXT", type=u, default="", help="custom errormessage when download size exceeds max (volflag=zipmaxt)")
+    ap2.add_argument("--zipmaxu", action="store_true", help="authenticated users bypass the zip size limit (volflag=zipmaxu)")
    ap2.add_argument("--zip-who", metavar="LVL", type=int, default=3, help="who can download as zip/tar? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=authenticated-with-read-access, [\033[32m3\033[0m]=everyone-with-read-access (volflag=zip_who)\n\033[1;31mWARNING:\033[0m if a nested volume has a more restrictive value than a parent volume, then this will be \033[33mignored\033[0m if the download is initiated from the parent, more lenient volume")
+    ap2.add_argument("--ua-nozip", metavar="PTN", type=u, default=BAD_BOTS, help="regex of user-agents to reject from download-as-zip/tar; disable with [\033[32mno\033[0m] or blank")
    ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar; same as \033[33m--zip-who=0\033[0m")
    ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
    ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
    ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
-    ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
+    ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader-IP into the database; will also disable unpost, you may want \033[32m--forget-ip\033[0m instead (volflag=no_db_ip)")


 def add_safety(ap):
@@ -1379,6 +1377,7 @@ def add_thumbnail(ap):
    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,cbz,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,mdgz,mdxz,mdz,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
+    ap2.add_argument("--th-spec-cnv", metavar="T,T", type=u, default="it,itgz,itxz,itz,mdgz,mdxz,mdz,mo3,mod,s3m,s3gz,s3xz,s3z,xm,xmgz,xmxz,xmz,xpk", help="audio formats which provoke https://trac.ffmpeg.org/ticket/10797 (huge ram usage for s3xmodit spectrograms)")
    ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz, cbz=jpg.cbz", help="audio/image formats to decompress before passing to ffmpeg")


@@ -1395,7 +1394,7 @@ def add_transcoding(ap):

 def add_rss(ap):
    ap2 = ap.add_argument_group('RSS options')
-    ap2.add_argument("--rss", action="store_true", help="enable RSS output (experimental)")
+    ap2.add_argument("--rss", action="store_true", help="enable RSS output (experimental) (volflag=rss)")
    ap2.add_argument("--rss-nf", metavar="HITS", type=int, default=250, help="default number of files to return (url-param 'nf')")
    ap2.add_argument("--rss-fext", metavar="E,E", type=u, default="", help="default list of file extensions to include (url-param 'fext'); blank=all")
    ap2.add_argument("--rss-sort", metavar="ORD", type=u, default="m", help="default sort order (url-param 'sort'); [\033[32mm\033[0m]=last-modified [\033[32mu\033[0m]=upload-time [\033[32mn\033[0m]=filename [\033[32ms\033[0m]=filesize; Uppercase=oldest-first. Note that upload-time is 0 for non-uploaded files")
@@ -1411,6 +1410,7 @@ def add_db_general(ap, hcores):
    ap2.add_argument("-e2vu", action="store_true", help="on hash mismatch: update the database with the new hash")
    ap2.add_argument("-e2vp", action="store_true", help="on hash mismatch: panic and quit copyparty")
    ap2.add_argument("--hist", metavar="PATH", type=u, default="", help="where to store volume data (db, thumbs); default is a folder named \".hist\" inside each volume (volflag=hist)")
+    ap2.add_argument("--dbpath", metavar="PATH", type=u, default="", help="override where the volume databases are to be placed; default is the same as \033[33m--hist\033[0m (volflag=dbpath)")
    ap2.add_argument("--no-hash", metavar="PTN", type=u, default="", help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
    ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
    ap2.add_argument("--no-dirsz", action="store_true", help="do not show total recursive size of folders in listings, show inode size instead; slightly faster (volflag=nodirsz)")
@@ -1418,6 +1418,7 @@ def add_db_general(ap, hcores):
    ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
    ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
    ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice -- only useful for offloading uploads to a cloud service or something (volflag=noforget)")
+    ap2.add_argument("--forget-ip", metavar="MIN", type=int, default=0, help="remove uploader-IP from database (and make unpost impossible) \033[33mMIN\033[0m minutes after upload, for GDPR reasons. Default [\033[32m0\033[0m] is never-forget. [\033[32m1440\033[0m]=day, [\033[32m10080\033[0m]=week, [\033[32m43200\033[0m]=month. (volflag=forget_ip)")
    ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see \033[33m--help-dbd\033[0m (volflag=dbd)")
    ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (probably buggy, not recommended) (volflag=xlink)")
    ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
@@ -1448,11 +1449,13 @@ def add_db_metadata(ap):

 def add_txt(ap):
    ap2 = ap.add_argument_group('textfile options')
+    ap2.add_argument("--md-hist", metavar="TXT", type=u, default="s", help="where to store old version of markdown files; [\033[32ms\033[0m]=subfolder, [\033[32mv\033[0m]=volume-histpath, [\033[32mn\033[0m]=nope/disabled (volflag=md_hist)")
    ap2.add_argument("-mcr", metavar="SEC", type=int, default=60, help="the textfile editor will check for serverside changes every \033[33mSEC\033[0m seconds")
    ap2.add_argument("-emp", action="store_true", help="enable markdown plugins -- neat but dangerous, big XSS risk")
    ap2.add_argument("--exp", action="store_true", help="enable textfile expansion -- replace {{self.ip}} and such; see \033[33m--help-exp\033[0m (volflag=exp)")
    ap2.add_argument("--exp-md", metavar="V,V,V", type=u, default=DEF_EXP, help="comma/space-separated list of placeholders to expand in markdown files; add/remove stuff on the default list with +hdr_foo or /vf.scan (volflag=exp_md)")
    ap2.add_argument("--exp-lg", metavar="V,V,V", type=u, default=DEF_EXP, help="comma/space-separated list of placeholders to expand in prologue/epilogue files (volflag=exp_lg)")
+    ap2.add_argument("--ua-nodoc", metavar="PTN", type=u, default=BAD_BOTS, help="regex of user-agents to reject from viewing documents through ?doc=[...]; disable with [\033[32mno\033[0m] or blank")


 def add_og(ap):
@@ -1486,7 +1489,9 @@ def add_ui(ap, retry):
    ap2.add_argument("--hsortn", metavar="N", type=int, default=2, help="number of sorting rules to include in media URLs by default (volflag=hsortn)")
    ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
    ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
+    ap2.add_argument("--ext-th", metavar="E=VP", type=u, action="append", help="use thumbnail-image \033[33mVP\033[0m for file-extension \033[33mE\033[0m, example: [\033[32mexe=/.res/exe.png\033[0m] (volflag=ext_th)")
    ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
+    ap2.add_argument("--spinner", metavar="TXT", type=u, default="🌲", help="\033[33memoji\033[0m or \033[33memoji,css\033[0m Example: [\033[32m🥖,padding:0\033[0m]")
    ap2.add_argument("--css-browser", metavar="L", type=u, default="", help="URL to additional CSS to include in the filebrowser html")
    ap2.add_argument("--js-browser", metavar="L", type=u, default="", help="URL to additional JS to include in the filebrowser html")
    ap2.add_argument("--js-other", metavar="L", type=u, default="", help="URL to additional JS to include in all other pages")
@@ -1547,9 +1552,9 @@ def run_argparse(

    cert_path = os.path.join(E.cfg, "cert.pem")

-    fk_salt = get_fk_salt()
-    dk_salt = get_dk_salt()
-    ah_salt = get_ah_salt()
+    fk_salt = get_salt("fk", 18)
+    dk_salt = get_salt("dk", 30)
+    ah_salt = get_salt("ah", 18)

    # alpine peaks at 5 threads for some reason,
    # all others scale past that (but try to avoid SMT),
--- a/copyparty/version.py
+++ b/copyparty/version.py
@@ -1,8 +1,8 @@
 # coding: utf-8

-VERSION = (1, 16, 12)
+VERSION = (1, 16, 21)
 CODENAME = "COPYparty"
-BUILD_DT = (2025, 2, 9)
+BUILD_DT = (2025, 4, 20)

 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
--- a/copyparty/authsrv.py
+++ b/copyparty/authsrv.py
@@ -33,6 +33,7 @@ from .util import (
    get_df,
    humansize,
    odfusion,
+    read_utf8,
    relchk,
    statdir,
    ub64enc,
@@ -46,7 +47,7 @@ from .util import (
 if True:  # pylint: disable=using-constant-test
    from collections.abc import Iterable

-    from typing import Any, Generator, Optional, Union
+    from typing import Any, Generator, Optional, Sequence, Union

    from .util import NamedLogger, RootLogger

@@ -71,6 +72,8 @@ SSEELOG = " ({})".format(SEE_LOG)
 BAD_CFG = "invalid config; {}".format(SEE_LOG)
 SBADCFG = " ({})".format(BAD_CFG)

+PTN_U_GRP = re.compile(r"\$\{u%([+-])([^}]+)\}")
+

 class CfgEx(Exception):
    pass
@@ -342,22 +345,27 @@ class VFS(object):
        log: Optional["RootLogger"],
        realpath: str,
        vpath: str,
+        vpath0: str,
        axs: AXS,
        flags: dict[str, Any],
    ) -> None:
        self.log = log
        self.realpath = realpath  # absolute path on host filesystem
        self.vpath = vpath  # absolute path in the virtual filesystem
+        self.vpath0 = vpath0  # original vpath (before idp expansion)
        self.axs = axs
        self.flags = flags  # config options
        self.root = self
        self.dev = 0  # st_dev
+        self.badcfg1 = False
        self.nodes: dict[str, VFS] = {}  # child nodes
        self.histtab: dict[str, str] = {}  # all realpath->histpath
+        self.dbpaths: dict[str, str] = {}  # all realpath->dbpath
        self.dbv: Optional[VFS] = None  # closest full/non-jump parent
        self.lim: Optional[Lim] = None  # upload limits; only set for dbv
        self.shr_src: Optional[tuple[VFS, str]] = None  # source vfs+rem of a share
        self.shr_files: set[str] = set()  # filenames to include from shr_src
+        self.shr_owner: str = ""  # uname
        self.aread: dict[str, list[str]] = {}
        self.awrite: dict[str, list[str]] = {}
        self.amove: dict[str, list[str]] = {}
@@ -374,12 +382,13 @@ class VFS(object):
            rp = realpath + ("" if realpath.endswith(os.sep) else os.sep)
            vp = vpath + ("/" if vpath else "")
            self.histpath = os.path.join(realpath, ".hist")  # db / thumbcache
+            self.dbpath = self.histpath
            self.all_vols = {vpath: self}  # flattened recursive
-            self.all_nodes = {vpath: self}  # also jumpvols
+            self.all_nodes = {vpath: self}  # also jumpvols/shares
            self.all_aps = [(rp, self)]
            self.all_vps = [(vp, self)]
        else:
-            self.histpath = ""
+            self.histpath = self.dbpath = ""
            self.all_vols = {}
            self.all_nodes = {}
            self.all_aps = []
@@ -415,7 +424,7 @@ class VFS(object):
        for v in self.nodes.values():
            v.get_all_vols(vols, nodes, aps, vps)

-    def add(self, src: str, dst: str) -> "VFS":
+    def add(self, src: str, dst: str, dst0: str) -> "VFS":
        """get existing, or add new path to the vfs"""
        assert src == "/" or not src.endswith("/")  # nosec
        assert not dst.endswith("/")  # nosec
@@ -423,20 +432,22 @@ class VFS(object):
        if "/" in dst:
            # requires breadth-first population (permissions trickle down)
            name, dst = dst.split("/", 1)
+            name0, dst0 = dst0.split("/", 1)
            if name in self.nodes:
                # exists; do not manipulate permissions
-                return self.nodes[name].add(src, dst)
+                return self.nodes[name].add(src, dst, dst0)

            vn = VFS(
                self.log,
                os.path.join(self.realpath, name) if self.realpath else "",
                "{}/{}".format(self.vpath, name).lstrip("/"),
+                "{}/{}".format(self.vpath0, name0).lstrip("/"),
                self.axs,
                self._copy_flags(name),
            )
            vn.dbv = self.dbv or self
            self.nodes[name] = vn
-            return vn.add(src, dst)
+            return vn.add(src, dst, dst0)

        if dst in self.nodes:
            # leaf exists; return as-is
@@ -444,24 +455,31 @@ class VFS(object):

        # leaf does not exist; create and keep permissions blank
        vp = "{}/{}".format(self.vpath, dst).lstrip("/")
-        vn = VFS(self.log, src, vp, AXS(), {})
+        vp0 = "{}/{}".format(self.vpath0, dst0).lstrip("/")
+        vn = VFS(self.log, src, vp, vp0, AXS(), {})
        vn.dbv = self.dbv or self
        self.nodes[dst] = vn
        return vn

    def _copy_flags(self, name: str) -> dict[str, Any]:
        flags = {k: v for k, v in self.flags.items()}
+
        hist = flags.get("hist")
        if hist and hist != "-":
            zs = "{}/{}".format(hist.rstrip("/"), name)
            flags["hist"] = os.path.expandvars(os.path.expanduser(zs))

+        dbp = flags.get("dbpath")
+        if dbp and dbp != "-":
+            zs = "{}/{}".format(dbp.rstrip("/"), name)
+            flags["dbpath"] = os.path.expandvars(os.path.expanduser(zs))
+
        return flags

    def bubble_flags(self) -> None:
        if self.dbv:
            for k, v in self.dbv.flags.items():
-                if k not in ["hist"]:
+                if k not in ("hist", "dbpath"):
                    self.flags[k] = v

        for n in self.nodes.values():
@@ -861,7 +879,7 @@ class AuthSrv(object):
        self.indent = ""

        # fwd-decl
-        self.vfs = VFS(log_func, "", "", AXS(), {})
+        self.vfs = VFS(log_func, "", "", "", AXS(), {})
        self.acct: dict[str, str] = {}  # uname->pw
        self.iacct: dict[str, str] = {}  # pw->uname
        self.ases: dict[str, str] = {}  # uname->session
@@ -929,7 +947,7 @@ class AuthSrv(object):
        self,
        src: str,
        dst: str,
-        mount: dict[str, str],
+        mount: dict[str, tuple[str, str]],
        daxs: dict[str, AXS],
        mflags: dict[str, dict[str, Any]],
        un_gns: dict[str, list[str]],
@@ -945,12 +963,24 @@ class AuthSrv(object):
            un_gn = [("", "")]

        for un, gn in un_gn:
+            m = PTN_U_GRP.search(dst0)
+            if m:
+                req, gnc = m.groups()
+                hit = gnc in (un_gns.get(un) or [])
+                if req == "+":
+                    if not hit:
+                        continue
+                elif hit:
+                    continue
+
            # if ap/vp has a user/group placeholder, make sure to keep
            # track so the same user/group is mapped when setting perms;
            # otherwise clear un/gn to indicate it's a regular volume

            src1 = src0.replace("${u}", un or "\n")
            dst1 = dst0.replace("${u}", un or "\n")
+            src1 = PTN_U_GRP.sub(un or "\n", src1)
+            dst1 = PTN_U_GRP.sub(un or "\n", dst1)
            if src0 == src1 and dst0 == dst1:
                un = ""

@@ -967,7 +997,7 @@ class AuthSrv(object):
                continue
            visited.add(label)

-            src, dst = self._map_volume(src, dst, mount, daxs, mflags)
+            src, dst = self._map_volume(src, dst, dst0, mount, daxs, mflags)
            if src:
                ret.append((src, dst, un, gn))
                if un or gn:
@@ -979,7 +1009,8 @@ class AuthSrv(object):
        self,
        src: str,
        dst: str,
-        mount: dict[str, str],
+        dst0: str,
+        mount: dict[str, tuple[str, str]],
        daxs: dict[str, AXS],
        mflags: dict[str, dict[str, Any]],
    ) -> tuple[str, str]:
@@ -989,13 +1020,13 @@ class AuthSrv(object):

        if dst in mount:
            t = "multiple filesystem-paths mounted at [/{}]:\n  [{}]\n  [{}]"
-            self.log(t.format(dst, mount[dst], src), c=1)
+            self.log(t.format(dst, mount[dst][0], src), c=1)
            raise Exception(BAD_CFG)

        if src in mount.values():
            t = "filesystem-path [{}] mounted in multiple locations:"
            t = t.format(src)
-            for v in [k for k, v in mount.items() if v == src] + [dst]:
+            for v in [k for k, v in mount.items() if v[0] == src] + [dst]:
                t += "\n  /{}".format(v)

            self.log(t, c=3)
@@ -1004,7 +1035,7 @@ class AuthSrv(object):
        if not bos.path.isdir(src):
            self.log("warning: filesystem-path does not exist: {}".format(src), 3)

-        mount[dst] = src
+        mount[dst] = (src, dst0)
        daxs[dst] = AXS()
        mflags[dst] = {}
        return (src, dst)
@@ -1065,7 +1096,7 @@ class AuthSrv(object):
        grps: dict[str, list[str]],
        daxs: dict[str, AXS],
        mflags: dict[str, dict[str, Any]],
-        mount: dict[str, str],
+        mount: dict[str, tuple[str, str]],
    ) -> None:
        self.line_ctr = 0

@@ -1090,7 +1121,7 @@ class AuthSrv(object):
        grps: dict[str, list[str]],
        daxs: dict[str, AXS],
        mflags: dict[str, dict[str, Any]],
-        mount: dict[str, str],
+        mount: dict[str, tuple[str, str]],
        npass: int,
    ) -> None:
        self.line_ctr = 0
@@ -1289,10 +1320,10 @@ class AuthSrv(object):
                # one or more bools before the final flag; eat them
                n1, uname = uname.split(",", 1)
                for _, vp, _, _ in vols:
-                    self._read_volflag(flags[vp], n1, True, False)
+                    self._read_volflag(vp, flags[vp], n1, True, False)

            for _, vp, _, _ in vols:
-                self._read_volflag(flags[vp], uname, cval, False)
+                self._read_volflag(vp, flags[vp], uname, cval, False)

            return

@@ -1379,20 +1410,42 @@ class AuthSrv(object):

    def _read_volflag(
        self,
+        vpath: str,
        flags: dict[str, Any],
        name: str,
        value: Union[str, bool, list[str]],
        is_list: bool,
    ) -> None:
+        if name not in flagdescs:
+            name = name.lower()
+
+            # volflags are snake_case, but a leading dash is the removal operator
+            stripped = name.lstrip("-")
+            zi = len(name) - len(stripped)
+            if zi > 1:
+                t = "WARNING: the config for volume [/%s] specified a volflag with multiple leading hyphens (%s); use one hyphen to remove, or zero hyphens to add a flag. Will now enable flag [%s]"
+                self.log(t % (vpath, name, stripped), 3)
+                name = stripped
+                zi = 0
+
+            if stripped not in flagdescs and "-" in stripped:
+                name = ("-" * zi) + stripped.replace("-", "_")
+
        desc = flagdescs.get(name.lstrip("-"), "?").replace("\n", " ")

+        if not name:
+            self._e("└─unreadable-line")
+            t = "WARNING: the config for volume [/%s] indicated that a volflag was to be defined, but the volflag name was blank"
+            self.log(t % (vpath,), 3)
+            return
+
        if re.match("^-[^-]+$", name):
            t = "└─unset volflag [{}]  ({})"
            self._e(t.format(name[1:], desc))
            flags[name] = True
            return

-        zs = "mtp on403 on404 xbu xau xiu xbc xac xbr xar xbd xad xm xban"
+        zs = "ext_th mtp on403 on404 xbu xau xiu xbc xac xbr xar xbd xad xm xban"
        if name not in zs.split():
            if value is True:
                t = "└─add volflag [{}] = {}  ({})"
@@ -1427,8 +1480,8 @@ class AuthSrv(object):
        acct: dict[str, str] = {}  # username:password
        grps: dict[str, list[str]] = {}  # groupname:usernames
        daxs: dict[str, AXS] = {}
-        mflags: dict[str, dict[str, Any]] = {}  # moutpoint:flags
-        mount: dict[str, str] = {}  # dst:src (mountpoint:realpath)
+        mflags: dict[str, dict[str, Any]] = {}  # vpath:flags
+        mount: dict[str, tuple[str, str]] = {}  # dst:src (vp:(ap,vp0))

        self.idp_vols = {}  # yolo

@@ -1507,8 +1560,8 @@ class AuthSrv(object):
        # case-insensitive; normalize
        if WINDOWS:
            cased = {}
-            for k, v in mount.items():
-                cased[k] = absreal(v)
+            for vp, (ap, vp0) in mount.items():
+                cased[vp] = (absreal(ap), vp0)

            mount = cased

@@ -1523,25 +1576,28 @@ class AuthSrv(object):
                t = "Read-access has been disabled due to failsafe: No volumes were defined by the config-file. This failsafe is to prevent unintended access if this is due to accidental loss of config. You can override this safeguard and allow read/write to the working-directory by adding the following arguments:  -v .::rw"
                self.log(t, 1)
                axs = AXS()
-            vfs = VFS(self.log_func, absreal("."), "", axs, {})
+            vfs = VFS(self.log_func, absreal("."), "", "", axs, {})
+            if not axs.uread:
+                vfs.badcfg1 = True
        elif "" not in mount:
            # there's volumes but no root; make root inaccessible
            zsd = {"d2d": True, "tcolor": self.args.tcolor}
-            vfs = VFS(self.log_func, "", "", AXS(), zsd)
+            vfs = VFS(self.log_func, "", "", "", AXS(), zsd)

        maxdepth = 0
        for dst in sorted(mount.keys(), key=lambda x: (x.count("/"), len(x))):
            depth = dst.count("/")
            assert maxdepth <= depth  # nosec
            maxdepth = depth
+            src, dst0 = mount[dst]

            if dst == "":
                # rootfs was mapped; fully replaces the default CWD vfs
-                vfs = VFS(self.log_func, mount[dst], dst, daxs[dst], mflags[dst])
+                vfs = VFS(self.log_func, src, dst, dst0, daxs[dst], mflags[dst])
                continue

            assert vfs  # type: ignore
-            zv = vfs.add(mount[dst], dst)
+            zv = vfs.add(src, dst, dst0)
            zv.axs = daxs[dst]
            zv.flags = mflags[dst]
            zv.dbv = None
@@ -1557,13 +1613,26 @@ class AuthSrv(object):
            vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
            vol.root = vfs

+        zs = "neversymlink"
+        k_ign = set(zs.split())
+        for vol in vfs.all_vols.values():
+            unknown_flags = set()
+            for k, v in vol.flags.items():
+                ks = k.lstrip("-")
+                if ks not in flagdescs and ks not in k_ign:
+                    unknown_flags.add(k)
+            if unknown_flags:
+                t = "WARNING: the config for volume [/%s] has unrecognized volflags; will ignore: '%s'"
+                self.log(t % (vol.vpath, "', '".join(unknown_flags)), 3)
+
        enshare = self.args.shr
        shr = enshare[1:-1]
        shrs = enshare[1:]
        if enshare:
            import sqlite3

-            shv = VFS(self.log_func, "", shr, AXS(), {})
+            zsd = {"d2d": True, "tcolor": self.args.tcolor}
+            shv = VFS(self.log_func, "", shr, shr, AXS(), zsd)

            db_path = self.args.shr_db
            db = sqlite3.connect(db_path)
@@ -1597,9 +1666,8 @@ class AuthSrv(object):

                # don't know the abspath yet + wanna ensure the user
                # still has the privs they granted, so nullmap it
-                shv.nodes[s_k] = VFS(
-                    self.log_func, "", "%s/%s" % (shr, s_k), s_axs, shv.flags.copy()
-                )
+                vp = "%s/%s" % (shr, s_k)
+                shv.nodes[s_k] = VFS(self.log_func, "", vp, vp, s_axs, shv.flags.copy())

            vfs.nodes[shr] = vfs.all_vols[shr] = shv
            for vol in shv.nodes.values():
@@ -1699,7 +1767,7 @@ class AuthSrv(object):
                pass
            elif vflag:
                vflag = os.path.expandvars(os.path.expanduser(vflag))
-                vol.histpath = uncyg(vflag) if WINDOWS else vflag
+                vol.histpath = vol.dbpath = uncyg(vflag) if WINDOWS else vflag
            elif self.args.hist:
                for nch in range(len(hid)):
                    hpath = os.path.join(self.args.hist, hid[: nch + 1])
@@ -1720,12 +1788,45 @@ class AuthSrv(object):
                        with open(powner, "wb") as f:
                            f.write(me)

-                    vol.histpath = hpath
+                    vol.histpath = vol.dbpath = hpath
                    break

            vol.histpath = absreal(vol.histpath)
+
+        for vol in vfs.all_vols.values():
+            hid = self.hid_cache[vol.realpath]
+            vflag = vol.flags.get("dbpath")
+            if vflag == "-":
+                pass
+            elif vflag:
+                vflag = os.path.expandvars(os.path.expanduser(vflag))
+                vol.dbpath = uncyg(vflag) if WINDOWS else vflag
+            elif self.args.dbpath:
+                for nch in range(len(hid)):
+                    hpath = os.path.join(self.args.dbpath, hid[: nch + 1])
+                    bos.makedirs(hpath)
+
+                    powner = os.path.join(hpath, "owner.txt")
+                    try:
+                        with open(powner, "rb") as f:
+                            owner = f.read().rstrip()
+                    except:
+                        owner = None
+
+                    me = afsenc(vol.realpath).rstrip()
+                    if owner not in [None, me]:
+                        continue
+
+                    if owner is None:
+                        with open(powner, "wb") as f:
+                            f.write(me)
+
+                    vol.dbpath = hpath
+                    break
+
+            vol.dbpath = absreal(vol.dbpath)
            if vol.dbv:
-                if bos.path.exists(os.path.join(vol.histpath, "up2k.db")):
+                if bos.path.exists(os.path.join(vol.dbpath, "up2k.db")):
                    promote.append(vol)
                    vol.dbv = None
                else:
@@ -1740,9 +1841,7 @@ class AuthSrv(object):
                "\n  the following jump-volumes were generated to assist the vfs.\n  As they contain a database (probably from v0.11.11 or older),\n  they are promoted to full volumes:"
            ]
            for vol in promote:
-                ta.append(
-                    "  /{}  ({})  ({})".format(vol.vpath, vol.realpath, vol.histpath)
-                )
+                ta.append("  /%s  (%s)  (%s)" % (vol.vpath, vol.realpath, vol.dbpath))

            self.log("\n\n".join(ta) + "\n", c=3)

@@ -1753,13 +1852,45 @@ class AuthSrv(object):
            is_shr = shr and zv.vpath.split("/")[0] == shr
            if histp and not is_shr and histp in rhisttab:
                zv2 = rhisttab[histp]
-                t = "invalid config; multiple volumes share the same histpath (database location):\n  histpath: %s\n  volume 1: /%s  [%s]\n  volume 2: %s  [%s]"
+                t = "invalid config; multiple volumes share the same histpath (database+thumbnails location):\n  histpath: %s\n  volume 1: /%s  [%s]\n  volume 2: %s  [%s]"
                t = t % (histp, zv2.vpath, zv2.realpath, zv.vpath, zv.realpath)
                self.log(t, 1)
                raise Exception(t)
            rhisttab[histp] = zv
            vfs.histtab[zv.realpath] = histp

+        rdbpaths = {}
+        vfs.dbpaths = {}
+        for zv in vfs.all_vols.values():
+            dbp = zv.dbpath
+            is_shr = shr and zv.vpath.split("/")[0] == shr
+            if dbp and not is_shr and dbp in rdbpaths:
+                zv2 = rdbpaths[dbp]
+                t = "invalid config; multiple volumes share the same dbpath (database location):\n  dbpath: %s\n  volume 1: /%s  [%s]\n  volume 2: %s  [%s]"
+                t = t % (dbp, zv2.vpath, zv2.realpath, zv.vpath, zv.realpath)
+                self.log(t, 1)
+                raise Exception(t)
+            rdbpaths[dbp] = zv
+            vfs.dbpaths[zv.realpath] = dbp
+
+        for vol in vfs.all_vols.values():
+            use = False
+            for k in ["zipmaxn", "zipmaxs"]:
+                try:
+                    zs = vol.flags[k]
+                except:
+                    zs = getattr(self.args, k)
+                if zs in ("", "0"):
+                    vol.flags[k] = 0
+                    continue
+
+                zf = unhumanize(zs)
+                vol.flags[k + "_v"] = zf
+                if zf:
+                    use = True
+            if use:
+                vol.flags["zipmax"] = True
+
        for vol in vfs.all_vols.values():
            lim = Lim(self.log_func)
            use = False
@@ -1918,11 +2049,8 @@ class AuthSrv(object):
                if vf not in vol.flags:
                    vol.flags[vf] = getattr(self.args, ga)

-            for k in ("nrand",):
-                if k not in vol.flags:
-                    vol.flags[k] = getattr(self.args, k)
-
-            for k in ("nrand", "u2abort", "ups_who", "zip_who"):
+            zs = "forget_ip nrand u2abort u2ow ups_who zip_who"
+            for k in zs.split():
                if k in vol.flags:
                    vol.flags[k] = int(vol.flags[k])

@@ -1974,8 +2102,10 @@ class AuthSrv(object):

            # append additive args from argv to volflags
            hooks = "xbu xau xiu xbc xac xbr xar xbd xad xm xban".split()
-            for name in "mtp on404 on403".split() + hooks:
-                self._read_volflag(vol.flags, name, getattr(self.args, name), True)
+            for name in "ext_th mtp on404 on403".split() + hooks:
+                self._read_volflag(
+                    vol.vpath, vol.flags, name, getattr(self.args, name), True
+                )

            for hn in hooks:
                cmds = vol.flags.get(hn)
@@ -2003,6 +2133,16 @@ class AuthSrv(object):
                    ncmds.append(ocmd)
                vol.flags[hn] = ncmds

+            ext_th = vol.flags["ext_th_d"] = {}
+            etv = "(?)"
+            try:
+                for etv in vol.flags.get("ext_th") or []:
+                    k, v = etv.split("=")
+                    ext_th[k] = v
+            except:
+                t = "WARNING: volume [/%s]: invalid value specified for ext-th: %s"
+                self.log(t % (vol.vpath, etv), 3)
+
            # d2d drops all database features for a volume
            for grp, rm in [["d2d", "e2d"], ["d2t", "e2t"], ["d2d", "e2v"]]:
                if not vol.flags.get(grp, False):
@@ -2134,8 +2274,13 @@ class AuthSrv(object):
        for vol in vfs.all_nodes.values():
            for k in list(vol.flags.keys()):
                if re.match("^-[^-]+$", k):
-                    vol.flags.pop(k[1:], None)
                    vol.flags.pop(k)
+                    zs = k[1:]
+                    if zs in vol.flags:
+                        vol.flags.pop(k[1:])
+                    else:
+                        t = "WARNING: the config for volume [/%s] tried to remove volflag [%s] by specifying [%s] but that volflag was not already set"
+                        self.log(t % (vol.vpath, zs, k), 3)

            if vol.flags.get("dots"):
                for name in vol.axs.uread:
@@ -2228,22 +2373,56 @@ class AuthSrv(object):
        except Pebkac:
            self.warn_anonwrite = True

-        idp_err = "WARNING! The following IdP volumes are mounted directly below another volume where anonymous users can read and/or write files. This is a SECURITY HAZARD!! When copyparty is restarted, it will not know about these IdP volumes yet. These volumes will then be accessible by anonymous users UNTIL one of the users associated with their volume sends a request to the server. RECOMMENDATION: You should create a restricted volume where nobody can read/write files, and make sure that all IdP volumes are configured to appear somewhere below that volume."
+        self.idp_warn = []
+        self.idp_err = []
        for idp_vp in self.idp_vols:
-            parent_vp = vsplit(idp_vp)[0]
-            vn, _ = vfs.get(parent_vp, "*", False, False)
-            zs = (
-                "READABLE"
-                if "*" in vn.axs.uread
-                else "WRITABLE"
-                if "*" in vn.axs.uwrite
-                else ""
-            )
-            if zs:
-                t = '\nWARNING: Volume "/%s" appears below "/%s" and would be WORLD-%s'
-                idp_err += t % (idp_vp, vn.vpath, zs)
-        if "\n" in idp_err:
-            self.log(idp_err, 1)
+            idp_vn, _ = vfs.get(idp_vp, "*", False, False)
+            idp_vp0 = idp_vn.vpath0
+
+            sigils = set(re.findall(r"(\${[ug][}%])", idp_vp0))
+            if len(sigils) > 1:
+                t = '\nWARNING: IdP-volume "/%s" created by "/%s" has multiple IdP placeholders: %s'
+                self.idp_warn.append(t % (idp_vp, idp_vp0, list(sigils)))
+                continue
+
+            sigil = sigils.pop()
+            par_vp = idp_vp
+            while par_vp:
+                par_vp = vsplit(par_vp)[0]
+                par_vn, _ = vfs.get(par_vp, "*", False, False)
+                if sigil in par_vn.vpath0:
+                    continue  # parent was spawned for and by same user
+
+                oth_read = []
+                oth_write = []
+                for usr in par_vn.axs.uread:
+                    if usr not in idp_vn.axs.uread:
+                        oth_read.append(usr)
+                for usr in par_vn.axs.uwrite:
+                    if usr not in idp_vn.axs.uwrite:
+                        oth_write.append(usr)
+
+                if "*" in oth_read:
+                    taxs = "WORLD-READABLE"
+                elif "*" in oth_write:
+                    taxs = "WORLD-WRITABLE"
+                elif oth_read:
+                    taxs = "READABLE BY %r" % (oth_read,)
+                elif oth_write:
+                    taxs = "WRITABLE BY %r" % (oth_write,)
+                else:
+                    break  # no sigil; not idp; safe to stop
+
+                t = '\nWARNING: IdP-volume "/%s" created by "/%s" has parent/grandparent "/%s" and would be %s'
+                self.idp_err.append(t % (idp_vp, idp_vp0, par_vn.vpath, taxs))
+
+        if self.idp_warn:
+            t = "WARNING! Some IdP volumes include multiple IdP placeholders; this is too complex to automatically determine if safe or not. To ensure that no users gain unintended access, please use only a single placeholder for each IdP volume."
+            self.log(t + "".join(self.idp_warn), 1)
+
+        if self.idp_err:
+            t = "WARNING! The following IdP volumes are mounted below another volume where other users can read and/or write files. This is a SECURITY HAZARD!! When copyparty is restarted, it will not know about these IdP volumes yet. These volumes will then be accessible by an unexpected set of permissions UNTIL one of the users associated with their volume sends a request to the server. RECOMMENDATION: You should create a restricted volume where nobody can read/write files, and make sure that all IdP volumes are configured to appear somewhere below that volume."
+            self.log(t + "".join(self.idp_err), 1)

        self.vfs = vfs
        self.acct = acct
@@ -2278,11 +2457,6 @@ class AuthSrv(object):
                for x, y in vfs.all_vols.items()
                if x != shr and not x.startswith(shrs)
            }
-            vfs.all_nodes = {
-                x: y
-                for x, y in vfs.all_nodes.items()
-                if x != shr and not x.startswith(shrs)
-            }

            assert db and cur and cur2 and shv  # type: ignore
            for row in cur.execute("select * from sh"):
@@ -2312,6 +2486,7 @@ class AuthSrv(object):
                else:
                    shn.ls = shn._ls

+                shn.shr_owner = s_un
                shn.shr_src = (s_vfs, s_rem)
                shn.realpath = s_vfs.canonical(s_rem)

@@ -2329,7 +2504,7 @@ class AuthSrv(object):
                    continue  # also fine
                for zs in svn.nodes.keys():
                    # hide subvolume
-                    vn.nodes[zs] = VFS(self.log_func, "", "", AXS(), {})
+                    vn.nodes[zs] = VFS(self.log_func, "", "", "", AXS(), {})

            cur2.close()
            cur.close()
@@ -2337,7 +2512,9 @@ class AuthSrv(object):

        self.js_ls = {}
        self.js_htm = {}
-        for vn in self.vfs.all_nodes.values():
+        for vp, vn in self.vfs.all_nodes.items():
+            if enshare and vp.startswith(shrs):
+                continue  # propagates later in this func
            vf = vn.flags
            vn.js_ls = {
                "idx": "e2d" in vf,
@@ -2354,6 +2531,7 @@ class AuthSrv(object):
                "sb_lg": "" if "no_sb_lg" in vf else (vf.get("lg_sbf") or "y"),
            }
            js_htm = {
+                "SPINNER": self.args.spinner,
                "s_name": self.args.bname,
                "have_up2k_idx": "e2d" in vf,
                "have_acode": not self.args.no_acode,
@@ -2363,6 +2541,7 @@ class AuthSrv(object):
                "have_del": not self.args.no_del,
                "have_unpost": int(self.args.unpost),
                "have_emp": self.args.emp,
+                "ext_th": vf.get("ext_th_d") or {},
                "sb_md": "" if "no_sb_md" in vf else (vf.get("md_sbf") or "y"),
                "sba_md": vf.get("md_sba") or "",
                "sba_lg": vf.get("lg_sba") or "",
@@ -2383,6 +2562,7 @@ class AuthSrv(object):
                "u2j": self.args.u2j,
                "u2sz": self.args.u2sz,
                "u2ts": vf["u2ts"],
+                "u2ow": vf["u2ow"],
                "frand": bool(vf.get("rand")),
                "lifetime": vn.js_ls["lifetime"],
                "u2sort": self.args.u2sort,
@@ -2392,8 +2572,12 @@ class AuthSrv(object):
        vols = list(vfs.all_nodes.values())
        if enshare:
            assert shv  # type: ignore  # !rm
-            vols.append(shv)
-            vols.extend(list(shv.nodes.values()))
+            for vol in shv.nodes.values():
+                if vol.vpath not in vfs.all_nodes:
+                    self.log("BUG: /%s not in all_nodes" % (vol.vpath,), 1)
+                    vols.append(vol)
+            if shr in vfs.all_nodes:
+                self.log("BUG: %s found in all_nodes" % (shr,), 1)

        for vol in vols:
            dbv = vol.get_dbv("")[0]
@@ -2496,8 +2680,8 @@ class AuthSrv(object):
            if not bos.path.exists(ap):
                pwdb = {}
            else:
-                with open(ap, "r", encoding="utf-8") as f:
-                    pwdb = json.load(f)
+                jtxt = read_utf8(self.log, ap, True)
+                pwdb = json.loads(jtxt)

            pwdb = [x for x in pwdb if x[0] != uname]
            pwdb.append((uname, self.defpw[uname], hpw))
@@ -2520,8 +2704,8 @@ class AuthSrv(object):
        if not self.args.chpw or not bos.path.exists(ap):
            return

-        with open(ap, "r", encoding="utf-8") as f:
-            pwdb = json.load(f)
+        jtxt = read_utf8(self.log, ap, True)
+        pwdb = json.loads(jtxt)

        useen = set()
        urst = set()
@@ -2635,7 +2819,7 @@ class AuthSrv(object):
    def dbg_ls(self) -> None:
        users = self.args.ls
        vol = "*"
-        flags: list[str] = []
+        flags: Sequence[str] = []

        try:
            users, vol = users.split(",", 1)
@@ -2767,7 +2951,9 @@ class AuthSrv(object):
        zs = "c ihead ohead mtm mtp on403 on404 xac xad xar xau xiu xban xbc xbd xbr xbu xm"
        lst = set(zs.split())
        askip = set("a v c vc cgen exp_lg exp_md theme".split())
-        fskip = set("exp_lg exp_md mv_re_r mv_re_t rm_re_r rm_re_t".split())
+
+        t = "exp_lg exp_md ext_th_d mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot"
+        fskip = set(t.split())

        # keymap from argv to vflag
        amap = vf_bmap()
@@ -3015,8 +3201,9 @@ def expand_config_file(
    ipath += " -> " + fp
    ret.append("#\033[36m opening cfg file{}\033[0m".format(ipath))

-    with open(fp, "rb") as f:
-        for oln in [x.decode("utf-8").rstrip() for x in f]:
+    cfg_lines = read_utf8(log, fp, True).split("\n")
+    if True:  # diff-golf
+        for oln in [x.rstrip() for x in cfg_lines]:
            ln = oln.split("  #")[0].strip()
            if ln.startswith("% "):
                pad = " " * len(oln.split("%")[0])
--- a/copyparty/cfg.py
+++ b/copyparty/cfg.py
@@ -5,6 +5,9 @@ from __future__ import print_function, unicode_literals
 zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
 onedash = set(zs.split())

+# verify that all volflags are documented here:
+# grep volflag= __main__.py | sed -r 's/.*volflag=//;s/\).*//' | sort | uniq | while IFS= read -r x; do grep -E "\"$x(=[^ \"]+)?\": \"" cfg.py || printf '%s\n' "$x"; done
+

 def vf_bmap() -> dict[str, str]:
    """argv-to-volflag: simple bools"""
@@ -40,6 +43,7 @@ def vf_bmap() -> dict[str, str]:
        "gsel",
        "hardlink",
        "magic",
+        "no_db_ip",
        "no_sb_md",
        "no_sb_lg",
        "nsort",
@@ -48,9 +52,11 @@ def vf_bmap() -> dict[str, str]:
        "og_s_title",
        "rand",
        "rss",
+        "wo_up_readme",
        "xdev",
        "xlink",
        "xvol",
+        "zipmaxu",
    ):
        ret[k] = k
    return ret
@@ -70,13 +76,16 @@ def vf_vmap() -> dict[str, str]:
    }
    for k in (
        "dbd",
+        "forget_ip",
        "hsortn",
        "html_head",
        "lg_sbf",
        "md_sbf",
        "lg_sba",
        "md_sba",
+        "md_hist",
        "nrand",
+        "u2ow",
        "og_desc",
        "og_site",
        "og_th",
@@ -95,6 +104,9 @@ def vf_vmap() -> dict[str, str]:
        "u2ts",
        "ups_who",
        "zip_who",
+        "zipmaxn",
+        "zipmaxs",
+        "zipmaxt",
    ):
        ret[k] = k
    return ret
@@ -106,6 +118,7 @@ def vf_cmap() -> dict[str, str]:
    for k in (
        "exp_lg",
        "exp_md",
+        "ext_th",
        "mte",
        "mth",
        "mtp",
@@ -152,7 +165,8 @@ flagcats = {
        "daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
        "nosub": "forces all uploads into the top folder of the vfs",
        "magic": "enables filetype detection for nameless uploads",
-        "gz": "allows server-side gzip of uploads with ?gz (also c,xz)",
+        "gz": "allows server-side gzip compression of uploads with ?gz",
+        "xz": "allows server-side lzma compression of uploads with ?xz",
        "pk": "forces server-side compression, optional arg: xz,9",
    },
    "upload rules": {
@@ -161,8 +175,10 @@ flagcats = {
        "vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
        "vmaxn=4k": "max 4096 files in volume (suffixes: b, k, m, g, t)",
        "medialinks": "return medialinks for non-up2k uploads (not hotlinks)",
+        "wo_up_readme": "write-only users can upload logues without getting renamed",
        "rand": "force randomized filenames, 9 chars long by default",
        "nrand=N": "randomized filenames are N chars long",
+        "u2ow=N": "overwrite existing files? 0=no 1=if-older 2=always",
        "u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
        "u2abort=1": "allow aborting unfinished uploads? 0=no 1=strict 2=ip-chk 3=acct-chk",
        "sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
@@ -179,17 +195,23 @@ flagcats = {
        "e2dsa": "scans all folders for new files on startup; also sets -e2d",
        "e2t": "enable multimedia indexing; makes it possible to search for tags",
        "e2ts": "scan existing files for tags on startup; also sets -e2t",
-        "e2tsa": "delete all metadata from DB (full rescan); also sets -e2ts",
+        "e2tsr": "delete all metadata from DB (full rescan); also sets -e2ts",
        "d2ts": "disables metadata collection for existing files",
+        "e2v": "verify integrity on startup by hashing files and comparing to db",
+        "e2vu": "when e2v fails, update the db (assume on-disk files are good)",
+        "e2vp": "when e2v fails, panic and quit copyparty",
        "d2ds": "disables onboot indexing, overrides -e2ds*",
        "d2t": "disables metadata collection, overrides -e2t*",
        "d2v": "disables file verification, overrides -e2v*",
        "d2d": "disables all database stuff, overrides -e2*",
        "hist=/tmp/cdb": "puts thumbnails and indexes at that location",
+        "dbpath=/tmp/cdb": "puts indexes at that location",
        "scan=60": "scan for new files every 60sec, same as --re-maxage",
        "nohash=\\.iso$": "skips hashing file contents if path matches *.iso",
        "noidx=\\.iso$": "fully ignores the contents at paths matching *.iso",
        "noforget": "don't forget files when deleted from disk",
+        "forget_ip=43200": "forget uploader-IP after 30 days (GDPR)",
+        "no_db_ip": "never store uploader-IP in the db; disables unpost",
        "fat32": "avoid excessive reindexing on android sdcardfs",
        "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
        "xlink": "cross-volume dupe detection / linking (dangerous)",
@@ -200,6 +222,8 @@ flagcats = {
        "srch_excl": "exclude search results with URL matching this regex",
    },
    'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
+        "mte=artist,title": "media-tags to index/display",
+        "mth=fmt,res,ac": "media-tags to hide by default",
        "mtp=.bpm=f,audio-bpm.py": 'uses the "audio-bpm.py" program to\ngenerate ".bpm" tags from uploads (f = overwrite tags)',
        "mtp=ahash,vhash=media-hash.py": "collects two tags at once",
    },
@@ -213,6 +237,7 @@ flagcats = {
        "crop": "center-cropping (y/n/fy/fn)",
        "th3x": "3x resolution (y/n/fy/fn)",
        "convt": "conversion timeout in seconds",
+        "ext_th=s=/b.png": "use /b.png as thumbnail for file-extension s",
    },
    "handlers\n(better explained in --help-handlers)": {
        "on404=PY": "handle 404s by executing PY file",
@@ -235,8 +260,12 @@ flagcats = {
        "grid": "show grid/thumbnails by default",
        "gsel": "select files in grid by ctrl-click",
        "sort": "default sort order",
+        "nsort": "natural-sort of leading digits in filenames",
+        "hsortn": "number of sort-rules to add to media URLs",
        "unlist": "dont list files matching REGEX",
        "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
+        "tcolor=#fc0": "theme color (a hint for webbrowsers, discord, etc.)",
+        "nodirsz": "don't show total folder size",
        "robots": "allows indexing by search engines (default)",
        "norobots": "kindly asks search engines to leave",
        "no_sb_md": "disable js sandbox for markdown files",
@@ -249,12 +278,41 @@ flagcats = {
        "lg_sba": "value of iframe allow-prop for *logue-sandbox",
        "nohtml": "return html and markdown as text/html",
    },
+    "opengraph (discord embeds)": {
+        "og": "enable OG (disables hotlinking)",
+        "og_site": "sitename; defaults to --name, disable with '-'",
+        "og_desc": "description text for all files; disable with '-'",
+        "og_th=jf": "thumbnail format; j / jf / jf3 / w / w3 / ...",
+        "og_title_a": "audio title format; default: {{ artist }} - {{ title }}",
+        "og_title_v": "video title format; default: {{ title }}",
+        "og_title_i": "image title format; default: {{ title }}",
+        "og_title=foo": "fallback title if there's nothing in the db",
+        "og_s_title": "force default title; do not read from tags",
+        "og_tpl": "custom html; see --og-tpl in --help",
+        "og_no_head": "you want to add tags manually with og_tpl",
+        "og_ua": "if defined: only send OG html if useragent matches this regex",
+    },
+    "textfiles": {
+        "md_hist": "where to put markdown backups; s=subfolder, v=volHist, n=nope",
+        "exp": "enable textfile expansion; see --help-exp",
+        "exp_md": "placeholders to expand in markdown files; see --help",
+        "exp_lg": "placeholders to expand in prologue/epilogue; see --help",
+    },
    "others": {
        "dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
        "fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
        "fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
+        "dk=8": 'generates per-directory accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
+        "dks": "per-directory accesskeys allow browsing into subdirs",
+        "dky": 'allow seeing files (not folders) inside a specific folder\nwith "g" perm, and does not require a valid dirkey to do so',
+        "rss": "allow '?rss' URL suffix (experimental)",
        "ups_who=2": "restrict viewing the list of recent uploads",
        "zip_who=2": "restrict access to download-as-zip/tar",
+        "zipmaxn=9k": "reject download-as-zip if more than 9000 files",
+        "zipmaxs=2g": "reject download-as-zip if size over 2 GiB",
+        "zipmaxt=no": "reply with 'no' if download-as-zip exceeds max",
+        "zipmaxu": "zip-size-limit does not apply to authenticated users",
+        "nopipe": "disable race-the-beam (download unfinished uploads)",
        "mv_retry": "ms-windows: timeout for renaming busy files",
        "rm_retry": "ms-windows: timeout for deleting busy files",
        "davauth": "ask webdav clients to login for all folders",
@@ -264,3 +322,10 @@ flagcats = {


 flagdescs = {k.split("=")[0]: v for tab in flagcats.values() for k, v in tab.items()}
+
+
+if True:  # so it gets removed in release-builds
+    for fun in [vf_bmap, vf_cmap, vf_vmap]:
+        for k in fun().values():
+            if k not in flagdescs:
+                raise Exception("undocumented volflag: " + k)
--- a/copyparty/fsutil.py
+++ b/copyparty/fsutil.py
@@ -78,7 +78,7 @@ class Fstab(object):
            return vid

    def build_fallback(self) -> None:
-        self.tab = VFS(self.log_func, "idk", "/", AXS(), {})
+        self.tab = VFS(self.log_func, "idk", "/", "/", AXS(), {})
        self.trusted = False

    def build_tab(self) -> None:
@@ -111,9 +111,10 @@ class Fstab(object):

        tab1.sort(key=lambda x: (len(x[0]), x[0]))
        path1, fs1 = tab1[0]
-        tab = VFS(self.log_func, fs1, path1, AXS(), {})
+        tab = VFS(self.log_func, fs1, path1, path1, AXS(), {})
        for path, fs in tab1[1:]:
-            tab.add(fs, path.lstrip("/"))
+            zs = path.lstrip("/")
+            tab.add(fs, zs, zs)

        self.tab = tab
        self.srctab = srctab
@@ -130,9 +131,10 @@ class Fstab(object):
        if not self.trusted:
            # no mtab access; have to build as we go
            if "/" in rem:
-                self.tab.add("idk", os.path.join(vn.vpath, rem.split("/")[0]))
+                zs = os.path.join(vn.vpath, rem.split("/")[0])
+                self.tab.add("idk", zs, zs)
            if rem:
-                self.tab.add(nval, path)
+                self.tab.add(nval, path, path)
            else:
                vn.realpath = nval

--- a/copyparty/ftpd.py
+++ b/copyparty/ftpd.py
@@ -19,6 +19,7 @@ from .__init__ import PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
+    FN_EMB,
    VF_CAREFUL,
    Daemon,
    ODict,
@@ -170,6 +171,16 @@ class FtpFs(AbstractedFS):
            fn = sanitize_fn(fn or "", "")
            vpath = vjoin(rd, fn)
            vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+            if (
+                w
+                and fn.lower() in FN_EMB
+                and self.h.uname not in vfs.axs.uread
+                and "wo_up_readme" not in vfs.flags
+            ):
+                fn = "_wo_" + fn
+                vpath = vjoin(rd, fn)
+                vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
+
            if not vfs.realpath:
                t = "No filesystem mounted at [{}]"
                raise FSE(t.format(vpath))
--- a/copyparty/httpcli.py
+++ b/copyparty/httpcli.py
@@ -4,7 +4,6 @@ from __future__ import print_function, unicode_literals
 import argparse  # typechk
 import copy
 import errno
-import gzip
 import hashlib
 import itertools
 import json
@@ -22,6 +21,7 @@ from datetime import datetime
 from operator import itemgetter

 import jinja2  # typechk
+from ipaddress import IPv6Network

 try:
    if os.environ.get("PRTY_NO_LZMA"):
@@ -45,6 +45,7 @@ from .util import (
    APPLESAN_RE,
    BITNESS,
    DAV_ALLPROPS,
+    FN_EMB,
    HAVE_SQLITE3,
    HTTPCODE,
    META_NOBOTS,
@@ -56,6 +57,7 @@ from .util import (
    UnrecvEOF,
    WrongPostKey,
    absreal,
+    afsenc,
    alltrace,
    atomic_move,
    b64dec,
@@ -68,6 +70,7 @@ from .util import (
    get_df,
    get_spd,
    guess_mime,
+    gzip,
    gzip_file_orig_sz,
    gzip_orig_sz,
    has_resource,
@@ -89,6 +92,7 @@ from .util import (
    read_socket,
    read_socket_chunked,
    read_socket_unbounded,
+    read_utf8,
    relchk,
    ren_open,
    runhook,
@@ -152,6 +156,8 @@ RE_HSAFE = re.compile(r"[\x00-\x1f<>\"'&]")  # search always much faster
 RE_HOST = re.compile(r"[^][0-9a-zA-Z.:_-]")  # search faster <=17ch
 RE_MHOST = re.compile(r"^[][0-9a-zA-Z.:_-]+$")  # match faster >=18ch
 RE_K = re.compile(r"[^0-9a-zA-Z_-]")  # search faster <=17ch
+RE_HR = re.compile(r"[<>\"'&]")
+RE_MDV = re.compile(r"(.*)\.([0-9]+\.[0-9]{3})(\.[Mm][Dd])$")

 UPARAM_CC_OK = set("doc move tree".split())

@@ -191,7 +197,7 @@ class HttpCli(object):
        self.is_vproxied = False
        self.in_hdr_recv = True
        self.headers: dict[str, str] = {}
-        self.mode = " "
+        self.mode = " "  # http verb
        self.req = " "
        self.http_ver = ""
        self.hint = ""
@@ -385,11 +391,12 @@ class HttpCli(object):
                        t += '  Note: if you are behind cloudflare, then this default header is not a good choice; please first make sure your local reverse-proxy (if any) does not allow non-cloudflare IPs from providing cf-* headers, and then add this additional global setting: "--xff-hdr=cf-connecting-ip"'
                    else:
                        t += '  Note: depending on your reverse-proxy, and/or WAF, and/or other intermediates, you may want to read the true client IP from another header by also specifying "--xff-hdr=SomeOtherHeader"'
-                    zs = (
-                        ".".join(pip.split(".")[:2]) + "."
-                        if "." in pip
-                        else ":".join(pip.split(":")[:4]) + ":"
-                    ) + "0.0/16"
+
+                    if "." in pip:
+                        zs = ".".join(pip.split(".")[:2]) + ".0.0/16"
+                    else:
+                        zs = IPv6Network(pip + "/64", False).compressed
+
                    zs2 = ' or "--xff-src=lan"' if self.conn.xff_lan.map(pip) else ""
                    self.log(t % (self.args.xff_hdr, pip, cli_ip, zso, zs, zs2), 3)
                    self.bad_xff = True
@@ -731,10 +738,10 @@ class HttpCli(object):
                return self.handle_unlock() and self.keepalive
            elif self.mode == "MKCOL":
                return self.handle_mkcol() and self.keepalive
-            elif self.mode == "MOVE":
-                return self.handle_move() and self.keepalive
+            elif self.mode in ("MOVE", "COPY"):
+                return self.handle_cpmv() and self.keepalive
            else:
-                raise Pebkac(400, 'invalid HTTP mode "{0}"'.format(self.mode))
+                raise Pebkac(400, 'invalid HTTP verb "{0}"'.format(self.mode))

        except Exception as ex:
            if not isinstance(ex, Pebkac):
@@ -866,8 +873,7 @@ class HttpCli(object):
            html = html.replace("%", "", 1)

        if html.startswith("@"):
-            with open(html[1:], "rb") as f:
-                html = f.read().decode("utf-8")
+            html = read_utf8(self.log, html[1:], True)

        if html.startswith("%"):
            html = html[1:]
@@ -1200,11 +1206,6 @@ class HttpCli(object):
                else:
                    return self.tx_res(res_path)

-            if res_path != undot(res_path):
-                t = "malicious user; attempted path traversal; req(%r) vp(%r) => %r"
-                self.log(t % (self.req, "/" + self.vpath, res_path), 1)
-                self.cbonk(self.conn.hsrv.gmal, self.req, "trav", "path traversal")
-
            self.tx_404()
            return False

@@ -1234,14 +1235,7 @@ class HttpCli(object):
                    return self.tx_404(True)
            else:
                vfs = self.asrv.vfs
-                if (
-                    not vfs.nodes
-                    and not vfs.axs.uread
-                    and not vfs.axs.uwrite
-                    and not vfs.axs.uget
-                    and not vfs.axs.uhtml
-                    and not vfs.axs.uadmin
-                ):
+                if vfs.badcfg1:
                    t = "<h2>access denied due to failsafe; check server log</h2>"
                    html = self.j2s("splash", this=self, msg=t)
                    self.reply(html.encode("utf-8", "replace"), 500)
@@ -1776,6 +1770,12 @@ class HttpCli(object):
            if "%" in self.req:
                self.log("  `-- %r" % (self.vpath,))

+        if self.args.no_dav:
+            raise Pebkac(405, "WebDAV is disabled in server config")
+
+        if not self.can_write:
+            raise Pebkac(401, "authenticate")
+
        try:
            return self._mkdir(self.vpath, True)
        except Pebkac as ex:
@@ -1785,14 +1785,36 @@ class HttpCli(object):
            self.reply(b"", ex.code)
            return True

-    def handle_move(self) -> bool:
+    def handle_cpmv(self) -> bool:
        dst = self.headers["destination"]
-        dst = re.sub("^https?://[^/]+", "", dst).lstrip()
-        dst = unquotep(dst)
-        if not self._mv(self.vpath, dst.lstrip("/")):
-            return False

-        return True
+        # dolphin (kioworker/6.10) "webdav://127.0.0.1:3923/a/b.txt"
+        dst = re.sub("^[a-zA-Z]+://[^/]+", "", dst).lstrip()
+
+        if self.is_vproxied and dst.startswith(self.args.SRS):
+            dst = dst[len(self.args.RS) :]
+
+        if self.do_log:
+            self.log("%s %s  --//>  %s @%s" % (self.mode, self.req, dst, self.uname))
+            if "%" in self.req:
+                self.log("  `-- %r" % (self.vpath,))
+
+        if self.args.no_dav:
+            raise Pebkac(405, "WebDAV is disabled in server config")
+
+        dst = unquotep(dst)
+
+        # overwrite=True is default; rfc4918 9.8.4
+        zs = self.headers.get("overwrite", "").lower()
+        overwrite = zs not in ["f", "false"]
+
+        try:
+            fun = self._cp if self.mode == "COPY" else self._mv
+            return fun(self.vpath, dst.lstrip("/"), overwrite)
+        except Pebkac as ex:
+            if ex.code == 403:
+                ex.code = 401
+            raise

    def _applesan(self) -> bool:
        if self.args.dav_mac or "Darwin/" not in self.ua:
@@ -2525,6 +2547,16 @@ class HttpCli(object):
        vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
        dbv, vrem = vfs.get_dbv(rem)

+        name = sanitize_fn(name, "")
+        if (
+            not self.can_read
+            and self.can_write
+            and name.lower() in FN_EMB
+            and "wo_up_readme" not in dbv.flags
+        ):
+            name = "_wo_" + name
+
+        body["name"] = name
        body["vtop"] = dbv.vpath
        body["ptop"] = dbv.realpath
        body["prel"] = vrem
@@ -2962,9 +2994,6 @@ class HttpCli(object):
        vfs, rem = self.asrv.vfs.get(vpath, self.uname, False, True)
        rem = sanitize_vpath(rem, "/")
        fn = vfs.canonical(rem)
-        if not fn.startswith(vfs.realpath):
-            self.log("invalid mkdir %r %r" % (self.gctx, vpath), 1)
-            raise Pebkac(422)

        if not nullwrite:
            fdir = os.path.dirname(fn)
@@ -3465,6 +3494,7 @@ class HttpCli(object):

        fp = os.path.join(fp, fn)
        rem = "{}/{}".format(rp, fn).strip("/")
+        dbv, vrem = vfs.get_dbv(rem)

        if not rem.endswith(".md") and not self.can_delete:
            raise Pebkac(400, "only markdown pls")
@@ -3519,13 +3549,27 @@ class HttpCli(object):
            mdir, mfile = os.path.split(fp)
            fname, fext = mfile.rsplit(".", 1) if "." in mfile else (mfile, "md")
            mfile2 = "{}.{:.3f}.{}".format(fname, srv_lastmod, fext)
-            try:
+
+            dp = ""
+            hist_cfg = dbv.flags["md_hist"]
+            if hist_cfg == "v":
+                vrd = vsplit(vrem)[0]
+                zb = hashlib.sha512(afsenc(vrd)).digest()
+                zs = ub64enc(zb).decode("ascii")[:24].lower()
+                dp = "%s/md/%s/%s/%s" % (dbv.histpath, zs[:2], zs[2:4], zs)
+                self.log("moving old version to %s/%s" % (dp, mfile2))
+                if bos.makedirs(dp):
+                    with open(os.path.join(dp, "dir.txt"), "wb") as f:
+                        f.write(afsenc(vrd))
+            elif hist_cfg == "s":
                dp = os.path.join(mdir, ".hist")
-                bos.mkdir(dp)
-                hidedir(dp)
-            except:
-                pass
-            wrename(self.log, fp, os.path.join(mdir, ".hist", mfile2), vfs.flags)
+                try:
+                    bos.mkdir(dp)
+                    hidedir(dp)
+                except:
+                    pass
+            if dp:
+                wrename(self.log, fp, os.path.join(dp, mfile2), vfs.flags)

        assert self.parser.gen  # !rm
        p_field, _, p_data = next(self.parser.gen)
@@ -3598,13 +3642,12 @@ class HttpCli(object):
            wunlink(self.log, fp, vfs.flags)
            raise Pebkac(403, t)

-        vfs, rem = vfs.get_dbv(rem)
        self.conn.hsrv.broker.say(
            "up2k.hash_file",
-            vfs.realpath,
-            vfs.vpath,
-            vfs.flags,
-            vsplit(rem)[0],
+            dbv.realpath,
+            dbv.vpath,
+            dbv.flags,
+            vsplit(vrem)[0],
            fn,
            self.ip,
            new_lastmod,
@@ -3708,8 +3751,7 @@ class HttpCli(object):
                    continue
                fn = "%s/%s" % (abspath, fn)
                if bos.path.isfile(fn):
-                    with open(fsenc(fn), "rb") as f:
-                        logues[n] = f.read().decode("utf-8")
+                    logues[n] = read_utf8(self.log, fsenc(fn), False)
                    if "exp" in vn.flags:
                        logues[n] = self._expand(
                            logues[n], vn.flags.get("exp_lg") or []
@@ -3730,9 +3772,8 @@ class HttpCli(object):
            for fn in fns:
                fn = "%s/%s" % (abspath, fn)
                if bos.path.isfile(fn):
-                    with open(fsenc(fn), "rb") as f:
-                        txt = f.read().decode("utf-8")
-                        break
+                    txt = read_utf8(self.log, fsenc(fn), False)
+                    break

            if txt and "exp" in vn.flags:
                txt = self._expand(txt, vn.flags.get("exp_md") or [])
@@ -3765,6 +3806,19 @@ class HttpCli(object):

        return txt

+    def _can_zip(self, volflags: dict[str, Any]) -> str:
+        lvl = volflags["zip_who"]
+        if self.args.no_zip or not lvl:
+            return "download-as-zip/tar is disabled in server config"
+        elif lvl <= 1 and not self.can_admin:
+            return "download-as-zip/tar is admin-only on this server"
+        elif lvl <= 2 and self.uname in ("", "*"):
+            return "you must be authenticated to download-as-zip/tar on this server"
+        elif self.args.ua_nozip and self.args.ua_nozip.search(self.ua):
+            t = "this URL contains no valuable information for bots/crawlers"
+            raise Pebkac(403, t)
+        return ""
+
    def tx_res(self, req_path: str) -> bool:
        status = 200
        logmsg = "{:4} {} ".format("", self.req)
@@ -4192,6 +4246,7 @@ class HttpCli(object):
                self.log(t % (data_end / M, lower / M, upper / M), 6)
                with self.u2mutex:
                    if data_end > self.u2fh.aps.get(ap_data, data_end):
+                        fhs: Optional[set[typing.BinaryIO]] = None
                        try:
                            fhs = self.u2fh.cache[ap_data].all_fhs
                            for fh in fhs:
@@ -4199,7 +4254,11 @@ class HttpCli(object):
                            self.u2fh.aps[ap_data] = data_end
                            self.log("pipe: flushed %d up2k-FDs" % (len(fhs),))
                        except Exception as ex:
-                            self.log("pipe: u2fh flush failed: %r" % (ex,))
+                            if fhs is None:
+                                err = "file is not being written to right now"
+                            else:
+                                err = repr(ex)
+                            self.log("pipe: u2fh flush failed: " + err)

            if lower >= data_end:
                if data_end:
@@ -4297,13 +4356,8 @@ class HttpCli(object):
        rem: str,
        items: list[str],
    ) -> bool:
-        lvl = vn.flags["zip_who"]
-        if self.args.no_zip or not lvl:
-            raise Pebkac(400, "download-as-zip/tar is disabled in server config")
-        elif lvl <= 1 and not self.can_admin:
-            raise Pebkac(400, "download-as-zip/tar is admin-only on this server")
-        elif lvl <= 2 and self.uname in ("", "*"):
-            t = "you must be authenticated to download-as-zip/tar on this server"
+        t = self._can_zip(vn.flags)
+        if t:
            raise Pebkac(400, t)

        logmsg = "{:4} {} ".format("", self.req)
@@ -4336,6 +4390,33 @@ class HttpCli(object):
        else:
            fn = self.host.split(":")[0]

+        if vn.flags.get("zipmax") and (not self.uname or not "zipmaxu" in vn.flags):
+            maxs = vn.flags.get("zipmaxs_v") or 0
+            maxn = vn.flags.get("zipmaxn_v") or 0
+            nf = 0
+            nb = 0
+            fgen = vn.zipgen(
+                vpath, rem, set(items), self.uname, False, not self.args.no_scandir
+            )
+            t = "total size exceeds a limit specified in server config"
+            t = vn.flags.get("zipmaxt") or t
+            if maxs and maxn:
+                for zd in fgen:
+                    nf += 1
+                    nb += zd["st"].st_size
+                    if maxs < nb or maxn < nf:
+                        raise Pebkac(400, t)
+            elif maxs:
+                for zd in fgen:
+                    nb += zd["st"].st_size
+                    if maxs < nb:
+                        raise Pebkac(400, t)
+            elif maxn:
+                for zd in fgen:
+                    nf += 1
+                    if maxn < nf:
+                        raise Pebkac(400, t)
+
        safe = (string.ascii_letters + string.digits).replace("%", "")
        afn = "".join([x if x in safe.replace('"', "") else "_" for x in fn])
        bascii = unicode(safe).encode("utf-8")
@@ -4801,7 +4882,7 @@ class HttpCli(object):
            self.reply(pt.encode("utf-8"), status=rc)
            return True

-        if "th" in self.ouparam:
+        if "th" in self.ouparam and str(self.ouparam["th"])[:1] in "jw":
            return self.tx_svg("e" + pt[:3])

        # most webdav clients will not send credentials until they
@@ -4809,9 +4890,12 @@ class HttpCli(object):
        # that the client is not a graphical browser
        if (
            rc == 403
-            and not self.pw
-            and not self.ua.startswith("Mozilla/")
+            and self.uname == "*"
            and "sec-fetch-site" not in self.headers
+            and (
+                not self.ua.startswith("Mozilla/")
+                or (self.args.dav_ua1 and self.args.dav_ua1.search(self.ua))
+            )
        ):
            rc = 401
            self.out_headers["WWW-Authenticate"] = 'Basic realm="a"'
@@ -4979,6 +5063,8 @@ class HttpCli(object):
    def get_dls(self) -> list[list[Any]]:
        ret = []
        dls = self.conn.hsrv.tdls
+        enshare = self.args.shr
+        shrs = enshare[1:]
        for dl_id, (t0, sz, vn, vp, uname) in self.conn.hsrv.tdli.items():
            t1, sent = dls[dl_id]
            if sent > 0x100000:  # 1m; buffers 2~4
@@ -4987,6 +5073,15 @@ class HttpCli(object):
                vp = ""
            elif self.uname not in vn.axs.udot and (vp.startswith(".") or "/." in vp):
                vp = ""
+            elif (
+                enshare
+                and vp.startswith(shrs)
+                and self.uname != vn.shr_owner
+                and self.uname not in vn.axs.uadmin
+                and self.uname not in self.args.shr_adm
+                and not dl_id.startswith(self.ip + ":")
+            ):
+                vp = ""
            if self.uname not in vn.axs.uadmin:
                dl_id = uname = ""

@@ -5441,6 +5536,8 @@ class HttpCli(object):

    def handle_rm(self, req: list[str]) -> bool:
        if not req and not self.can_delete:
+            if self.mode == "DELETE" and self.uname == "*":
+                raise Pebkac(401, "authenticate")  # webdav
            raise Pebkac(403, "'delete' not allowed for user " + self.uname)

        if self.args.no_del:
@@ -5475,14 +5572,22 @@ class HttpCli(object):
        if not dst:
            raise Pebkac(400, "need dst vpath")

-        return self._mv(self.vpath, dst.lstrip("/"))
+        return self._mv(self.vpath, dst.lstrip("/"), False)

-    def _mv(self, vsrc: str, vdst: str) -> bool:
+    def _mv(self, vsrc: str, vdst: str, overwrite: bool) -> bool:
        if self.args.no_mv:
            raise Pebkac(403, "the rename/move feature is disabled in server config")

-        self.asrv.vfs.get(vsrc, self.uname, True, False, True)
-        self.asrv.vfs.get(vdst, self.uname, False, True)
+        # `handle_cpmv` will catch 403 from these and raise 401
+        svn, srem = self.asrv.vfs.get(vsrc, self.uname, True, False, True)
+        dvn, drem = self.asrv.vfs.get(vdst, self.uname, False, True)
+
+        if overwrite:
+            dabs = dvn.canonical(drem)
+            if bos.path.exists(dabs):
+                self.log("overwriting %s" % (dabs,))
+                self.asrv.vfs.get(vdst, self.uname, False, True, False, True)
+                wunlink(self.log, dabs, dvn.flags)

        x = self.conn.hsrv.broker.ask("up2k.handle_mv", self.uname, self.ip, vsrc, vdst)
        self.loud_reply(x.get(), status=201)
@@ -5498,14 +5603,21 @@ class HttpCli(object):
        if not dst:
            raise Pebkac(400, "need dst vpath")

-        return self._cp(self.vpath, dst.lstrip("/"))
+        return self._cp(self.vpath, dst.lstrip("/"), False)

-    def _cp(self, vsrc: str, vdst: str) -> bool:
+    def _cp(self, vsrc: str, vdst: str, overwrite: bool) -> bool:
        if self.args.no_cp:
            raise Pebkac(403, "the copy feature is disabled in server config")

-        self.asrv.vfs.get(vsrc, self.uname, True, False)
-        self.asrv.vfs.get(vdst, self.uname, False, True)
+        svn, srem = self.asrv.vfs.get(vsrc, self.uname, True, False)
+        dvn, drem = self.asrv.vfs.get(vdst, self.uname, False, True)
+
+        if overwrite:
+            dabs = dvn.canonical(drem)
+            if bos.path.exists(dabs):
+                self.log("overwriting %s" % (dabs,))
+                self.asrv.vfs.get(vdst, self.uname, False, True, False, True)
+                wunlink(self.log, dabs, dvn.flags)

        x = self.conn.hsrv.broker.ask("up2k.handle_cp", self.uname, self.ip, vsrc, vdst)
        self.loud_reply(x.get(), status=201)
@@ -5698,7 +5810,13 @@ class HttpCli(object):

                thp = None
                if self.thumbcli and not nothumb:
-                    thp = self.thumbcli.get(dbv, vrem, int(st.st_mtime), th_fmt)
+                    try:
+                        thp = self.thumbcli.get(dbv, vrem, int(st.st_mtime), th_fmt)
+                    except Pebkac as ex:
+                        if ex.code == 500 and th_fmt[:1] in "jw":
+                            self.log("failed to convert [%s]:\n%s" % (abspath, ex), 3)
+                            return self.tx_svg("--error--\ncheck\nserver\nlog")
+                        raise

                if thp:
                    return self.tx_file(thp)
@@ -5920,9 +6038,11 @@ class HttpCli(object):
        # check for old versions of files,
        # [num-backups, most-recent, hist-path]
        hist: dict[str, tuple[int, float, str]] = {}
-        histdir = os.path.join(fsroot, ".hist")
-        ptn = re.compile(r"(.*)\.([0-9]+\.[0-9]{3})(\.[^\.]+)$")
        try:
+            if vf["md_hist"] != "s":
+                raise Exception()
+            histdir = os.path.join(fsroot, ".hist")
+            ptn = RE_MDV
            for hfn in bos.listdir(histdir):
                m = ptn.match(hfn)
                if not m:
@@ -5952,8 +6072,11 @@ class HttpCli(object):
                zs = self.gen_fk(2, self.args.dk_salt, abspath, 0, 0)[:add_dk]
                ls_ret["dk"] = cgv["dk"] = zs

+        no_zip = bool(self._can_zip(vf))
+
        dirs = []
        files = []
+        ptn_hr = RE_HR
        for fn in ls_names:
            base = ""
            href = fn
@@ -5976,7 +6099,7 @@ class HttpCli(object):
            is_dir = stat.S_ISDIR(inf.st_mode)
            if is_dir:
                href += "/"
-                if self.args.no_zip:
+                if no_zip:
                    margin = "DIR"
                elif add_dk:
                    zs = absreal(fspath)
@@ -5989,7 +6112,7 @@ class HttpCli(object):
                        quotep(href),
                    )
            elif fn in hist:
-                margin = '<a href="%s.hist/%s">#%s</a>' % (
+                margin = '<a href="%s.hist/%s" rel="nofollow">#%s</a>' % (
                    base,
                    html_escape(hist[fn][2], quot=True, crlf=True),
                    hist[fn][0],
@@ -6008,11 +6131,13 @@ class HttpCli(object):
                zd.second,
            )

-            try:
-                ext = "---" if is_dir else fn.rsplit(".", 1)[1]
+            if is_dir:
+                ext = "---"
+            elif "." in fn:
+                ext = ptn_hr.sub("@", fn.rsplit(".", 1)[1])
                if len(ext) > 16:
                    ext = ext[:16]
-            except:
+            else:
                ext = "%"

            if add_fk and not is_dir:
@@ -6189,6 +6314,10 @@ class HttpCli(object):

        doc = self.uparam.get("doc") if self.can_read else None
        if doc:
+            zp = self.args.ua_nodoc
+            if zp and zp.search(self.ua):
+                t = "this URL contains no valuable information for bots/crawlers"
+                raise Pebkac(403, t)
            j2a["docname"] = doc
            doctxt = None
            dfn = lnames.get(doc.lower())
@@ -6199,9 +6328,7 @@ class HttpCli(object):
                docpath = os.path.join(abspath, doc)
                sz = bos.path.getsize(docpath)
                if sz < 1024 * self.args.txt_max:
-                    with open(fsenc(docpath), "rb") as f:
-                        doctxt = f.read().decode("utf-8", "replace")
-
+                    doctxt = read_utf8(self.log, fsenc(docpath), False)
                    if doc.lower().endswith(".md") and "exp" in vn.flags:
                        doctxt = self._expand(doctxt, vn.flags.get("exp_md") or [])
                else:
--- a/copyparty/ico.py
+++ b/copyparty/ico.py
@@ -94,10 +94,21 @@ class Ico(object):
 <?xml version="1.0" encoding="UTF-8"?>
 <svg version="1.1" viewBox="0 0 100 {}" xmlns="http://www.w3.org/2000/svg"><g>
 <rect width="100%" height="100%" fill="#{}" />
-<text x="50%" y="50%" dominant-baseline="middle" text-anchor="middle" xml:space="preserve"
+<text x="50%" y="{}" dominant-baseline="middle" text-anchor="middle" xml:space="preserve"
  fill="#{}" font-family="monospace" font-size="14px" style="letter-spacing:.5px">{}</text>
 </g></svg>
 """
-        svg = svg.format(h, c[:6], c[6:], html_escape(ext, True))
+
+        txt = html_escape(ext, True)
+        if "\n" in txt:
+            lines = txt.split("\n")
+            n = len(lines)
+            y = "20%" if n == 2 else "10%" if n == 3 else "0"
+            zs = '<tspan x="50%%" dy="1.2em">%s</tspan>'
+            txt = "".join([zs % (x,) for x in lines])
+        else:
+            y = "50%"
+
+        svg = svg.format(h, c[:6], y, c[6:], txt)

        return "image/svg+xml", svg.encode("utf-8")
--- a/copyparty/mtag.py
+++ b/copyparty/mtag.py
@@ -18,6 +18,7 @@ from .util import (
    REKOBO_LKEY,
    VF_CAREFUL,
    fsenc,
+    gzip,
    min_ex,
    pybin,
    retchk,
@@ -138,8 +139,6 @@ def au_unpk(
        fd, ret = tempfile.mkstemp("." + au)

        if pk == "gz":
-            import gzip
-
            fi = gzip.GzipFile(abspath, mode="rb")

        elif pk == "xz":
--- a/copyparty/pwhash.py
+++ b/copyparty/pwhash.py
@@ -15,7 +15,7 @@ try:
        raise Exception()

    HAVE_ARGON2 = True
-    from argon2 import __version__ as argon2ver
+    from argon2 import exceptions as argon2ex
 except:
    HAVE_ARGON2 = False

--- a/copyparty/svchub.py
+++ b/copyparty/svchub.py
@@ -3,7 +3,6 @@ from __future__ import print_function, unicode_literals

 import argparse
 import errno
-import gzip
 import logging
 import os
 import re
@@ -63,7 +62,9 @@ from .util import (
    ansi_re,
    build_netmap,
    expat_ver,
+    gzip,
    load_ipu,
+    lock_file,
    min_ex,
    mp,
    odfusion,
@@ -73,6 +74,9 @@ from .util import (
    ub64enc,
 )

+if HAVE_SQLITE3:
+    import sqlite3
+
 if TYPE_CHECKING:
    try:
        from .mdns import MDNS
@@ -84,6 +88,10 @@ if PY2:
    range = xrange  # type: ignore


+VER_SESSION_DB = 1
+VER_SHARES_DB = 2
+
+
 class SvcHub(object):
    """
    Hosts all services which cannot be parallelized due to reliance on monolithic resources.
@@ -186,8 +194,14 @@ class SvcHub(object):

        if not args.use_fpool and args.j != 1:
            args.no_fpool = True
-            t = "multithreading enabled with -j {}, so disabling fpool -- this can reduce upload performance on some filesystems"
-            self.log("root", t.format(args.j))
+            t = "multithreading enabled with -j {}, so disabling fpool -- this can reduce upload performance on some filesystems, and make some antivirus-softwares "
+            c = 0
+            if ANYWIN:
+                t += "(especially Microsoft Defender) stress your CPU and HDD severely during big uploads"
+                c = 3
+            else:
+                t += "consume more resources (CPU/HDD) than normal"
+            self.log("root", t.format(args.j), c)

        if not args.no_fpool and args.j != 1:
            t = "WARNING: ignoring --use-fpool because multithreading (-j{}) is enabled"
@@ -406,25 +420,49 @@ class SvcHub(object):
            self.log("root", t, 3)
            return

-        import sqlite3
+        assert sqlite3  # type: ignore  # !rm
+
+        # policy:
+        # the sessions-db is whatever, if something looks broken then just nuke it

-        create = True
        db_path = self.args.ses_db
-        self.log("root", "opening sessions-db %s" % (db_path,))
-        for n in range(2):
+        db_lock = db_path + ".lock"
+        try:
+            create = not os.path.getsize(db_path)
+        except:
+            create = True
+        zs = "creating new" if create else "opening"
+        self.log("root", "%s sessions-db %s" % (zs, db_path))
+
+        for tries in range(2):
+            sver = 0
            try:
                db = sqlite3.connect(db_path)
                cur = db.cursor()
                try:
+                    zs = "select v from kv where k='sver'"
+                    sver = cur.execute(zs).fetchall()[0][0]
+                    if sver > VER_SESSION_DB:
+                        zs = "this version of copyparty only understands session-db v%d and older; the db is v%d"
+                        raise Exception(zs % (VER_SESSION_DB, sver))
+
                    cur.execute("select count(*) from us").fetchone()
-                    create = False
-                    break
                except:
-                    pass
+                    if sver:
+                        raise
+                    sver = 1
+                    self._create_session_db(cur)
+                err = self._verify_session_db(cur, sver, db_path)
+                if err:
+                    tries = 99
+                    self.args.no_ses = True
+                    self.log("root", err, 3)
+                break
+
            except Exception as ex:
-                if n:
+                if tries or sver > VER_SESSION_DB:
                    raise
-                t = "sessions-db corrupt; deleting and recreating: %r"
+                t = "sessions-db is unusable; deleting and recreating: %r"
                self.log("root", t % (ex,), 3)
                try:
                    cur.close()  # type: ignore
@@ -434,8 +472,13 @@ class SvcHub(object):
                    db.close()  # type: ignore
                except:
                    pass
+                try:
+                    os.unlink(db_lock)
+                except:
+                    pass
                os.unlink(db_path)

+    def _create_session_db(self, cur: "sqlite3.Cursor") -> None:
        sch = [
            r"create table kv (k text, v int)",
            r"create table us (un text, si text, t0 int)",
@@ -445,17 +488,44 @@ class SvcHub(object):
            r"create index us_t0 on us(t0)",
            r"insert into kv values ('sver', 1)",
        ]
+        for cmd in sch:
+            cur.execute(cmd)
+        self.log("root", "created new sessions-db")

-        assert db  # type: ignore  # !rm
-        assert cur  # type: ignore  # !rm
-        if create:
-            for cmd in sch:
-                cur.execute(cmd)
-            self.log("root", "created new sessions-db")
-            db.commit()
+    def _verify_session_db(self, cur: "sqlite3.Cursor", sver: int, db_path: str) -> str:
+        # ensure writable (maybe owned by other user)
+        db = cur.connection

+        try:
+            zil = cur.execute("select v from kv where k='pid'").fetchall()
+            if len(zil) > 1:
+                raise Exception()
+            owner = zil[0][0]
+        except:
+            owner = 0
+
+        if not lock_file(db_path + ".lock"):
+            t = "the sessions-db [%s] is already in use by another copyparty instance (pid:%d). This is not supported; please provide another database with --ses-db or give this copyparty-instance its entirely separate config-folder by setting another path in the XDG_CONFIG_HOME env-var. You can also disable this safeguard by setting env-var PRTY_NO_DB_LOCK=1. Will now disable sessions and instead use plaintext passwords in cookies."
+            return t % (db_path, owner)
+
+        vars = (("pid", os.getpid()), ("ts", int(time.time() * 1000)))
+        if owner:
+            # wear-estimate: 2 cells; offsets 0x10, 0x50, 0x19720
+            for k, v in vars:
+                cur.execute("update kv set v=? where k=?", (v, k))
+        else:
+            # wear-estimate: 3~4 cells; offsets 0x10, 0x50, 0x19180, 0x19710, 0x36000, 0x360b0, 0x36b90
+            for k, v in vars:
+                cur.execute("insert into kv values(?, ?)", (k, v))
+
+        if sver < VER_SESSION_DB:
+            cur.execute("delete from kv where k='sver'")
+            cur.execute("insert into kv values('sver',?)", (VER_SESSION_DB,))
+
+        db.commit()
        cur.close()
        db.close()
+        return ""

    def setup_share_db(self) -> None:
        al = self.args
@@ -464,7 +534,7 @@ class SvcHub(object):
            al.shr = ""
            return

-        import sqlite3
+        assert sqlite3  # type: ignore  # !rm

        al.shr = al.shr.strip("/")
        if "/" in al.shr or not al.shr:
@@ -475,34 +545,48 @@ class SvcHub(object):
        al.shr = "/%s/" % (al.shr,)
        al.shr1 = al.shr[1:]

-        create = True
-        modified = False
+        # policy:
+        # the shares-db is important, so panic if something is wrong
+
        db_path = self.args.shr_db
-        self.log("root", "opening shares-db %s" % (db_path,))
-        for n in range(2):
-            try:
-                db = sqlite3.connect(db_path)
-                cur = db.cursor()
-                try:
-                    cur.execute("select count(*) from sh").fetchone()
-                    create = False
-                    break
-                except:
-                    pass
-            except Exception as ex:
-                if n:
-                    raise
-                t = "shares-db corrupt; deleting and recreating: %r"
-                self.log("root", t % (ex,), 3)
-                try:
-                    cur.close()  # type: ignore
-                except:
-                    pass
-                try:
-                    db.close()  # type: ignore
-                except:
-                    pass
-                os.unlink(db_path)
+        db_lock = db_path + ".lock"
+        try:
+            create = not os.path.getsize(db_path)
+        except:
+            create = True
+        zs = "creating new" if create else "opening"
+        self.log("root", "%s shares-db %s" % (zs, db_path))
+
+        sver = 0
+        try:
+            db = sqlite3.connect(db_path)
+            cur = db.cursor()
+            if not create:
+                zs = "select v from kv where k='sver'"
+                sver = cur.execute(zs).fetchall()[0][0]
+                if sver > VER_SHARES_DB:
+                    zs = "this version of copyparty only understands shares-db v%d and older; the db is v%d"
+                    raise Exception(zs % (VER_SHARES_DB, sver))
+
+                cur.execute("select count(*) from sh").fetchone()
+        except Exception as ex:
+            t = "could not open shares-db; will now panic...\nthe following database must be repaired or deleted before you can launch copyparty:\n%s\n\nERROR: %s\n\nadditional details:\n%s\n"
+            self.log("root", t % (db_path, ex, min_ex()), 1)
+            raise
+
+        try:
+            zil = cur.execute("select v from kv where k='pid'").fetchall()
+            if len(zil) > 1:
+                raise Exception()
+            owner = zil[0][0]
+        except:
+            owner = 0
+
+        if not lock_file(db_lock):
+            t = "the shares-db [%s] is already in use by another copyparty instance (pid:%d). This is not supported; please provide another database with --shr-db or give this copyparty-instance its entirely separate config-folder by setting another path in the XDG_CONFIG_HOME env-var. You can also disable this safeguard by setting env-var PRTY_NO_DB_LOCK=1. Will now panic."
+            t = t % (db_path, owner)
+            self.log("root", t, 1)
+            raise Exception(t)

        sch1 = [
            r"create table kv (k text, v int)",
@@ -514,34 +598,37 @@ class SvcHub(object):
            r"create index sf_k on sf(k)",
            r"create index sh_k on sh(k)",
            r"create index sh_t1 on sh(t1)",
+            r"insert into kv values ('sver', 2)",
        ]

        assert db  # type: ignore  # !rm
        assert cur  # type: ignore  # !rm
-        if create:
-            dver = 2
-            modified = True
+        if not sver:
+            sver = VER_SHARES_DB
            for cmd in sch1 + sch2:
                cur.execute(cmd)
            self.log("root", "created new shares-db")
-        else:
-            (dver,) = cur.execute("select v from kv where k = 'sver'").fetchall()[0]

-        if dver == 1:
-            modified = True
+        if sver == 1:
            for cmd in sch2:
                cur.execute(cmd)
            cur.execute("update sh set st = 0")
            self.log("root", "shares-db schema upgrade ok")

-        if modified:
-            for cmd in [
-                r"delete from kv where k = 'sver'",
-                r"insert into kv values ('sver', %d)" % (2,),
-            ]:
-                cur.execute(cmd)
-            db.commit()
+        if sver < VER_SHARES_DB:
+            cur.execute("delete from kv where k='sver'")
+            cur.execute("insert into kv values('sver',?)", (VER_SHARES_DB,))

+        vars = (("pid", os.getpid()), ("ts", int(time.time() * 1000)))
+        if owner:
+            # wear-estimate: same as sessions-db
+            for k, v in vars:
+                cur.execute("update kv set v=? where k=?", (v, k))
+        else:
+            for k, v in vars:
+                cur.execute("insert into kv values(?, ?)", (k, v))
+
+        db.commit()
        cur.close()
        db.close()

@@ -679,10 +766,11 @@ class SvcHub(object):
                t += ", "
            t += "\033[0mNG: \033[35m" + sng

-        t += "\033[0m, see --deps"
-        self.log("dependencies", t, 6)
+        t += "\033[0m, see --deps (this is fine btw)"
+        self.log("optional-dependencies", t, 6)

    def _check_env(self) -> None:
+        al = self.args
        try:
            files = os.listdir(E.cfg)
        except:
@@ -699,6 +787,21 @@ class SvcHub(object):
            if self.args.bauth_last:
                self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)

+        have_tcp = False
+        for zs in al.i:
+            if not zs.startswith("unix:"):
+                have_tcp = True
+        if not have_tcp:
+            zb = False
+            zs = "z zm zm4 zm6 zmv zmvv zs zsv zv"
+            for zs in zs.split():
+                if getattr(al, zs, False):
+                    setattr(al, zs, False)
+                    zb = True
+            if zb:
+                t = "only listening on unix-sockets; cannot enable zeroconf/mdns/ssdp as requested"
+                self.log("root", t, 3)
+
        if not self.args.no_dav:
            from .dxml import DXML_OK

@@ -763,13 +866,14 @@ class SvcHub(object):
            vl = [os.path.expandvars(os.path.expanduser(x)) for x in vl]
            setattr(al, k, vl)

-        for k in "lo hist ssl_log".split(" "):
+        for k in "lo hist dbpath ssl_log".split(" "):
            vs = getattr(al, k)
            if vs:
                vs = os.path.expandvars(os.path.expanduser(vs))
                setattr(al, k, vs)

-        for k in "sus_urls nonsus_urls".split(" "):
+        zs = "dav_ua1 sus_urls nonsus_urls ua_nodoc ua_nozip"
+        for k in zs.split(" "):
            vs = getattr(al, k)
            if not vs or vs == "no":
                setattr(al, k, None)
@@ -1260,7 +1364,7 @@ class SvcHub(object):
                raise

    def check_mp_support(self) -> str:
-        if MACOS:
+        if MACOS and not os.environ.get("PRTY_FORCE_MP"):
            return "multiprocessing is wonky on mac osx;"
        elif sys.version_info < (3, 3):
            return "need python 3.3 or newer for multiprocessing;"
@@ -1280,7 +1384,7 @@ class SvcHub(object):
            return False

        try:
-            if mp.cpu_count() <= 1:
+            if mp.cpu_count() <= 1 and not os.environ.get("PRTY_FORCE_MP"):
                raise Exception()
        except:
            self.log("svchub", "only one CPU detected; multiprocessing disabled")
--- a/copyparty/szip.py
+++ b/copyparty/szip.py
@@ -4,12 +4,11 @@ from __future__ import print_function, unicode_literals
 import calendar
 import stat
 import time
-import zlib

 from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
-from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile
+from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile, zlib

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Generator, Optional
@@ -55,6 +54,7 @@ def gen_fdesc(sz: int, crc32: int, z64: bool) -> bytes:

 def gen_hdr(
    h_pos: Optional[int],
+    z64: bool,
    fn: str,
    sz: int,
    lastmod: int,
@@ -71,7 +71,6 @@ def gen_hdr(
    # appnote 4.5 / zip 3.0 (2008) / unzip 6.0 (2009) says to add z64
    # extinfo for values which exceed H, but that becomes an off-by-one
    # (can't tell if it was clamped or exactly maxval), make it obvious
-    z64 = sz >= 0xFFFFFFFF
    z64v = [sz, sz] if z64 else []
    if h_pos and h_pos >= 0xFFFFFFFF:
        # central, also consider ptr to original header
@@ -245,6 +244,7 @@ class StreamZip(StreamArc):

        sz = st.st_size
        ts = st.st_mtime
+        h_pos = self.pos

        crc = 0
        if self.pre_crc:
@@ -253,8 +253,12 @@ class StreamZip(StreamArc):

            crc &= 0xFFFFFFFF

-        h_pos = self.pos
-        buf = gen_hdr(None, name, sz, ts, self.utf8, crc, self.pre_crc)
+        # some unzip-programs expect a 64bit data-descriptor
+        # even if the only 32bit-exceeding value is the offset,
+        # so force that by placeholdering the filesize too
+        z64 = h_pos >= 0xFFFFFFFF or sz >= 0xFFFFFFFF
+
+        buf = gen_hdr(None, z64, name, sz, ts, self.utf8, crc, self.pre_crc)
        yield self._ct(buf)

        for buf in yieldfile(src, self.args.iobuf):
@@ -267,8 +271,6 @@ class StreamZip(StreamArc):

        self.items.append((name, sz, ts, crc, h_pos))

-        z64 = sz >= 4 * 1024 * 1024 * 1024
-
        if z64 or not self.pre_crc:
            buf = gen_fdesc(sz, crc, z64)
            yield self._ct(buf)
@@ -307,7 +309,8 @@ class StreamZip(StreamArc):

            cdir_pos = self.pos
            for name, sz, ts, crc, h_pos in self.items:
-                buf = gen_hdr(h_pos, name, sz, ts, self.utf8, crc, self.pre_crc)
+                z64 = h_pos >= 0xFFFFFFFF or sz >= 0xFFFFFFFF
+                buf = gen_hdr(h_pos, z64, name, sz, ts, self.utf8, crc, self.pre_crc)
                mbuf += self._ct(buf)
                if len(mbuf) >= 16384:
                    yield mbuf
--- a/copyparty/tcpsrv.py
+++ b/copyparty/tcpsrv.py
@@ -151,9 +151,15 @@ class TcpSrv(object):
            if just_ll or self.args.ll:
                ll_ok.add(ip.split("/")[0])

+        listening_on = []
+        for ip, ports in sorted(ok.items()):
+            for port in sorted(ports):
+                listening_on.append("%s %s" % (ip, port))
+
        qr1: dict[str, list[int]] = {}
        qr2: dict[str, list[int]] = {}
        msgs = []
+        accessible_on = []
        title_tab: dict[str, dict[str, int]] = {}
        title_vars = [x[1:] for x in self.args.wintitle.split(" ") if x.startswith("$")]
        t = "available @ {}://{}:{}/  (\033[33m{}\033[0m)"
@@ -169,6 +175,10 @@ class TcpSrv(object):
                ):
                    continue

+                zs = "%s %s" % (ip, port)
+                if zs not in accessible_on:
+                    accessible_on.append(zs)
+
                proto = " http"
                if self.args.http_only:
                    pass
@@ -219,6 +229,14 @@ class TcpSrv(object):
        else:
            print("\n", end="")

+        for fn, ls in (
+            (self.args.wr_h_eps, listening_on),
+            (self.args.wr_h_aon, accessible_on),
+        ):
+            if fn:
+                with open(fn, "wb") as f:
+                    f.write(("\n".join(ls)).encode("utf-8"))
+
        if self.args.qr or self.args.qrs:
            self.qr = self._qr(qr1, qr2)

@@ -548,7 +566,7 @@ class TcpSrv(object):
        ip = None
        ips = list(t1) + list(t2)
        qri = self.args.qri
-        if self.args.zm and not qri:
+        if self.args.zm and not qri and ips:
            name = self.args.name + ".local"
            t1[name] = next(v for v in (t1 or t2).values())
            ips = [name] + ips
--- a/copyparty/tftpd.py
+++ b/copyparty/tftpd.py
@@ -36,7 +36,19 @@ from partftpy.TftpShared import TftpException
 from .__init__ import EXE, PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
-from .util import UTC, BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
+from .util import (
+    FN_EMB,
+    UTC,
+    BytesIO,
+    Daemon,
+    ODict,
+    exclude_dotfiles,
+    min_ex,
+    runhook,
+    undot,
+    vjoin,
+    vsplit,
+)

 if True:  # pylint: disable=using-constant-test
    from typing import Any, Union
@@ -244,16 +256,25 @@ class Tftpd(object):
        for srv in srvs:
            srv.stop()

-    def _v2a(self, caller: str, vpath: str, perms: list, *a: Any) -> tuple[VFS, str]:
+    def _v2a(
+        self, caller: str, vpath: str, perms: list, *a: Any
+    ) -> tuple[VFS, str, str]:
        vpath = vpath.replace("\\", "/").lstrip("/")
        if not perms:
            perms = [True, True]

        debug('%s("%s", %s) %s\033[K\033[0m', caller, vpath, str(a), perms)
        vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
+        if perms[1] and "*" not in vfs.axs.uread and "wo_up_readme" not in vfs.flags:
+            zs, fn = vsplit(vpath)
+            if fn.lower() in FN_EMB:
+                vpath = vjoin(zs, "_wo_" + fn)
+                vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
+
        if not vfs.realpath:
            raise Exception("unmapped vfs")
-        return vfs, vfs.canonical(rem)
+
+        return vfs, vpath, vfs.canonical(rem)

    def _ls(self, vpath: str, raddress: str, rport: int, force=False) -> Any:
        # generate file listing if vpath is dir.txt and return as file object
@@ -331,7 +352,7 @@ class Tftpd(object):
        else:
            raise Exception("bad mode %s" % (mode,))

-        vfs, ap = self._v2a("open", vpath, [rd, wr])
+        vfs, vpath, ap = self._v2a("open", vpath, [rd, wr])
        if wr:
            if "*" not in vfs.axs.uwrite:
                yeet("blocked write; folder not world-writable: /%s" % (vpath,))
@@ -368,7 +389,7 @@ class Tftpd(object):
        return open(ap, mode, *a, **ka)

    def _mkdir(self, vpath: str, *a) -> None:
-        vfs, ap = self._v2a("mkdir", vpath, [])
+        vfs, _, ap = self._v2a("mkdir", vpath, [False, True])
        if "*" not in vfs.axs.uwrite:
            yeet("blocked mkdir; folder not world-writable: /%s" % (vpath,))

@@ -376,7 +397,7 @@ class Tftpd(object):

    def _unlink(self, vpath: str) -> None:
        # return bos.unlink(self._v2a("stat", vpath, *a)[1])
-        vfs, ap = self._v2a("delete", vpath, [True, False, False, True])
+        vfs, _, ap = self._v2a("delete", vpath, [True, False, False, True])

        try:
            inf = bos.stat(ap)
@@ -400,7 +421,7 @@ class Tftpd(object):

    def _p_exists(self, vpath: str) -> bool:
        try:
-            ap = self._v2a("p.exists", vpath, [False, False])[1]
+            ap = self._v2a("p.exists", vpath, [False, False])[2]
            bos.stat(ap)
            return True
        except:
@@ -408,7 +429,7 @@ class Tftpd(object):

    def _p_isdir(self, vpath: str) -> bool:
        try:
-            st = bos.stat(self._v2a("p.isdir", vpath, [False, False])[1])
+            st = bos.stat(self._v2a("p.isdir", vpath, [False, False])[2])
            ret = stat.S_ISDIR(st.st_mode)
            return ret
        except:
--- a/copyparty/th_cli.py
+++ b/copyparty/th_cli.py
@@ -1,13 +1,15 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals

+import errno
 import os
+import stat

 from .__init__ import TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .th_srv import EXTS_AC, HAVE_WEBP, thumb_path
-from .util import Cooldown
+from .util import Cooldown, Pebkac

 if True:  # pylint: disable=using-constant-test
    from typing import Optional, Union
@@ -16,6 +18,9 @@ if TYPE_CHECKING:
    from .httpsrv import HttpSrv


+IOERROR = "reading the file was denied by the server os; either due to filesystem permissions, selinux, apparmor, or similar:\n%r"
+
+
 class ThumbCli(object):
    def __init__(self, hsrv: "HttpSrv") -> None:
        self.broker = hsrv.broker
@@ -124,7 +129,7 @@ class ThumbCli(object):

        tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
        tpaths = [tpath]
-        if fmt == "w":
+        if fmt[:1] == "w":
            # also check for jpg (maybe webp is unavailable)
            tpaths.append(tpath.rsplit(".", 1)[0] + ".jpg")

@@ -157,8 +162,22 @@ class ThumbCli(object):
        if abort:
            return None

-        if not bos.path.getsize(os.path.join(ptop, rem)):
-            return None
+        ap = os.path.join(ptop, rem)
+        try:
+            st = bos.stat(ap)
+            if not st.st_size or not stat.S_ISREG(st.st_mode):
+                return None
+
+            with open(ap, "rb", 4) as f:
+                if not f.read(4):
+                    raise Exception()
+        except OSError as ex:
+            if ex.errno == errno.ENOENT:
+                raise Pebkac(404)
+            else:
+                raise Pebkac(500, IOERROR % (ex,))
+        except Exception as ex:
+            raise Pebkac(500, IOERROR % (ex,))

        x = self.broker.ask("thumbsrv.get", ptop, rem, mtime, fmt)
        return x.get()  # type: ignore
--- a/copyparty/th_srv.py
+++ b/copyparty/th_srv.py
@@ -4,8 +4,10 @@ from __future__ import print_function, unicode_literals
 import hashlib
 import logging
 import os
+import re
 import shutil
 import subprocess as sp
+import tempfile
 import threading
 import time

@@ -18,6 +20,7 @@ from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, au_unpk, ffprobe
 from .util import BytesIO  # type: ignore
 from .util import (
    FFMPEG_URL,
+    VF_CAREFUL,
    Cooldown,
    Daemon,
    afsenc,
@@ -48,6 +51,10 @@ HAVE_WEBP = False

 EXTS_TH = set(["jpg", "webp", "png"])
 EXTS_AC = set(["opus", "owa", "caf", "mp3"])
+EXTS_SPEC_SAFE = set("aif aiff flac mp3 opus wav".split())
+
+PTN_TS = re.compile("^-?[0-9a-f]{8,10}$")
+

 try:
    if os.environ.get("PRTY_NO_PIL"):
@@ -163,12 +170,15 @@ class ThumbSrv(object):

        self.mutex = threading.Lock()
        self.busy: dict[str, list[threading.Condition]] = {}
+        self.untemp: dict[str, list[str]] = {}
        self.ram: dict[str, float] = {}
        self.memcond = threading.Condition(self.mutex)
        self.stopping = False
        self.rm_nullthumbs = True  # forget failed conversions on startup
        self.nthr = max(1, self.args.th_mt)

+        self.exts_spec_unsafe = set(self.args.th_spec_cnv.split(","))
+
        self.q: Queue[Optional[tuple[str, str, str, VFS]]] = Queue(self.nthr * 4)
        for n in range(self.nthr):
            Daemon(self.worker, "thumb-{}-{}".format(n, self.nthr))
@@ -385,8 +395,12 @@ class ThumbSrv(object):
                    self.log(msg, c)
                    if getattr(ex, "returncode", 0) != 321:
                        if fun == funs[-1]:
-                            with open(ttpath, "wb") as _:
-                                pass
+                            try:
+                                with open(ttpath, "wb") as _:
+                                    pass
+                            except Exception as ex:
+                                t = "failed to create the file [%s]: %r"
+                                self.log(t % (ttpath, ex), 3)
                    else:
                        # ffmpeg may spawn empty files on windows
                        try:
@@ -399,13 +413,24 @@ class ThumbSrv(object):

            try:
                wrename(self.log, ttpath, tpath, vn.flags)
-            except:
+            except Exception as ex:
+                if not os.path.exists(tpath):
+                    t = "failed to move  [%s]  to  [%s]:  %r"
+                    self.log(t % (ttpath, tpath, ex), 3)
                pass

+            untemp = []
            with self.mutex:
                subs = self.busy[tpath]
                del self.busy[tpath]
                self.ram.pop(ttpath, None)
+                untemp = self.untemp.pop(ttpath, None) or []
+
+            for ap in untemp:
+                try:
+                    wunlink(self.log, ap, VF_CAREFUL)
+                except:
+                    pass

            for x in subs:
                with x:
@@ -659,15 +684,43 @@ class ThumbSrv(object):
        if "ac" not in ret:
            raise Exception("not audio")

+        fext = abspath.split(".")[-1].lower()
+
        # https://trac.ffmpeg.org/ticket/10797
        # expect 1 GiB every 600 seconds when duration is tricky;
        # simple filetypes are generally safer so let's special-case those
-        safe = ("flac", "wav", "aif", "aiff", "opus")
-        coeff = 1800 if abspath.split(".")[-1].lower() in safe else 600
-        dur = ret[".dur"][1] if ".dur" in ret else 300
+        coeff = 1800 if fext in EXTS_SPEC_SAFE else 600
+        dur = ret[".dur"][1] if ".dur" in ret else 900
        need = 0.2 + dur / coeff
        self.wait4ram(need, tpath)

+        infile = abspath
+        if dur >= 900 or fext in self.exts_spec_unsafe:
+            with tempfile.NamedTemporaryFile(suffix=".spec.flac", delete=False) as f:
+                f.write(b"h")
+                infile = f.name
+                try:
+                    self.untemp[tpath].append(infile)
+                except:
+                    self.untemp[tpath] = [infile]
+
+            # fmt: off
+            cmd = [
+                b"ffmpeg",
+                b"-nostdin",
+                b"-v", b"error",
+                b"-hide_banner",
+                b"-i", fsenc(abspath),
+                b"-map", b"0:a:0",
+                b"-ac", b"1",
+                b"-ar", b"48000",
+                b"-sample_fmt", b"s16",
+                b"-t", b"900",
+                b"-y", fsenc(infile),
+            ]
+            # fmt: on
+            self._run_ff(cmd, vn)
+
        fc = "[0:a:0]aresample=48000{},showspectrumpic=s="
        if "3" in fmt:
            fc += "1280x1024,crop=1420:1056:70:48[o]"
@@ -687,7 +740,7 @@ class ThumbSrv(object):
            b"-nostdin",
            b"-v", b"error",
            b"-hide_banner",
-            b"-i", fsenc(abspath),
+            b"-i", fsenc(infile),
            b"-filter_complex", fc.encode("utf-8"),
            b"-map", b"[o]",
            b"-frames:v", b"1",
@@ -991,6 +1044,8 @@ class ThumbSrv(object):
            # thumb file
            try:
                b64, ts, ext = f.split(".")
+                if len(ts) > 8 and PTN_TS.match(ts):
+                    ts = "yeahokay"
                if len(b64) != 24 or len(ts) != 8 or ext not in exts:
                    raise Exception()
            except:
--- a/copyparty/u2idx.py
+++ b/copyparty/u2idx.py
@@ -134,9 +134,9 @@ class U2idx(object):
        assert sqlite3  # type: ignore  # !rm

        ptop = vn.realpath
-        histpath = self.asrv.vfs.histtab.get(ptop)
+        histpath = self.asrv.vfs.dbpaths.get(ptop)
        if not histpath:
-            self.log("no histpath for %r" % (ptop,))
+            self.log("no dbpath for %r" % (ptop,))
            return None

        db_path = os.path.join(histpath, "up2k.db")
--- a/copyparty/up2k.py
+++ b/copyparty/up2k.py
@@ -2,7 +2,6 @@
 from __future__ import print_function, unicode_literals

 import errno
-import gzip
 import hashlib
 import json
 import math
@@ -42,6 +41,7 @@ from .util import (
    fsenc,
    gen_filekey,
    gen_filekey_dbg,
+    gzip,
    hidedir,
    humansize,
    min_ex,
@@ -94,7 +94,7 @@ VF_AFFECTS_INDEXING = set(zsg.split(" "))

 SBUSY = "cannot receive uploads right now;\nserver busy with %s.\nPlease wait; the client will retry..."

-HINT_HISTPATH = "you could try moving the database to another location (preferably an SSD or NVME drive) using either the --hist argument (global option for all volumes), or the hist volflag (just for this volume)"
+HINT_HISTPATH = "you could try moving the database to another location (preferably an SSD or NVME drive) using either the --hist argument (global option for all volumes), or the hist volflag (just for this volume), or, if you want to keep the thumbnails in the current location and only move the database itself, then use --dbpath or volflag dbpath"


 NULLSTAT = os.stat_result((0, -1, -1, 0, 0, 0, 0, 0, 0, 0))
@@ -557,6 +557,7 @@ class Up2k(object):
            else:
                # important; not deferred by db_act
                timeout = self._check_lifetimes()
+                timeout = min(self._check_forget_ip(), timeout)
                try:
                    if self.args.shr:
                        timeout = min(self._check_shares(), timeout)
@@ -617,6 +618,43 @@ class Up2k(object):
                for v in vols:
                    volage[v] = now

+    def _check_forget_ip(self) -> float:
+        now = time.time()
+        timeout = now + 9001
+        for vp, vol in sorted(self.vfs.all_vols.items()):
+            maxage = vol.flags["forget_ip"]
+            if not maxage:
+                continue
+
+            cur = self.cur.get(vol.realpath)
+            if not cur:
+                continue
+
+            cutoff = now - maxage * 60
+
+            for _ in range(2):
+                q = "select ip, at from up where ip > '' order by +at limit 1"
+                hits = cur.execute(q).fetchall()
+                if not hits:
+                    break
+
+                remains = hits[0][1] - cutoff
+                if remains > 0:
+                    timeout = min(timeout, now + remains)
+                    break
+
+                q = "update up set ip = '' where ip > '' and at <= %d"
+                cur.execute(q % (cutoff,))
+                zi = cur.rowcount
+                cur.connection.commit()
+
+                t = "forget-ip(%d) removed %d IPs from db [/%s]"
+                self.log(t % (maxage, zi, vol.vpath))
+
+            timeout = min(timeout, now + 900)
+
+        return timeout
+
    def _check_lifetimes(self) -> float:
        now = time.time()
        timeout = now + 9001
@@ -1058,9 +1096,9 @@ class Up2k(object):
        self, ptop: str, flags: dict[str, Any]
    ) -> Optional[tuple["sqlite3.Cursor", str]]:
        """mutex(main,reg) me"""
-        histpath = self.vfs.histtab.get(ptop)
+        histpath = self.vfs.dbpaths.get(ptop)
        if not histpath:
-            self.log("no histpath for %r" % (ptop,))
+            self.log("no dbpath for %r" % (ptop,))
            return None

        db_path = os.path.join(histpath, "up2k.db")
@@ -1081,7 +1119,7 @@ class Up2k(object):
        ft = "\033[0;32m{}{:.0}"
        ff = "\033[0;35m{}{:.0}"
        fv = "\033[0;36m{}:\033[90m{}"
-        zs = "html_head mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot"
+        zs = "ext_th_d html_head mv_re_r mv_re_t rm_re_r rm_re_t srch_re_dots srch_re_nodot zipmax zipmaxn_v zipmaxs_v"
        fx = set(zs.split())
        fd = vf_bmap()
        fd.update(vf_cmap())
@@ -1306,12 +1344,15 @@ class Up2k(object):
            ]
            excl += [absreal(x) for x in excl]
            excl += list(self.vfs.histtab.values())
+            excl += list(self.vfs.dbpaths.values())
            if WINDOWS:
                excl = [x.replace("/", "\\") for x in excl]
            else:
                # ~/.wine/dosdevices/z:/ and such
                excl.extend(("/dev", "/proc", "/run", "/sys"))

+            excl = list({k: 1 for k in excl})
+
            if self.args.re_dirsz:
                db.c.execute("delete from ds")
                db.n += 1
@@ -2880,7 +2921,6 @@ class Up2k(object):
            if ptop not in self.registry:
                raise Pebkac(410, "location unavailable")

-        cj["name"] = sanitize_fn(cj["name"], "")
        cj["poke"] = now = self.db_act = self.vol_act[ptop] = time.time()
        wark = dwark = self._get_wark(cj)
        job = None
@@ -2916,9 +2956,14 @@ class Up2k(object):
                    self.salt, cj["size"], cj["lmod"], cj["prel"], cj["name"]
                )

-            if vfs.flags.get("up_ts", "") == "fu" or not cj["lmod"]:
+            zi = cj["lmod"]
+            bad_mt = zi <= 0 or zi > 0xAAAAAAAA
+            if bad_mt or vfs.flags.get("up_ts", "") == "fu":
                # force upload time rather than last-modified
                cj["lmod"] = int(time.time())
+                if zi and bad_mt:
+                    t = "ignoring impossible last-modified time from client: %s"
+                    self.log(t % (zi,), 6)

            alts: list[tuple[int, int, dict[str, Any], "sqlite3.Cursor", str, str]] = []
            for ptop, cur in vols:
@@ -3193,6 +3238,7 @@ class Up2k(object):
                                    job["ptop"] = vfs.realpath
                                    job["vtop"] = vfs.vpath
                                    job["prel"] = rem
+                                    job["name"] = sanitize_fn(job["name"], "")
                                    if zvfs.vpath != vfs.vpath:
                                        # print(json.dumps(job, sort_keys=True, indent=4))
                                        job["hash"] = cj["hash"]
@@ -3335,7 +3381,17 @@ class Up2k(object):
            return fname

        fp = djoin(fdir, fname)
-        if job.get("replace") and bos.path.exists(fp):
+
+        ow = job.get("replace") and bos.path.exists(fp)
+        if ow and "mt" in str(job["replace"]).lower():
+            mts = bos.stat(fp).st_mtime
+            mtc = job["lmod"]
+            if mtc < mts:
+                t = "will not overwrite; server %d sec newer than client; %d > %d %r"
+                self.log(t % (mts - mtc, mts, mtc, fp))
+                ow = False
+
+        if ow:
            self.log("replacing existing file at %r" % (fp,))
            cur = None
            ptop = job["ptop"]
@@ -3373,6 +3429,7 @@ class Up2k(object):
        rm: bool = False,
        lmod: float = 0,
        fsrc: Optional[str] = None,
+        is_mv: bool = False,
    ) -> None:
        if src == dst or (fsrc and fsrc == dst):
            t = "symlinking a file to itself?? orig(%s) fsrc(%s) link(%s)"
@@ -3389,7 +3446,7 @@ class Up2k(object):

        linked = False
        try:
-            if not flags.get("dedup"):
+            if not is_mv and not flags.get("dedup"):
                raise Exception("dedup is disabled in config")

            lsrc = src
@@ -3655,8 +3712,9 @@ class Up2k(object):
        if self.idx_wark(vflags, *z2):
            del self.registry[ptop][wark]
        else:
-            for k in "host tnam busy sprs poke t0c".split():
+            for k in "host tnam busy sprs poke".split():
                del job[k]
+            job.pop("t0c", None)
            job["t0"] = int(job["t0"])
            job["hash"] = []
            job["done"] = 1
@@ -3789,7 +3847,7 @@ class Up2k(object):
            db_ip = ""
        else:
            # plugins may expect this to look like an actual IP
-            db_ip = "1.1.1.1" if self.args.no_db_ip else ip
+            db_ip = "1.1.1.1" if "no_db_ip" in vflags else ip

        sql = "insert into up values (?,?,?,?,?,?,?)"
        v = (dwark, int(ts), sz, rd, fn, db_ip, int(at or 0))
@@ -4548,7 +4606,7 @@ class Up2k(object):
                dlink = bos.readlink(sabs)
                dlink = os.path.join(os.path.dirname(sabs), dlink)
                dlink = bos.path.abspath(dlink)
-                self._symlink(dlink, dabs, dvn.flags, lmod=ftime)
+                self._symlink(dlink, dabs, dvn.flags, lmod=ftime, is_mv=True)
                wunlink(self.log, sabs, svn.flags)
            else:
                atomic_move(self.log, sabs, dabs, svn.flags)
@@ -4628,12 +4686,12 @@ class Up2k(object):
        Optional[str],
        Optional[int],
        Optional[int],
-        Optional[str],
+        str,
        Optional[int],
    ]:
        cur = self.cur.get(ptop)
        if not cur:
-            return None, None, None, None, None, None
+            return None, None, None, None, "", None

        rd, fn = vsplit(vrem)
        q = "select w, mt, sz, ip, at from up where rd=? and fn=? limit 1"
@@ -4647,7 +4705,7 @@ class Up2k(object):
        if hit:
            wark, ftime, fsize, ip, at = hit
            return cur, wark, ftime, fsize, ip, at
-        return cur, None, None, None, None, None
+        return cur, None, None, None, "", None

    def _forget_file(
        self,
@@ -4767,7 +4825,7 @@ class Up2k(object):
            flags = self.flags.get(ptop) or {}
            atomic_move(self.log, sabs, slabs, flags)
            bos.utime(slabs, (int(time.time()), int(mt)), False)
-            self._symlink(slabs, sabs, flags, False)
+            self._symlink(slabs, sabs, flags, False, is_mv=True)
            full[slabs] = (ptop, rem)
            sabs = slabs

@@ -4826,7 +4884,9 @@ class Up2k(object):
            # (for example a volume with symlinked dupes but no --dedup);
            # fsrc=sabs is then a source that currently resolves to copy

-            self._symlink(dabs, alink, flags, False, lmod=lmod or 0, fsrc=sabs)
+            self._symlink(
+                dabs, alink, flags, False, lmod=lmod or 0, fsrc=sabs, is_mv=True
+            )

        return len(full) + len(links)

@@ -4940,6 +5000,7 @@ class Up2k(object):
                    job["ptop"] = vfs.realpath
                    job["vtop"] = vfs.vpath
                    job["prel"] = rem
+                    job["name"] = sanitize_fn(job["name"], "")
                    if zvfs.vpath != vfs.vpath:
                        self.log("xbu reloc2:%d..." % (depth,), 6)
                        return self._handle_json(job, depth + 1)
@@ -5044,7 +5105,7 @@ class Up2k(object):

    def _snap_reg(self, ptop: str, reg: dict[str, dict[str, Any]]) -> None:
        now = time.time()
-        histpath = self.vfs.histtab.get(ptop)
+        histpath = self.vfs.dbpaths.get(ptop)
        if not histpath:
            return

--- a/copyparty/util.py
+++ b/copyparty/util.py
@@ -31,6 +31,17 @@ from collections import Counter
 from ipaddress import IPv4Address, IPv4Network, IPv6Address, IPv6Network
 from queue import Queue

+try:
+    from zlib_ng import gzip_ng as gzip
+    from zlib_ng import zlib_ng as zlib
+
+    sys.modules["gzip"] = gzip
+    # sys.modules["zlib"] = zlib
+    # `- somehow makes tarfile 3% slower with default malloc, and barely faster with mimalloc
+except:
+    import gzip
+    import zlib
+
 from .__init__ import (
    ANYWIN,
    EXE,
@@ -103,8 +114,14 @@ IP6ALL = "0:0:0:0:0:0:0:0"


 try:
-    import ctypes
    import fcntl
+
+    HAVE_FCNTL = True
+except:
+    HAVE_FCNTL = False
+
+try:
+    import ctypes
    import termios
 except:
    pass
@@ -234,6 +251,9 @@ SYMTIME = PY36 and os.utime in os.supports_follow_symlinks

 META_NOBOTS = '<meta name="robots" content="noindex, nofollow">\n'

+# smart enough to understand javascript while also ignoring rel="nofollow"
+BAD_BOTS = r"Barkrowler|bingbot|BLEXBot|Googlebot|GoogleOther|GPTBot|PetalBot|SeekportBot|SemrushBot|YandexBot"
+
 FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"

 URL_PRJ = "https://github.com/9001/copyparty"
@@ -448,6 +468,8 @@ UNHUMANIZE_UNITS = {

 VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}

+FN_EMB = set([".prologue.html", ".epilogue.html", "readme.md", "preadme.md"])
+

 def read_ram() -> tuple[float, float]:
    a = b = 0
@@ -594,6 +616,38 @@ except Exception as ex:
        print("using fallback base64 codec due to %r" % (ex,))


+class NotUTF8(Exception):
+    pass
+
+
+def read_utf8(log: Optional["NamedLogger"], ap: Union[str, bytes], strict: bool) -> str:
+    with open(ap, "rb") as f:
+        buf = f.read()
+
+    try:
+        return buf.decode("utf-8", "strict")
+    except UnicodeDecodeError as ex:
+        eo = ex.start
+        eb = buf[eo : eo + 1]
+
+    if not strict:
+        t = "WARNING: The file [%s] is not using the UTF-8 character encoding; some characters in the file will be skipped/ignored. The first unreadable character was byte %r at offset %d. Please convert this file to UTF-8 by opening the file in your text-editor and saving it as UTF-8."
+        t = t % (ap, eb, eo)
+        if log:
+            log(t, 3)
+        else:
+            print(t)
+        return buf.decode("utf-8", "replace")
+
+    t = "ERROR: The file [%s] is not using the UTF-8 character encoding, and cannot be loaded. The first unreadable character was byte %r at offset %d. Please convert this file to UTF-8 by opening the file in your text-editor and saving it as UTF-8."
+    t = t % (ap, eb, eo)
+    if log:
+        log(t, 3)
+    else:
+        print(t)
+    raise NotUTF8(t)
+
+
 class Daemon(threading.Thread):
    def __init__(
        self,
@@ -1419,8 +1473,6 @@ def stackmon(fp: str, ival: float, suffix: str) -> None:
        buf = st.encode("utf-8", "replace")

        if fp.endswith(".gz"):
-            import gzip
-
            # 2459b 2304b 2241b 2202b 2194b 2191b lv3..8
            # 0.06s 0.08s 0.11s 0.13s 0.16s 0.19s
            buf = gzip.compress(buf, compresslevel=6)
@@ -1500,6 +1552,12 @@ def vol_san(vols: list["VFS"], txt: bytes) -> bytes:
        txt = txt.replace(bap.replace(b"\\", b"\\\\"), bvp)
        txt = txt.replace(bhp.replace(b"\\", b"\\\\"), bvph)

+        if vol.histpath != vol.dbpath:
+            bdp = vol.dbpath.encode("utf-8")
+            bdph = b"$db(/" + bvp + b")"
+            txt = txt.replace(bdp, bdph)
+            txt = txt.replace(bdp.replace(b"\\", b"\\\\"), bdph)
+
    if txt != txt0:
        txt += b"\r\nNOTE: filepaths sanitized; see serverlog for correct values"

@@ -3888,8 +3946,75 @@ def hidedir(dp) -> None:
            pass


+_flocks = {}
+
+
+def _lock_file_noop(ap: str) -> bool:
+    return True
+
+
+def _lock_file_ioctl(ap: str) -> bool:
+    assert fcntl  # type: ignore  # !rm
+    try:
+        fd = _flocks.pop(ap)
+        os.close(fd)
+    except:
+        pass
+
+    fd = os.open(ap, os.O_RDWR | os.O_CREAT, 438)
+    # NOTE: the fcntl.lockf identifier is (pid,node);
+    #  the lock will be dropped if os.close(os.open(ap))
+    #  is performed anywhere else in this thread
+
+    try:
+        fcntl.lockf(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        _flocks[ap] = fd
+        return True
+    except Exception as ex:
+        eno = getattr(ex, "errno", -1)
+        try:
+            os.close(fd)
+        except:
+            pass
+        if eno in (errno.EAGAIN, errno.EACCES):
+            return False
+        print("WARNING: unexpected errno %d from fcntl.lockf; %r" % (eno, ex))
+        return True
+
+
+def _lock_file_windows(ap: str) -> bool:
+    try:
+        import msvcrt
+
+        try:
+            fd = _flocks.pop(ap)
+            os.close(fd)
+        except:
+            pass
+
+        fd = os.open(ap, os.O_RDWR | os.O_CREAT, 438)
+        msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
+        return True
+    except Exception as ex:
+        eno = getattr(ex, "errno", -1)
+        if eno == errno.EACCES:
+            return False
+        print("WARNING: unexpected errno %d from msvcrt.locking; %r" % (eno, ex))
+        return True
+
+
+if os.environ.get("PRTY_NO_DB_LOCK"):
+    lock_file = _lock_file_noop
+elif ANYWIN:
+    lock_file = _lock_file_windows
+elif HAVE_FCNTL:
+    lock_file = _lock_file_ioctl
+else:
+    lock_file = _lock_file_noop
+
+
 try:
-    if sys.version_info < (3, 10):
+    if sys.version_info < (3, 10) or os.environ.get("PRTY_NO_IMPRESO"):
        # py3.8 doesn't have .files
        # py3.9 has broken .is_file
        raise ImportError()
@@ -4021,9 +4146,22 @@ class WrongPostKey(Pebkac):
        self.datagen = datagen


-_: Any = (mp, BytesIO, quote, unquote, SQLITE_VER, JINJA_VER, PYFTPD_VER, PARTFTPY_VER)
+_: Any = (
+    gzip,
+    mp,
+    zlib,
+    BytesIO,
+    quote,
+    unquote,
+    SQLITE_VER,
+    JINJA_VER,
+    PYFTPD_VER,
+    PARTFTPY_VER,
+)
 __all__ = [
+    "gzip",
    "mp",
+    "zlib",
    "BytesIO",
    "quote",
    "unquote",
--- a/copyparty/web/browser.css
+++ b/copyparty/web/browser.css
@@ -1695,15 +1695,24 @@ html.y #tree.nowrap .ntree a+a:hover {
 	line-height: 0;
 }
 .dumb_loader_thing {
-	display: inline-block;
+	display: block;
 	margin: 1em .3em 1em 1em;
 	padding: 0 1.2em 0 0;
 	font-size: 4em;
+	min-width: 1em;
+	min-height: 1em;
 	opacity: 0;
 	animation: 1s linear .15s infinite forwards spin, .2s ease .15s 1 forwards fadein;
-	position: absolute;
+	position: fixed;
+	top: .3em;
 	z-index: 9;
 }
+#dlt_t {
+	left: 0;
+}
+#dlt_f {
+	right: .5em;
+}
 #files .cfg {
 	display: none;
 	font-size: 2em;
--- a/copyparty/web/browser.js
+++ b/copyparty/web/browser.js
@@ -151,7 +151,8 @@ var Ls = {

 		"ul_par": "parallel uploads:",
 		"ut_rand": "randomize filenames",
-		"ut_u2ts": "copy the last-modified timestamp$Nfrom your filesystem to the server",
+		"ut_u2ts": "copy the last-modified timestamp$Nfrom your filesystem to the server\">📅",
+		"ut_ow": "overwrite existing files on the server?$N🛡️: never (will generate a new filename instead)$N🕒: overwrite if server-file is older than yours$N♻️: always overwrite if the files are different",
 		"ut_mt": "continue hashing other files while uploading$N$Nmaybe disable if your CPU or HDD is a bottleneck",
 		"ut_ask": 'ask for confirmation before upload starts">💭',
 		"ut_pot": "improve upload speed on slow devices$Nby making the UI less complex",
@@ -270,6 +271,7 @@ var Ls = {
 		"ml_eq": "audio equalizer",
 		"ml_drc": "dynamic range compressor",

+		"mt_loop": "loop/repeat one song\">🔁",
 		"mt_shuf": "shuffle the songs in each folder\">🔀",
 		"mt_aplay": "autoplay if there is a song-ID in the link you clicked to access the server$N$Ndisabling this will also stop the page URL from being updated with song-IDs when playing music, to prevent autoplay if these settings are lost but the URL remains\">a▶",
 		"mt_preload": "start loading the next song near the end for gapless playback\">preload",
@@ -315,6 +317,7 @@ var Ls = {
 		"mm_eunk": "Unknown Errol",
 		"mm_e404": "Could not play audio; error 404: File not found.",
 		"mm_e403": "Could not play audio; error 403: Access denied.\n\nTry pressing F5 to reload, maybe you got logged out",
+		"mm_e500": "Could not play audio; error 500: Check server logs.",
 		"mm_e5xx": "Could not play audio; server error ",
 		"mm_nof": "not finding any more audio files nearby",
 		"mm_prescan": "Looking for music to play next...",
@@ -329,6 +332,7 @@ var Ls = {
 		"f_bigtxt": "this file is {0} MiB large -- really view as text?",
 		"fbd_more": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_more">show {2}</a> or <a href="#" id="bd_all">show all</a></div>',
 		"fbd_all": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_all">show all</a></div>',
+		"f_anota": "only {0} of the {1} items were selected;\nto select the full folder, first scroll to the bottom",

 		"f_dls": 'the file links in the current folder have\nbeen changed into download links',

@@ -538,8 +542,10 @@ var Ls = {
 		"u_ewrite": 'you do not have write-access to this folder',
 		"u_eread": 'you do not have read-access to this folder',
 		"u_enoi": 'file-search is not enabled in server config',
+		"u_enoow": "overwrite will not work here; need Delete-permission",
 		"u_badf": 'These {0} files (of {1} total) were skipped, possibly due to filesystem permissions:\n\n',
 		"u_blankf": 'These {0} files (of {1} total) are blank / empty; upload them anyways?\n\n',
+		"u_applef": 'These {0} files (of {1} total) are probably undesirable;\nPress <code>OK/Enter</code> to SKIP the following files,\nPress <code>Cancel/ESC</code> to NOT exclude, and UPLOAD those as well:\n\n',
 		"u_just1": '\nMaybe it works better if you select just one file',
 		"u_ff_many": "if you're using <b>Linux / MacOS / Android,</b> then this amount of files <a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1790500\" target=\"_blank\"><em>may</em> crash Firefox!</a>\nif that happens, please try again (or use Chrome).",
 		"u_up_life": "This upload will be deleted from the server\n{0} after it completes",
@@ -751,7 +757,8 @@ var Ls = {

 		"ul_par": "samtidige handl.:",
 		"ut_rand": "finn opp nye tilfeldige filnavn",
-		"ut_u2ts": "gi filen på serveren samme$Ntidsstempel som lokalt hos deg",
+		"ut_u2ts": "gi filen på serveren samme$Ntidsstempel som lokalt hos deg\">📅",
+		"ut_ow": "overskrive eksisterende filer på serveren?$N🛡️: aldri (finner på et nytt filnavn istedenfor)$N🕒: overskriv hvis serverens fil er eldre$N♻️: alltid, gitt at innholdet er forskjellig",
 		"ut_mt": "fortsett å befare køen mens opplastning foregår$N$Nskru denne av dersom du har en$Ntreg prosessor eller harddisk",
 		"ut_ask": 'bekreft filutvalg før opplastning starter">💭',
 		"ut_pot": "forbedre ytelsen på trege enheter ved å$Nforenkle brukergrensesnittet",
@@ -870,6 +877,7 @@ var Ls = {
 		"ml_eq": "audio equalizer (tonejustering)",
 		"ml_drc": "compressor (volum-utjevning)",

+		"mt_loop": "spill den samme sangen om og om igjen\">🔁",
 		"mt_shuf": "sangene i hver mappe$Nspilles i tilfeldig rekkefølge\">🔀",
 		"mt_aplay": "forsøk å starte avspilling hvis linken du klikket på for å åpne nettsiden inneholder en sang-ID$N$Nhvis denne deaktiveres så vil heller ikke nettside-URLen bli oppdatert med sang-ID'er når musikk spilles, i tilfelle innstillingene skulle gå tapt og nettsiden lastes på ny\">a▶",
 		"mt_preload": "hent ned litt av neste sang i forkant,$Nslik at pausen i overgangen blir mindre\">forles",
@@ -915,6 +923,7 @@ var Ls = {
 		"mm_eunk": "Ukjent feil",
 		"mm_e404": "Avspilling feilet: Fil ikke funnet.",
 		"mm_e403": "Avspilling feilet: Tilgang nektet.\n\nKanskje du ble logget ut?\nPrøv å trykk F5 for å laste siden på nytt.",
+		"mm_e500": "Avspilling feilet: Rusk i maskineriet, sjekk serverloggen.",
 		"mm_e5xx": "Avspilling feilet: ",
 		"mm_nof": "finner ikke flere sanger i nærheten",
 		"mm_prescan": "Leter etter neste sang...",
@@ -929,6 +938,7 @@ var Ls = {
 		"f_bigtxt": "denne filen er hele {0} MiB -- vis som tekst?",
 		"fbd_more": '<div id="blazy">viser <code>{0}</code> av <code>{1}</code> filer; <a href="#" id="bd_more">vis {2}</a> eller <a href="#" id="bd_all">vis alle</a></div>',
 		"fbd_all": '<div id="blazy">viser <code>{0}</code> av <code>{1}</code> filer; <a href="#" id="bd_all">vis alle</a></div>',
+		"f_anota": "kun {0} av totalt {1} elementer ble markert;\nfor å velge alt må du bla til bunnen av mappen først",

 		"f_dls": 'linkene i denne mappen er nå\nomgjort til nedlastningsknapper',

@@ -1138,8 +1148,10 @@ var Ls = {
 		"u_ewrite": 'du har ikke skrivetilgang i denne mappen',
 		"u_eread": 'du har ikke lesetilgang i denne mappen',
 		"u_enoi": 'filsøk er deaktivert i serverkonfigurasjonen',
+		"u_enoow": "kan ikke overskrive filer her (Delete-rettigheten er nødvendig)",
 		"u_badf": 'Disse {0} filene (av totalt {1}) kan ikke leses, kanskje pga rettighetsproblemer i filsystemet på datamaskinen din:\n\n',
 		"u_blankf": 'Disse {0} filene (av totalt {1}) er blanke / uten innhold; ønsker du å laste dem opp uansett?\n\n',
+		"u_applef": 'Disse {0} filene (av totalt {1}) er antagelig uønskede;\nTrykk <code>OK/Enter</code> for å HOPPE OVER disse filene,\nTrykk <code>Avbryt/ESC</code> for å LASTE OPP disse filene også:\n\n',
 		"u_just1": '\nFunker kanskje bedre hvis du bare tar én fil om gangen',
 		"u_ff_many": 'Hvis du bruker <b>Linux / MacOS / Android,</b> så kan dette antallet filer<br /><a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1790500" target="_blank"><em>kanskje</em> krasje Firefox!</a> Hvis det skjer, så prøv igjen (eller bruk Chrome).',
 		"u_up_life": "Filene slettes fra serveren {0}\netter at opplastningen er fullført",
@@ -1351,7 +1363,8 @@ var Ls = {

 		"ul_par": "并行上传：",
 		"ut_rand": "随机化文件名",
-		"ut_u2ts": "将最后修改的时间戳$N从你的文件系统复制到服务器",
+		"ut_u2ts": "将最后修改的时间戳$N从你的文件系统复制到服务器\">📅",
+		"ut_ow": "覆盖服务器上的现有文件？$N🛡️: 从不（会生成一个新文件名）$N🕒: 服务器文件较旧则覆盖$N♻️: 总是覆盖，如果文件内容不同", //m
 		"ut_mt": "在上传时继续哈希其他文件$N$N如果你的 CPU 或硬盘是瓶颈，可能需要禁用",
 		"ut_ask": '上传开始前询问确认">💭',
 		"ut_pot": "通过简化 UI 来$N提高慢设备上的上传速度",
@@ -1470,6 +1483,7 @@ var Ls = {
 		"ml_eq": "音频均衡器",
 		"ml_drc": "动态范围压缩器",

+		"mt_loop": "循环播放当前的歌曲\">🔁", //m
 		"mt_shuf": "在每个文件夹中随机播放歌曲\">🔀",
 		"mt_aplay": "如果链接中有歌曲 ID，则自动播放,禁用此选项将停止在播放音乐时更新页面 URL 中的歌曲 ID，以防止在设置丢失但 URL 保留时自动播放\">自动播放▶",
 		"mt_preload": "在歌曲快结束时开始加载下一首歌，以实现无缝播放\">预加载",
@@ -1515,6 +1529,7 @@ var Ls = {
 		"mm_eunk": "未知错误",
 		"mm_e404": "无法播放音频；错误 404：文件未找到。",
 		"mm_e403": "无法播放音频；错误 403：访问被拒绝。\n\n尝试按 F5 重新加载，也许你已被注销",
+		"mm_e500": "无法播放音频；错误 500：检查服务器日志。", //m
 		"mm_e5xx": "无法播放音频；服务器错误",
 		"mm_nof": "附近找不到更多音频文件",
 		"mm_prescan": "正在寻找下一首音乐...",
@@ -1738,8 +1753,10 @@ var Ls = {
 		"u_ewrite": '你对这个文件夹没有写入权限',
 		"u_eread": '你对这个文件夹没有读取权限',
 		"u_enoi": '文件搜索在服务器配置中未启用',
+		"u_enoow": "无法覆盖此处的文件；需要删除权限", //m
 		"u_badf": '这些 {0} 个文件（共 {1} 个）被跳过，可能是由于文件系统权限：\n\n',
 		"u_blankf": '这些 {0} 个文件（共 {1} 个）是空白的；是否仍然上传？\n\n',
+		"u_applef": "这些 {0} 个文件（共 {1} 个）可能是不需要的；\n按 <code>确定/Enter</code> 跳过以下文件，\n按 <code>取消/ESC</code> 取消排除，并上传这些文件：\n\n", //m
 		"u_just1": '\n也许如果你只选择一个文件会更好',
 		"u_ff_many": "如果你使用的是 <b>Linux / MacOS / Android，</b> 那么这个文件数量 <a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1790500\" target=\"_blank\"><em>可能</em> 崩溃 Firefox!</a>\n如果发生这种情况，请再试一次（或使用 Chrome）。",
 		"u_up_life": "此上传将在 {0} 后从服务器删除",
@@ -1918,8 +1935,8 @@ ebi('op_up2k').innerHTML = (
 	'			<label for="u2rand" tt="' + L.ut_rand + '">🎲</label>\n' +
 	'		</td>\n' +
 	'		<td class="c" rowspan="2">\n' +
-	'			<input type="checkbox" id="u2ts" />\n' +
-	'			<label for="u2ts" tt="' + L.ut_u2ts + '">📅</a>\n' +
+	'			<input type="checkbox" id="u2ow" />\n' +
+	'			<label for="u2ow" tt="' + L.ut_ow + '">?</a>\n' +
 	'		</td>\n' +
 	'		<td class="c" data-perm="read" data-dep="idx" rowspan="2">\n' +
 	'			<input type="checkbox" id="fsearch" />\n' +
@@ -2037,6 +2054,7 @@ ebi('op_cfg').innerHTML = (
 	'	<h3>' + L.cl_uopts + '</h3>\n' +
 	'	<div>\n' +
 	'		<a id="ask_up" class="tgl btn" href="#" tt="' + L.ut_ask + '</a>\n' +
+	'		<a id="u2ts" class="tgl btn" href="#" tt="' + L.ut_u2ts + '</a>\n' +
 	'		<a id="umod" class="tgl btn" href="#" tt="' + L.cut_umod + '</a>\n' +
 	'		<a id="hashw" class="tgl btn" href="#" tt="' + L.cut_mt + '</a>\n' +
 	'		<a id="u2turbo" class="tgl btn ttb" href="#" tt="' + L.cut_turbo + '</a>\n' +
@@ -2173,6 +2191,11 @@ function goto(dest) {
 }


+var m = SPINNER.split(','),
+	SPINNER_CSS = SPINNER.slice(1 + m[0].length);
+SPINNER = m[0];
+
+
 var SBW, SBH;  // scrollbar size
 (function () {
 	var el = mknod('div');
@@ -2274,6 +2297,7 @@ var mpl = (function () {

 	ebi('op_player').innerHTML = (
 		'<div><h3>' + L.cl_opts + '</h3><div>' +
+		'<a href="#" class="tgl btn" id="au_loop" tt="' + L.mt_loop + '</a>' +
 		'<a href="#" class="tgl btn" id="au_shuf" tt="' + L.mt_shuf + '</a>' +
 		'<a href="#" class="tgl btn" id="au_aplay" tt="' + L.mt_aplay + '</a>' +
 		'<a href="#" class="tgl btn" id="au_preload" tt="' + L.mt_preload + '</a>' +
@@ -2325,6 +2349,10 @@ var mpl = (function () {
 		"os_ctl": bcfg_get('au_os_ctl', have_mctl) && have_mctl,
 		'traversals': 0,
 	};
+	bcfg_bind(r, 'loop', 'au_loop', false, function (v) {
+		if (mp.au)
+			mp.au.loop = v;
+	});
 	bcfg_bind(r, 'shuf', 'au_shuf', false, function () {
 		mp.read_order();  // don't bind
 	});
@@ -2537,7 +2565,7 @@ var mpl = (function () {
 		ebi('np_artist').textContent = np.artist || (fns.length > 1 ? fns[0] : '');
 		ebi('np_title').textContent = np.title || '';
 		ebi('np_dur').textContent = np['.dur'] || '';
-		ebi('np_url').textContent = get_vpath() + np.file.split('?')[0];
+		ebi('np_url').textContent = uricom_dec(get_evpath()) + np.file.split('?')[0];
 		if (!MOBILE && cover)
 			ebi('np_img').setAttribute('src', cover);
 		else
@@ -2803,6 +2831,14 @@ function MPlayer() {
 		r.fau.loop = true;
 		r.fau.play();
 	};
+
+	r.set_ev = function () {
+		mp.au.onended = evau_end;
+		mp.au.onerror = evau_error;
+		mp.au.onprogress = pbar.drawpos;
+		mp.au.onplaying = mpui.progress_updater;
+		mp.au.onloadeddata = mp.au.onloadedmetadata = mp.nopause;
+	};
 }


@@ -4040,10 +4076,7 @@ function play(tid, is_ev, seek) {
 	else {
 		mp.au = new Audio();
 		mp.au2 = new Audio();
-		mp.au.onerror = evau_error;
-		mp.au.onprogress = pbar.drawpos;
-		mp.au.onplaying = mpui.progress_updater;
-		mp.au.onended = next_song;
+		mp.set_ev();
 		widget.open();
 	}
 	mp.init_fau();
@@ -4056,13 +4089,9 @@ function play(tid, is_ev, seek) {
 		var t = mp.au;
 		mp.au = mp.au2;
 		mp.au2 = t;
-		t.onerror = t.onprogress = t.onended = null;
+		t.onerror = t.onprogress = t.onended = t.loop = null;
 		t.ld = 0; //owa
-		mp.au.onerror = evau_error;
-		mp.au.onprogress = pbar.drawpos;
-		mp.au.onplaying = mpui.progress_updater;
-		mp.au.onloadeddata = mp.au.onloadedmetadata = mp.nopause;
-		mp.au.onended = next_song;
+		mp.set_ev();
 		t = mp.au.currentTime;
 		if (isNum(t) && t > 0.1)
 			mp.au.currentTime = 0;
@@ -4100,6 +4129,7 @@ function play(tid, is_ev, seek) {

 	try {
 		mp.nopause();
+		mp.au.loop = mpl.loop;
 		if (mpl.aplay || is_ev !== -1)
 			mp.au.play();

@@ -4144,6 +4174,15 @@ function scroll2playing() {
 }


+function evau_end(e) {
+	if (!mpl.loop)
+		return next_song(e);
+	ev(e);
+	mp.au.currentTime = 0;
+	mp.au.play();
+}
+
+
 // event from the audio object if something breaks
 function evau_error(e) {
 	var err = '',
@@ -4187,6 +4226,7 @@ function evau_error(e) {
 	}
 	var em = '' + eplaya.error.message,
 		mfile = '\n\nFile: «' + uricom_dec(eplaya.src.split('/').pop()) + '»',
+		e500 = L.mm_e500,
 		e404 = L.mm_e404,
 		e403 = L.mm_e403;

@@ -4199,6 +4239,9 @@ function evau_error(e) {
 	if (em.startsWith('404: '))
 		err = e404;

+	if (em.startsWith('500: '))
+		err = e500;
+
 	toast.warn(15, esc(basenames(err + mfile)));
 	console.log(basenames(err + mfile));

@@ -4210,7 +4253,9 @@ function evau_error(e) {
 			if (this.status < 400)
 				return;

-			err = this.status == 403 ? e403 : this.status == 404 ? e404 :
+			err = this.status == 403 ? e403 :
+				this.status == 404 ? e404 :
+				this.status == 500 ? e500 :
 				L.mm_e5xx + this.status;

 			toast.warn(15, esc(basenames(err + mfile)));
@@ -4409,7 +4454,8 @@ function eval_hash() {

 function read_dsort(txt) {
 	dnsort = dnsort ? 1 : 0;
-	clmod(ebi('nsort'), 'on', (sread('nsort') || dnsort) == 1);
+	ENATSORT = NATSORT && (sread('nsort') || dnsort) == 1;
+	clmod(ebi('nsort'), 'on', ENATSORT);
 	try {
 		var zt = (('' + txt).trim() || 'href').split(/,+/g);
 		dsort = [];
@@ -4426,7 +4472,7 @@ function read_dsort(txt) {
 		}
 	}
 	catch (ex) {
-		toast.warn(10, 'failed to apply default sort order [' + txt + ']:\n' + ex);
+		toast.warn(10, 'failed to apply default sort order [' + esc('' + txt) + ']:\n' + ex);
 		dsort = [['href', 1, '']];
 	}
 }
@@ -4455,9 +4501,6 @@ function sortfiles(nodes) {

 	sopts = sopts && sopts.length ? sopts : jcp(dsort);

-	var collator = !clgot(ebi('nsort'), 'on') ? null :
-		new Intl.Collator([], {numeric: true});
-
 	try {
 		var is_srch = false;
 		if (nodes[0]['rp']) {
@@ -4509,8 +4552,9 @@ function sortfiles(nodes) {
 				}
 				if (v2 === undefined) return 1 * rev;

-				var ret = rev * (typ == 'int' ? (v1 - v2) : collator ?
-					collator.compare(v1, v2) : v1.localeCompare(v2));
+				var ret = rev * (typ == 'int' ? (v1 - v2) :
+					ENATSORT ? NATSORT.compare(v1, v2) :
+					v1.localeCompare(v2));

 				if (ret === 0)
 					ret = onodes.indexOf(n1) - onodes.indexOf(n2);
@@ -5759,7 +5803,7 @@ var showfile = (function () {

 			td.innerHTML = '<a href="#" id="t' +
 				link.id + '" class="doc bri" hl="' +
-				link.id + '">-txt-</a>';
+				link.id + '" rel="nofollow">-txt-</a>';

 			td.getElementsByTagName('a')[0].setAttribute('href', '?doc=' + fn);
 		}
@@ -5948,7 +5992,8 @@ var showfile = (function () {
 	};

 	r.mktree = function () {
-		var html = ['<li class="bn">' + L.tv_lst + '<br />' + linksplit(get_vpath()).join('<span>/</span>') + '</li>'];
+		var crumbs = linksplit(get_evpath()).join('<span>/</span>'),
+			html = ['<li class="bn">' + L.tv_lst + '<br />' + crumbs + '</li>'];
 		for (var a = 0; a < r.files.length; a++) {
 			var file = r.files[a];
 			html.push('<li><a href="?doc=' +
@@ -6294,19 +6339,40 @@ var thegrid = (function () {
 		var html = [],
 			svgs = new Set(),
 			max_svgs = CHROME ? 500 : 5000,
+			need_ext = !r.thumbs || !!ext_th,
+			use_ext_th = r.thumbs && ext_th,
 			files = QSA('#files>tbody>tr>td:nth-child(2) a[id]');

 		for (var a = 0, aa = files.length; a < aa; a++) {
 			var ao = files[a],
 				ohref = esc(ao.getAttribute('href')),
 				href = ohref.split('?')[0],
+				ext = '',
 				name = uricom_dec(vsplit(href)[1]),
 				ref = ao.getAttribute('id'),
 				isdir = href.endsWith('/'),
 				ac = isdir ? ' class="dir"' : '',
 				ihref = ohref;

-			if (r.thumbs) {
+			if (need_ext && href != "#") {
+				var ar = href.split('.');
+				if (ar.length > 1)
+					ar.shift();
+
+				ar.reverse();
+				for (var b = 0; b < Math.min(2, ar.length); b++) {
+					if (ar[b].length > 7)
+						break;
+
+					ext = ar[b] + '.' + ext;
+				}
+				ext = (ext || 'unk.').slice(0, -1);
+			}
+
+			if (use_ext_th && ext_th[ext]) {
+				ihref = ext_th[ext];
+			}
+			else if (r.thumbs) {
 				ihref = addq(ihref, 'th=' + (have_webp ? 'w' : 'j'));
 				if (!r.crop)
 					ihref += 'f';
@@ -6319,22 +6385,6 @@ var thegrid = (function () {
 				ihref = SR + '/.cpr/ico/folder';
 			}
 			else {
-				var ar = href.split('.');
-				if (ar.length > 1)
-					ar = ar.slice(1);
-
-				ihref = '';
-				ar.reverse();
-				for (var b = 0; b < ar.length; b++) {
-					if (ar[b].length > 7)
-						break;
-
-					ihref = ar[b] + '.' + ihref;
-				}
-				if (!ihref) {
-					ihref = 'unk.';
-				}
-				var ext = ihref.slice(0, -1);
 				if (!svgs.has(ext)) {
 					if (svgs.size < max_svgs)
 						svgs.add(ext);
@@ -6518,8 +6568,8 @@ function tree_scrolltoo(q) {
 	var ctr = ebi('tree'),
 		em = parseFloat(getComputedStyle(act).fontSize),
 		top = act.offsetTop + ul.offsetTop,
-		min = top - 11 * em,
-		max = top - (ctr.offsetHeight - 10 * em);
+		min = top - 20 * em,
+		max = top - (ctr.offsetHeight - 16 * em);

 	if (ctr.scrollTop > min)
 		ctr.scrollTop = Math.floor(min);
@@ -6690,7 +6740,8 @@ var ahotkeys = function (e) {
 			return ebi('griden').click();
 	}

-	if ((aet == 'tr' || aet == 'td') && ae.closest('#files')) {
+	var in_ftab = (aet == 'tr' || aet == 'td') && ae.closest('#files');
+	if (in_ftab) {
 		var d = '', rem = 0;
 		if (aet == 'td') ae = ae.closest('tr'); //ie11
 		if (k == 'ArrowUp' || k == 'Up') d = 'previous';
@@ -6707,12 +6758,19 @@ var ahotkeys = function (e) {
 			msel.selui();
 			return ev(e);
 		}
+	}
+	if (in_ftab || !aet || (ae && ae.closest('#ggrid'))) {
 		if ((k == 'KeyA' || k == 'a') && ctrl(e)) {
-			var sel = msel.getsel(),
+			var ntot = treectl.lsc.files.length + treectl.lsc.dirs.length,
+				sel = msel.getsel(),
 				all = msel.getall();

 			msel.evsel(e, sel.length < all.length);
 			msel.origin_id(null);
+			if (ntot > all.length)
+				toast.warn(10, L.f_anota.format(all.length, ntot), L.f_anota);
+			else if (toast.tag == L.f_anota)
+				toast.hide();
 			return ev(e);
 		}
 	}
@@ -7238,6 +7296,7 @@ var treectl = (function () {
 		treesz = clamp(icfg_get('treesz', 16), 10, 50);

 	var resort = function () {
+		ENATSORT = NATSORT && clgot(ebi('nsort'), 'on');
 		treectl.gentab(get_evpath(), treectl.lsc);
 	};
 	bcfg_bind(r, 'ireadme', 'ireadme', true);
@@ -7493,7 +7552,7 @@ var treectl = (function () {
 		xhr.open('GET', addq(dst, 'tree=' + top + (r.dots ? '&dots' : '') + k), true);
 		xhr.onload = xhr.onerror = r.recvtree;
 		xhr.send();
-		enspin('#tree');
+		enspin('t');
 	}

 	r.recvtree = function () {
@@ -7541,7 +7600,7 @@ var treectl = (function () {
 				}
 			}
 		}
-		despin('#tree');
+		qsr('#dlt_t');

 		try {
 			QS('#treeul>li>a+a').textContent = '[root]';
@@ -7566,8 +7625,8 @@ var treectl = (function () {
 	};

 	function reload_tree() {
-		var cdir = r.nextdir || get_vpath(),
-			cevp = get_evpath(),
+		var cevp = get_evpath(),
+			cdir = r.nextdir || uricom_dec(cevp),
 			links = QSA('#treeul a+a'),
 			nowrap = QS('#tree.nowrap') && QS('#hovertree.on'),
 			act = null;
@@ -7701,8 +7760,8 @@ var treectl = (function () {
 		r.sb_msg = false;
 		r.nextdir = xhr.top;
 		clearTimeout(mpl.t_eplay);
-		enspin('#tree');
-		enspin(thegrid.en ? '#gfiles' : '#files');
+		enspin('t');
+		enspin('f');
 		window.removeEventListener('scroll', r.tscroll);
 	}

@@ -7797,9 +7856,8 @@ var treectl = (function () {
 		}

 		r.gentab(this.top, res);
-		despin('#tree');
-		despin('#files');
-		despin('#gfiles');
+		qsr('#dlt_t');
+		qsr('#dlt_f');

 		var lg0 = res.logues ? res.logues[0] || "" : "",
 			lg1 = res.logues ? res.logues[1] || "" : "",
@@ -7917,7 +7975,7 @@ var treectl = (function () {

 			if (tn.lead == '-')
 				tn.lead = '<a href="?doc=' + bhref + '" id="t' + id +
-					'" class="doc' + (lang ? ' bri' : '') +
+					'" rel="nofollow" class="doc' + (lang ? ' bri' : '') +
 					'" hl="' + id + '" name="' + hname + '">-txt-</a>';

 			var cl = /\.PARTIAL$/.exec(fname) ? ' class="fade"' : '',
@@ -8111,9 +8169,16 @@ var treectl = (function () {
 		}
 		delete res['a'];
 		var keys = Object.keys(res);
-		keys.sort(function (a, b) { return a.localeCompare(b); });
+		for (var a = 0; a < keys.length; a++)
+			keys[a] = [uricom_dec(keys[a]), keys[a]];
+
+		if (ENATSORT)
+			keys.sort(function (a, b) { return NATSORT.compare(a[0], b[0]); });
+		else
+			keys.sort(function (a, b) { return a[0].localeCompare(b[0]); });
+
 		for (var a = 0; a < keys.length; a++) {
-			var kk = keys[a],
+			var kk = keys[a][1],
 				m = /(\?k=[^\n]+)/.exec(kk),
 				kdk = m ? m[1] : '',
 				ks = kk.replace(kdk, '').slice(1),
@@ -8188,20 +8253,15 @@ var treectl = (function () {
 })();


-function enspin(sel) {
-	despin(sel);
-	var d = mknod('div');
+function enspin(i) {
+	i = 'dlt_' + i;
+	if (ebi(i))
+		return;
+	var d = mknod('div', i, SPINNER);
 	d.className = 'dumb_loader_thing';
-	d.innerHTML = '🌲';
-	var tgt = QS(sel);
-	tgt.insertBefore(d, tgt.childNodes[0]);
-}
-
-
-function despin(sel) {
-	var o = QSA(sel + '>.dumb_loader_thing');
-	for (var a = o.length - 1; a >= 0; a--)
-		o[a].parentNode.removeChild(o[a]);
+	if (SPINNER_CSS)
+		d.style.cssText = SPINNER_CSS;
+	document.body.appendChild(d);
 }


@@ -8368,7 +8428,7 @@ function mk_files_header(taglist) {
 		var tag = taglist[a],
 			c1 = tag.slice(0, 1).toUpperCase();

-		tag = c1 + tag.slice(1);
+		tag = esc(c1 + tag.slice(1));
 		if (c1 == '.')
 			tag = '<th name="tags/' + tag + '" sort="int"><span>' + tag.slice(1);
 		else
@@ -8685,7 +8745,17 @@ var mukey = (function () {

 var light, theme, themen;
 var settheme = (function () {
-	var ax = 'abcdefghijklmnopqrstuvwx';
+	var r = {},
+		ax = 'abcdefghijklmnopqrstuvwx',
+		tre = '🌲',
+		chldr = !SPINNER_CSS && SPINNER == tre;
+
+	r.ldr = {
+		'4':['🌴'],
+		'5':['🌭', 'padding:0 0 .7em .7em;filter:saturate(3)'],
+		'6':['📞', 'padding:0;filter:brightness(2) sepia(1) saturate(3) hue-rotate(60deg)'],
+		'7':['▲', 'font-size:3em'], //cp437
+	};

 	theme = sread('cpp_thm') || 'a';
 	if (!/^[a-x][yz]/.exec(theme))
@@ -8715,13 +8785,19 @@ var settheme = (function () {
 		ebi('themes').innerHTML = html.join('');
 		var btns = QSA('#themes a');
 		for (var a = 0; a < themes; a++)
-			btns[a].onclick = settheme;
+			btns[a].onclick = r.go;
+
+		if (chldr) {
+			var x = r.ldr[itheme] || [tre];
+			SPINNER = x[0];
+			SPINNER_CSS = x[1];
+		}

 		bcfg_set('light', light);
 		tt.att(ebi('themes'));
 	}

-	function settheme(e) {
+	r.go = function (e) {
 		var i = e;
 		try { ev(e); i = e.target.textContent; } catch (ex) { }
 		light = i % 2 == 1;
@@ -8734,7 +8810,7 @@ var settheme = (function () {
 	}

 	freshen();
-	return settheme;
+	return r;
 })();


@@ -9742,7 +9818,7 @@ function wintitle(txt, noname) {
 	if (s_name && !noname)
 		txt = s_name + ' ' + txt;

-	txt += get_vpath().slice(1, -1).split('/').pop();
+	txt += uricom_dec(get_evpath()).slice(1, -1).split('/').pop();

 	document.title = txt;
 }
--- a/copyparty/web/md2.js
+++ b/copyparty/web/md2.js
@@ -1078,26 +1078,28 @@ action_stack = (function () {
        var p1 = from.length,
            p2 = to.length;

-        while (p1-- > 0 && p2-- > 0)
+        while (p1 --> 0 && p2 --> 0)
            if (from[p1] != to[p2])
                break;

-        if (car > ++p1) {
+        if (car > ++p1)
            car = p1;
-        }

        var txt = from.substring(car, p1)
        return {
            car: car,
-            cdr: ++p2,
+            cdr: p2 + (car && 1),
            txt: txt,
            cpos: cpos
        };
    }

    var undiff = function (from, change) {
+        var t1 = from.substring(0, change.car),
+            t2 = from.substring(change.cdr);
+
        return {
-            txt: from.substring(0, change.car) + change.txt + from.substring(change.cdr),
+            txt: t1 + change.txt + t2,
            cpos: change.cpos
        };
    }
--- a/copyparty/web/splash.html
+++ b/copyparty/web/splash.html
@@ -122,7 +122,7 @@
 				<input type="hidden" id="la" name="act" value="login" />
 				<input type="password" id="lp" name="cppwd" placeholder=" password" />
 				<input type="hidden" name="uhash" id="uhash" value="x" />
-				<input type="submit" id="ls" value="Login" />
+				<input type="submit" id="ls" value="login" />
 				{% if chpw %}
 				<a id="x" href="#">change password</a>
 				{% endif %}
--- a/copyparty/web/ui.css
+++ b/copyparty/web/ui.css
@@ -381,6 +381,9 @@ html.y .btn:focus {
 	box-shadow: 0 .1em .2em #037 inset;
 	outline: #037 solid .1em;
 }
+input, button {
+	font-family: var(--font-main), sans-serif;
+}
 input[type="submit"] {
 	cursor: pointer;
 }
--- a/copyparty/web/up2k.js
+++ b/copyparty/web/up2k.js
@@ -885,6 +885,25 @@ function up2k_init(subtle) {
    bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
    bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);

+    uc.ow = parseInt(sread('u2ow', ['0', '1', '2']) || u2ow);
+    uc.owt = ['🛡️', '🕒', '♻️'];
+    function set_ow() {
+        QS('label[for="u2ow"]').innerHTML = uc.owt[uc.ow];
+        ebi('u2ow').checked  = true; //cosmetic
+    }
+    ebi('u2ow').onclick = function (e) {
+        ev(e);
+        if (++uc.ow > 2)
+            uc.ow = 0;
+        swrite('u2ow', uc.ow);
+        set_ow();
+        if (uc.ow && !has(perms, 'delete'))
+            toast.warn(10, L.u_enoow, 'noow');
+        else if (toast.tag == 'noow')
+            toast.hide();
+    };
+    set_ow();
+
    var st = {
        "files": [],
        "nfile": {
@@ -1300,7 +1319,7 @@ function up2k_init(subtle) {
        if (bad_files.length) {
            var msg = L.u_badf.format(bad_files.length, ntot);
            for (var a = 0, aa = Math.min(20, bad_files.length); a < aa; a++)
-                msg += '-- ' + bad_files[a][1] + '\n';
+                msg += '-- ' + esc(bad_files[a][1]) + '\n';

            msg += L.u_just1;
            return modal.alert(msg, function () {
@@ -1312,7 +1331,7 @@ function up2k_init(subtle) {
        if (nil_files.length) {
            var msg = L.u_blankf.format(nil_files.length, ntot);
            for (var a = 0, aa = Math.min(20, nil_files.length); a < aa; a++)
-                msg += '-- ' + nil_files[a][1] + '\n';
+                msg += '-- ' + esc(nil_files[a][1]) + '\n';

            msg += L.u_just1;
            return modal.confirm(msg, function () {
@@ -1324,10 +1343,68 @@ function up2k_init(subtle) {
            });
        }

+        var fps = new Set(), pdp = '';
+        for (var a = 0; a < good_files.length; a++) {
+            var fp = good_files[a][1],
+                dp = vsplit(fp)[0];
+            fps.add(fp);
+            if (pdp != dp) {
+                pdp = dp;
+                dp = dp.slice(0, -1);
+                while (dp) {
+                    fps.add(dp);
+                    dp = vsplit(dp)[0].slice(0, -1);
+                }
+            }
+        }
+
+        var junk = [], rmi = [];
+        for (var a = 0; a < good_files.length; a++) {
+            var fn = good_files[a][1];
+            if (fn.indexOf("/.") < 0 && fn.indexOf("/__MACOS") < 0)
+                continue;
+
+            if (/\/__MACOS|\/\.(DS_Store|AppleDouble|LSOverride|DocumentRevisions-|fseventsd|Spotlight-V[0-9]|TemporaryItems|Trashes|VolumeIcon\.icns|com\.apple\.timemachine\.donotpresent|AppleDB|AppleDesktop|apdisk)/.exec(fn)) {
+                junk.push(good_files[a]);
+                rmi.push(a);
+                continue;
+            }
+
+            if (fn.indexOf("/._") + 1 &&
+                fps.has(fn.replace("/._", "/")) &&
+                fn.split("/").pop().startsWith("._") &&
+                !has(rmi, a)
+            ) {
+                junk.push(good_files[a]);
+                rmi.push(a);
+            }
+        }
+
+        if (!junk.length)
+            return gotallfiles2(good_files);
+
+        junk.sort();
+        rmi.sort(function (a, b) { return a - b; });
+
+        var msg = L.u_applef.format(junk.length, good_files.length);
+        for (var a = 0, aa = Math.min(1000, junk.length); a < aa; a++)
+            msg += '-- ' + esc(junk[a][1]) + '\n';
+
+        return modal.confirm(msg, function () {
+            for (var a = rmi.length - 1; a >= 0; a--)
+                good_files.splice(rmi[a], 1);
+
+            start_actx();
+            gotallfiles2(good_files);
+        }, function () {
+            start_actx();
+            gotallfiles2(good_files);
+        });
+    }
+
+    function gotallfiles2(good_files) {
        good_files.sort(function (a, b) {
-            a = a[1];
-            b = b[1];
-            return a < b ? -1 : a > b ? 1 : 0;
+            return a[1] < b[1] ? -1 : 1;
        });

        var msg = [];
@@ -1338,7 +1415,7 @@ function up2k_init(subtle) {
        if (FIREFOX && good_files.length > 3000)
            msg.push(L.u_ff_many + "\n\n");

-        msg.push(L.u_asku.format(good_files.length, esc(get_vpath())) + '<ul>');
+        msg.push(L.u_asku.format(good_files.length, esc(uricom_dec(get_evpath()))) + '<ul>');
        for (var a = 0, aa = Math.min(20, good_files.length); a < aa; a++)
            msg.push('<li>' + esc(good_files[a][1]) + '</li>');

@@ -1380,9 +1457,7 @@ function up2k_init(subtle) {

        if (!uc.az)
            good_files.sort(function (a, b) {
-                a = a[0].size;
-                b = b[0].size;
-                return a < b ? -1 : a > b ? 1 : 0;
+                return a[0].size - b[0].size;
            });

        for (var a = 0; a < good_files.length; a++) {
@@ -1390,7 +1465,7 @@ function up2k_init(subtle) {
                name = good_files[a][1],
                fdir = evpath,
                now = Date.now(),
-                lmod = uc.u2ts ? (fobj.lastModified || now) : 0,
+                lmod = (uc.u2ts && fobj.lastModified) || 0,
                ofs = name.lastIndexOf('/') + 1;

            if (ofs) {
@@ -2054,8 +2129,8 @@ function up2k_init(subtle) {
                try { orz(e); } catch (ex) { vis_exh(ex + '', 'up2k.js', '', '', ex); }
            };
            reader.onerror = function () {
-                var err = reader.error + '';
-                var handled = false;
+                var err = esc('' + reader.error),
+                    handled = false;

                if (err.indexOf('NotReadableError') !== -1 || // win10-chrome defender
                    err.indexOf('NotFoundError') !== -1  // macos-firefox permissions
@@ -2279,7 +2354,7 @@ function up2k_init(subtle) {
        xhr.onerror = xhr.ontimeout = function () {
            console.log('head onerror, retrying', t.name, t);
            if (!toast.visible)
-                toast.warn(9.98, L.u_enethd + "\n\nfile: " + t.name, t);
+                toast.warn(9.98, L.u_enethd + "\n\nfile: " + esc(t.name), t);

            apop(st.busy.head, t);
            st.todo.head.unshift(t);
@@ -2354,7 +2429,7 @@ function up2k_init(subtle) {
                return console.log('zombie handshake onerror', t.name, t);

            if (!toast.visible)
-                toast.warn(9.98, L.u_eneths + "\n\nfile: " + t.name, t);
+                toast.warn(9.98, L.u_eneths + "\n\nfile: " + esc(t.name), t);

            console.log('handshake onerror, retrying', t.name, t);
            apop(st.busy.handshake, t);
@@ -2459,7 +2534,7 @@ function up2k_init(subtle) {
                    var idx = t.hash.indexOf(missing[a]);
                    if (idx < 0)
                        return modal.alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
-                            missing[a], JSON.stringify(t)));
+                            missing[a], esc(JSON.stringify(t))));

                    t.postlist.push(idx);
                    cbd[idx] = 0;
@@ -2613,7 +2688,7 @@ function up2k_init(subtle) {
                    return toast.err(0, L.u_ehsdf + "\n\n" + rsp.replace(/.*; /, ''));

                err = t.t_uploading ? L.u_ehsfin : t.srch ? L.u_ehssrch : L.u_ehsinit;
-                xhrchk(xhr, err + "\n\nfile: " + t.name + "\n\nerror ", "404, target folder not found", "warn", t);
+                xhrchk(xhr, err + "\n\nfile: " + esc(t.name) + "\n\nerror ", "404, target folder not found", "warn", t);
            }
        }
        xhr.onload = function (e) {
@@ -2634,6 +2709,13 @@ function up2k_init(subtle) {
        else if (t.umod)
            req.umod = true;

+        if (!t.srch) {
+            if (uc.ow == 1)
+                req.replace = 'mt';
+            if (uc.ow == 2)
+                req.replace = true;
+        }
+
        xhr.open('POST', t.purl, true);
        xhr.responseType = 'text';
        xhr.timeout = 42000 + (t.srch || t.t_uploaded ? 0 :
@@ -2763,7 +2845,7 @@ function up2k_init(subtle) {
                    toast.inf(10, L.u_cbusy);
            }
            else {
-                xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), t.name), "404, target folder not found (???)", "warn", t);
+                xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), "404, target folder not found (???)", "warn", t);
                chill(t);
            }
            orz2(xhr);
@@ -2807,7 +2889,7 @@ function up2k_init(subtle) {
                xhr.bsent = 0;

                if (!toast.visible)
-                    toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), t.name), t);
+                    toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), esc(t.name)), t);

                t.nojoin = t.nojoin || t.postlist.length;  // maybe rproxy postsize limit
                console.log('chunkpit onerror,', t.name, t);
--- a/copyparty/web/util.js
+++ b/copyparty/web/util.js
@@ -461,6 +461,13 @@ function namesan(txt, win, fslash) {
 }


+var NATSORT, ENATSORT;
+try {
+    NATSORT = new Intl.Collator([], {numeric: true});
+}
+catch (ex) { }
+
+
 var crctab = (function () {
    var c, tab = [];
    for (var n = 0; n < 256; n++) {
@@ -614,6 +621,33 @@ function showsort(tab) {
        }
    }
 }
+function st_cmp_num(a, b) {
+    a = a[0];
+    b = b[0];
+    return (
+        a === null ? -1 :
+        b === null ? 1 :
+        (a - b)
+    );
+}
+function st_cmp_nat(a, b) {
+    a = a[0];
+    b = b[0];
+    return (
+        a === null ? -1 :
+        b === null ? 1 :
+        NATSORT.compare(a, b)
+    );
+}
+function st_cmp_gen(a, b) {
+    a = a[0];
+    b = b[0];
+    return (
+        a === null ? -1 :
+        b === null ? 1 :
+        a.localeCompare(b)
+    );
+}
 function sortTable(table, col, cb) {
    var tb = table.tBodies[0],
        th = table.tHead.rows[0].cells,
@@ -659,19 +693,17 @@ function sortTable(table, col, cb) {
        }
        vl.push([v, a]);
    }
-    vl.sort(function (a, b) {
-        a = a[0];
-        b = b[0];
-        if (a === null)
-            return -1;
-        if (b === null)
-            return 1;

-        if (stype == 'int') {
-            return reverse * (a - b);
-        }
-        return reverse * (a.localeCompare(b));
-    });
+    if (stype == 'int')
+        vl.sort(st_cmp_num);
+    else if (ENATSORT)
+        vl.sort(st_cmp_nat);
+    else
+        vl.sort(st_cmp_gen);
+
+    if (reverse < 0)
+        vl.reverse();
+
    if (sread('dir1st') !== '0') {
        var r1 = [], r2 = [];
        for (var i = 0; i < tr.length; i++) {
@@ -857,11 +889,6 @@ function get_evpath() {
 }


-function get_vpath() {
-    return uricom_dec(get_evpath());
-}
-
-
 function noq_href(el) {
    return el.getAttribute('href').split('?')[0];
 }
--- a/copyparty/web/w.hash.js
+++ b/copyparty/web/w.hash.js
@@ -64,7 +64,7 @@ onmessage = (d) => {
    };
    reader.onerror = function () {
        busy = false;
-        var err = reader.error + '';
+        var err = esc('' + reader.error);

        if (err.indexOf('NotReadableError') !== -1 || // win10-chrome defender
            err.indexOf('NotFoundError') !== -1  // macos-firefox permissions
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -1,3 +1,336 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0413-2151  `v1.16.20`  all sorted
+
+## 🧪 new features
+
+* when enabled, natural-sort will now also apply to tags, not just filenames 7b2bd6da
+
+## 🩹 bugfixes
+
+* some sorting-related stuff 7b2bd6da
+  * folders with non-ascii names would sort incorrectly in the navpane/sidebar
+  * natural-sort didn't apply correctly after changing the sort order
+* workaround [ffmpeg-bug 10797](https://trac.ffmpeg.org/ticket/10797) 98dcaee2
+  * reduces ram usage from 1534 to 230 MiB when generating spectrograms of s3xmodit songs (amiga chiptunes)
+* disable mdns if only listening on uds (unix-sockets) ffc16109 361aebf8
+
+## 🔧 other changes
+
+* hotkey CTRL-A will now select all files in gridview 233075ae
+  * and it toggles (just like in list-view) so try pressing it again
+* copyparty.exe: upgrade to pillow v11.2.1 c7aa1a35
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0408-2132  `v1.16.19`  GHOST
+
+did you know that every song named `GHOST` is a banger? it's true! [ghost](https://www.youtube.com/watch?v=NoUAwC4yiAw) // [ghost](https://www.youtube.com/watch?v=IKKar5SS29E) // [ghost](https://www.youtube.com/watch?v=tFSFlgm_tsw)
+
+## 🧪 new features
+
+* option to store markdown backups out-of-volume fc883418
+  * the default is still a subfolder named `.hist` next to the markdown file
+  * `--md-hist v` puts them in the volume's hist-folder instead
+  * `--md-hist n` disables markdown-backups entirely
+* #149 option to store the volume sqlite databases at a custom locations outside the hist-folder e1b9ac63
+  * new option `--dbpath` works like `--hist` but it only moves the database file, not the thumbnails
+  * they can be combined, in which case `--hist` is applied to thumbnails, `--dbpath` to the db
+  * useful when you're squeezing every last drop of performance out of your filesystem (see the issue)
+* actively prevent sharing certain databases (sessions/shares) between multiple copyparty instances acfaacbd
+  * an errormessage was added to explain some different alternatives for doing this safely
+    * for example by setting `XDG_CONFIG_HOME` which now works on all platforms b17ccc38
+
+## 🩹 bugfixes
+
+* #151 mkdir did not work in locations outside the volume root (via symlinks) 2b50fc20
+* improve the ui feedback when trying to play an audio file which failed to transcode f9954bc4
+  * also helps with server-filesystem issues, including image-thumbs
+
+## 🔧 other changes
+
+* #152 custom fonts are also applied to textboxes and buttons (thx @thaddeuskkr) d450f615
+* be more careful with the shares-db 8e0364ef
+* be less careful with the sessions-db 8e0364ef
+* update deps c0becc64
+  * web: dompurify
+  * copyparty.exe: python 3.12.10
+* rephrase `-j0` warning on windows to also mention that Microsoft Defender will freak out c0becc64
+* #149 add [a script](https://github.com/9001/copyparty/tree/hovudstraum/contrib#zfs-tunepy) to optimize the sqlite databases for storage on zfs 4f397b9b
+* block `GoogleOther` (another recalcitrant bot) from zip-downloads c2034f7b
+* rephrase `-j0` warning on windows to also mention that Microsoft Defender will freak out c0becc64
+* update [contributing.md](https://github.com/9001/copyparty/blob/hovudstraum/CONTRIBUTING.md) with a section regarding LLM/AI-written code cec3bee0
+* the [helptext](https://ocv.me/copyparty/helptext.html) will also be uploaded to each github release from now on, [permalink](https://github.com/9001/copyparty/releases/latest/download/helptext.html)
+* add review from ixbt forums b383c08c
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0323-2216  `v1.16.18`  zlib-ng
+
+## 🧪 new features
+
+* prefer zlib-ng when available 57a56073
+  * download-as-tar-gz becomes 2.5x faster
+  * default-enabled in docker-images
+  * not enabled in copyparty.exe yet; coming in a future python version
+* docker: add mimalloc (optional, default-disabled) de2c9788
+  * gives twice the speed, and twice the ram usage
+
+## 🩹 bugfixes
+
+* small up2k glitch 3c90cec0
+
+## 🔧 other changes
+
+* rename logues/readmes when uploaded with write-only access 2525d594
+  * since they are used as helptext when viewing the page
+* try to block google and other bad bots from `?doc` and `?zip` 99f63adf
+  * apparently `rel="nofollow"` means nothing these days
+
+### the docker images for this release were built from e1dea7ef
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0316-2002  `v1.16.17`  boot2party
+
+## NEW: make it a bootable usb flashdrive
+
+get the party going anywhere, anytime, no OS required! [download flashdrive image](https://a.ocv.me/pub/stuff/edcd001/enterprise-edition/) or watch the [low-effort demo video](https://a.ocv.me/pub/stuff/edcd001/enterprise-edition/hub-demo-hq.webm) which eventually gets to the copyparty part after showing off a bunch of other stuff on there
+
+* there is [source code](https://github.com/9001/asm/tree/hovudstraum/p/hub) and [build instructions](https://github.com/9001/asm/tree/hovudstraum/p/hub/sm/how2build) too
+* please don't take this too seriously
+
+## 🧪 new features
+
+* option to specify max-size for download-as-zip/tar 494179bd 0a33336d
+  * either the total download size (`--zipmaxs 500M`), and/or max number of files (`--zipmaxn 9k`)
+  * applies to all uesrs by default; can also ignore limits for authorized users (`--zipmaxu`)
+  * errormessage can be customized with `--zipmaxt "winter is coming... but this download isn't"`
+* [appledoubles](https://a.ocv.me/pub/stuff/?doc=appledoubles-and-friends.txt) are detected and skipped when uploading with the browser-UI 78208405
+* IdP-volumes can be filtered by group 9c2c4237
+  * `[/users/${u}]` in a config-file creates the volume for all users like before
+  * `[/users/${u%+canwrite}]` only if the user is in the `canwrite` group
+  * `[/users/${u%-admins}]` only if the user is NOT in the `admins` group
+
+## 🩹 bugfixes
+
+* when moving a folder with symlinks, don't expand them into full files 5ab09769
+  * absolute symlinks are moved as-is; relative symlinks are rewritten so they still point to the same file when possible (if both source and destination are indexed in the db)
+  * the previous behavior was good for un-deduplicating files after changing the server-settings, but was too inconvenient for all other usecases
+* #146 fix downloading from shares when `-j0` enabled 8417098c
+* only show the download-as-zip link when the user is actually allowed to 14bb2999
+* the suggestions in the serverlog regarding how to fix incorrect X-Forwarded-For settings would be incorrect if the reverse-proxy used IPv6 to communicate with copyparty 16462ee5
+* set nofollow on `?doc` links so crawlers don't download binary files as text 6a2644fe
+
+## 🔧 other changes
+
+* #147 IdP: fix the warning about dangerous misconfigurations to be more accurate 29a17ae2
+* #143 print a warning on incorrect character-encoding in textfiles (config-files, logues, readmes etc.) 25974d66
+* copyparty.exe: update to jinja 3.1.6 (copyparty was *not affected* by the jinja-3.1.5 vuln)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0228-1846  `v1.16.16`  lemon melon cookie
+
+<img src="https://github.com/9001/copyparty/raw/hovudstraum/docs/logo.svg" width="250" align="right"/>
+
+webdev is [like a lemon](https://youtu.be/HPURbfKb7to) sometimes
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+there is a [discord server](https://discord.gg/25J8CdTT6G) with an `@everyone` in case of future important updates, such as [vulnerabilities](https://github.com/9001/copyparty/security) (most recently 2025-02-25)
+
+## recent important news
+
+* [v1.16.15 (2025-02-25)](https://github.com/9001/copyparty/releases/tag/v1.16.15) fixed low-severity xss when uploading maliciously-named files
+* [v1.15.0 (2024-09-08)](https://github.com/9001/copyparty/releases/tag/v1.15.0) changed upload deduplication to be default-disabled
+* [v1.14.3 (2024-08-30)](https://github.com/9001/copyparty/releases/tag/v1.14.3) fixed a bug that was introduced in v1.13.8 (2024-08-13); this bug could lead to **data loss** -- see the v1.14.3 release-notes for details
+
+## 🧪 new features
+
+* #142 workaround android-chrome timestamp bug 5e12abbb
+  * all files were uploaded with last-modified year 1601 in specific recent versions of chrome
+  * https://issues.chromium.org/issues/393149335 has the actual fix; will be out soon
+
+## 🩹 bugfixes
+
+* add helptext for volflags `dk`, `dks`, `dky` 65a7706f
+* fix false-positive warning when disabling a global option per-volume by unsetting the volflag
+
+## 🔧 other changes
+
+* #140 nixos: @daimond113 fixed a warning in the nixpkg (thx!) e0fe2b97
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0225-0017  `v1.16.15`  fix low-severity vuln
+
+<img src="https://github.com/9001/copyparty/raw/hovudstraum/docs/logo.svg" width="250" align="right"/>
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+## ⚠️ this fixes a minor vulnerability; CVE-score `3.6`/`10`
+
+[GHSA-m2jw-cj8v-937r](https://github.com/9001/copyparty/security/advisories/GHSA-m2jw-cj8v-937r) aka [CVE-2025-27145](https://www.cve.org/CVERecord?id=CVE-2025-27145) could let an attacker run arbitrary javascript by tricking an authenticated user into uploading files with malicious filenames
+
+* ...but it required some clever social engineering, and is **not likely** to be a cause for concern... ah, better safe than sorry
+
+there is a [discord server](https://discord.gg/25J8CdTT6G) with an `@everyone` in case of future important updates, such as [vulnerabilities](https://github.com/9001/copyparty/security) (most recently 2025-02-25)
+
+## recent important news
+
+* [v1.15.0 (2024-09-08)](https://github.com/9001/copyparty/releases/tag/v1.15.0) changed upload deduplication to be default-disabled
+* [v1.14.3 (2024-08-30)](https://github.com/9001/copyparty/releases/tag/v1.14.3) fixed a bug that was introduced in v1.13.8 (2024-08-13); this bug could lead to **data loss** -- see the v1.14.3 release-notes for details
+
+## 🧪 new features
+
+* nothing this time
+
+## 🩹 bugfixes
+
+* fix [GHSA-m2jw-cj8v-937r](https://github.com/9001/copyparty/security/advisories/GHSA-m2jw-cj8v-937r) / [CVE-2025-27145](https://www.cve.org/CVERecord?id=CVE-2025-27145) in 438ea6cc
+  * when trying to upload an empty files by dragging it into the browser, the filename would be rendered as HTML, allowing javascript injection if the filename was malicious
+  * issue discovered and reported by @JayPatel48 (thx!)
+* related issues in errorhandling of uploads 499ae1c7 36866f1d
+  * these all had the same consequences as the GHSA above, but a network outage was necessary to trigger them
+    * which would probably have the lucky side-effect of blocking the javascript download, nice
+* paranoid fixing of probably-not-even-issues 3adbb2ff
+* fix some markdown / texteditor bugs 407531bc
+  * only indicate file-versions for markdown files in listings, since it's tricky to edit non-textfiles otherwise
+  * CTRL-C followed by CTRL-V and CTRL-Z in a single-line file would make a character fall off
+  * ensure safety of extensions
+
+## 🔧 other changes
+
+* readme:
+  * mention support for running the server on risc-v 6d102fc8
+  * mention that the [sony psp](https://github.com/user-attachments/assets/9d21f020-1110-4652-abeb-6fc09c533d4f) can browse and upload 598a29a7
+
+----
+
+# 💾 what to download?
+| download link | is it good? | description |
+| -- | -- | -- |
+| **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** | ✅ the best 👍 | runs anywhere! only needs python |
+| [a docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker/README.md) | it's ok | good if you prefer docker 🐋 |
+| [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) |  ⚠️ [acceptable](https://github.com/9001/copyparty#copypartyexe) | for [win8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png) or later; built-in thumbnailer |
+| [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.16.14/u2c.exe) | ⚠️ acceptable | [CLI uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/u2c.py) as a win7+ exe ([video](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm)) |
+| [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) | ⚠️ acceptable | similar to the regular sfx, [mostly worse](https://github.com/9001/copyparty#zipapp) |
+| [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) | ⛔️ [dangerous](https://github.com/9001/copyparty#copypartyexe) | for [win7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png) -- never expose to the internet! |
+| [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.16.5/copyparty-winpe64.exe) | ⛔️ dangerous | runs on [64bit WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png), otherwise useless |
+
+* except for [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.16.14/u2c.exe), all of the options above are mostly equivalent
+* the zip and tar.gz files below are just source code
+* python packages are available at [PyPI](https://pypi.org/project/copyparty/#files)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0219-2309  `v1.16.14`  overwrite by upload
+
+## 🧪 new features
+
+* #139 overwrite existing files by uploading over them e9f78ea7
+  * default-disabled; a new togglebutton in the upload-UI configures it
+  * can optionally compare last-modified-time and only overwrite older files
+* [GDPR compliance](https://github.com/9001/copyparty#GDPR-compliance) (maybe/probably) 4be0d426
+
+## 🩹 bugfixes
+
+* some cosmetic volflag stuff, all harmless b190e676
+  * disabling a volflag `foo` with `-foo` shows a warning that `-foo` was not a recognized volflag, but it still does the right thing
+  * some volflags give the *"unrecognized volflag, will ignore"* warning, but not to worry, they still work just fine:
+    * `xz` to allow serverside xz-compression of uploaded files 
+* the option to customize the loader-spinner would glitch out during the initial page load 7d7d5d6c
+
+## 🔧 other changes
+
+* [randpic.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/handlers/randpic.py), new 404-handler example, returns a random pic from a folder 60d5f271
+* readme: [howto permanent cloudflare tunnel](https://github.com/9001/copyparty#permanent-cloudflare-tunnel) for easy hosting from home 2beb2acc
+* [synology-dsm](https://github.com/9001/copyparty/blob/hovudstraum/docs/synology-dsm.md): mention how to update the docker image 56ce5919
+* spinner improvements 6858cb06
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0213-2057  `v1.16.13`  configure with confidence
+
+## 🧪 new features
+
+* make the config-parser more helpful regarding volflags a255db70
+  * if an unrecognized volflag is specified, print a warning instead of silently ignoring it
+  * understand volflag-names with Uppercase and/or kebab-case (dashes), and not just snake_case (underscores)
+  * improve `--help-flags` to mention and explain all available flags
+* #136 WebDAV: support COPY 62ee7f69
+  * also support overwrite of existing target files (default-enabled according to the spec)
+    * the user must have the delete-permission to actually replace files
+* option to specify custom icons for certain file extensions 7e4702cf
+  * see `--ext-th` mentioned briefly in the [thumbnails section](https://github.com/9001/copyparty/#thumbnails)
+* option to replace the loading-spinner animation 685f0869
+  * including how to [make it exceptionally normal-looking](https://github.com/9001/copyparty/tree/hovudstraum/docs/rice#boring-loader-spinner)
+
+## 🩹 bugfixes
+
+* #136 WebDAV fixes 62ee7f69
+  * COPY/MOVE/MKCOL: challenge clients to provide the password as necessary
+    * most clients only need this in PROPFIND, but KDE-Dolphin is more picky
+  * MOVE: support `webdav://` Destination prefix as used by Dolphin, probably others
+* #136 WebDAV: improve support for KDE-Dolphin as client 9d769027
+  * it masquerades as a graphical browser yet still expects 401, so special-case it with a useragent scan
+
+## 🔧 other changes
+
+* Docker-only: quick hacky fix for the [musl CVE](https://www.openwall.com/lists/musl/2025/02/13/1) until the official fix is out 4d6626b0
+  * the docker images will be rebuilt when `musl-1.2.5-r9.apk` is released, in 6~24h or so
+  * until then, there is no support for reading korean XML files when running in docker
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2025-0209-2331  `v1.16.12`  RTT
+
+## 🧪 new features
+
+* show rtt (network latency to server, including request processing time) in the top status text d27f1104 
+  * and log the client-reported RTT to serverlog 20ddeb6e
+* remember file selection when changing folders c7db08ed
+  * good for when you accidentally navigate elsewhere
+* option to restrict download-as-zip/tar to admins-only c87af9e8
+* #135 add [bubbleparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/README.md#bubblepartysh), thx @coderofsalvation! 3582a100
+  * runs copyparty in a [sandbox](https://github.com/containers/bubblewrap), making it harder to gain unintended access through bugs in python or copyparty
+  * better alternative to [prisonparty](https://github.com/9001/copyparty/tree/hovudstraum/bin#prisonpartysh), more similar to [the sandboxing in the nixos package](https://github.com/9001/copyparty/blob/7dda77dcb/contrib/nixos/modules/copyparty.nix#L232-L272)
+* new plugin: [quickmove](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/quickmove.js) 46f9e9ef
+  * adds hotkey `W` to quickly move selected files into a subfolder
+* #133 new plugin: [graft-thumbs.js](https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/graft-thumbs.js) 6c202eff
+  * in folders with foobar.mp3 and foobar.png, can copy the thumbnail from the png to the jpg (and then hide the png)
+* handlers: add [http-redirect example](https://github.com/9001/copyparty/blob/hovudstraum/bin/handlers/redirect.py) 22cbd2db
+* add [ping.html](https://github.com/9001/copyparty/blob/hovudstraum/srv/ping.html) 7de9d15a 910797cc
+
+## 🩹 bugfixes
+
+* improve iPad detection so they get opus instead of mp3 12dcea4f
+
+## 🔧 other changes
+
+* safeguard against accidental config loss cd71b505
+  * while no copyparty servers have ended up in this unfortunate situation yet (afaik), be proactive and borrow some experience from other docker-based services
+* readme: improve config examples 32e90859
+* improve serverlog entries regarding 403s b020fd4a
+* #132 mention fuse permissions in readme d9d2a092
+* traefik-example: fix disconnect during big uploads 6a9ffe7e
+* try to show an appropriate warning for media that the browser doesn't support playing 4ef35263
+  * was an attempt at detecting iphones failing to play high-color-precision webm files, but safari doesn't seem to realize itself that playback has failed, ah well 
+* copyparty.exe: update to python 3.12.9
+* update deps: dompurify 3.2.4
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2025-0127-0140  `v1.16.11`  fix no-acode

--- a/docs/devnotes.md
+++ b/docs/devnotes.md
@@ -281,8 +281,11 @@ on writing your own [hooks](../README.md#event-hooks)
 hooks can cause intentional side-effects,  such as redirecting an upload into another location, or creating+indexing additional files, or deleting existing files, by returning json on stdout

 * `reloc` can redirect uploads before/after uploading has finished, based on filename, extension, file contents, uploader ip/name etc.
+  * example: [reloc-by-ext](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/reloc-by-ext.py)
 * `idx` informs copyparty about a new file to index as a consequence of this upload
+  * example: [podcast-normalizer.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/podcast-normalizer.py)
 * `del` tells copyparty to delete an unrelated file by vpath
+  * example: (　´・ω・) nyoro~n

 for these to take effect, the hook must be defined with the `c1` flag; see example [reloc-by-ext](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/reloc-by-ext.py)

--- a/docs/examples/docker/basic-docker-compose/docker-compose.yml
+++ b/docs/examples/docker/basic-docker-compose/docker-compose.yml
@@ -11,9 +11,14 @@ services:
      - ./:/cfg:z
      - /path/to/your/fileshare/top/folder:/w:z

+    # enabling mimalloc by replacing "NOPE" with "2" will make some stuff twice as fast, but everything will use twice as much ram:
+    environment:
+      LD_PRELOAD: /usr/lib/libmimalloc-secure.so.NOPE
+
    stop_grace_period: 15s  # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
    healthcheck:
-      test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset"]
+      # hide it from logs with "/._" so it matches the default --lf-url filter 
+      test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset=/._"]
      interval: 1m
      timeout: 2s
      retries: 5
--- a/docs/examples/docker/idp-authelia-traefik/docker-compose.yml
+++ b/docs/examples/docker/idp-authelia-traefik/docker-compose.yml
@@ -23,6 +23,9 @@ services:
      - 'traefik.http.routers.copyparty.tls=true'
      - 'traefik.http.routers.copyparty.middlewares=authelia@docker'
    stop_grace_period: 15s  # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
+    environment:
+      LD_PRELOAD: /usr/lib/libmimalloc-secure.so.NOPE
+      # enable mimalloc by replacing "NOPE" with "2" for a nice speed-boost (will use twice as much ram)

  authelia:
    image: authelia/authelia:v4.38.0-beta3  # the config files in the authelia folder use the new syntax
--- a/docs/examples/docker/idp-authentik-traefik/docker-compose.yml
+++ b/docs/examples/docker/idp-authentik-traefik/docker-compose.yml
@@ -22,13 +22,10 @@ services:
      - 'traefik.http.routers.fs.rule=Host(`fs.example.com`)'
      - 'traefik.http.routers.fs.entrypoints=http'
      #- 'traefik.http.routers.fs.middlewares=authelia@docker'  # TODO: ???
-    healthcheck:
-      test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset"]
-      interval: 1m
-      timeout: 2s
-      retries: 5
-      start_period: 15s
    stop_grace_period: 15s  # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
+    environment:
+      LD_PRELOAD: /usr/lib/libmimalloc-secure.so.NOPE
+      # enable mimalloc by replacing "NOPE" with "2" for a nice speed-boost (will use twice as much ram)

  traefik:
    image: traefik:v2.11
--- a/docs/examples/docker/idp/copyparty.conf
+++ b/docs/examples/docker/idp/copyparty.conf
@@ -13,6 +13,8 @@
 # because that is the data-volume in the docker containers,
 # because a deployment like this (with an IdP) is more commonly
 # seen in containerized environments -- but this is not required
+#
+# the example group "su" (super-user) is the admins group


 [global]
@@ -78,6 +80,18 @@
    rwmda: @${g}, @su  # read-write-move-delete-admin for that group + the "su" group


+[/sus/${u%+su}]  # users which ARE members of group "su" gets /sus/username
+  /w/tank1/${u}  # which will be "tank1/username" in the docker data volume
+  accs:
+    rwmda: ${u}  # read-write-move-delete-admin for that username
+
+
+[/m8s/${u%-su}]  # users which are NOT members of group "su" gets /m8s/username
+  /w/tank2/${u}  # which will be "tank2/username" in the docker data volume
+  accs:
+    rwmda: ${u}  # read-write-move-delete-admin for that username
+
+
 # and create some strategic volumes to prevent anyone from gaining
 # unintended access to priv folders if the users/groups db is lost
 [/u]
@@ -88,3 +102,7 @@
  /w/lounge
  accs:
    rwmda: @su
+[/sus]
+  /w/tank1
+[/m8s]
+  /w/tank2
--- a/docs/rice/README.md
+++ b/docs/rice/README.md
@@ -33,12 +33,6 @@ if you are introducing a new ttf/woff font, don't forget to declare the font its
 }
 ```

-and because textboxes don't inherit fonts by default, you can force it like this:
-
-```css
-input[type=text], input[type=submit], input[type=button] { font-family: var(--font-main) }
-```
-
 and if you want to have a monospace font in the fancy markdown editor, do this:

 ```css
@@ -48,6 +42,20 @@ and if you want to have a monospace font in the fancy markdown editor, do this:
 NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative


+# boring loader spinner
+
+replace the 🌲 with a spinning circle using commandline args:
+
+`--spinner ',padding:0;border-radius:9em;border:.2em solid #444;border-top:.2em solid #fc0'`
+
+or config file example:
+
+```yaml
+[global]
+  spinner: ,padding:0;border-radius:9em;border:.2em solid #444;border-top:.2em solid #fc0
+```
+
+
 # `<head>`

 to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
--- a/docs/synology-dsm.md
+++ b/docs/synology-dsm.md
@@ -115,6 +115,16 @@ note that if you only want to share some folders inside your data volume, and no



+## updating
+
+to update to a new copyparty version: `Container Manager` » `Images` » `Update available` » `Update`
+
+* DSM checks for updates every 12h; you can force a check with `sudo /var/packages/ContainerManager/target/tool/image_upgradable_checker`
+
+* there is no auto-update feature, and beware that watchtower does not support DSM
+
+
+
 ## regarding ram usage

 the ram usage indicator in both `Docker` and `Container Manager` is misleading  because it also counts the kernel disk cache which makes the number insanely high -- the synology resource monitor shows the correct values, usually less than 100 MiB
--- a/docs/versus.md
+++ b/docs/versus.md
@@ -131,6 +131,7 @@ symbol legend,
 | runs on Linux           | █ | ╱ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
 | runs on Macos           | █ |   | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |   |
 | runs on FreeBSD         | █ |   |   | • | █ | █ | █ | • | █ | █ |   | █ |   |
+| runs on Risc-V          | █ |   |   | █ | █ | █ |   | • |   | █ |   |   |   |
 | portable binary         | █ | █ | █ |   |   | █ | █ |   |   | █ |   | █ | █ |
 | zero setup, just go     | █ | █ | █ |   |   | ╱ | █ |   |   | █ |   | ╱ | █ |
 | android app             | ╱ |   |   | █ | █ |   |   |   |   |   |   |   |   |
--- a/scripts/deps-docker/Dockerfile
+++ b/scripts/deps-docker/Dockerfile
@@ -3,7 +3,7 @@ WORKDIR /z
 ENV     ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
        ver_hashwasm=4.12.0 \
        ver_marked=4.3.0 \
-        ver_dompf=3.2.4 \
+        ver_dompf=3.2.5 \
        ver_mde=2.18.0 \
        ver_codemirror=5.65.18 \
        ver_fontawesome=5.13.0 \
@@ -12,7 +12,7 @@ ENV     ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \

 # versioncheck:
 # https://github.com/markedjs/marked/releases
-# https://github.com/Ionaru/easy-markdown-editor/tags
+# https://github.com/Ionaru/easy-markdown-editor/tags  # ignore 2.20.0
 # https://github.com/codemirror/codemirror5/releases
 # https://github.com/cure53/DOMPurify/releases
 # https://github.com/Daninet/hash-wasm/releases
--- a/scripts/docker/Dockerfile.ac
+++ b/scripts/docker/Dockerfile.ac
@@ -8,12 +8,13 @@ LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
 ENV     XDG_CONFIG_HOME=/cfg

 RUN     apk --no-cache add !pyc \
-            tzdata wget \
+            tzdata wget mimalloc2 mimalloc2-insecure \
            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
            ffmpeg

 COPY    i/dist/copyparty-sfx.py innvikler.sh ./
-RUN     ash innvikler.sh && rm innvikler.sh
+ADD     base ./base
+RUN     ash innvikler.sh ac

 WORKDIR /w
 EXPOSE  3923
--- a/scripts/docker/Dockerfile.dj
+++ b/scripts/docker/Dockerfile.dj
@@ -11,7 +11,7 @@ COPY    i/bin/mtag/install-deps.sh ./
 COPY    i/bin/mtag/audio-bpm.py /mtag/
 COPY    i/bin/mtag/audio-key.py /mtag/
 RUN     apk add -U !pyc \
-            tzdata wget \
+            tzdata wget mimalloc2 mimalloc2-insecure \
            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
            py3-pip py3-cffi \
            ffmpeg \
@@ -31,7 +31,8 @@ RUN     apk add -U !pyc \
        && ln -s /root/vamp /root/.local /

 COPY    i/dist/copyparty-sfx.py innvikler.sh ./
-RUN     ash innvikler.sh && rm innvikler.sh
+ADD     base ./base
+RUN     ash innvikler.sh dj

 WORKDIR /w
 EXPOSE  3923
--- a/scripts/docker/Dockerfile.im
+++ b/scripts/docker/Dockerfile.im
@@ -8,11 +8,12 @@ LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
 ENV     XDG_CONFIG_HOME=/cfg

 RUN     apk --no-cache add !pyc \
-            tzdata wget \
+            tzdata wget mimalloc2 mimalloc2-insecure \
            py3-jinja2 py3-argon2-cffi py3-pillow py3-mutagen

 COPY    i/dist/copyparty-sfx.py innvikler.sh ./
-RUN     ash innvikler.sh && rm innvikler.sh
+ADD     base ./base
+RUN     ash innvikler.sh im

 WORKDIR /w
 EXPOSE  3923
--- a/scripts/docker/Dockerfile.iv
+++ b/scripts/docker/Dockerfile.iv
@@ -8,7 +8,7 @@ LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
 ENV     XDG_CONFIG_HOME=/cfg

 RUN     apk add -U !pyc \
-            tzdata wget \
+            tzdata wget mimalloc2 mimalloc2-insecure \
            py3-jinja2 py3-argon2-cffi py3-pyzmq py3-pillow \
            py3-pip py3-cffi \
            ffmpeg \
@@ -21,7 +21,8 @@ RUN     apk add -U !pyc \
        && apk del py3-pip .bd

 COPY    i/dist/copyparty-sfx.py innvikler.sh ./
-RUN     ash innvikler.sh && rm innvikler.sh
+ADD     base ./base
+RUN     ash innvikler.sh iv

 WORKDIR /w
 EXPOSE  3923
--- a/scripts/docker/Dockerfile.min
+++ b/scripts/docker/Dockerfile.min
@@ -11,7 +11,7 @@ RUN     apk --no-cache add !pyc \
            py3-jinja2

 COPY    i/dist/copyparty-sfx.py innvikler.sh ./
-RUN     ash innvikler.sh && rm innvikler.sh
+RUN     ash innvikler.sh min

 WORKDIR /w
 EXPOSE  3923
--- a/scripts/docker/README.md
+++ b/scripts/docker/README.md
@@ -101,6 +101,14 @@ the following advice is best-effort and not guaranteed to be entirely correct

 * copyparty will generally create a `.hist` folder at the top of each volume, which contains the filesystem index, thumbnails and such. For performance reasons, but also just to keep things tidy, it might be convenient to store these inside the config folder instead. Add the line `hist: /cfg/hists/` inside the `[global]` section of your `copyparty.conf` to do this

+* if you want more performance, and you're OK with doubling the RAM usage, then consider enabling mimalloc **(maybe buggy)** with one of these:
+
+  * `-e LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2` makes download-as-zip **3x** as fast, filesystem-indexing **1.5x** as fast, etc.
+
+  * `-e LD_PRELOAD=/usr/lib/libmimalloc-insecure.so.2` adds another 10% speed but makes it easier to exploit future vulnerabilities
+
+  * complete example: `podman run --rm -it -p 3923:3923 -v "$PWD:/w:z" -e LD_PRELOAD=/usr/lib/libmimalloc-secure.so.2 copyparty/ac -v /w::r`
+

 ## enabling the ftp server

--- a/scripts/docker/base/Dockerfile.zlibng
+++ b/scripts/docker/base/Dockerfile.zlibng
@@ -0,0 +1,5 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk add py3-pip make gcc musl-dev python3-dev
+RUN     pip wheel https://files.pythonhosted.org/packages/c4/a7/0b7673be5945071e99364a3ac1987b02fc1d416617e97f3e8816d275174e/zlib_ng-0.5.1.tar.gz
--- a/scripts/docker/base/Makefile
+++ b/scripts/docker/base/Makefile
@@ -0,0 +1,15 @@
+self := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+
+all:
+# 	build zlib-ng from source so we know how the sausage was made
+#	(still only doing the archs which are officially supported/tested)
+
+	podman build --arch amd64 -t localhost/cpp-zlibng-amd64:latest -f Dockerfile.zlibng .
+	podman run --arch amd64 --rm --log-driver=none -i localhost/cpp-zlibng-amd64:latest tar -cC/z . | tar -xv
+
+	podman build --arch arm64 -t localhost/cpp-zlibng-amd64:latest -f Dockerfile.zlibng .
+	podman run --arch arm64 --rm --log-driver=none -i localhost/cpp-zlibng-amd64:latest tar -cC/z . | tar -xv
+
+sh:
+	@printf "\n\033[1;31mopening a shell in the most recently created docker image\033[0m\n"
+	docker run --rm -it --entrypoint /bin/ash `docker images -aq | head -n 1`
--- a/scripts/docker/innvikler.sh
+++ b/scripts/docker/innvikler.sh
@@ -1,6 +1,16 @@
 #!/bin/ash
 set -ex

+# use zlib-ng if available
+f=/z/base/zlib_ng-0.5.1-cp312-cp312-linux_$(cat /etc/apk/arch).whl
+[ "$1" != min ] && [ -e $f ] && {
+  apk add -t .bd !pyc py3-pip
+  rm -f /usr/lib/python3*/EXTERNALLY-MANAGED
+  pip install $f
+  apk del .bd
+}
+rm -rf /z/base
+
 # cleanup for flavors with python build steps (dj/iv)
 rm -rf /var/cache/apk/* /root/.cache

@@ -22,6 +32,9 @@ rm -rf \
  /tmp/pe-* /z/copyparty-sfx.py \
  ensurepip pydoc_data turtle.py turtledemo lib2to3

+# speedhack
+sed -ri 's/os.environ.get\("PRTY_NO_IMPRESO"\)/"1"/' /usr/lib/python3.*/site-packages/copyparty/util.py
+
 # drop bytecode
 find / -xdev -name __pycache__ -print0 | xargs -0 rm -rf

@@ -40,7 +53,33 @@ find -name __pycache__ |
 cd /z
 python3 -m copyparty \
  --ign-ebind -p$((1024+RANDOM)),$((1024+RANDOM)),$((1024+RANDOM)) \
-  --no-crt -qi127.1 --exit=idx -e2dsa -e2ts
+  -v .::r --no-crt -qi127.1 --exit=idx -e2dsa -e2ts
+
+########################################################################
+# test download-as-tar.gz
+
+t=$(mktemp)
+python3 -m copyparty \
+  --ign-ebind -p$((1024+RANDOM)),$((1024+RANDOM)),$((1024+RANDOM)) \
+  -v .::r --no-crt -qi127.1 --wr-h-eps $t & pid=$!
+
+for n in $(seq 1 200); do sleep 0.2
+  v=$(awk '/^127/{print;n=1;exit}END{exit n-1}' $t) && break
+done
+[ -z "$v" ] && echo SNAAAAAKE && exit 1
+
+for n in $(seq 1 200); do sleep 0.2
+  wget -O- http://${v/ /:}/?tar=gz:1 >tf && break
+done
+tar -xzO top/innvikler.sh <tf | cmp innvikler.sh
+rm tf
+
+kill $pid; wait $pid
+
+########################################################################

 # output from -e2d
 rm -rf .hist
+
+# goodbye
+exec rm innvikler.sh
--- a/scripts/make-sfx.sh
+++ b/scripts/make-sfx.sh
@@ -237,6 +237,8 @@ necho() {
 	tar -zxf $f
 	mv partftpy-*/partftpy .
 	rm -rf partftpy-* partftpy/bin
+	#(cd partftpy && "$pybin" ../../scripts/strip_hints/a.py; rm uh)  # dont need the full thing, just this:
+	sed -ri 's/from typing import TYPE_CHECKING$/TYPE_CHECKING = False/' partftpy/TftpShared.py

 	necho collecting python-magic
 	v=0.4.27
--- a/scripts/pyinstaller/build.sh
+++ b/scripts/pyinstaller/build.sh
@@ -79,7 +79,6 @@ excl=(
    email.parser
    importlib.resources
    importlib_resources
-    inspect
    multiprocessing
    packaging
    pdb
@@ -99,6 +98,7 @@ excl=(
    PIL.ImageWin
    PIL.PdfParser
 ) || excl+=(
+    inspect
    PIL
    PIL.ExifTags
    PIL.Image
--- a/scripts/pyinstaller/deps.sha512
+++ b/scripts/pyinstaller/deps.sha512
@@ -23,11 +23,11 @@ ac96786e5d35882e0c5b724794329c9125c2b86ae7847f17acfc49f0d294312c6afc1c3f248655de
 # win10
 0a2cd4cadf0395f0374974cd2bc2407e5cc65c111275acdffb6ecc5a2026eee9e1bb3da528b35c7f0ff4b64563a74857d5c2149051e281cc09ebd0d1968be9aa  en-us_windows_10_enterprise_ltsc_2021_x64_dvd_d289cf96.iso
 16cc0c58b5df6c7040893089f3eb29c074aed61d76dae6cd628d8a89a05f6223ac5d7f3f709a12417c147594a87a94cc808d1e04a6f1e407cc41f7c9f47790d1  virtio-win-0.1.248.iso
-18b9e8cfa682da51da1b682612652030bd7f10e4a1d5ea5220ab32bde734b0e6fe1c7dbd903ac37928c0171fd45d5ca602952054de40a4e55e9ed596279516b5  jinja2-3.1.5-py3-none-any.whl
+9a7f40edc6f9209a2acd23793f3cbd6213c94f36064048cb8bf6eb04f1bdb2c2fe991cb09f77fe8b13e5cd85c618ef23573e79813b2fef899ab2f290cd129779  jinja2-3.1.6-py3-none-any.whl
 6df21f0da408a89f6504417c7cdf9aaafe4ed88cfa13e9b8fa8414f604c0401f885a04bbad0484dc51a29284af5d1548e33c6cc6bfb9896d9992c1b1074f332d  MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl
 8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e  mutagen-1.47.0-py3-none-any.whl
 0203ec2551c4836696cfab0b2c9fff603352f03fa36e7476e2e1ca7ec57a3a0c24bd791fcd92f342bf817f0887854d9f072e0271c643de4b313d8c9569ba8813  packaging-24.1-py3-none-any.whl
-12d7921dc7dfd8a4b0ea0fa2bae8f1354fcdd59ece3d7f4e075aed631f9ba791dc142c70b1ccd1e6287c43139df1db26bd57a7a217c8da3a77326036495cdb57  pillow-11.1.0-cp312-cp312-win_amd64.whl
+c9051daaf34ec934962c743a5ac2dbe55a9b0cababb693a8cde0001d24d4a50b67bd534d714d935def6ca7b898ec0a352e58bd9ccdce01c54eaf2281b18e478d  pillow-11.2.1-cp312-cp312-win_amd64.whl
 f0463895e9aee97f31a2003323de235fed1b26289766dc0837261e3f4a594a31162b69e9adbb0e9a31e2e2d4b5f25c762ed1669553df7dc89a8ba4f85d297873  pyinstaller-6.11.1-py3-none-win_amd64.whl
 d550a0a14428386945533de2220c4c2e37c0c890fc51a600f626c6ca90a32d39572c121ec04c157ba3a8d6601cb021f8433d871b5c562a3d342c804fffec90c1  pyinstaller_hooks_contrib-2024.11-py3-none-any.whl
-17b64ff6744004a05d475c8f6de3e48286db4069afad4cae690f83b3555f8e35ceafb210eeba69a11e983d0da3001099de284b6696ed0f1bf9cd791938a7f2cd  python-3.12.9-amd64.exe
+4f9a4d9f65c93e2d851e2674057343a9599f30f5dc582ffca485522237d4fcf43653b3d393ed5eb11e518c4ba93714a07134bbb13a97d421cce211e1da34682e  python-3.12.10-amd64.exe
--- a/scripts/pyinstaller/notes.txt
+++ b/scripts/pyinstaller/notes.txt
@@ -34,14 +34,14 @@ fns=(
  upx-4.2.4-win32.zip
 )
 [ $w10 ] && fns+=(
-  jinja2-3.1.4-py3-none-any.whl
+  jinja2-3.1.6-py3-none-any.whl
  MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl
  mutagen-1.47.0-py3-none-any.whl
  packaging-24.1-py3-none-any.whl
-  pillow-11.1.0-cp312-cp312-win_amd64.whl
+  pillow-11.2.1-cp312-cp312-win_amd64.whl
  pyinstaller-6.10.0-py3-none-win_amd64.whl
  pyinstaller_hooks_contrib-2024.8-py3-none-any.whl
-  python-3.12.9-amd64.exe
+  python-3.12.10-amd64.exe
 )
 [ $w7 ] && fns+=(
  future-1.0.0-py3-none-any.whl
--- a/scripts/sfx.py
+++ b/scripts/sfx.py
@@ -413,6 +413,9 @@ def run_i(ld):
    for x in ld:
        sys.path.insert(0, x)

+    e = os.environ
+    e["PRTY_NO_IMPRESO"] = "1"
+
    from copyparty.__main__ import main as p

    p()
--- a/scripts/strip_hints/a.py
+++ b/scripts/strip_hints/a.py
@@ -84,6 +84,9 @@ def uh2(fp):
        if "  # !rm" in ln:
            continue

+        if ln.endswith("TYPE_CHECKING"):
+            ln = ln.replace("from typing import TYPE_CHECKING", "TYPE_CHECKING = False")
+
        lns.append(ln)

    cs = "\n".join(lns)
--- a/scripts/tl.js
+++ b/scripts/tl.js
@@ -148,6 +148,7 @@ var tl_browser = {
 				["U/O", "skip 10sec back/fwd"],
 				["0..9", "jump to 0%..90%"],
 				["P", "play/pause (also initiates)"],
+				["S", "select playing song"],
 				["Y", "download song"],
 			], [
 				"image-viewer",
@@ -156,6 +157,7 @@ var tl_browser = {
 				["F", "fullscreen"],
 				["R", "rotate clockwise"],
 				["🡅 R", "rotate ccw"],
+				["S", "select pic"],
 				["Y", "download pic"],
 			], [
 				"video-player",
@@ -235,7 +237,8 @@ var tl_browser = {

 		"ul_par": "parallel uploads:",
 		"ut_rand": "randomize filenames",
-		"ut_u2ts": "copy the last-modified timestamp$Nfrom your filesystem to the server",
+		"ut_u2ts": "copy the last-modified timestamp$Nfrom your filesystem to the server\">📅",
+		"ut_ow": "overwrite existing files on the server?$N🛡️: never (will generate a new filename instead)$N🕒: overwrite if server-file is older than yours$N♻️: always overwrite if the files are different",
 		"ut_mt": "continue hashing other files while uploading$N$Nmaybe disable if your CPU or HDD is a bottleneck",
 		"ut_ask": 'ask for confirmation before upload starts">💭',
 		"ut_pot": "improve upload speed on slow devices$Nby making the UI less complex",
@@ -327,7 +330,7 @@ var tl_browser = {
 		"cut_nag": "OS notification when upload completes$N(only if the browser or tab is not active)",
 		"cut_sfx": "audible alert when upload completes$N(only if the browser or tab is not active)",

-		"cut_mt": "use multithreading to accelerate file hashing$N$Nthis uses web-workers and requires$Nmore RAM (up to 512 MiB extra)$N$N30% faster https, 4.5x faster http,$Nand 5.3x faster on android phones\">mt",
+		"cut_mt": "use multithreading to accelerate file hashing$N$Nthis uses web-workers and requires$Nmore RAM (up to 512 MiB extra)$N$Nmakes https 30% faster, http 4.5x faster\">mt",

 		"cft_text": "favicon text (blank and refresh to disable)",
 		"cft_fg": "foreground color",
@@ -349,10 +352,12 @@ var tl_browser = {
 		"ml_pmode": "at end of folder...",
 		"ml_btns": "cmds",
 		"ml_tcode": "transcode",
+		"ml_tcode2": "transcode to",
 		"ml_tint": "tint",
 		"ml_eq": "audio equalizer",
 		"ml_drc": "dynamic range compressor",

+		"mt_loop": "loop/repeat one song\">🔁",
 		"mt_shuf": "shuffle the songs in each folder\">🔀",
 		"mt_aplay": "autoplay if there is a song-ID in the link you clicked to access the server$N$Ndisabling this will also stop the page URL from being updated with song-IDs when playing music, to prevent autoplay if these settings are lost but the URL remains\">a▶",
 		"mt_preload": "start loading the next song near the end for gapless playback\">preload",
@@ -372,6 +377,14 @@ var tl_browser = {
 		"mt_cflac": "convert flac / wav to opus\">flac",
 		"mt_caac": "convert aac / m4a to opus\">aac",
 		"mt_coth": "convert all others (not mp3) to opus\">oth",
+		"mt_c2opus": "best choice for desktops, laptops, android\">opus",
+		"mt_c2owa": "opus-weba, for iOS 17.5 and newer\">owa",
+		"mt_c2caf": "opus-caf, for iOS 11 through 17\">caf",
+		"mt_c2mp3": "use this on very old devices\">mp3",
+		"mt_c2ok": "nice, good choice",
+		"mt_c2nd": "that's not the recommended output format for your device, but that's fine",
+		"mt_c2ng": "your device does not seem to support this output format, but let's try anyways",
+		"mt_xowa": "there are bugs in iOS preventing background playback using this format; please use caf or mp3 instead",
 		"mt_tint": "background level (0-100) on the seekbar$Nto make buffering less distracting",
 		"mt_eq": "enables the equalizer and gain control;$N$Nboost &lt;code&gt;0&lt;/code&gt; = standard 100% volume (unmodified)$N$Nwidth &lt;code&gt;1 &nbsp;&lt;/code&gt; = standard stereo (unmodified)$Nwidth &lt;code&gt;0.5&lt;/code&gt; = 50% left-right crossfeed$Nwidth &lt;code&gt;0 &nbsp;&lt;/code&gt; = mono$N$Nboost &lt;code&gt;-0.8&lt;/code&gt; &amp; width &lt;code&gt;10&lt;/code&gt; = vocal removal :^)$N$Nenabling the equalizer makes gapless albums fully gapless, so leave it on with all the values at zero (except width = 1) if you care about that",
 		"mt_drc": "enables the dynamic range compressor (volume flattener / brickwaller); will also enable EQ to balance the spaghetti, so set all EQ fields except for 'width' to 0 if you don't want it$N$Nlowers the volume of audio above THRESHOLD dB; for every RATIO dB past THRESHOLD there is 1 dB of output, so default values of tresh -24 and ratio 12 means it should never get louder than -22 dB and it is safe to increase the equalizer boost to 0.8, or even 1.8 with ATK 0 and a huge RLS like 90 (only works in firefox; RLS is max 1 in other browsers)$N$N(see wikipedia, they explain it much better)",
@@ -390,6 +403,7 @@ var tl_browser = {
 		"mm_eunk": "Unknown Errol",
 		"mm_e404": "Could not play audio; error 404: File not found.",
 		"mm_e403": "Could not play audio; error 403: Access denied.\n\nTry pressing F5 to reload, maybe you got logged out",
+		"mm_e500": "Could not play audio; error 500: Check server logs.",
 		"mm_e5xx": "Could not play audio; server error ",
 		"mm_nof": "not finding any more audio files nearby",
 		"mm_prescan": "Looking for music to play next...",
@@ -404,6 +418,7 @@ var tl_browser = {
 		"f_bigtxt": "this file is {0} MiB large -- really view as text?",
 		"fbd_more": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_more">show {2}</a> or <a href="#" id="bd_all">show all</a></div>',
 		"fbd_all": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_all">show all</a></div>',
+		"f_anota": "only {0} of the {1} items were selected;\nto select the full folder, first scroll to the bottom",

 		"f_dls": 'the file links in the current folder have\nbeen changed into download links',

@@ -613,8 +628,10 @@ var tl_browser = {
 		"u_ewrite": 'you do not have write-access to this folder',
 		"u_eread": 'you do not have read-access to this folder',
 		"u_enoi": 'file-search is not enabled in server config',
+		"u_enoow": "overwrite will not work here; need Delete-permission",
 		"u_badf": 'These {0} files (of {1} total) were skipped, possibly due to filesystem permissions:\n\n',
 		"u_blankf": 'These {0} files (of {1} total) are blank / empty; upload them anyways?\n\n',
+		"u_applef": 'These {0} files (of {1} total) are probably undesirable;\nPress <code>OK/Enter</code> to SKIP the following files,\nPress <code>Cancel/ESC</code> to NOT exclude, and UPLOAD those as well:\n\n',
 		"u_just1": '\nMaybe it works better if you select just one file',
 		"u_ff_many": "if you're using <b>Linux / MacOS / Android,</b> then this amount of files <a href=\"https://bugzilla.mozilla.org/show_bug.cgi?id=1790500\" target=\"_blank\"><em>may</em> crash Firefox!</a>\nif that happens, please try again (or use Chrome).",
 		"u_up_life": "This upload will be deleted from the server\n{0} after it completes",
--- a/tests/test_dots.py
+++ b/tests/test_dots.py
@@ -371,7 +371,7 @@ class TestDots(unittest.TestCase):
        return " ".join(tar)

    def curl(self, url, uname, binary=False, req=b""):
-        req = req or hdr(url, uname)
+        req = req or hdr(url.replace("th=x", "th=j"), uname)
        conn = self.conn.setbuf(req)
        HttpCli(conn).run()
        if binary:
--- a/tests/util.py
+++ b/tests/util.py
@@ -129,13 +129,13 @@ class Cfg(Namespace):
    def __init__(self, a=None, v=None, c=None, **ka0):
        ka = {}

-        ex = "chpw daw dav_auth dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid gsel hardlink ih ihead magic hardlink_only nid nih no_acode no_athumb no_bauth no_clone no_cp no_dav no_db_ip no_del no_dirsz no_dupe no_lifetime no_logues no_mv no_pipe no_poll no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nsort nw og og_no_head og_s_title ohead q rand re_dirsz rss smb srch_dbg srch_excl stats uqe vague_403 vc ver write_uplog xdev xlink xvol zs"
+        ex = "chpw daw dav_auth dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid gsel hardlink ih ihead magic hardlink_only nid nih no_acode no_athumb no_bauth no_clone no_cp no_dav no_db_ip no_del no_dirsz no_dupe no_lifetime no_logues no_mv no_pipe no_poll no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nsort nw og og_no_head og_s_title ohead q rand re_dirsz rss smb srch_dbg srch_excl stats uqe vague_403 vc ver wo_up_readme write_uplog xdev xlink xvol zipmaxu zs"
        ka.update(**{k: False for k in ex.split()})

        ex = "dav_inf dedup dotpart dotsrch hook_v no_dhash no_fastboot no_fpool no_htp no_rescan no_sendfile no_ses no_snap no_up_list no_voldump re_dhash plain_ip"
        ka.update(**{k: True for k in ex.split()})

-        ex = "ah_cli ah_gen css_browser hist ipu js_browser js_other mime mimes no_forget no_hash no_idx nonsus_urls og_tpl og_ua"
+        ex = "ah_cli ah_gen css_browser dbpath hist ipu js_browser js_other mime mimes no_forget no_hash no_idx nonsus_urls og_tpl og_ua ua_nodoc ua_nozip"
        ka.update(**{k: None for k in ex.split()})

        ex = "hash_mt hsortn safe_dedup srch_time u2abort u2j u2sz"
@@ -144,16 +144,16 @@ class Cfg(Namespace):
        ex = "au_vol dl_list mtab_age reg_cap s_thead s_tbody th_convt ups_who zip_who"
        ka.update(**{k: 9 for k in ex.split()})

-        ex = "db_act k304 loris no304 re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
+        ex = "db_act forget_ip k304 loris no304 re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo u2ow zipmaxn zipmaxs"
        ka.update(**{k: 0 for k in ex.split()})

-        ex = "ah_alg bname chpw_db doctitle df exit favico idp_h_usr ipa html_head lg_sba lg_sbf log_fk md_sba md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i shr tcolor textfiles unlist vname xff_src R RS SR"
+        ex = "ah_alg bname chpw_db doctitle df exit favico idp_h_usr ipa html_head lg_sba lg_sbf log_fk md_sba md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i shr tcolor textfiles unlist vname xff_src zipmaxt R RS SR"
        ka.update(**{k: "" for k in ex.split()})

-        ex = "ban_403 ban_404 ban_422 ban_pw ban_url"
+        ex = "ban_403 ban_404 ban_422 ban_pw ban_url spinner"
        ka.update(**{k: "no" for k in ex.split()})

-        ex = "grp on403 on404 xac xad xar xau xban xbc xbd xbr xbu xiu xm"
+        ex = "ext_th grp on403 on404 xac xad xar xau xban xbc xbd xbr xbu xiu xm"
        ka.update(**{k: [] for k in ex.split()})

        ex = "exp_lg exp_md"
@@ -174,6 +174,7 @@ class Cfg(Namespace):
            lang="eng",
            log_badpwd=1,
            logout=573,
+            md_hist="s",
            mte={"a": True},
            mth={},
            mtp=[],
Author	SHA1	Message	Date
ed	38c2dcce3e	v1.16.21	2025-04-20 18:36:32 +00:00
ed	5b3a5fe76b	show warning on ctrl-a in lazyloaded folders	2025-04-20 13:33:01 +00:00
ed	d5a9bd80b2	docker: hide healthcheck from logs	2025-04-20 12:26:56 +00:00
ed	71c5565949	add button to loop/repeat music; closes #156	2025-04-20 11:45:48 +00:00
ed	db33d68d42	zip-download: eagerly 64bit data-descriptors; closes #155 this avoids a false-positive in the info-zip unzip zipbomb detector. unfortunately, * now impossible to extract large (4 GiB) zipfiles using old software (WinXP, macos 10.12) * now less viable to stream download-as-zip into a zipfile unpacker (please use download-as-tar for that purpose) context: the zipfile specification (APPNOTE.TXT) is slightly ambiguous as to when data-descriptor (0x504b0708) filesize-fields change from 32bit to 64bit; both copyparty and libarchive independently made the same interpretation that this is only when the local header is zip64, AND the size-fields are both 0xFFFFFFFF. This makes sense because the data descriptor is only necessary when that particular file-to-be-added exceeds 4 GiB, and/or when the crc32 is not known ahead of time. another interpretation, seen in an early version of the patchset to fix CVE-2019-13232 (zip-bombs) in the info-zip unzip command, believes the only requirement is that the local header is zip64. in many linux distributions, the unzip command would thus fail on zipfiles created by copyparty, since they (by default) satisfy the three requirements to hit the zipbomb false-positive: * total filesize exceeds 4 GiB, and... * a mix of regular (32bit) and zip64 entries, and... * streaming-mode zipfile (not made with ?zip=crc) this issue no longer exists in a more recent version of that patchset, https://github.com/madler/unzip/commit/af0d07f95809653b but this fix has not yet made it into most linux distros	2025-04-17 18:52:47 +00:00
ed	e1c20c7a18	readme: mention bootable flashdrive / cdrom	2025-04-17 18:45:50 +00:00
ed	d3f1b45ce3	update pkgs to 1.16.20	2025-04-13 22:32:06 +00:00
ed	c7aa1a3558	v1.16.20	2025-04-13 21:51:39 +00:00
ed	7b2bd6da83	fix sorting of japanese folders directory-tree sidebar did not sort correctly for non-ascii names also fix a natural-sort bug; it only took effect for the initial folder load, and not when changing the sort-order also, natural-sort will now apply to all non-numeric fields, not just the filename like before	2025-04-13 21:11:07 +00:00
ed	2bd955ba9f	race-the-beam: improve phrasing	2025-04-13 18:51:45 +00:00
ed	98dcaee210	workaround ffmpeg-bug 10797 reduces ram usage from 1534 to 230 MiB when generating spectrograms of files which are decoded by libopenmpt, so most s3xmodit formats	2025-04-13 18:51:35 +00:00
ed	361aebf877	warn on zeroconf with uds-only	2025-04-13 16:38:29 +00:00
ed	ffc1610980	dont crash if qrcode + mdns + uds	2025-04-13 16:11:36 +00:00
ed	233075aee7	ctrl-a selects all files in gridview too	2025-04-13 16:09:49 +00:00
ed	d1a4d335df	increase treenav scroll-margins was too small in deep folders, and/or long foldernames	2025-04-13 16:09:14 +00:00
ed	96acbd3593	cleanup * remove cpr bonk (deadcode) * remove get_vpath (wasteful)	2025-04-13 16:08:44 +00:00
thaddeus kuah	4b876dd133	full lowercase on login button to match the page Signed-off-by: thaddeus kuah <tk@tkkr.dev>	2025-04-11 23:56:51 +02:00
ed	a06c5eb048	new xau hook: podcast-normalizer.py	2025-04-09 19:44:13 +00:00
ed	c9cdc3e1c1	update pkgs to 1.16.19	2025-04-08 21:52:43 +00:00
ed	c0becc6418	v1.16.19	2025-04-08 21:32:51 +00:00
ed	b17ccc38ee	prefer XDG_CONFIG_HOME on all platforms convenient escape-hatch	2025-04-08 19:23:14 +00:00
ed	acfaacbd46	enforce single-instance for session/shares db use file-locking to detect and prevent misconfigurations which could lead to subtle unexpected behavior	2025-04-08 19:08:12 +00:00
ed	8e0364efad	if this is wrong i blame suzubrah for playing entirely too hype music at 6am in the fkn morning improve shares/session-db smoketests and error semantics	2025-04-08 05:42:21 +00:00
ed	e3043004ba	improve u2ow phrasing	2025-04-07 20:48:43 +00:00
ed	b2aaf40a3e	speedgolf in some envs (unsure which), importlib.resources is an expensive import; drop it when we know it's useless	2025-04-07 20:34:55 +00:00
ed	21db8833dc	tests: fix for `f9954bc4e5`	2025-04-07 18:59:43 +00:00
ed	ec14c3944e	fix DeprecationWarning: Accessing argon2.__version__ is deprecated and will be removed in a future release. Use importlib.metadata directly to query for structlog's packaging metadata.	2025-04-07 18:51:13 +00:00
ed	20920e844f	svg newlines + fix cleaner warning: * support newlines in svg files; * `--error--\ncheck\nserver\nlog` * `upload\nonly` * thumbnails of files with lastmodified year 1601 would make the cleaner print a harmless but annoying warning	2025-04-07 18:47:20 +00:00
ed	f9954bc4e5	smoketest fs-access when transcoding the thumbnailer / audio transcoder could return misleading errors if the operation fails due to insufficient filesystem permissions try reading a few bytes from the file and bail early if it fails, and detect/log unwritable output folders for thumbnails also fixes http-response to only return svg-formatted errors if the initial request expects a picture in response, not audio	2025-04-07 18:41:37 +00:00
thaddeus kuah	d450f61534	Apply custom fonts to buttons and input fields (#152 ) * set custom font for inputs and buttons Signed-off-by: Thaddeus Kuah <tk@tkkr.dev>	2025-04-06 19:15:10 +00:00
ed	2b50fc2010	fix mkdir in symlinked folders; closes #151 remove an overly careful safety-check which would refuse creating directories if the location was outside of the volume's base-path it is safe to trust `rem` due to `vpath = undot(vpath)` and a similar check being performed inside `vfs.get` as well, so this served no purpose	2025-04-06 09:18:40 +00:00
ed	c2034f7bc5	add GoogleOther to bad-crawlers list	2025-04-01 21:29:58 +02:00
ed	cec3bee020	forbid all use of LLM / AI when writing code	2025-03-31 17:25:56 +00:00
ed	e1b9ac631f	separate histpath and dbpath options (#149 ) the up2k databases are, by default, stored in a `.hist` subfolder inside each volume, next to thumbnails and transcoded audio add a new option for storing the databases in a separate location, making it possible to tune the underlying filesystem for optimal performance characteristics the `--hist` global-option and `hist` volflag still behave like before, but `--dbpath` and volflag `dbpath` will override the histpath for the up2k-db and up2k-snap exclusivey	2025-03-30 16:08:28 +00:00
ed	19ee64e5e3	clarify that all dependencies are optional (#149 )	2025-03-30 13:30:52 +00:00
ed	4f397b9b5b	add zfs-tune (#149 )	2025-03-30 13:30:15 +00:00
ed	71775dcccb	mention mimalloc	2025-03-30 13:17:12 +00:00
ed	b383c08cc3	add review from ixbt forums	2025-03-29 13:57:35 +00:00
ed	fc88341820	add option to store markdown backups elsewhere `--md-hist` / volflag `md_hist` specifies where to put old versions of markdown files when edited using the web-ui; * `s` = create `.hist` subfolder next to the markdown file (the default, both previously and now) * `v` = use the volume's hist-path, either according to `--hist` or the `hist` volflag. NOTE: old versions will not be retrievable through the web-ui * `n` = nope / disabled; overwrite without backup	2025-03-26 20:07:35 +00:00
ed	43bbd566d7	mention mimalloc in docker-compose examples (thx thad)	2025-03-24 23:19:17 +00:00
ed	e1dea7ef3e	dangit	2025-03-23 23:28:05 +00:00
ed	de2fedd2cd	update pkgs to 1.16.18	2025-03-23 23:04:53 +00:00
ed	6aaafeee6d	v1.16.18	2025-03-23 22:16:40 +00:00
ed	99f63adf58	google isn't taking the hint specifically google, but also some others, have started ignoring rel="nofollow" while also understanding just enough javascript to try viewing binary files as text	2025-03-23 21:21:41 +00:00
ed	de2c978842	docker: suggest mimalloc	2025-03-23 20:45:03 +00:00
ed	3c90cec0cd	forgot these pyinstaller/build.sh: fix jinja2 after upgrade up2k.py: fix double-hs after dupe finalize	2025-03-23 20:19:18 +00:00
ed	57a56073d8	use zlib-ng when available download-as-tar-gz becomes 2.4x faster in docker segfaults on windows, so don't use it there does not affect fedora or gentoo, since zlib-ng is already system-default on those also adds a global-option to write list of successful binds to a textfile, for automation / smoketest purposes	2025-03-23 20:15:21 +00:00
ed	2525d594c5	`19a5985f` removed the restriction on uploading logues, as it was too restrictive, blocking editing through webdav and ftp but since logues and readmes can be used as helptext for users with write-only access, it makes sense to block logue/readme uploads from write-only users users with write-only access can still upload any file as before, but the filename prefix `_wo_` is added onto files named either README.md \| PREADME.md \| .prologue.html \| .epilogue.html the new option `--wo-up-readme` restores previous behavior, and will not add the filename-prefix for readmes/logues	2025-03-22 14:21:35 +00:00
ed	a0ecc4d88e	update pkgs to 1.16.17	2025-03-16 21:13:23 +00:00
ed	accd003d15	v1.16.17	2025-03-16 20:02:51 +00:00
ed	9c2c423761	IdP: extend ${u} with syntax to exclude by group just like before, if vpath contains ${u} then the IdP-volume is created unconditionally but this is new: ${u%+foo} creates the vol only if user is member of group foo ${u%-foo} creates the vol if user is NOT member of group foo	2025-03-16 19:28:23 +00:00
ed	999789c742	improve accuracy of failsafe-check also fix fsutil relabel after `8417098c`	2025-03-16 18:49:42 +00:00
ed	14bb299918	hide zip-link when user not allowed	2025-03-16 18:08:20 +00:00
ed	0a33336dd4	cosmetic: fix zipmax in up2k volume-listing	2025-03-16 17:51:50 +00:00
ed	6a2644fece	set nofollow on ?doc links google keeps trying to read binaries as text, maybe now it won't	2025-03-16 11:57:42 +00:00
ed	5ab09769e1	move symlinks as-is; don't expand into full files previously, when moving or renaming a symlink to a file (or a folder with symlinks inside), the dedup setting would decide whether those links would be expanded into full files or not with dedup disabled (which is the default), all symlinks would be expanded during a move operation now, the dedup-setting is ignored when files/folders are moved, but it still applies when uploading or copying files/folders * absolute symlinks are moved as-is * relative symlinks are rewritten as necessary, assuming both source and destination is known in db	2025-03-15 23:54:32 +00:00
ed	782084056d	filter appledoubles from uploads should catch all the garbage that macs sprinkle onto flashdrives; https://a.ocv.me/pub/stuff/?doc=appledoubles-and-friends.txt will notice and suggest to skip the following files/dirs: * __MACOSX * .DS_Store * .AppleDouble * .LSOverride * .DocumentRevisions-* * .fseventsd * .Spotlight-V* * .TemporaryItems * .Trashes * .VolumeIcon.icns * .com.apple.timemachine.donotpresent * .AppleDB * .AppleDesktop * .apdisk and conditionally ._foo.jpg if foo.jpg is also being uploaded	2025-03-15 21:16:54 +00:00
ed	494179bd1c	optional max-size for download-as-zip/tar	2025-03-14 23:36:01 +00:00
ed	29a17ae2b7	fix detection of unsafe IdP volumes; closes #147 was overly aggressive until now, thinking the following was unsafe: -v 'x::' # no-anonymous-access -v 'x/${u}:${u}:r:A,${u}' # world-readable,user-admin -v 'x/${u}/priv:${u}/priv:A,${u}' # only-user-admin now it realizes that this is safe because both IdP volumes will be created/owned by the same user however, if the first volume is 'x::r' then this is NOT safe, and is now still correctly detected as being dangerous also add a separate warning if `${g}` and `${u}` is mixed in a volpath, since that is PROBABLY (not provably) unsafe	2025-03-14 21:08:21 +00:00
ed	815d46f2c4	this keyboard sure is bouncy	2025-03-09 21:14:53 +01:00
ed	8417098c68	fix dl from shares with `-j0`; closes #146 `write_dls` assumed `vfs.all_nodes` included shares; make it so shares now also appear in the active-downloads list, but the URL is hidden unless the viewer definitely already knows the share exists (which is why vfs-nodes now have `shr_owner`) also adds PRTY_FORCE_MP, a beefybit (opposite of chickenbit) to allow multiprocessing on known-buggy platforms (macos)	2025-03-09 21:10:31 +01:00
ed	25974d660d	improve errmsg when reading non-utf8 files (#143 ) previously, the native python-error was printed when reading the contents of a textfile using the wrong character encoding while technically correct, it could be confusing for end-users add a helper to produce a more helpful errormessage when someone (for example) tries to load a latin-1 config file	2025-03-09 11:59:33 +01:00
ed	12fcb42201	github: mention preferred language	2025-03-08 23:58:07 +02:00
ed	16462ee573	xff-warning: suggest proper /64 for ipv6	2025-03-06 19:57:20 +01:00
ed	540664e0c2	usb-eject nitpicks * fix navpane reload * strip trailing newlines in toasts	2025-03-02 23:51:35 +00:00
ed	b5cb763ab1	usb-eject: treepar fix since this dumb plugin found an actual usecase, fix the most glaring issue when nodes overflow from treeul into treepar, the eject-button is cloned over as well, but the clone does nothing (as expected), though this will also cause a flood of new eject-buttons appearing, and that's worth fixing NB: check treeul + treepar explicitly; avoid docul	2025-03-01 22:03:18 +00:00
ed	c24a0ec364	update pkgs to 1.16.16	2025-02-28 19:16:27 +00:00
ed	4accef00fb	v1.16.16	2025-02-28 18:46:32 +00:00
ed	d779525500	move `-volflag` warning to avoid false positives	2025-02-28 18:13:23 +00:00
ed	65a7706f77	add helptext for volflags dk, dks, dky	2025-02-28 17:56:51 +00:00
ed	5e12abbb9b	ignore impossible lastmod on upload; closes #142 android-chrome bug https://issues.chromium.org/issues/393149335 sends last-modified time `-11644473600` for all uploads this has been fixed in chromium, but there might be similar bugs in other browsers, so add server-side and client-side detection for unreasonable lastmod times previously, if the js detected a similar situation, it would substitute the lastmod-time with the client's wallclock, but now the server's wallclock is always preferrred as fallback	2025-02-28 17:48:14 +00:00
daimond113	e0fe2b97be	nix: add mainProgram Silences warnings like "getExe: Package "copyparty-1.16.15" does not have the meta.mainProgram attribute. We'll assume that the main program has the same name for now, but this behavior is deprecated, because it leads to surprising errors when the assumption does not hold. If the package has a main program, please set `meta.mainProgram` in its definition to make this warning go away. Otherwise, if the package does not have a main program, or if you don't control its definition, use getExe' to specify the name to the program, such as lib.getExe' foo "bar"."	2025-02-26 23:07:19 +01:00
ed	bd33863f9f	update pkgs to 1.16.15	2025-02-25 01:25:15 +00:00
ed	a011139894	v1.16.15	2025-02-25 00:17:58 +00:00
ed	36866f1d36	dangit.wav	2025-02-25 00:11:57 +00:00
ed	407531bcb1	fix markdown / text-editor jank * only indicate file-history for markdown files since other files won't load into the editor which makes that entirely pointless; do file extension instead * text-editor: in files containing one single line, ^C followed by ^V ^Z would accidentally a letter and fix unhydrated extensions	2025-02-25 00:03:22 +00:00
ed	3adbb2ff41	https://youtu.be/WyXebd3I3Vo	2025-02-24 23:32:03 +00:00
ed	499ae1c7a1	other minor html-escaping fixes mostly related to error-handling for uploads, network-loss etc, nothing worse than the dom-xss just now	2025-02-24 22:42:05 +00:00
ed	438ea6ccb0	fix GHSA-m2jw-cj8v-937r ; this fixes a DOM-Based XSS when preparing files for upload; empty files would have their filenames rendered as HTML in a messagebox, making it possible to trick users into running arbitrary javascript by giving them maliciously-named files note that, being a general-purpose webserver, it is still intentionally possible to upload and execute arbitrary javascript, just not in this unexpected manner	2025-02-24 21:23:13 +00:00
ed	598a29a733	mention sony psp support (thx dwarf)	2025-02-23 21:37:21 +00:00
ed	6d102fc826	mention risc-v support	2025-02-20 04:51:04 +00:00
ed	fca07fbb62	update pkgs to 1.16.14	2025-02-19 23:35:05 +00:00
ed	cdedcc24b8	v1.16.14	2025-02-19 23:09:14 +00:00
ed	60d5f27140	new example: randpic.py	2025-02-19 22:41:30 +00:00
ed	cb413bae49	webdav: a healthy dash of paranoia there's probably at least one client sending `Overwrite: False` instead of the spec-correct `Overwrite: F`	2025-02-19 22:07:26 +00:00
ed	e9f78ea70c	up2k: tristate option for overwriting files; closes #139 adds a third possible value for the `replace` property in handshakes: * absent or False: never overwrite an existing file on the server, and instead generate a new filename to avoid collision * True: always overwrite existing files on the server * "mt": only overwrite if client's last-modified is more recent (this is the new option) the new UI button toggles between all three options, defaulting to never-overwrite	2025-02-19 21:58:56 +00:00
ed	6858cb066f	spinner: themes + improve positioning loading-spinner is either `#dlt_t` or `#dlt_f` (tree or files), appearing top-left or top-right, regardless of page/tree scroll (position:fixed)	2025-02-19 18:55:33 +00:00
ed	4be0d426f4	option to forget uploader-IP from db after some time does this mean copyparty is GDPR-compliant now? idklol	2025-02-17 23:47:59 +00:00
ed	7d7d5d6c3c	fix custom spinner css on initial page load	2025-02-17 23:26:21 +00:00
ed	0422387e90	readme: changing the loading spinner (#138 )	2025-02-16 19:28:57 +00:00
ed	2ed5fd9ac4	readme: diagnosing broken thumbnails (#137 )	2025-02-16 19:22:17 +00:00
ed	2beb2acc24	readme: permanent cloudflare tunnel (#137 )	2025-02-16 18:59:18 +00:00
ed	56ce591908	synology dsm: add updating	2025-02-16 18:12:35 +00:00
ed	b190e676b4	fix cosmetic volflag stuff: * `xz` would show the "unrecognized volflag" warning, but it still applied correctly * removing volflags with `-foo` would also show the warning but it would still get removed correctly * hide `ext_th_d` in the startup volume-listing	2025-02-14 20:54:13 +00:00
ed	19520b2ec9	remove patch for musl cve (no longer necessary)	2025-02-14 09:15:52 +00:00
ed	eeb96ae8b5	update pkgs to 1.16.13	2025-02-13 21:43:32 +00:00
ed	cddedd37d5	v1.16.13	2025-02-13 20:57:04 +00:00
ed	4d6626b099	workaround musl 1.2.5 cve	2025-02-13 20:53:47 +00:00
ed	7a55833bb2	silence linter	2025-02-13 18:34:41 +00:00
ed	7e4702cf09	file-extension icons global-option / volflag `ext_th` specifies custom thumbnail for a given file extension	2025-02-13 18:32:01 +00:00
ed	685f08697a	alternative loader spinners	2025-02-13 17:07:48 +00:00
ed	a255db706d	make volflags less confusing 1. warn about unrecognized volflags previously, when specifying an unknown volflag, it would be silently ignored, giving the impression that it applied 2. also allow uppercase, kebab-case (previously, only snake_case was accepted) 3. mention every volflag in --help-flags (some volflags were missing)	2025-02-13 00:34:46 +00:00
ed	9d76902710	WebDAV: adjust 401-mask for GETs (#136 ) some clients, including KDE Dolphin (kioworker/6.10) keeps sending requests without the basic-auth header, expecting the server to respond with a 401 before it does most clients only do this for the initial request, which is usually a PROPFIND, which makes this nice and simple -- but turns out we need to consider this for GET as well... this is tricky because a graphical webbrowser must never receive a 401 lest it becomes near-impossible to deauth, and that's exactly what Dolphin pretends to be in its UA man （　´_ゝ`） note: `KIO/` hits konqueror so don't	2025-02-11 23:32:44 +00:00
ed	62ee7f6980	WebDAV: support COPY, KDE-Dolphin (#136 ): * add support for the COPY verb * COPY/MOVE: add overwrite support; default is True according to rfc (only applies to single files for now) * COPY/MOVE/MKCOL: return 401 as necessary for clients which rechallenge frequently such as KDE Dolphin (KIO/6.10) * MOVE: support webdav:// Destination prefix as used by KDE Dolphin (KIO/6.10) * MOVE: vproxy support	2025-02-11 21:34:24 +00:00
ed	2f6707825a	improve usb-eject hook: * don't crash internet explorer * support running as root * support old linuxen	2025-02-10 04:52:57 +00:00
ed	7dda77dcb4	update pkgs to 1.16.12	2025-02-10 00:04:04 +00:00