v1.13.4

hooks can be restricted to users with certain permissions, for example
`--xm aw,notify-send` will only `notify-send` if user has write-access

the user's list of permissions are now also included in the json
that is passed to the hook if enabled; `--xm aw,j,notify-send`

will now also stop parsing flags when encountering a blank value,
allowing to specify any initial arguments to the command:
`--xm aw,j,,notify-send,hey` would run `notify-send` with `hey`
as its first argument, and the json would be the 2nd argument,
similarly `--xm ,notify-send,hey` when no flags specified

this is somewhat explained in `--help-hooks`, but
additional related features are planned in the near future
and will all be better documented when the dust settles

README.md

@@ -83,6 +83,8 @@ turn almost any device into a file server with resumable uploads/downloads using
     * [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
         * [real-ip](#real-ip) - teaching copyparty how to see client IPs
     * [prometheus](#prometheus) - metrics/stats can be enabled
+    * [other extremely specific features](#other-extremely-specific-features) - you'll never find a use for these
+        * [custom mimetypes](#custom-mimetypes) - change the association of a file extension
 * [packages](#packages) - the party might be closer than you think
     * [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
     * [fedora package](#fedora-package) - does not exist yet
@@ -573,6 +575,7 @@ it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video f
 audio files are covnerted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
 
 images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
+* the order is significant, so if both `cover.png` and `folder.jpg` exist in a folder, it will pick the first matching `--th-covers` entry (`folder.jpg`)
 * and, if you enable [file indexing](#file-indexing), it will also try those names as dotfiles (`.folder.jpg` and so), and then fallback on the first picture in the folder (if it has any pictures at all)
 
 in the grid/thumbnail view, if the audio player panel is open, songs will start playing when clicked
@@ -580,6 +583,7 @@ in the grid/thumbnail view, if the audio player panel is open, songs will start
 
 enabling `multiselect` lets you click files to select them, and then shift-click another file for range-select
 * `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
+  * the `sel` option can be made default globally with `--gsel` or per-volume with volflag `gsel`
 
 
 ## zip downloads
@@ -709,7 +713,7 @@ uploads can be given a lifetime,  afer which they expire / self-destruct
 
 the feature must be enabled per-volume with the `lifetime` [upload rule](#upload-rules) which sets the upper limit for how long a file gets to stay on the server
 
-clients can specify a shorter expiration time using the [up2k ui](#uploading) -- the relevant options become visible upon navigating into a folder with `lifetimes` enabled -- or by using the `life` [upload modifier](#write)
+clients can specify a shorter expiration time using the [up2k ui](#uploading) -- the relevant options become visible upon navigating into a folder with `lifetimes` enabled -- or by using the `life` [upload modifier](./docs/devnotes.md#write)
 
 specifying a custom expiration time client-side will affect the timespan in which unposts are permitted, so keep an eye on the estimates in the up2k ui
 
@@ -877,6 +881,8 @@ see [./srv/expand/](./srv/expand/) for usage and examples
 
 * files named `.prologue.html` / `.epilogue.html` will be rendered before/after directory listings unless `--no-logues`
 
+* files named `descript.ion` / `DESCRIPT.ION` are parsed and displayed in the file listing, or as the epilogue if nonstandard
+
 * files named `README.md` / `readme.md` will be rendered after directory listings unless `--no-readme` (but `.epilogue.html` takes precedence)
 
 * `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see `--help-exp`
@@ -1445,8 +1451,9 @@ you can either:
 * or do location-based proxying, using `--rp-loc=/stuff` to tell copyparty where it is mounted -- has a slight performance cost and higher chance of bugs
   * if copyparty says `incorrect --rp-loc or webserver config; expected vpath starting with [...]` it's likely because the webserver is stripping away the proxy location from the request URLs -- see the `ProxyPass` in the apache example below
 
-some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which could be a nice speed boost
-* **warning:** nginx-QUIC is still experimental and can make uploads much slower, so HTTP/2 is recommended for now
+some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which *could* be a nice speed boost, depending on a lot of factors
+* **warning:** nginx-QUIC (HTTP/3) is still experimental and can make uploads much slower, so HTTP/1.1 is recommended for now
+* depending on server/client, HTTP/1.1 can also be 5x faster than HTTP/2
 
 example webserver configs:
 
@@ -1526,6 +1533,28 @@ the following options are available to disable some of the metrics:
 note: the following metrics are counted incorrectly if multiprocessing is enabled with `-j`: `cpp_http_conns`, `cpp_http_reqs`, `cpp_sus_reqs`, `cpp_active_bans`, `cpp_total_bans`
 
 
+## other extremely specific features
+
+you'll never find a use for these:
+
+
+### custom mimetypes
+
+change the association of a file extension
+
+using commandline args, you can do something like `--mime gif=image/jif` and `--mime ts=text/x.typescript` (can be specified multiple times)
+
+in a config-file, this is the same as:
+
+```yaml
+[global]
+  mime: gif=image/jif
+  mime: ts=text/x.typescript
+```
+
+run copyparty with `--mimes` to list all the default mappings
+
+
 # packages
 
 the party might be closer than you think
@@ -1775,6 +1804,8 @@ alternatively, some alternatives roughly sorted by speed (unreproducible benchma
 
 most clients will fail to mount the root of a copyparty server unless there is a root volume (so you get the admin-panel instead of a browser when accessing it) -- in that case, mount a specific volume instead
 
+if you have volumes that are accessible without a password, then some webdav clients (such as davfs2) require the global-option `--dav-auth` to access any password-protected areas
+
 
 # android app
 
@@ -1803,6 +1834,7 @@ defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
 
 below are some tweaks roughly ordered by usefulness:
 
+* disabling HTTP/2 and HTTP/3 can make uploads 5x faster, depending on server/client software
 * `-q` disables logging and can help a bunch, even when combined with `-lo` to redirect logs to file
 * `--hist` pointing to a fast location (ssd) will make directory listings and searches faster when `-e2d` or `-e2t` is set
   * and also makes thumbnails load faster, regardless of e2d/e2t
@@ -1918,7 +1950,7 @@ volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, grant
 
 volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
 
-* `dk` + `dky` gives the same behavior as if all users with `g` access have full read-access, but subfolders are hidden files (their names start with a dot), so `dky` is an alternative to renaming all the folders for that purpose, maybe just for some users
+* `dk` + `dky` gives the same behavior as if all users with `g` access have full read-access, but subfolders are hidden files (as if their names start with a dot), so `dky` is an alternative to renaming all the folders for that purpose, maybe just for some users
 
 volflag `dks` lets people enter subfolders as well, and also enables download-as-zip/tar
 
@@ -1943,7 +1975,7 @@ the default configs take about 0.4 sec and 256 MiB RAM to process a new password
 
 both HTTP and HTTPS are accepted  by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
 
-copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well
+copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3
 
 if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, copyparty will automatically create a CA and server-cert on startup
 * the certs are written to `--crt-dir` for distribution, see `--help` for the other `--crt` options

bin/hooks/README.md

@@ -27,3 +27,5 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
 
 # on message
 * [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
+* [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
+* [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder

bin/hooks/discord-announce.py

@@ -12,19 +12,28 @@ announces a new upload on discord
 example usage as global config:
     --xau f,t5,j,bin/hooks/discord-announce.py
 
+parameters explained,
+    xau = execute after upload
+    f  = fork; don't delay other hooks while this is running
+    t5 = timeout if it's still running after 5 sec
+    j  = this hook needs upload information as json (not just the filename)
+
 example usage as a volflag (per-volume config):
     -v srv/inc:inc:r:rw,ed:c,xau=f,t5,j,bin/hooks/discord-announce.py
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
     (share filesystem-path srv/inc as volume /inc,
      readable by everyone, read-write for user 'ed',
-     running this plugin on all uploads with the params listed below)
+     running this plugin on all uploads with the params explained above)
 
-parameters explained,
-    xbu = execute after upload
-    f  = fork; don't wait for it to finish
-    t5 = timeout if it's still running after 5 sec
-    j  = provide upload information as json; not just the filename
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        xau: f,t5,j,bin/hooks/discord-announce.py
 
 replace "xau" with "xbu" to announce Before upload starts instead of After completion
 

bin/hooks/msg-log.py

@@ -14,19 +14,32 @@ except:
     from datetime import datetime
 
 
-"""
+_ = r"""
 use copyparty as a dumb messaging server / guestbook thing;
+accepts guestbook entries from 📟 (message-to-server-log) in the web-ui
 initially contributed by @clach04 in https://github.com/9001/copyparty/issues/35 (thanks!)
 
-Sample usage:
-
+example usage as global config:
     python copyparty-sfx.py --xm j,bin/hooks/msg-log.py
 
-Where:
+parameters explained,
+    xm = execute on message (📟)
+    j  = this hook needs message information as json (not just the message-text)
 
-    xm = execute on message-to-server-log
-    j = provide message information as json; not just the text - this script REQUIRES json
-    t10 = timeout and kill download after 10 secs
+example usage as a volflag (per-volume config):
+    python copyparty-sfx.py -v srv/log:log:r:c,xm=j,bin/hooks/msg-log.py
+                                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/log as volume /log, readable by everyone,
+     running this plugin on all messages with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/log]
+      srv/log
+      accs:
+        r: *
+      flags:
+        xm: j,bin/hooks/msg-log.py
 """
 
 

bin/hooks/qbittorrent-magnet.py

@@ -0,0 +1,127 @@
+#!/usr/bin/env python3
+# coding: utf-8
+
+import os
+import sys
+import json
+import shutil
+import subprocess as sp
+
+
+_ = r"""
+start downloading a torrent by POSTing a magnet URL to copyparty,
+for example using 📟 (message-to-server-log) in the web-ui
+
+by default it will download the torrent to the folder you were in
+when you pasted the magnet into the message-to-server-log field
+
+you can optionally specify another location by adding a whitespace
+after the magnet URL followed by the name of the subfolder to DL into,
+or for example "anime/airing" would download to /srv/media/anime/airing
+because the keyword "anime" is in the DESTS config below
+
+needs python3
+
+example usage as global config (not a good idea):
+    python copyparty-sfx.py --xm f,j,t60,bin/hooks/qbittorrent-magnet.py
+
+parameters explained,
+    xm = execute on message (📟)
+    f = fork; don't delay other hooks while this is running
+    j = provide message information as json (not just the text)
+    t60 = abort if qbittorrent has to think about it for more than 1 min
+
+example usage as a volflag (per-volume config, much better):
+    -v srv/qb:qb:A,ed:c,xm=f,j,t60,bin/hooks/qbittorrent-magnet.py
+                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+    (share filesystem-path srv/qb as volume /qb with Admin for user 'ed',
+     running this plugin on all messages with the params explained above)
+
+example usage as a volflag in a copyparty config file:
+    [/qb]
+      srv/qb
+      accs:
+        A: ed
+      flags:
+        xm: f,j,t60,bin/hooks/qbittorrent-magnet.py
+
+the volflag examples only kicks in if you send the torrent magnet
+while you're in the /qb folder (or any folder below there)
+"""
+
+
+# list of usernames to allow
+ALLOWLIST = [ "ed", "morpheus" ]
+
+
+# list of destination aliases to translate into full filesystem
+# paths; takes effect if the first folder component in the
+# custom download location matches anything in this dict
+DESTS = {
+    "iso": "/srv/pub/linux-isos",
+    "anime": "/srv/media/anime",
+}
+
+
+def main():
+    inf = json.loads(sys.argv[1])
+    url = inf["txt"]
+    if not url.lower().startswith("magnet:?"):
+        # not a magnet, abort
+        return
+
+    if inf["user"] not in ALLOWLIST:
+        print("🧲 denied for user", inf["user"])
+        return
+
+    # might as well run the command inside the filesystem folder
+    # which matches the URL that the magnet message was sent to
+    os.chdir(inf["ap"])
+
+    # is there is a custom download location in the url?
+    dst = ""
+    if " " in url:
+        url, dst = url.split(" ", 1)
+
+        # is the location in the predefined list of locations?
+        parts = dst.replace("\\", "/").split("/")
+        if parts[0] in DESTS:
+            dst = os.path.join(DESTS[parts[0]], *(parts[1:]))
+
+    else:
+        # nope, so download to the current folder instead;
+        # comment the dst line below to instead use the default
+        # download location from your qbittorrent settings
+        dst = inf["ap"]
+        pass
+
+    # archlinux has a -nox suffix for qbittorrent if headless
+    # so check if we should be using that
+    if shutil.which("qbittorrent-nox"):
+        torrent_bin = "qbittorrent-nox"
+    else:
+        torrent_bin = "qbittorrent"
+
+    # the command to add a new torrent, adjust if necessary
+    cmd = [torrent_bin, url]
+    if dst:
+        cmd += ["--save-path=%s" % (dst,)]
+
+    # if copyparty and qbittorrent are running as different users
+    # you may have to do something like the following
+    # (assuming qbittorrent* is nopasswd-allowed in sudoers):
+    #
+    # cmd = ["sudo", "-u", "qbitter"] + cmd
+
+    print("🧲", cmd)
+
+    try:
+        sp.check_call(cmd)
+    except:
+        print("🧲 FAILED TO ADD", url)
+
+
+if __name__ == "__main__":
+    main()
+

bin/hooks/wget.py

@@ -9,25 +9,37 @@ import subprocess as sp
 _ = r"""
 use copyparty as a file downloader by POSTing URLs as
 application/x-www-form-urlencoded (for example using the
-message/pager function on the website)
+📟 message-to-server-log in the web-ui)
 
 example usage as global config:
     --xm f,j,t3600,bin/hooks/wget.py
 
+parameters explained,
+    xm = execute on message-to-server-log
+    f = fork; don't delay other hooks while this is running
+    j = provide message information as json (not just the text)
+    c3 = mute all output
+    t3600 = timeout and abort download after 1 hour
+
 example usage as a volflag (per-volume config):
     -v srv/inc:inc:r:rw,ed:c,xm=f,j,t3600,bin/hooks/wget.py
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
     (share filesystem-path srv/inc as volume /inc,
      readable by everyone, read-write for user 'ed',
-     running this plugin on all messages with the params listed below)
+     running this plugin on all messages with the params explained above)
 
-parameters explained,
-    xm = execute on message-to-server-log
-    f = fork so it doesn't block uploads
-    j = provide message information as json; not just the text
-    c3 = mute all output
-    t3600 = timeout and kill download after 1 hour
+example usage as a volflag in a copyparty config file:
+    [/inc]
+      srv/inc
+      accs:
+        r: *
+        rw: ed
+      flags:
+        xm: f,j,t3600,bin/hooks/wget.py     
+
+the volflag examples only kicks in if you send the message
+while you're in the /inc folder (or any folder below there)
 """
 
 

bin/u2c.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
 from __future__ import print_function, unicode_literals
 
-S_VERSION = "1.17"
-S_BUILD_DT = "2024-05-09"
+S_VERSION = "1.18"
+S_BUILD_DT = "2024-06-01"
 
 """
 u2c.py: upload to copyparty
@@ -1144,7 +1144,7 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("url", type=unicode, help="server url, including destination folder")
     ap.add_argument("files", type=unicode, nargs="+", help="files and/or folders to process")
     ap.add_argument("-v", action="store_true", help="verbose")
-    ap.add_argument("-a", metavar="PASSWORD", help="password or $filepath")
+    ap.add_argument("-a", metavar="PASSWD", help="password or $filepath")
     ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
     ap.add_argument("-x", type=unicode, metavar="REGEX", default="", help="skip file if filesystem-abspath matches REGEX, example: '.*/\\.hist/.*'")
     ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
@@ -1162,8 +1162,8 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("--drd", action="store_true", help="delete remote files during upload instead of afterwards; reduces peak disk space usage, but will reupload instead of detecting renames")
 
     ap = app.add_argument_group("performance tweaks")
-    ap.add_argument("-j", type=int, metavar="THREADS", default=4, help="parallel connections")
-    ap.add_argument("-J", type=int, metavar="THREADS", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
+    ap.add_argument("-j", type=int, metavar="CONNS", default=2, help="parallel connections")
+    ap.add_argument("-J", type=int, metavar="CORES", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
     ap.add_argument("-nh", action="store_true", help="disable hashing while uploading")
     ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles and macos)")
     ap.add_argument("--cd", type=float, metavar="SEC", default=5, help="delay before reattempting a failed handshake/upload")
@@ -1171,7 +1171,7 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")
 
     ap = app.add_argument_group("tls")
-    ap.add_argument("-te", metavar="PEM_FILE", help="certificate to expect/verify")
+    ap.add_argument("-te", metavar="PATH", help="path to ca.pem or cert.pem to expect/verify")
     ap.add_argument("-td", action="store_true", help="disable certificate check")
     # fmt: on
 
@@ -1208,6 +1208,14 @@ source file/folder selection uses rsync syntax, meaning that:
     ar.url = ar.url.rstrip("/") + "/"
     if "://" not in ar.url:
         ar.url = "http://" + ar.url
+    
+    if "https://" in ar.url.lower():
+        try:
+            import ssl, zipfile
+        except:
+            t = "ERROR: https is not available for some reason; please use http"
+            print("\n\n   %s\n\n" % (t,))
+            raise
 
     if ar.a and ar.a.startswith("$"):
         fn = ar.a[1:]

contrib/package/arch/PKGBUILD

@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.13.1"
+pkgver="1.13.3"
 pkgrel=1
 pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("f103b784c423a45fbab47c584e4cc53d887fe0616f803bffe009fbfdab3963d7")
+sha256sums=("35845d6335fba4a13d153d7062f365dad529202bc865b93267d899e19a0a6da3")
 
 build() {
     cd "${srcdir}/${pkgname}-${pkgver}"

contrib/package/nix/copyparty/pin.json

@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.13.1/copyparty-sfx.py",
-    "version": "1.13.1",
-    "hash": "sha256-NFfnveCrR1SbiNlibVyU3UPePLUGJMc4XZvWdksXNd8="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.13.3/copyparty-sfx.py",
+    "version": "1.13.3",
+    "hash": "sha256-LtbdioAYtWGC4+5frzUjXwm0thubkyMhc86YU/rXIuo="
 }

contrib/systemd/copyparty.service

@@ -4,7 +4,7 @@
 #
 # installation:
 #   wget https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py -O /usr/local/bin/copyparty-sfx.py
-#   useradd -r -s /sbin/nologin -d /var/lib/copyparty copyparty
+#   useradd -r -s /sbin/nologin -m -d /var/lib/copyparty copyparty
 #   firewall-cmd --permanent --add-port=3923/tcp  # --zone=libvirt
 #   firewall-cmd --reload
 #   cp -pv copyparty.service /etc/systemd/system/
@@ -12,11 +12,18 @@
 #   restorecon -vr /etc/systemd/system/copyparty.service  # on fedora/rhel
 #   systemctl daemon-reload && systemctl enable --now copyparty
 #
+# every time you edit this file, you must "systemctl daemon-reload"
+# for the changes to take effect and then "systemctl restart copyparty"
+#
 # if it fails to start, first check this: systemctl status copyparty
 # then try starting it while viewing logs:
 #   journalctl -fan 100
 #   tail -Fn 100 /var/log/copyparty/$(date +%Y-%m%d.log)
 #
+# if you run into any issues, for example thumbnails not working,
+# try removing the "some quick hardening" section and then please
+# let me know if that actually helped so we can look into it
+#
 # you may want to:
 #  - change "User=copyparty" and "/var/lib/copyparty/" to another user
 #  - edit /etc/copyparty.conf to configure copyparty

copyparty/__main__.py

@@ -13,6 +13,7 @@ import base64
 import locale
 import os
 import re
+import select
 import socket
 import sys
 import threading
@@ -41,6 +42,7 @@ from .util import (
     DEF_EXP,
     DEF_MTE,
     DEF_MTH,
+    HAVE_IPV6,
     IMPLICATIONS,
     JINJA_VER,
     MIMES,
@@ -173,8 +175,10 @@ def init_E(EE: EnvParams) -> None:
             (os.environ.get, "TMP"),
             (unicode, "/tmp"),
         ]
+        errs = []
         for chk in [os.listdir, os.mkdir]:
-            for pf, pa in paths:
+            for npath, (pf, pa) in enumerate(paths):
+                p = ""
                 try:
                     p = pf(pa)
                     # print(chk.__name__, p, pa)
@@ -187,9 +191,20 @@ def init_E(EE: EnvParams) -> None:
                     if not os.path.isdir(p):
                         os.mkdir(p)
 
+                    if npath > 1:
+                        t = "Using [%s] for config; filekeys/dirkeys will change on every restart. Consider setting XDG_CONFIG_HOME or giving the unix-user a ~/.config/"
+                        errs.append(t % (p,))
+                    elif errs:
+                        errs.append("Using [%s] instead" % (p,))
+
+                    if errs:
+                        print("WARNING: " + ". ".join(errs))
+
                     return p  # type: ignore
-                except:
-                    pass
+                except Exception as ex:
+                    if p and npath < 2:
+                        t = "Unable to store config in [%s] due to %r"
+                        errs.append(t % (p, ex))
 
         raise Exception("could not find a writable path for config")
 
@@ -279,6 +294,9 @@ def get_ah_salt() -> str:
 
 
 def ensure_locale() -> None:
+    if ANYWIN and PY2:
+        return  # maybe XP, so busted 65001
+
     safe = "en_US.UTF-8"
     for x in [
         safe,
@@ -616,12 +634,12 @@ def get_sects():
              \033[36mxban\033[35m executes CMD if someone gets banned
             \033[0m
             can be defined as --args or volflags; for example \033[36m
-             --xau notify-send
-             -v .::r:c,xau=notify-send
+             --xau foo.py
+             -v .::r:c,xau=bar.py
             \033[0m
-            commands specified as --args are appended to volflags;
-            each --arg and volflag can be specified multiple times,
-            each command will execute in order unless one returns non-zero
+            hooks specified as commandline --args are appended to volflags;
+            each commandline --arg and volflag can be specified multiple times,
+            each hook will execute in order unless one returns non-zero
 
             optionally prefix the command with comma-sep. flags similar to -mtp:
 
@@ -632,6 +650,10 @@ def get_sects():
              \033[36mtN\033[35m sets an N sec timeout before the command is abandoned
              \033[36miN\033[35m xiu only: volume must be idle for N sec (default = 5)
 
+             \033[36mar\033[35m only run hook if user has read-access
+             \033[36marw\033[35m only run hook if user has read-write-access
+             \033[36marwmd\033[35m ...and so on... (doesn't work for xiu or xban)
+
              \033[36mkt\033[35m kills the entire process tree on timeout (default),
              \033[36mkm\033[35m kills just the main process
              \033[36mkn\033[35m lets it continue running until copyparty is terminated
@@ -641,6 +663,21 @@ def get_sects():
              \033[36mc2\033[35m show only stdout
              \033[36mc3\033[35m mute all process otput
             \033[0m
+            examples:
+
+             \033[36m--xm some.py\033[35m runs \033[33msome.py msgtxt\033[35m on each 📟 message;
+              \033[33mmsgtxt\033[35m is the message that was written into the web-ui
+
+             \033[36m--xm j,some.py\033[35m runs \033[33msome.py jsontext\033[35m on each 📟 message;
+              \033[33mjsontext\033[35m is the message info (ip, user, ..., msg-text)
+
+             \033[36m--xm aw,j,some.py\033[35m requires user to have write-access
+
+             \033[36m--xm aw,,notify-send,hey,--\033[35m shows an OS alert on linux;
+              the \033[33m,,\033[35m stops copyparty from reading the rest as flags and
+              the \033[33m--\033[35m stops notify-send from reading the message as args
+              and the alert will be "hey" followed by the messagetext
+            \033[0m
             each hook is executed once for each event, except for \033[36mxiu\033[0m
             which builds up a backlog of uploads, running the hook just once
             as soon as the volume has been idle for iN seconds (5 by default)
@@ -667,7 +704,10 @@ def get_sects():
               \033[36mstash\033[35m dumps the data to file and returns length + checksum
               \033[36msave,get\033[35m dumps to file and returns the page like a GET
               \033[36mprint,get\033[35m prints the data in the log and returns GET
-              (leave out the ",get" to return an error instead)
+              (leave out the ",get" to return an error instead)\033[0m
+
+            note that the \033[35m--xm\033[0m hook will only run if \033[35m--urlform\033[0m
+              is either \033[36mprint\033[0m or the default \033[36mprint,get\033[0m
             """
             ),
         ],
@@ -893,12 +933,12 @@ def add_upload(ap):
     ap2.add_argument("--no-dupe", action="store_true", help="reject duplicate files during upload; only matches within the same volume (volflag=nodupe)")
     ap2.add_argument("--no-snap", action="store_true", help="disable snapshots -- forget unfinished uploads on shutdown; don't create .hist/up2k.snap files -- abandoned/interrupted uploads must be cleaned up manually")
     ap2.add_argument("--snap-wri", metavar="SEC", type=int, default=300, help="write upload state to ./hist/up2k.snap every \033[33mSEC\033[0m seconds; allows resuming incomplete uploads after a server crash")
-    ap2.add_argument("--snap-drop", metavar="MIN", type=float, default=1440, help="forget unfinished uploads after \033[33mMIN\033[0m minutes; impossible to resume them after that (360=6h, 1440=24h)")
+    ap2.add_argument("--snap-drop", metavar="MIN", type=float, default=1440.0, help="forget unfinished uploads after \033[33mMIN\033[0m minutes; impossible to resume them after that (360=6h, 1440=24h)")
     ap2.add_argument("--u2ts", metavar="TXT", type=u, default="c", help="how to timestamp uploaded files; [\033[32mc\033[0m]=client-last-modified, [\033[32mu\033[0m]=upload-time, [\033[32mfc\033[0m]=force-c, [\033[32mfu\033[0m]=force-u (volflag=u2ts)")
     ap2.add_argument("--rand", action="store_true", help="force randomized filenames, \033[33m--nrand\033[0m chars long (volflag=rand)")
     ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
     ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
-    ap2.add_argument("--df", metavar="GiB", type=float, default=0, help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests")
+    ap2.add_argument("--df", metavar="GiB", type=u, default="0", help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests; assumes gigabytes unless a unit suffix is given: [\033[32m256m\033[0m], [\033[32m4\033[0m], [\033[32m2T\033[0m] (volflag=df)")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
     ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
     ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16 for cross-atlantic (max=64)")
@@ -921,12 +961,12 @@ def add_network(ap):
     else:
         ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
     ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
-    ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
+    ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186.0, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
     ap2.add_argument("--s-rd-sz", metavar="B", type=int, default=256*1024, help="socket read size in bytes (indirectly affects filesystem writes; recommendation: keep equal-to or lower-than \033[33m--iobuf\033[0m)")
     ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
-    ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0, help="debug: socket write delay in seconds")
-    ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0, help="debug: response delay in seconds")
-    ap2.add_argument("--rsp-jtr", metavar="SEC", type=float, default=0, help="debug: response delay, random duration 0..\033[33mSEC\033[0m")
+    ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0.0, help="debug: socket write delay in seconds")
+    ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0.0, help="debug: response delay in seconds")
+    ap2.add_argument("--rsp-jtr", metavar="SEC", type=float, default=0.0, help="debug: response delay, random duration 0..\033[33mSEC\033[0m")
 
 
 def add_tls(ap, cert_path):
@@ -934,10 +974,10 @@ def add_tls(ap, cert_path):
     ap2.add_argument("--http-only", action="store_true", help="disable ssl/tls -- force plaintext")
     ap2.add_argument("--https-only", action="store_true", help="disable plaintext -- force tls")
     ap2.add_argument("--cert", metavar="PATH", type=u, default=cert_path, help="path to TLS certificate")
-    ap2.add_argument("--ssl-ver", metavar="LIST", type=u, help="set allowed ssl/tls versions; [\033[32mhelp\033[0m] shows available versions; default is what your python version considers safe")
-    ap2.add_argument("--ciphers", metavar="LIST", type=u, help="set allowed ssl/tls ciphers; [\033[32mhelp\033[0m] shows available ciphers")
+    ap2.add_argument("--ssl-ver", metavar="LIST", type=u, default="", help="set allowed ssl/tls versions; [\033[32mhelp\033[0m] shows available versions; default is what your python version considers safe")
+    ap2.add_argument("--ciphers", metavar="LIST", type=u, default="", help="set allowed ssl/tls ciphers; [\033[32mhelp\033[0m] shows available ciphers")
     ap2.add_argument("--ssl-dbg", action="store_true", help="dump some tls info")
-    ap2.add_argument("--ssl-log", metavar="PATH", type=u, help="log master secrets for later decryption in wireshark")
+    ap2.add_argument("--ssl-log", metavar="PATH", type=u, default="", help="log master secrets for later decryption in wireshark")
 
 
 def add_cert(ap, cert_path):
@@ -950,12 +990,12 @@ def add_cert(ap, cert_path):
     ap2.add_argument("--crt-nolo", action="store_true", help="do not add 127.0.0.1 / localhost into cert")
     ap2.add_argument("--crt-nohn", action="store_true", help="do not add mDNS names / hostname into cert")
     ap2.add_argument("--crt-dir", metavar="PATH", default=cert_dir, help="where to save the CA cert")
-    ap2.add_argument("--crt-cdays", metavar="D", type=float, default=3650, help="ca-certificate expiration time in days")
-    ap2.add_argument("--crt-sdays", metavar="D", type=float, default=365, help="server-cert expiration time in days")
+    ap2.add_argument("--crt-cdays", metavar="D", type=float, default=3650.0, help="ca-certificate expiration time in days")
+    ap2.add_argument("--crt-sdays", metavar="D", type=float, default=365.0, help="server-cert expiration time in days")
     ap2.add_argument("--crt-cn", metavar="TXT", type=u, default="partyco", help="CA/server-cert common-name")
     ap2.add_argument("--crt-cnc", metavar="TXT", type=u, default="--crt-cn", help="override CA name")
     ap2.add_argument("--crt-cns", metavar="TXT", type=u, default="--crt-cn cpp", help="override server-cert name")
-    ap2.add_argument("--crt-back", metavar="HRS", type=float, default=72, help="backdate in hours")
+    ap2.add_argument("--crt-back", metavar="HRS", type=float, default=72.0, help="backdate in hours")
     ap2.add_argument("--crt-alg", metavar="S-N", type=u, default="ecdsa-256", help="algorithm and keysize; one of these: \033[32mecdsa-256 rsa-4096 rsa-2048\033[0m")
 
 
@@ -996,7 +1036,7 @@ def add_zc_mdns(ap):
     ap2.add_argument("--zm-mnic", action="store_true", help="merge NICs which share subnets; assume that same subnet means same network")
     ap2.add_argument("--zm-msub", action="store_true", help="merge subnets on each NIC -- always enabled for ipv6 -- reduces network load, but gnome-gvfs clients may stop working, and clients cannot be in subnets that the server is not")
     ap2.add_argument("--zm-noneg", action="store_true", help="disable NSEC replies -- try this if some clients don't see copyparty")
-    ap2.add_argument("--zm-spam", metavar="SEC", type=float, default=0, help="send unsolicited announce every \033[33mSEC\033[0m; useful if clients have IPs in a subnet which doesn't overlap with the server, or to avoid some firewall issues")
+    ap2.add_argument("--zm-spam", metavar="SEC", type=float, default=0.0, help="send unsolicited announce every \033[33mSEC\033[0m; useful if clients have IPs in a subnet which doesn't overlap with the server, or to avoid some firewall issues")
 
 
 def add_zc_ssdp(ap):
@@ -1011,14 +1051,15 @@ def add_zc_ssdp(ap):
 
 def add_ftp(ap):
     ap2 = ap.add_argument_group('FTP options (TCP only)')
-    ap2.add_argument("--ftp", metavar="PORT", type=int, help="enable FTP server on \033[33mPORT\033[0m, for example \033[32m3921")
-    ap2.add_argument("--ftps", metavar="PORT", type=int, help="enable FTPS server on \033[33mPORT\033[0m, for example \033[32m3990")
+    ap2.add_argument("--ftp", metavar="PORT", type=int, default=0, help="enable FTP server on \033[33mPORT\033[0m, for example \033[32m3921")
+    ap2.add_argument("--ftps", metavar="PORT", type=int, default=0, help="enable FTPS server on \033[33mPORT\033[0m, for example \033[32m3990")
     ap2.add_argument("--ftpv", action="store_true", help="verbose")
     ap2.add_argument("--ftp4", action="store_true", help="only listen on IPv4")
     ap2.add_argument("--ftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m; specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
+    ap2.add_argument("--ftp-no-ow", action="store_true", help="if target file exists, reject upload instead of overwrite")
     ap2.add_argument("--ftp-wt", metavar="SEC", type=int, default=7, help="grace period for resuming interrupted uploads (any client can write to any file last-modified more recently than \033[33mSEC\033[0m seconds ago)")
-    ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, help="the NAT address to use for passive connections")
-    ap2.add_argument("--ftp-pr", metavar="P-P", type=u, help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
+    ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, default="", help="the NAT address to use for passive connections")
+    ap2.add_argument("--ftp-pr", metavar="P-P", type=u, default="", help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
 
 
 def add_webdav(ap):
@@ -1032,14 +1073,15 @@ def add_webdav(ap):
 
 def add_tftp(ap):
     ap2 = ap.add_argument_group('TFTP options (UDP only)')
-    ap2.add_argument("--tftp", metavar="PORT", type=int, help="enable TFTP server on \033[33mPORT\033[0m, for example \033[32m69 \033[0mor \033[32m3969")
+    ap2.add_argument("--tftp", metavar="PORT", type=int, default=0, help="enable TFTP server on \033[33mPORT\033[0m, for example \033[32m69 \033[0mor \033[32m3969")
+    ap2.add_argument("--tftp4", action="store_true", help="only listen on IPv4")
     ap2.add_argument("--tftpv", action="store_true", help="verbose")
     ap2.add_argument("--tftpvv", action="store_true", help="verboser")
     ap2.add_argument("--tftp-no-fast", action="store_true", help="debug: disable optimizations")
     ap2.add_argument("--tftp-lsf", metavar="PTN", type=u, default="\\.?(dir|ls)(\\.txt)?", help="return a directory listing if a file with this name is requested and it does not exist; defaults matches .ls, dir, .dir.txt, ls.txt, ...")
     ap2.add_argument("--tftp-nols", action="store_true", help="if someone tries to download a directory, return an error instead of showing its directory listing")
     ap2.add_argument("--tftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m; specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
-    ap2.add_argument("--tftp-pr", metavar="P-P", type=u, help="the range of UDP ports to use for data transfer, for example \033[32m12000-13000")
+    ap2.add_argument("--tftp-pr", metavar="P-P", type=u, default="", help="the range of UDP ports to use for data transfer, for example \033[32m12000-13000")
 
 
 def add_smb(ap):
@@ -1115,7 +1157,7 @@ def add_safety(ap):
     ap2.add_argument("-s", action="count", default=0, help="increase safety: Disable thumbnails / potentially dangerous software (ffmpeg/pillow/vips), hide partial uploads, avoid crawlers.\n └─Alias of\033[32m --dotpart --no-thumb --no-mtag-ff --no-robots --force-js")
     ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --hardlink --vague-403 -nih")
     ap2.add_argument("-sss", action="store_true", help="further increase safety: Enable logging to disk, scan for dangerous symlinks.\n └─Alias of\033[32m -ss --no-dav --no-logues --no-readme -lo=cpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz --ls=**,*,ln,p,r")
-    ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m (see \033[33m--help-ls\033[0m); example [\033[32m**,*,ln,p,r\033[0m]")
+    ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, default="", help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m (see \033[33m--help-ls\033[0m); example [\033[32m**,*,ln,p,r\033[0m]")
     ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user has similar access (volflag=xvol)")
     ap2.add_argument("--xdev", action="store_true", help="stay within the filesystem of the volume root; do not descend into other devices (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
     ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
@@ -1125,7 +1167,7 @@ def add_safety(ap):
     ap2.add_argument("--vague-403", action="store_true", help="send 404 instead of 403 (security through ambiguity, very enterprise)")
     ap2.add_argument("--force-js", action="store_true", help="don't send folder listings as HTML, force clients to use the embedded json instead -- slight protection against misbehaving search engines which ignore \033[33m--no-robots\033[0m")
     ap2.add_argument("--no-robots", action="store_true", help="adds http and html headers asking search engines to not index anything (volflag=norobots)")
-    ap2.add_argument("--logout", metavar="H", type=float, default="8086", help="logout clients after \033[33mH\033[0m hours of inactivity; [\033[32m0.0028\033[0m]=10sec, [\033[32m0.1\033[0m]=6min, [\033[32m24\033[0m]=day, [\033[32m168\033[0m]=week, [\033[32m720\033[0m]=month, [\033[32m8760\033[0m]=year)")
+    ap2.add_argument("--logout", metavar="H", type=float, default=8086.0, help="logout clients after \033[33mH\033[0m hours of inactivity; [\033[32m0.0028\033[0m]=10sec, [\033[32m0.1\033[0m]=6min, [\033[32m24\033[0m]=day, [\033[32m168\033[0m]=week, [\033[32m720\033[0m]=month, [\033[32m8760\033[0m]=year)")
     ap2.add_argument("--ban-pw", metavar="N,W,B", type=u, default="9,60,1440", help="more than \033[33mN\033[0m wrong passwords in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; disable with [\033[32mno\033[0m]")
     ap2.add_argument("--ban-404", metavar="N,W,B", type=u, default="50,60,1440", help="hitting more than \033[33mN\033[0m 404's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; only affects users who cannot see directory listings because their access is either g/G/h")
     ap2.add_argument("--ban-403", metavar="N,W,B", type=u, default="9,2,1440", help="hitting more than \033[33mN\033[0m 403's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; [\033[32m1440\033[0m]=day, [\033[32m10080\033[0m]=week, [\033[32m43200\033[0m]=month")
@@ -1161,7 +1203,7 @@ def add_shutdown(ap):
 def add_logging(ap):
     ap2 = ap.add_argument_group('logging options')
     ap2.add_argument("-q", action="store_true", help="quiet; disable most STDOUT messages")
-    ap2.add_argument("-lo", metavar="PATH", type=u, help="logfile, example: \033[32mcpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz\033[0m (NB: some errors may appear on STDOUT only)")
+    ap2.add_argument("-lo", metavar="PATH", type=u, default="", help="logfile, example: \033[32mcpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz\033[0m (NB: some errors may appear on STDOUT only)")
     ap2.add_argument("--no-ansi", action="store_true", default=not VT100, help="disable colors; same as environment-variable NO_COLOR")
     ap2.add_argument("--ansi", action="store_true", help="force colors; overrides environment-variable NO_COLOR")
     ap2.add_argument("--no-logflush", action="store_true", help="don't flush the logfile after each write; tiny bit faster")
@@ -1188,10 +1230,10 @@ def add_thumbnail(ap):
     ap2.add_argument("--no-athumb", action="store_true", help="disable audio thumbnails (spectrograms) (volflag=dathumb)")
     ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res (volflag=thsize)")
     ap2.add_argument("--th-mt", metavar="CORES", type=int, default=CORES, help="num cpu cores to use for generating thumbnails")
-    ap2.add_argument("--th-convt", metavar="SEC", type=float, default=60, help="conversion timeout in seconds (volflag=convt)")
-    ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=6, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
-    ap2.add_argument("--th-crop", metavar="TXT", type=u, default="y", help="crop thumbnails to 4:3 or keep dynamic height; client can override in UI unless force. [\033[32mfy\033[0m]=crop, [\033[32mfn\033[0m]=nocrop, [\033[32mfy\033[0m]=force-y, [\033[32mfn\033[0m]=force-n (volflag=crop)")
-    ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32mfy\033[0m]=yes, [\033[32mfn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
+    ap2.add_argument("--th-convt", metavar="SEC", type=float, default=60.0, help="conversion timeout in seconds (volflag=convt)")
+    ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=6.0, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
+    ap2.add_argument("--th-crop", metavar="TXT", type=u, default="y", help="crop thumbnails to 4:3 or keep dynamic height; client can override in UI unless force. [\033[32my\033[0m]=crop, [\033[32mn\033[0m]=nocrop, [\033[32mfy\033[0m]=force-y, [\033[32mfn\033[0m]=force-n (volflag=crop)")
+    ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32my\033[0m]=yes, [\033[32mn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
     ap2.add_argument("--th-dec", metavar="LIBS", default="vips,pil,ff", help="image decoders, in order of preference")
     ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
     ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
@@ -1230,8 +1272,8 @@ def add_db_general(ap, hcores):
     ap2.add_argument("-e2v", action="store_true", help="verify file integrity; rehash all files and compare with db")
     ap2.add_argument("-e2vu", action="store_true", help="on hash mismatch: update the database with the new hash")
     ap2.add_argument("-e2vp", action="store_true", help="on hash mismatch: panic and quit copyparty")
-    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs) (volflag=hist)")
-    ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
+    ap2.add_argument("--hist", metavar="PATH", type=u, default="", help="where to store volume data (db, thumbs); default is a folder named \".hist\" inside each volume (volflag=hist)")
+    ap2.add_argument("--no-hash", metavar="PTN", type=u, default="", help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
     ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
     ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
     ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
@@ -1240,7 +1282,7 @@ def add_db_general(ap, hcores):
     ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
     ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
     ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="rescan filesystem for changes every \033[33mSEC\033[0m seconds; 0=off (volflag=scan)")
-    ap2.add_argument("--db-act", metavar="SEC", type=float, default=10, help="defer any scheduled volume reindexing until \033[33mSEC\033[0m seconds after last db write (uploads, renames, ...)")
+    ap2.add_argument("--db-act", metavar="SEC", type=float, default=10.0, help="defer any scheduled volume reindexing until \033[33mSEC\033[0m seconds after last db write (uploads, renames, ...)")
     ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than \033[33mSEC\033[0m seconds")
     ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
     ap2.add_argument("--dotsrch", action="store_true", help="show dotfiles in search results (volflags: dotsrch | nodotsrch)")
@@ -1293,6 +1335,7 @@ def add_og(ap):
 def add_ui(ap, retry):
     ap2 = ap.add_argument_group('ui options')
     ap2.add_argument("--grid", action="store_true", help="show grid/thumbnails by default (volflag=grid)")
+    ap2.add_argument("--gsel", action="store_true", help="select files in grid by ctrl-click (volflag=gsel)")
     ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language; one of the following: \033[32meng nor\033[0m")
     ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use (0..7)")
     ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
@@ -1301,8 +1344,8 @@ def add_ui(ap, retry):
     ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
     ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
     ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
-    ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
-    ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
+    ap2.add_argument("--js-browser", metavar="L", type=u, default="", help="URL to additional JS to include")
+    ap2.add_argument("--css-browser", metavar="L", type=u, default="", help="URL to additional CSS to include")
     ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages; can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template (volflag=html_head)")
     ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
     ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
@@ -1322,14 +1365,16 @@ def add_debug(ap):
     ap2 = ap.add_argument_group('debug options')
     ap2.add_argument("--vc", action="store_true", help="verbose config file parser (explain config)")
     ap2.add_argument("--cgen", action="store_true", help="generate config file from current config (best-effort; probably buggy)")
+    if hasattr(select, "poll"):
+        ap2.add_argument("--no-poll", action="store_true", help="kernel-bug workaround: disable poll; use select instead (limits max num clients to ~700)")
     ap2.add_argument("--no-sendfile", action="store_true", help="kernel-bug workaround: disable sendfile; do a safe and slow read-send-loop instead")
     ap2.add_argument("--no-scandir", action="store_true", help="kernel-bug workaround: disable scandir; do a listdir + stat on each file instead")
     ap2.add_argument("--no-fastboot", action="store_true", help="wait for initial filesystem indexing before accepting client requests")
     ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead")
     ap2.add_argument("--srch-dbg", action="store_true", help="explain search processing, and do some extra expensive sanity checks")
     ap2.add_argument("--rclone-mdns", action="store_true", help="use mdns-domain instead of server-ip on /?hc")
-    ap2.add_argument("--stackmon", metavar="P,S", type=u, help="write stacktrace to \033[33mP\033[0math every \033[33mS\033[0m second, for example --stackmon=\033[32m./st/%%Y-%%m/%%d/%%H%%M.xz,60")
-    ap2.add_argument("--log-thrs", metavar="SEC", type=float, help="list active threads every \033[33mSEC\033[0m")
+    ap2.add_argument("--stackmon", metavar="P,S", type=u, default="", help="write stacktrace to \033[33mP\033[0math every \033[33mS\033[0m second, for example --stackmon=\033[32m./st/%%Y-%%m/%%d/%%H%%M.xz,60")
+    ap2.add_argument("--log-thrs", metavar="SEC", type=float, default=0.0, help="list active threads every \033[33mSEC\033[0m")
     ap2.add_argument("--log-fk", metavar="REGEX", type=u, default="", help="log filekey params for files where path matches \033[33mREGEX\033[0m; [\033[32m.\033[0m] (a single dot) = all files")
     ap2.add_argument("--bak-flips", action="store_true", help="[up2k] if a client uploads a bitflipped/corrupted chunk, store a copy according to \033[33m--bf-nc\033[0m and \033[33m--bf-dir\033[0m")
     ap2.add_argument("--bf-nc", metavar="NUM", type=int, default=200, help="bak-flips: stop if there's more than \033[33mNUM\033[0m files at \033[33m--kf-dir\033[0m already; default: 6.3 GiB max (200*32M)")
@@ -1532,7 +1577,7 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
         if hard > 0:  # -1 == infinite
             nc = min(nc, int(hard / 4))
     except:
-        nc = 512
+        nc = 486  # mdns/ssdp restart headroom; select() maxfd is 512 on windows
 
     retry = False
     for fmtr in [RiceFormatter, RiceFormatter, Dodge11874, BasicDodge11874]:
@@ -1577,6 +1622,9 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
         if getattr(al, k1):
             setattr(al, k2, False)
 
+    if not HAVE_IPV6 and al.i == "::":
+        al.i = "0.0.0.0"
+
     al.i = al.i.split(",")
     try:
         if "-" in al.p:
@@ -1625,6 +1673,9 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
     if not hasattr(os, "sendfile"):
         al.no_sendfile = True
 
+    if not hasattr(select, "poll"):
+        al.no_poll = True
+
     # signal.signal(signal.SIGINT, sighandler)
 
     SvcHub(al, dal, argv, "".join(printed)).run()

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 13, 2)
+VERSION = (1, 13, 4)
 CODENAME = "race the beam"
-BUILD_DT = (2024, 5, 10)
+BUILD_DT = (2024, 7, 16)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -17,6 +17,8 @@ from .bos import bos
 from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
 from .pwhash import PWHash
 from .util import (
+    DEF_MTE,
+    DEF_MTH,
     EXTS,
     IMPLICATIONS,
     MIMES,
@@ -475,6 +477,13 @@ class VFS(object):
         )
         # skip uhtml because it's rarely needed
 
+    def get_perms(self, vpath: str, uname: str) -> str:
+        zbl = self.can_access(vpath, uname)
+        ret = "".join(ch for ch, ok in zip("rwmdgGa.", zbl) if ok)
+        if "rwmd" in ret and "a." in ret:
+            ret += "A"
+        return ret
+
     def get(
         self,
         vpath: str,
@@ -1617,11 +1626,14 @@ class AuthSrv(object):
                 use = True
                 lim.nosub = True
 
-            zs = vol.flags.get("df") or (
-                "{}g".format(self.args.df) if self.args.df else ""
-            )
-            if zs:
+            zs = vol.flags.get("df") or self.args.df or ""
+            if zs not in ("", "0"):
                 use = True
+                try:
+                    _ = float(zs)
+                    zs = "%sg" % (zs)
+                except:
+                    pass
                 lim.dfl = unhumanize(zs)
 
             zs = vol.flags.get("sz")
@@ -2264,10 +2276,11 @@ class AuthSrv(object):
             "",
         ]
 
-        csv = set("i p".split())
+        csv = set("i p th_covers zm_on zm_off zs_on zs_off".split())
         zs = "c ihead mtm mtp on403 on404 xad xar xau xiu xban xbd xbr xbu xm"
         lst = set(zs.split())
-        askip = set("a v c vc cgen theme".split())
+        askip = set("a v c vc cgen exp_lg exp_md theme".split())
+        fskip = set("exp_lg exp_md mv_re_r mv_re_t rm_re_r rm_re_t".split())
 
         # keymap from argv to vflag
         amap = vf_bmap()
@@ -2288,11 +2301,35 @@ class AuthSrv(object):
             for k, v in args.items():
                 if k in askip:
                     continue
+
+                try:
+                    v = v.pattern
+                    if k in ("idp_gsep", "tftp_lsf"):
+                        v = v[1:-1]  # close enough
+                except:
+                    pass
+
+                skip = False
+                for k2, defstr in (("mte", DEF_MTE), ("mth", DEF_MTH)):
+                    if k != k2:
+                        continue
+                    s1 = list(sorted(list(v)))
+                    s2 = list(sorted(defstr.split(",")))
+                    if s1 == s2:
+                        skip = True
+                        break
+                    v = ",".join(s1)
+
+                if skip:
+                    continue
+
                 if k in csv:
                     v = ", ".join([str(za) for za in v])
                 try:
                     v2 = getattr(self.dargs, k)
-                    if v == v2:
+                    if k == "tcolor" and len(v2) == 3:
+                        v2 = "".join([x * 2 for x in v2])
+                    if v == v2 or v.replace(", ", ",") == v2:
                         continue
                 except:
                     continue
@@ -2351,6 +2388,7 @@ class AuthSrv(object):
                         pstr += pchar
                 if "g" in pstr and "G" in pstr:
                     pstr = pstr.replace("g", "")
+                pstr = pstr.replace("rwmd.a", "A")
                 try:
                     vperms[pstr].append(uname)
                 except:
@@ -2360,24 +2398,48 @@ class AuthSrv(object):
             trues = []
             vals = []
             for k, v in sorted(vol.flags.items()):
+                if k in fskip:
+                    continue
+
+                try:
+                    v = v.pattern
+                except:
+                    pass
+
                 try:
                     ak = vmap[k]
-                    if getattr(self.args, ak) is v:
+                    v2 = getattr(self.args, ak)
+
+                    try:
+                        v2 = v2.pattern
+                    except:
+                        pass
+
+                    if v2 is v:
                         continue
                 except:
                     pass
 
+                skip = False
+                for k2, defstr in (("mte", DEF_MTE), ("mth", DEF_MTH)):
+                    if k != k2:
+                        continue
+                    s1 = list(sorted(list(v)))
+                    s2 = list(sorted(defstr.split(",")))
+                    if s1 == s2:
+                        skip = True
+                        break
+                    v = ",".join(s1)
+
+                if skip:
+                    continue
+
                 if k in lst:
                     for ve in v:
                         vals.append("{}: {}".format(k, ve))
                 elif v is True:
                     trues.append(k)
                 elif v is not False:
-                    try:
-                        v = v.pattern
-                    except:
-                        pass
-
                     vals.append("{}: {}".format(k, v))
             pops = []
             for k1, k2 in IMPLICATIONS:

copyparty/cert.py

@@ -12,7 +12,7 @@ from .util import Netdev, runcmd, wrename, wunlink
 HAVE_CFSSL = True
 
 if True:  # pylint: disable=using-constant-test
-    from .util import RootLogger
+    from .util import NamedLogger, RootLogger
 
 
 if ANYWIN:
@@ -83,6 +83,8 @@ def _read_crt(args, fn):
 
 
 def _gen_ca(log: "RootLogger", args):
+    nlog: "NamedLogger" = lambda msg, c=0: log("cert-gen-ca", msg, c)
+
     expiry = _read_crt(args, "ca.pem")[0]
     if time.time() + args.crt_cdays * 60 * 60 * 24 * 0.1 < expiry:
         return
@@ -113,16 +115,18 @@ def _gen_ca(log: "RootLogger", args):
 
     bname = os.path.join(args.crt_dir, "ca")
     try:
-        wunlink(log, bname + ".key", VF)
+        wunlink(nlog, bname + ".key", VF)
     except:
         pass
-    wrename(log, bname + "-key.pem", bname + ".key", VF)
-    wunlink(log, bname + ".csr", VF)
+    wrename(nlog, bname + "-key.pem", bname + ".key", VF)
+    wunlink(nlog, bname + ".csr", VF)
 
     log("cert", "new ca OK", 2)
 
 
 def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
+    nlog: "NamedLogger" = lambda msg, c=0: log("cert-gen-srv", msg, c)
+
     names = args.crt_ns.split(",") if args.crt_ns else []
     if not args.crt_exact:
         for n in names[:]:
@@ -196,11 +200,11 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
 
     bname = os.path.join(args.crt_dir, "srv")
     try:
-        wunlink(log, bname + ".key", VF)
+        wunlink(nlog, bname + ".key", VF)
     except:
         pass
-    wrename(log, bname + "-key.pem", bname + ".key", VF)
-    wunlink(log, bname + ".csr", VF)
+    wrename(nlog, bname + "-key.pem", bname + ".key", VF)
+    wunlink(nlog, bname + ".csr", VF)
 
     with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:
         ca = f.read()

copyparty/cfg.py

@@ -35,6 +35,7 @@ def vf_bmap() -> dict[str, str]:
         "e2vp",
         "exp",
         "grid",
+        "gsel",
         "hardlink",
         "magic",
         "no_sb_md",
@@ -144,6 +145,7 @@ flagcats = {
         "maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g, t)",
         "vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
         "vmaxn=4k": "max 4096 files in volume (suffixes: b, k, m, g, t)",
+        "medialinks": "return medialinks for non-up2k uploads (not hotlinks)",
         "rand": "force randomized filenames, 9 chars long by default",
         "nrand=N": "randomized filenames are N chars long",
         "u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
@@ -213,6 +215,7 @@ flagcats = {
     },
     "client and ux": {
         "grid": "show grid/thumbnails by default",
+        "gsel": "select files in grid by ctrl-click",
         "sort": "default sort order",
         "unlist": "dont list files matching REGEX",
         "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",

copyparty/ftpd.py

@@ -19,6 +19,7 @@ from .__init__ import PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import (
+    VF_CAREFUL,
     Daemon,
     ODict,
     Pebkac,
@@ -30,6 +31,7 @@ from .util import (
     runhook,
     sanitize_fn,
     vjoin,
+    wunlink,
 )
 
 if TYPE_CHECKING:
@@ -37,7 +39,7 @@ if TYPE_CHECKING:
 
 if True:  # pylint: disable=using-constant-test
     import typing
-    from typing import Any, Optional
+    from typing import Any, Optional, Union
 
 
 class FSE(FilesystemError):
@@ -139,6 +141,9 @@ class FtpFs(AbstractedFS):
         self.listdirinfo = self.listdir
         self.chdir(".")
 
+    def log(self, msg: str, c: Union[int, str] = 0) -> None:
+        self.hub.log("ftpd", msg, c)
+
     def v2a(
         self,
         vpath: str,
@@ -207,17 +212,37 @@ class FtpFs(AbstractedFS):
         w = "w" in mode or "a" in mode or "+" in mode
 
         ap = self.rv2a(filename, r, w)[0]
+        self.validpath(ap)
         if w:
             try:
                 st = bos.stat(ap)
                 td = time.time() - st.st_mtime
+                need_unlink = True
             except:
+                need_unlink = False
                 td = 0
 
-            if td < -1 or td > self.args.ftp_wt:
-                raise FSE("Cannot open existing file for writing")
+        if w and need_unlink:
+            if td >= -1 and td <= self.args.ftp_wt:
+                # within permitted timeframe; unlink and accept
+                do_it = True
+            elif self.args.no_del or self.args.ftp_no_ow:
+                # file too old, or overwrite not allowed; reject
+                do_it = False
+            else:
+                # allow overwrite if user has delete permission
+                # (avoids win2000 freaking out and deleting the server copy without uploading its own)
+                try:
+                    self.rv2a(filename, False, True, False, True)
+                    do_it = True
+                except:
+                    do_it = False
+
+            if not do_it:
+                raise FSE("File already exists")
+
+            wunlink(self.log, ap, VF_CAREFUL)
 
-        self.validpath(ap)
         return open(fsenc(ap), mode, self.args.iobuf)
 
     def chdir(self, path: str) -> None:
@@ -282,9 +307,20 @@ class FtpFs(AbstractedFS):
                 # display write-only folders as empty
                 return []
 
-            # return list of volumes
-            r = {x.split("/")[0]: 1 for x in self.hub.asrv.vfs.all_vols.keys()}
-            return list(sorted(list(r.keys())))
+            # return list of accessible volumes
+            ret = []
+            for vn in self.hub.asrv.vfs.all_vols.values():
+                if "/" in vn.vpath or not vn.vpath:
+                    continue  # only include toplevel-mounted vols
+
+                try:
+                    self.hub.asrv.vfs.get(vn.vpath, self.uname, True, False)
+                    ret.append(vn.vpath)
+                except:
+                    pass
+
+            ret.sort()
+            return ret
 
     def rmdir(self, path: str) -> None:
         ap = self.rv2a(path, d=True)[0]
@@ -434,9 +470,10 @@ class FtpHandler(FTPHandler):
             None,
             xbu,
             ap,
-            vfs.canonical(rem),
+            vp,
             "",
             self.uname,
+            self.hub.asrv.vfs.get_perms(vp, self.uname),
             0,
             0,
             self.cli_ip,

copyparty/httpcli.py

@@ -646,8 +646,12 @@ class HttpCli(object):
                 if not self._check_nonfatal(pex, post):
                     self.keepalive = False
 
-                em = str(ex)
-                msg = em if pex is ex else min_ex()
+                if pex is ex:
+                    em = msg = str(ex)
+                else:
+                    em = repr(ex)
+                    msg = min_ex()
+
                 if pex.code != 404 or self.do_log:
                     self.log(
                         "%s\033[0m, %s" % (msg, self.vpath),
@@ -695,6 +699,7 @@ class HttpCli(object):
             self.vpath,
             self.host,
             self.uname,
+            "",
             time.time(),
             0,
             self.ip,
@@ -744,7 +749,7 @@ class HttpCli(object):
             or ("; Trident/" in self.ua and not k304)
         )
 
-    def _build_html_head(self, maybe_html: Any, kv: dict[str, Any]) -> bool:
+    def _build_html_head(self, maybe_html: Any, kv: dict[str, Any]) -> None:
         html = str(maybe_html)
         is_jinja = html[:2] in "%@%"
         if is_jinja:
@@ -759,7 +764,6 @@ class HttpCli(object):
             is_jinja = True
 
         if is_jinja:
-            print("applying jinja")
             with self.conn.hsrv.mutex:
                 if html not in self.conn.hsrv.j2:
                     j2env = jinja2.Environment()
@@ -1632,6 +1636,7 @@ class HttpCli(object):
                                     self.vpath,
                                     self.host,
                                     self.uname,
+                                    self.asrv.vfs.get_perms(self.vpath, self.uname),
                                     time.time(),
                                     len(buf),
                                     self.ip,
@@ -1675,6 +1680,8 @@ class HttpCli(object):
         remains = int(self.headers.get("content-length", -1))
         if remains == -1:
             self.keepalive = False
+            self.in_hdr_recv = True
+            self.s.settimeout(max(self.args.s_tbody // 20, 1))
             return read_socket_unbounded(self.sr, bufsz), remains
         else:
             return read_socket(self.sr, bufsz, remains), remains
@@ -1779,6 +1786,7 @@ class HttpCli(object):
                 self.vpath,
                 self.host,
                 self.uname,
+                self.asrv.vfs.get_perms(self.vpath, self.uname),
                 at,
                 remains,
                 self.ip,
@@ -1869,6 +1877,7 @@ class HttpCli(object):
             self.vpath,
             self.host,
             self.uname,
+            self.asrv.vfs.get_perms(self.vpath, self.uname),
             mt,
             post_sz,
             self.ip,
@@ -1905,6 +1914,9 @@ class HttpCli(object):
                 0 if ANYWIN else bos.stat(path).st_ino,
             )[: vfs.flags["fk"]]
 
+        if "media" in self.uparam or "medialinks" in vfs.flags:
+            vsuf += "&v" if vsuf else "?v"
+
         vpath = "/".join([x for x in [vfs.vpath, rem, fn] if x])
         vpath = quotep(vpath)
 
@@ -2026,7 +2038,7 @@ class HttpCli(object):
 
         v = self.uparam[k]
 
-        if self._use_dirkey():
+        if self._use_dirkey(self.vn, ""):
             vn = self.vn
             rem = self.rem
         else:
@@ -2548,6 +2560,7 @@ class HttpCli(object):
                         self.vpath,
                         self.host,
                         self.uname,
+                        self.asrv.vfs.get_perms(self.vpath, self.uname),
                         at,
                         0,
                         self.ip,
@@ -2611,6 +2624,7 @@ class HttpCli(object):
                         self.vpath,
                         self.host,
                         self.uname,
+                        self.asrv.vfs.get_perms(self.vpath, self.uname),
                         at,
                         sz,
                         self.ip,
@@ -2686,6 +2700,9 @@ class HttpCli(object):
                     0 if ANYWIN or not ap else bos.stat(ap).st_ino,
                 )[: vfs.flags["fk"]]
 
+            if "media" in self.uparam or "medialinks" in vfs.flags:
+                vsuf += "&v" if vsuf else "?v"
+
             vpath = "{}/{}".format(upload_vpath, lfn).strip("/")
             rel_url = quotep(self.args.RS + vpath) + vsuf
             msg += 'sha512: {} // {} // {} bytes // <a href="/{}">{}</a> {}\n'.format(
@@ -2852,6 +2869,7 @@ class HttpCli(object):
                 self.vpath,
                 self.host,
                 self.uname,
+                self.asrv.vfs.get_perms(self.vpath, self.uname),
                 time.time(),
                 0,
                 self.ip,
@@ -2890,6 +2908,7 @@ class HttpCli(object):
             self.vpath,
             self.host,
             self.uname,
+            self.asrv.vfs.get_perms(self.vpath, self.uname),
             new_lastmod,
             sz,
             self.ip,
@@ -2944,22 +2963,24 @@ class HttpCli(object):
 
         return file_lastmod, True
 
-    def _use_dirkey(self, ap: str = "") -> bool:
+    def _use_dirkey(self, vn: VFS, ap: str) -> bool:
         if self.can_read or not self.can_get:
             return False
 
-        if self.vn.flags.get("dky"):
+        if vn.flags.get("dky"):
             return True
 
         req = self.uparam.get("k") or ""
         if not req:
             return False
 
-        dk_len = self.vn.flags.get("dk")
+        dk_len = vn.flags.get("dk")
         if not dk_len:
             return False
 
-        ap = ap or self.vn.canonical(self.rem)
+        if not ap:
+            ap = vn.canonical(self.rem)
+
         zs = self.gen_fk(2, self.args.dk_salt, ap, 0, 0)[:dk_len]
         if req == zs:
             return True
@@ -2968,6 +2989,71 @@ class HttpCli(object):
         self.log(t % (zs, req, self.req, ap), 6)
         return False
 
+    def _use_filekey(self, vn: VFS, ap: str, st: os.stat_result) -> bool:
+        if self.can_read or not self.can_get:
+            return False
+
+        req = self.uparam.get("k") or ""
+        if not req:
+            return False
+
+        fk_len = vn.flags.get("fk")
+        if not fk_len:
+            return False
+
+        if not ap:
+            ap = self.vn.canonical(self.rem)
+
+        alg = 2 if "fka" in vn.flags else 1
+
+        zs = self.gen_fk(
+            alg, self.args.fk_salt, ap, st.st_size, 0 if ANYWIN else st.st_ino
+        )[:fk_len]
+
+        if req == zs:
+            return True
+
+        t = "wrong filekey, want %s, got %s\n  vp: %s\n  ap: %s"
+        self.log(t % (zs, req, self.req, ap), 6)
+        return False
+
+    def _add_logues(
+        self, vn: VFS, abspath: str, lnames: Optional[dict[str, str]]
+    ) -> tuple[list[str], str]:
+        logues = ["", ""]
+        if not self.args.no_logues:
+            for n, fn in enumerate([".prologue.html", ".epilogue.html"]):
+                if lnames is not None and fn not in lnames:
+                    continue
+                fn = os.path.join(abspath, fn)
+                if bos.path.exists(fn):
+                    with open(fsenc(fn), "rb") as f:
+                        logues[n] = f.read().decode("utf-8")
+                    if "exp" in vn.flags:
+                        logues[n] = self._expand(
+                            logues[n], vn.flags.get("exp_lg") or []
+                        )
+
+        readme = ""
+        if not self.args.no_readme and not logues[1]:
+            if lnames is None:
+                fns = ["README.md", "readme.md"]
+            elif "readme.md" in lnames:
+                fns = [lnames["readme.md"]]
+            else:
+                fns = []
+
+            for fn in fns:
+                fn = os.path.join(abspath, fn)
+                if bos.path.isfile(fn):
+                    with open(fsenc(fn), "rb") as f:
+                        readme = f.read().decode("utf-8")
+                        break
+            if readme and "exp" in vn.flags:
+                readme = self._expand(readme, vn.flags.get("exp_md") or [])
+
+        return logues, readme
+
     def _expand(self, txt: str, phs: list[str]) -> str:
         for ph in phs:
             if ph.startswith("hdr."):
@@ -2997,6 +3083,7 @@ class HttpCli(object):
         logtail = ""
 
         if ptop is not None:
+            ap_data = "<%s>" % (req_path,)
             try:
                 dp, fn = os.path.split(req_path)
                 tnam = fn + ".PARTIAL"
@@ -3194,7 +3281,14 @@ class HttpCli(object):
 
             sendfun = sendfile_kern if use_sendfile else sendfile_py
             remains = sendfun(
-                self.log, lower, upper, f, self.s, self.args.s_wr_sz, self.args.s_wr_slp
+                self.log,
+                lower,
+                upper,
+                f,
+                self.s,
+                self.args.s_wr_sz,
+                self.args.s_wr_slp,
+                not self.args.no_poll,
             )
 
         if remains > 0:
@@ -3336,7 +3430,16 @@ class HttpCli(object):
 
         if lower < upper and not broken:
             with open(req_path, "rb") as f:
-                remains = sendfile_py(self.log, lower, upper, f, self.s, wr_sz, wr_slp)
+                remains = sendfile_py(
+                    self.log,
+                    lower,
+                    upper,
+                    f,
+                    self.s,
+                    wr_sz,
+                    wr_slp,
+                    not self.args.no_poll,
+                )
 
         spd = self._spd((upper - lower) - remains)
         if self.do_log:
@@ -3423,7 +3526,7 @@ class HttpCli(object):
 
         bgen = packer(
             self.log,
-            self.args,
+            self.asrv,
             fgen,
             utf8="utf" in uarg,
             pre_crc="crc" in uarg,
@@ -3848,7 +3951,7 @@ class HttpCli(object):
         dk_sz = False
         if dk:
             vn, rem = vfs.get(top, self.uname, False, False)
-            if vn.flags.get("dks") and self._use_dirkey(vn.canonical(rem)):
+            if vn.flags.get("dks") and self._use_dirkey(vn, vn.canonical(rem)):
                 dk_sz = vn.flags.get("dk")
 
         dots = False
@@ -4157,6 +4260,9 @@ class HttpCli(object):
                 add_og = True
             og_fn = ""
 
+        if "v" in self.uparam:
+            add_og = og_ua = True
+
         if "b" in self.uparam:
             self.out_headers["X-Robots-Tag"] = "noindex, nofollow"
 
@@ -4169,9 +4275,20 @@ class HttpCli(object):
             if idx and hasattr(idx, "p_end"):
                 icur = idx.get_cur(dbv)
 
+        if "k" in self.uparam or "dky" in vn.flags:
+            if is_dir:
+                use_dirkey = self._use_dirkey(vn, abspath)
+                use_filekey = False
+            else:
+                use_filekey = self._use_filekey(vn, abspath, st)
+                use_dirkey = False
+        else:
+            use_dirkey = use_filekey = False
+
         th_fmt = self.uparam.get("th")
         if self.can_read or (
-            self.can_get and (vn.flags.get("dk") or "fk" not in vn.flags)
+            self.can_get
+            and (use_filekey or use_dirkey or (not is_dir and "fk" not in vn.flags))
         ):
             if th_fmt is not None:
                 nothumb = "dthumb" in dbv.flags
@@ -4188,18 +4305,21 @@ class HttpCli(object):
                             if cfn:
                                 fn = cfn[0]
                                 fp = os.path.join(abspath, fn)
-                                if bos.path.exists(fp):
-                                    vrem = "{}/{}".format(vrem, fn).strip("/")
-                                    is_dir = False
+                                st = bos.stat(fp)
+                                vrem = "{}/{}".format(vrem, fn).strip("/")
+                                is_dir = False
                         except:
                             pass
                     else:
                         for fn in self.args.th_covers:
                             fp = os.path.join(abspath, fn)
-                            if bos.path.exists(fp):
+                            try:
+                                st = bos.stat(fp)
                                 vrem = "{}/{}".format(vrem, fn).strip("/")
                                 is_dir = False
                                 break
+                            except:
+                                pass
 
                     if is_dir:
                         return self.tx_svg("folder")
@@ -4251,21 +4371,10 @@ class HttpCli(object):
 
         if not is_dir and (self.can_read or self.can_get):
             if not self.can_read and not fk_pass and "fk" in vn.flags:
-                alg = 2 if "fka" in vn.flags else 1
-                correct = self.gen_fk(
-                    alg,
-                    self.args.fk_salt,
-                    abspath,
-                    st.st_size,
-                    0 if ANYWIN else st.st_ino,
-                )[: vn.flags["fk"]]
-                got = self.uparam.get("k")
-                if got != correct:
-                    t = "wrong filekey, want %s, got %s\n  vp: %s\n  ap: %s"
-                    self.log(t % (correct, got, self.req, abspath), 6)
+                if not use_filekey:
                     return self.tx_404()
 
-            if add_og:
+            if add_og and not abspath.lower().endswith(".md"):
                 if og_ua or self.host not in self.headers.get("referer", ""):
                     self.vpath, og_fn = vsplit(self.vpath)
                     vpath = self.vpath
@@ -4280,7 +4389,7 @@ class HttpCli(object):
                 (abspath.endswith(".md") or self.can_delete)
                 and "nohtml" not in vn.flags
                 and (
-                    "v" in self.uparam
+                    ("v" in self.uparam and abspath.endswith(".md"))
                     or "edit" in self.uparam
                     or "edit2" in self.uparam
                 )
@@ -4293,7 +4402,7 @@ class HttpCli(object):
                 )
 
         elif is_dir and not self.can_read:
-            if self._use_dirkey(abspath):
+            if use_dirkey:
                 is_dk = True
             elif not self.can_write:
                 return self.tx_404(True)
@@ -4350,29 +4459,6 @@ class HttpCli(object):
             tpl = "browser2"
             is_js = False
 
-        logues = ["", ""]
-        if not self.args.no_logues:
-            for n, fn in enumerate([".prologue.html", ".epilogue.html"]):
-                fn = os.path.join(abspath, fn)
-                if bos.path.exists(fn):
-                    with open(fsenc(fn), "rb") as f:
-                        logues[n] = f.read().decode("utf-8")
-                    if "exp" in vn.flags:
-                        logues[n] = self._expand(
-                            logues[n], vn.flags.get("exp_lg") or []
-                        )
-
-        readme = ""
-        if not self.args.no_readme and not logues[1]:
-            for fn in ["README.md", "readme.md"]:
-                fn = os.path.join(abspath, fn)
-                if bos.path.isfile(fn):
-                    with open(fsenc(fn), "rb") as f:
-                        readme = f.read().decode("utf-8")
-                        break
-            if readme and "exp" in vn.flags:
-                readme = self._expand(readme, vn.flags.get("exp_md") or [])
-
         vf = vn.flags
         unlist = vf.get("unlist", "")
         ls_ret = {
@@ -4391,8 +4477,6 @@ class HttpCli(object):
             "frand": bool(vn.flags.get("rand")),
             "unlist": unlist,
             "perms": perms,
-            "logues": logues,
-            "readme": readme,
         }
         cgv = {
             "ls0": None,
@@ -4410,8 +4494,8 @@ class HttpCli(object):
             "have_zip": (not self.args.no_zip),
             "have_unpost": int(self.args.unpost),
             "sb_md": "" if "no_sb_md" in vf else (vf.get("md_sbf") or "y"),
-            "readme": readme,
             "dgrid": "grid" in vf,
+            "dgsel": "gsel" in vf,
             "dsort": vf["sort"],
             "dcrop": vf["crop"],
             "dth3x": vf["th3x"],
@@ -4432,7 +4516,6 @@ class HttpCli(object):
             "have_b_u": (self.can_write and self.uparam.get("b") == "u"),
             "sb_lg": "" if "no_sb_lg" in vf else (vf.get("lg_sbf") or "y"),
             "url_suf": url_suf,
-            "logues": logues,
             "title": html_escape("%s %s" % (self.args.bname, self.vpath), crlf=True),
             "srv_info": srv_infot,
             "dtheme": self.args.theme,
@@ -4452,6 +4535,10 @@ class HttpCli(object):
             j2a["no_prism"] = True
 
         if not self.can_read and not is_dk:
+            logues, readme = self._add_logues(vn, abspath, None)
+            ls_ret["logues"] = j2a["logues"] = logues
+            ls_ret["readme"] = cgv["readme"] = readme
+
             if is_ls:
                 return self.tx_ls(ls_ret)
 
@@ -4508,6 +4595,8 @@ class HttpCli(object):
         ):
             ls_names = exclude_dotfiles(ls_names)
 
+        lnames = {x.lower(): x for x in ls_names}
+
         add_dk = vf.get("dk")
         add_fk = vf.get("fk")
         fk_alg = 2 if "fka" in vf else 1
@@ -4697,9 +4786,45 @@ class HttpCli(object):
         else:
             taglist = list(tagset)
 
+        logues, readme = self._add_logues(vn, abspath, lnames)
+        ls_ret["logues"] = j2a["logues"] = logues
+        ls_ret["readme"] = cgv["readme"] = readme
+
         if not files and not dirs and not readme and not logues[0] and not logues[1]:
             logues[1] = "this folder is empty"
 
+        if "descript.ion" in lnames and os.path.isfile(
+            os.path.join(abspath, lnames["descript.ion"])
+        ):
+            rem = []
+            with open(os.path.join(abspath, lnames["descript.ion"]), "rb") as f:
+                for bln in [x.strip() for x in f]:
+                    try:
+                        if bln.endswith(b"\x04\xc2"):
+                            # multiline comment; replace literal r"\n" with " // "
+                            bln = bln.replace(br"\\n", b" // ")[:-2]
+                        ln = bln.decode("utf-8", "replace")
+                        if ln.startswith('"'):
+                            fn, desc = ln.split('" ', 1)
+                            fn = fn[1:]
+                        else:
+                            fn, desc = ln.split(" ", 1)
+                        fe = next(
+                            (x for x in files if x["name"].lower() == fn.lower()), None
+                        )
+                        if fe:
+                            fe["tags"]["descript.ion"] = desc
+                        else:
+                            t = "<li><code>%s</code> %s</li>"
+                            rem.append(t % (html_escape(fn), html_escape(desc)))
+                    except:
+                        pass
+            if "descript.ion" not in taglist:
+                taglist.insert(0, "descript.ion")
+            if rem and not logues[1]:
+                t = "<h3>descript.ion</h3><ul>\n"
+                logues[1] = t + "\n".join(rem) + "</ul>"
+
         if is_ls:
             ls_ret["dirs"] = dirs
             ls_ret["files"] = files
@@ -4769,14 +4894,13 @@ class HttpCli(object):
                         self.conn.hsrv.j2[tpl] = j2env.get_template(tname)
             thumb = ""
             is_pic = is_vid = is_au = False
-            covernames = self.args.th_coversd
-            for fn in ls_names:
-                if fn.lower() in covernames:
-                    thumb = fn
+            for fn in self.args.th_coversd:
+                if fn in lnames:
+                    thumb = lnames[fn]
                     break
             if og_fn:
                 ext = og_fn.split(".")[-1].lower()
-                if ext in self.thumbcli.thumbable:
+                if self.thumbcli and ext in self.thumbcli.thumbable:
                     is_pic = (
                         ext in self.thumbcli.fmt_pil
                         or ext in self.thumbcli.fmt_vips

copyparty/httpsrv.py

@@ -12,7 +12,7 @@ import time
 
 import queue
 
-from .__init__ import ANYWIN, CORES, EXE, MACOS, TYPE_CHECKING, EnvParams
+from .__init__ import ANYWIN, CORES, EXE, MACOS, TYPE_CHECKING, EnvParams, unicode
 
 try:
     MNFE = ModuleNotFoundError
@@ -335,11 +335,11 @@ class HttpSrv(object):
 
             try:
                 sck, saddr = srv_sck.accept()
-                cip, cport = saddr[:2]
+                cip = unicode(saddr[0])
                 if cip.startswith("::ffff:"):
                     cip = cip[7:]
 
-                addr = (cip, cport)
+                addr = (cip, saddr[1])
             except (OSError, socket.error) as ex:
                 if self.stopping:
                     break

copyparty/mdns.py

@@ -292,6 +292,22 @@ class MDNS(MCast):
     def run2(self) -> None:
         last_hop = time.time()
         ihop = self.args.mc_hop
+
+        try:
+            if self.args.no_poll:
+                raise Exception()
+            fd2sck = {}
+            srvpoll = select.poll()
+            for sck in self.srv:
+                fd = sck.fileno()
+                fd2sck[fd] = sck
+                srvpoll.register(fd, select.POLLIN)
+        except Exception as ex:
+            srvpoll = None
+            if not self.args.no_poll:
+                t = "WARNING: failed to poll(), will use select() instead: %r"
+                self.log(t % (ex,), 3)
+
         while self.running:
             timeout = (
                 0.02 + random.random() * 0.07
@@ -300,8 +316,13 @@ class MDNS(MCast):
                 if self.unsolicited
                 else (last_hop + ihop if ihop else 180)
             )
-            rdy = select.select(self.srv, [], [], timeout)
-            rx: list[socket.socket] = rdy[0]  # type: ignore
+            if srvpoll:
+                pr = srvpoll.poll(timeout * 1000)
+                rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
+            else:
+                rdy = select.select(self.srv, [], [], timeout)
+                rx: list[socket.socket] = rdy[0]  # type: ignore
+
             self.rx4.cln()
             self.rx6.cln()
             buf = b""
@@ -340,7 +361,7 @@ class MDNS(MCast):
             except:
                 pass
 
-        self.srv = {}
+        self.srv.clear()
 
     def eat(self, buf: bytes, addr: tuple[str, int], sck: socket.socket) -> None:
         cip = addr[0]

copyparty/mtag.py

@@ -111,7 +111,9 @@ class MParser(object):
             raise Exception()
 
 
-def au_unpk(log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None) -> str:
+def au_unpk(
+    log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
+) -> str:
     ret = ""
     try:
         ext = abspath.split(".")[-1].lower()
@@ -137,6 +139,9 @@ def au_unpk(log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optio
             zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
             fi = zf.open(zil[0])
 
+        else:
+            raise Exception("unknown compression %s" % (pk,))
+
         with os.fdopen(fd, "wb") as fo:
             while True:
                 buf = fi.read(32768)

copyparty/smbd.py

@@ -240,7 +240,7 @@ class SMB(object):
 
             xbu = vfs.flags.get("xbu")
             if xbu and not runhook(
-                self.nlog, xbu, ap, vpath, "", "", 0, 0, "1.7.6.2", 0, ""
+                self.nlog, xbu, ap, vpath, "", "", "", 0, 0, "1.7.6.2", 0, ""
             ):
                 yeet("blocked by xbu server config: " + vpath)
 

copyparty/ssdp.py

@@ -141,9 +141,29 @@ class SSDPd(MCast):
         self.log("stopped", 2)
 
     def run2(self) -> None:
+        try:
+            if self.args.no_poll:
+                raise Exception()
+            fd2sck = {}
+            srvpoll = select.poll()
+            for sck in self.srv:
+                fd = sck.fileno()
+                fd2sck[fd] = sck
+                srvpoll.register(fd, select.POLLIN)
+        except Exception as ex:
+            srvpoll = None
+            if not self.args.no_poll:
+                t = "WARNING: failed to poll(), will use select() instead: %r"
+                self.log(t % (ex,), 3)
+
         while self.running:
-            rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
-            rx: list[socket.socket] = rdy[0]  # type: ignore
+            if srvpoll:
+                pr = srvpoll.poll((self.args.z_chk or 180) * 1000)
+                rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
+            else:
+                rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
+                rx: list[socket.socket] = rdy[0]  # type: ignore
+
             self.rxc.cln()
             buf = b""
             addr = ("0", 0)
@@ -168,7 +188,7 @@ class SSDPd(MCast):
             except:
                 pass
 
-        self.srv = {}
+        self.srv.clear()
 
     def eat(self, buf: bytes, addr: tuple[str, int]) -> None:
         cip = addr[0]

copyparty/star.py

@@ -1,13 +1,13 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
-import argparse
 import re
 import stat
 import tarfile
 
 from queue import Queue
 
+from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
 from .util import Daemon, fsenc, min_ex
@@ -45,12 +45,12 @@ class StreamTar(StreamArc):
     def __init__(
         self,
         log: "NamedLogger",
-        args: argparse.Namespace,
+        asrv: AuthSrv,
         fgen: Generator[dict[str, Any], None, None],
         cmp: str = "",
         **kwargs: Any
     ):
-        super(StreamTar, self).__init__(log, args, fgen)
+        super(StreamTar, self).__init__(log, asrv, fgen)
 
         self.ci = 0
         self.co = 0
@@ -148,7 +148,7 @@ class StreamTar(StreamArc):
                 errors.append((f["vp"], ex))
 
         if errors:
-            self.errf, txt = errdesc(errors)
+            self.errf, txt = errdesc(self.asrv.vfs, errors)
             self.log("\n".join(([repr(self.errf)] + txt[1:])))
             self.ser(self.errf)
 

copyparty/sutil.py

@@ -1,15 +1,15 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
-import argparse
 import os
 import tempfile
 from datetime import datetime
 
 from .__init__ import CORES
+from .authsrv import AuthSrv, VFS
 from .bos import bos
 from .th_cli import ThumbCli
-from .util import UTC, vjoin
+from .util import UTC, vjoin, vol_san
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Generator, Optional
@@ -21,12 +21,13 @@ class StreamArc(object):
     def __init__(
         self,
         log: "NamedLogger",
-        args: argparse.Namespace,
+        asrv: AuthSrv,
         fgen: Generator[dict[str, Any], None, None],
         **kwargs: Any
     ):
         self.log = log
-        self.args = args
+        self.asrv = asrv
+        self.args = asrv.args
         self.fgen = fgen
         self.stopped = False
 
@@ -103,15 +104,20 @@ def enthumb(
     return f
 
 
-def errdesc(errors: list[tuple[str, str]]) -> tuple[dict[str, Any], list[str]]:
+def errdesc(
+    vfs: VFS, errors: list[tuple[str, str]]
+) -> tuple[dict[str, Any], list[str]]:
     report = ["copyparty failed to add the following files to the archive:", ""]
 
     for fn, err in errors:
         report.extend([" file: {}".format(fn), "error: {}".format(err), ""])
 
+    btxt = "\r\n".join(report).encode("utf-8", "replace")
+    btxt = vol_san(list(vfs.all_vols.values()), btxt)
+
     with tempfile.NamedTemporaryFile(prefix="copyparty-", delete=False) as tf:
         tf_path = tf.name
-        tf.write("\r\n".join(report).encode("utf-8", "replace"))
+        tf.write(btxt)
 
     dt = datetime.now(UTC).strftime("%Y-%m%d-%H%M%S")
 

copyparty/svchub.py

@@ -479,8 +479,10 @@ class SvcHub(object):
         zsl = al.th_covers.split(",")
         zsl = [x.strip() for x in zsl]
         zsl = [x for x in zsl if x]
-        al.th_covers = set(zsl)
-        al.th_coversd = set(zsl + ["." + x for x in zsl])
+        al.th_covers = zsl
+        al.th_coversd = zsl + ["." + x for x in zsl]
+        al.th_covers_set = set(al.th_covers)
+        al.th_coversd_set = set(al.th_coversd)
 
         for k in "c".split(" "):
             vl = getattr(al, k)

copyparty/szip.py

@@ -1,12 +1,12 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
-import argparse
 import calendar
 import stat
 import time
 import zlib
 
+from .authsrv import AuthSrv
 from .bos import bos
 from .sutil import StreamArc, errdesc
 from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile
@@ -219,13 +219,13 @@ class StreamZip(StreamArc):
     def __init__(
         self,
         log: "NamedLogger",
-        args: argparse.Namespace,
+        asrv: AuthSrv,
         fgen: Generator[dict[str, Any], None, None],
         utf8: bool = False,
         pre_crc: bool = False,
         **kwargs: Any
     ) -> None:
-        super(StreamZip, self).__init__(log, args, fgen)
+        super(StreamZip, self).__init__(log, asrv, fgen)
 
         self.utf8 = utf8
         self.pre_crc = pre_crc
@@ -302,7 +302,7 @@ class StreamZip(StreamArc):
                 mbuf = b""
 
             if errors:
-                errf, txt = errdesc(errors)
+                errf, txt = errdesc(self.asrv.vfs, errors)
                 self.log("\n".join(([repr(errf)] + txt[1:])))
                 for x in self.ser(errf):
                     yield x

copyparty/tcpsrv.py

@@ -15,6 +15,7 @@ from .util import (
     E_ADDR_IN_USE,
     E_ADDR_NOT_AVAIL,
     E_UNREACH,
+    HAVE_IPV6,
     IP6ALL,
     Netdev,
     min_ex,
@@ -111,8 +112,10 @@ class TcpSrv(object):
 
         eps = {
             "127.0.0.1": Netdev("127.0.0.1", 0, "", "local only"),
-            "::1": Netdev("::1", 0, "", "local only"),
         }
+        if HAVE_IPV6:
+            eps["::1"] = Netdev("::1", 0, "", "local only")
+
         nonlocals = [x for x in self.args.i if x not in [k.split("/")[0] for k in eps]]
         if nonlocals:
             try:

copyparty/tftpd.py

@@ -33,7 +33,7 @@ from partftpy import (
 )
 from partftpy.TftpShared import TftpException
 
-from .__init__ import EXE, TYPE_CHECKING
+from .__init__ import EXE, PY2, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .util import BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
@@ -95,7 +95,7 @@ class Tftpd(object):
                 TftpServer,
             ]
             cbak = []
-            if not self.args.tftp_no_fast and not EXE:
+            if not self.args.tftp_no_fast and not EXE and not PY2:
                 try:
                     ptn = re.compile(r"(^\s*)log\.debug\(.*\)$")
                     for C in Cs:
@@ -105,7 +105,7 @@ class Tftpd(object):
                         cfn = C.__spec__.origin
                         exec (compile(src2, filename=cfn, mode="exec"), C.__dict__)
                 except Exception:
-                    t = "failed to optimize tftp code; run with --tftp-noopt if there are issues:\n"
+                    t = "failed to optimize tftp code; run with --tftp-no-fast if there are issues:\n"
                     self.log("tftp", t + min_ex(), 3)
                     for n, zd in enumerate(cbak):
                         Cs[n].__dict__ = zd
@@ -150,11 +150,6 @@ class Tftpd(object):
 
         self._disarm(fos)
 
-        ip = next((x for x in self.args.i if ":" not in x), None)
-        if not ip:
-            self.log("tftp", "IPv6 not supported for tftp; listening on 0.0.0.0", 3)
-            ip = "0.0.0.0"
-
         self.port = int(self.args.tftp)
         self.srv = []
         self.ips = []
@@ -168,7 +163,7 @@ class Tftpd(object):
         if "::" in ips:
             ips.append("0.0.0.0")
 
-        if self.args.ftp4:
+        if self.args.tftp4:
             ips = [x for x in ips if ":" not in x]
 
         ips = list(ODict.fromkeys(ips))  # dedup
@@ -333,7 +328,7 @@ class Tftpd(object):
 
             xbu = vfs.flags.get("xbu")
             if xbu and not runhook(
-                self.nlog, xbu, ap, vpath, "", "", 0, 0, "8.3.8.7", 0, ""
+                self.nlog, xbu, ap, vpath, "", "", "", 0, 0, "8.3.8.7", 0, ""
             ):
                 yeet("blocked by xbu server config: " + vpath)
 

copyparty/th_cli.py

@@ -106,7 +106,7 @@ class ThumbCli(object):
                 sfmt += "3" if "3" in fmt else ""
 
             fmt = sfmt
-        
+
         elif fmt[:1] == "p" and not is_au:
             t = "cannot thumbnail [%s]: png only allowed for waveforms"
             self.log(t % (rem), 6)

copyparty/th_srv.py

@@ -599,13 +599,14 @@ class ThumbSrv(object):
                 b"pngquant",
                 b"--strip",
                 b"--nofs",
-                b"--output", fsenc(wtpath),
-                fsenc(tpath)
+                b"--output",
+                fsenc(wtpath),
+                fsenc(tpath),
             ]
             ret = runcmd(cmd, timeout=vn.flags["convt"], nice=True, oom=400)[0]
             if ret:
                 try:
-                    wunlink(self.log, wtpath,  vn.flags)
+                    wunlink(self.log, wtpath, vn.flags)
                 except:
                     pass
             else:
@@ -673,8 +674,8 @@ class ThumbSrv(object):
             raise Exception("disabled in server config")
 
         self.wait4ram(0.2, tpath)
-        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
-        if "ac" not in ret:
+        tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        if "ac" not in tags:
             raise Exception("not audio")
 
         if quality.endswith("k"):
@@ -695,7 +696,7 @@ class ThumbSrv(object):
             b"-v", b"error",
             b"-hide_banner",
             b"-i", fsenc(abspath),
-            b"-map_metadata", b"-1",
+        ] + self.big_tags(rawtags) + [
             b"-map", b"0:a:0",
             b"-ar", b"44100",
             b"-ac", b"2",
@@ -711,16 +712,16 @@ class ThumbSrv(object):
             raise Exception("disabled in server config")
 
         self.wait4ram(0.2, tpath)
-        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
-        if "ac" not in ret:
+        tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        if "ac" not in tags:
             raise Exception("not audio")
 
         try:
-            dur = ret[".dur"][1]
+            dur = tags[".dur"][1]
         except:
             dur = 0
 
-        src_opus = abspath.lower().endswith(".opus") or ret["ac"][1] == "opus"
+        src_opus = abspath.lower().endswith(".opus") or tags["ac"][1] == "opus"
         want_caf = tpath.endswith(".caf")
         tmp_opus = tpath
         if want_caf:
@@ -741,7 +742,7 @@ class ThumbSrv(object):
                 b"-v", b"error",
                 b"-hide_banner",
                 b"-i", fsenc(abspath),
-                b"-map_metadata", b"-1",
+            ] + self.big_tags(rawtags) + [
                 b"-map", b"0:a:0",
                 b"-c:a", b"libopus",
                 b"-b:a", bq,
@@ -798,6 +799,16 @@ class ThumbSrv(object):
             except:
                 pass
 
+    def big_tags(self, raw_tags: dict[str, list[str]]) -> list[bytes]:
+        ret = []
+        for k, vs in raw_tags.items():
+            for v in vs:
+                if len(str(v)) >= 1024:
+                    bv = k.encode("utf-8", "replace")
+                    ret += [b"-metadata", bv + b"="]
+                    break
+        return ret
+
     def poke(self, tdir: str) -> None:
         if not self.poke_cd.poke(tdir):
             return

copyparty/up2k.py

@@ -654,7 +654,7 @@ class Up2k(object):
             return False, flags
 
         ret = {k: v for k, v in flags.items() if not k.startswith("e2t")}
-        if ret.keys() == flags.keys():
+        if len(ret) == len(flags):
             return False, flags
 
         return True, ret
@@ -709,9 +709,9 @@ class Up2k(object):
                 try:
                     bos.makedirs(vol.realpath)  # gonna happen at snap anyways
                     dir_is_empty(self.log_func, not self.args.no_scandir, vol.realpath)
-                except:
+                except Exception as ex:
                     self.volstate[vol.vpath] = "OFFLINE (cannot access folder)"
-                    self.log("cannot access " + vol.realpath, c=1)
+                    self.log("cannot access %s: %r" % (vol.realpath, ex), c=1)
                     continue
 
                 if scan_vols and vol.vpath not in scan_vols:
@@ -1195,6 +1195,9 @@ class Up2k(object):
         fat32 = True
         cv = ""
 
+        th_cvd = self.args.th_coversd
+        th_cvds = self.args.th_coversd_set
+
         assert self.pp and self.mem_cur
         self.pp.msg = "a%d %s" % (self.pp.n, cdir)
 
@@ -1279,12 +1282,21 @@ class Up2k(object):
 
                 files.append((sz, lmod, iname))
                 liname = iname.lower()
-                if sz and (
-                    iname in self.args.th_coversd
-                    or (
+                if (
+                    sz
+                    and (
+                        liname in th_cvds
+                        or (
+                            not cv
+                            and liname.rsplit(".", 1)[-1] in CV_EXTS
+                            and not iname.startswith(".")
+                        )
+                    )
+                    and (
                         not cv
-                        and liname.rsplit(".", 1)[-1] in CV_EXTS
-                        and not iname.startswith(".")
+                        or liname not in th_cvds
+                        or cv.lower() not in th_cvds
+                        or th_cvd.index(iname) < th_cvd.index(cv)
                     )
                 ):
                     cv = iname
@@ -2758,6 +2770,7 @@ class Up2k(object):
                             job["vtop"],
                             job["host"],
                             job["user"],
+                            self.asrv.vfs.get_perms(job["vtop"], job["user"]),
                             job["lmod"],
                             job["size"],
                             job["addr"],
@@ -3080,7 +3093,7 @@ class Up2k(object):
 
         return ret, dst
 
-    def finish_upload(self, ptop: str, wark: str, busy_aps: set[str]) -> None:
+    def finish_upload(self, ptop: str, wark: str, busy_aps: dict[str, int]) -> None:
         self.busy_aps = busy_aps
         with self.mutex, self.reg_mutex:
             self._finish_upload(ptop, wark)
@@ -3285,6 +3298,7 @@ class Up2k(object):
             djoin(vtop, rd, fn),
             host,
             usr,
+            self.asrv.vfs.get_perms(djoin(vtop, rd, fn), usr),
             int(ts),
             sz,
             ip,
@@ -3319,15 +3333,29 @@ class Up2k(object):
                 with self.rescan_cond:
                     self.rescan_cond.notify_all()
 
-        if rd and sz and fn.lower() in self.args.th_coversd:
+        if rd and sz and fn.lower() in self.args.th_coversd_set:
             # wasteful; db_add will re-index actual covers
             # but that won't catch existing files
             crd, cdn = rd.rsplit("/", 1) if "/" in rd else ("", rd)
             try:
-                db.execute("delete from cv where rd=? and dn=?", (crd, cdn))
-                db.execute("insert into cv values (?,?,?)", (crd, cdn, fn))
+                q = "select fn from cv where rd=? and dn=?"
+                db_cv = db.execute(q, (crd, cdn)).fetchone()[0]
+                db_lcv = db_cv.lower()
+                if db_lcv in self.args.th_coversd_set:
+                    idx_db = self.args.th_coversd.index(db_lcv)
+                    idx_fn = self.args.th_coversd.index(fn.lower())
+                    add_cv = idx_fn < idx_db
+                else:
+                    add_cv = True
             except:
-                pass
+                add_cv = True
+
+            if add_cv:
+                try:
+                    db.execute("delete from cv where rd=? and dn=?", (crd, cdn))
+                    db.execute("insert into cv values (?,?,?)", (crd, cdn, fn))
+                except:
+                    pass
 
     def handle_rm(
         self,
@@ -3470,6 +3498,7 @@ class Up2k(object):
                         vpath,
                         "",
                         uname,
+                        self.asrv.vfs.get_perms(vpath, uname),
                         stl.st_mtime,
                         st.st_size,
                         ip,
@@ -3503,6 +3532,7 @@ class Up2k(object):
                         vpath,
                         "",
                         uname,
+                        self.asrv.vfs.get_perms(vpath, uname),
                         stl.st_mtime,
                         st.st_size,
                         ip,
@@ -3635,7 +3665,18 @@ class Up2k(object):
         xar = dvn.flags.get("xar")
         if xbr:
             if not runhook(
-                self.log, xbr, sabs, svp, "", uname, stl.st_mtime, st.st_size, "", 0, ""
+                self.log,
+                xbr,
+                sabs,
+                svp,
+                "",
+                uname,
+                self.asrv.vfs.get_perms(svp, uname),
+                stl.st_mtime,
+                st.st_size,
+                "",
+                0,
+                "",
             ):
                 t = "move blocked by xbr server config: {}".format(svp)
                 self.log(t, 1)
@@ -3660,7 +3701,20 @@ class Up2k(object):
                 self.rescan_cond.notify_all()
 
             if xar:
-                runhook(self.log, xar, dabs, dvp, "", uname, 0, 0, "", 0, "")
+                runhook(
+                    self.log,
+                    xar,
+                    dabs,
+                    dvp,
+                    "",
+                    uname,
+                    self.asrv.vfs.get_perms(dvp, uname),
+                    0,
+                    0,
+                    "",
+                    0,
+                    "",
+                )
 
             return "k"
 
@@ -3759,7 +3813,20 @@ class Up2k(object):
             wunlink(self.log, sabs, svn.flags)
 
         if xar:
-            runhook(self.log, xar, dabs, dvp, "", uname, 0, 0, "", 0, "")
+            runhook(
+                self.log,
+                xar,
+                dabs,
+                dvp,
+                "",
+                uname,
+                self.asrv.vfs.get_perms(dvp, uname),
+                0,
+                0,
+                "",
+                0,
+                "",
+            )
 
         return "k"
 
@@ -4048,6 +4115,7 @@ class Up2k(object):
             vp_chk,
             job["host"],
             job["user"],
+            self.asrv.vfs.get_perms(vp_chk, job["user"]),
             int(job["lmod"]),
             job["size"],
             job["addr"],

copyparty/util.py

@@ -158,6 +158,18 @@ else:
     from urllib import unquote  # type: ignore # pylint: disable=no-name-in-module
 
 
+try:
+    socket.inet_pton(socket.AF_INET6, "::1")
+    HAVE_IPV6 = True
+except:
+
+    def inet_pton(fam, ip):
+        return socket.inet_aton(ip)
+
+    socket.inet_pton = inet_pton
+    HAVE_IPV6 = False
+
+
 try:
     struct.unpack(b">i", b"idgi")
     spack = struct.pack  # type: ignore
@@ -231,6 +243,7 @@ IMPLICATIONS = [
     ["e2vu", "e2v"],
     ["e2vp", "e2v"],
     ["e2v", "e2d"],
+    ["tftpvv", "tftpv"],
     ["smbw", "smb"],
     ["smb1", "smb"],
     ["smbvvv", "smbvv"],
@@ -358,6 +371,18 @@ APPLESAN_TXT = r"/(__MACOS|Icon\r\r)|/\.(_|DS_Store|AppleDouble|LSOverride|Docum
 APPLESAN_RE = re.compile(APPLESAN_TXT)
 
 
+HUMANSIZE_UNITS = ("B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB")
+
+UNHUMANIZE_UNITS = {
+    "b": 1,
+    "k": 1024,
+    "m": 1024 * 1024,
+    "g": 1024 * 1024 * 1024,
+    "t": 1024 * 1024 * 1024 * 1024,
+    "p": 1024 * 1024 * 1024 * 1024 * 1024,
+    "e": 1024 * 1024 * 1024 * 1024 * 1024 * 1024,
+}
+
 VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
 
 
@@ -1808,7 +1833,7 @@ def gencookie(k: str, v: str, r: str, tls: bool, dur: int = 0, txt: str = "") ->
 
 
 def humansize(sz: float, terse: bool = False) -> str:
-    for unit in ["B", "KiB", "MiB", "GiB", "TiB"]:
+    for unit in HUMANSIZE_UNITS:
         if sz < 1024:
             break
 
@@ -1829,12 +1854,7 @@ def unhumanize(sz: str) -> int:
         pass
 
     mc = sz[-1:].lower()
-    mi = {
-        "k": 1024,
-        "m": 1024 * 1024,
-        "g": 1024 * 1024 * 1024,
-        "t": 1024 * 1024 * 1024 * 1024,
-    }.get(mc, 1)
+    mi = UNHUMANIZE_UNITS.get(mc, 1)
     return int(float(sz[:-1]) * mi)
 
 
@@ -2451,6 +2471,9 @@ def build_netmap(csv: str):
         csv += ", 127.0.0.0/8, ::1/128"  # loopback
 
     srcs = [x.strip() for x in csv.split(",") if x.strip()]
+    if not HAVE_IPV6:
+        srcs = [x for x in srcs if ":" not in x]
+
     cidrs = []
     for zs in srcs:
         if not zs.endswith("."):
@@ -2488,7 +2511,7 @@ def yieldfile(fn: str, bufsz: int) -> Generator[bytes, None, None]:
 def hashcopy(
     fin: Generator[bytes, None, None],
     fout: Union[typing.BinaryIO, typing.IO[Any]],
-    slp: int = 0,
+    slp: float = 0,
     max_sz: int = 0,
 ) -> tuple[int, str, str]:
     hashobj = hashlib.sha512()
@@ -2516,7 +2539,8 @@ def sendfile_py(
     f: typing.BinaryIO,
     s: socket.socket,
     bufsz: int,
-    slp: int,
+    slp: float,
+    use_poll: bool,
 ) -> int:
     remains = upper - lower
     f.seek(lower)
@@ -2544,23 +2568,32 @@ def sendfile_kern(
     f: typing.BinaryIO,
     s: socket.socket,
     bufsz: int,
-    slp: int,
+    slp: float,
+    use_poll: bool,
 ) -> int:
     out_fd = s.fileno()
     in_fd = f.fileno()
     ofs = lower
     stuck = 0.0
+    if use_poll:
+        poll = select.poll()
+        poll.register(out_fd, select.POLLOUT)
+
     while ofs < upper:
         stuck = stuck or time.time()
         try:
             req = min(2 ** 30, upper - ofs)
-            select.select([], [out_fd], [], 10)
+            if use_poll:
+                poll.poll(10000)
+            else:
+                select.select([], [out_fd], [], 10)
             n = os.sendfile(out_fd, in_fd, ofs, req)
             stuck = 0
         except OSError as ex:
             # client stopped reading; do another select
             d = time.time() - stuck
             if d < 3600 and ex.errno == errno.EWOULDBLOCK:
+                time.sleep(0.02)
                 continue
 
             n = 0
@@ -2959,7 +2992,8 @@ def retchk(
 
 def _parsehook(
     log: Optional["NamedLogger"], cmd: str
-) -> tuple[bool, bool, bool, float, dict[str, Any], str]:
+) -> tuple[str, bool, bool, bool, float, dict[str, Any], list[str]]:
+    areq = ""
     chk = False
     fork = False
     jtxt = False
@@ -2984,8 +3018,12 @@ def _parsehook(
             cap = int(arg[1:])  # 0=none 1=stdout 2=stderr 3=both
         elif arg.startswith("k"):
             kill = arg[1:]  # [t]ree [m]ain [n]one
+        elif arg.startswith("a"):
+            areq = arg[1:]  # required perms
         elif arg.startswith("i"):
             pass
+        elif not arg:
+            break
         else:
             t = "hook: invalid flag {} in {}"
             (log or print)(t.format(arg, ocmd))
@@ -3012,9 +3050,11 @@ def _parsehook(
         "capture": cap,
     }
 
-    cmd = os.path.expandvars(os.path.expanduser(cmd))
+    argv = cmd.split(",") if "," in cmd else [cmd]
 
-    return chk, fork, jtxt, wait, sp_ka, cmd
+    argv[0] = os.path.expandvars(os.path.expanduser(argv[0]))
+
+    return areq, chk, fork, jtxt, wait, sp_ka, argv
 
 
 def runihook(
@@ -3023,10 +3063,9 @@ def runihook(
     vol: "VFS",
     ups: list[tuple[str, int, int, str, str, str, int]],
 ) -> bool:
-    ocmd = cmd
-    chk, fork, jtxt, wait, sp_ka, cmd = _parsehook(log, cmd)
-    bcmd = [sfsenc(cmd)]
-    if cmd.endswith(".py"):
+    _, chk, fork, jtxt, wait, sp_ka, acmd = _parsehook(log, cmd)
+    bcmd = [sfsenc(x) for x in acmd]
+    if acmd[0].endswith(".py"):
         bcmd = [sfsenc(pybin)] + bcmd
 
     vps = [vjoin(*list(s3dec(x[3], x[4]))) for x in ups]
@@ -3051,7 +3090,7 @@ def runihook(
 
     t0 = time.time()
     if fork:
-        Daemon(runcmd, ocmd, [bcmd], ka=sp_ka)
+        Daemon(runcmd, cmd, bcmd, ka=sp_ka)
     else:
         rc, v, err = runcmd(bcmd, **sp_ka)  # type: ignore
         if chk and rc:
@@ -3072,14 +3111,20 @@ def _runhook(
     vp: str,
     host: str,
     uname: str,
+    perms: str,
     mt: float,
     sz: int,
     ip: str,
     at: float,
     txt: str,
 ) -> bool:
-    ocmd = cmd
-    chk, fork, jtxt, wait, sp_ka, cmd = _parsehook(log, cmd)
+    areq, chk, fork, jtxt, wait, sp_ka, acmd = _parsehook(log, cmd)
+    if areq:
+        for ch in areq:
+            if ch not in perms:
+                t = "user %s not allowed to run hook %s; need perms %s, have %s"
+                log(t % (uname, cmd, areq, perms))
+                return True  # fallthrough to next hook
     if jtxt:
         ja = {
             "ap": ap,
@@ -3090,21 +3135,22 @@ def _runhook(
             "at": at or time.time(),
             "host": host,
             "user": uname,
+            "perms": perms,
             "txt": txt,
         }
         arg = json.dumps(ja)
     else:
         arg = txt or ap
 
-    acmd = [cmd, arg]
-    if cmd.endswith(".py"):
+    acmd += [arg]
+    if acmd[0].endswith(".py"):
         acmd = [pybin] + acmd
 
     bcmd = [fsenc(x) if x == ap else sfsenc(x) for x in acmd]
 
     t0 = time.time()
     if fork:
-        Daemon(runcmd, ocmd, [bcmd], ka=sp_ka)
+        Daemon(runcmd, cmd, [bcmd], ka=sp_ka)
     else:
         rc, v, err = runcmd(bcmd, **sp_ka)  # type: ignore
         if chk and rc:
@@ -3125,6 +3171,7 @@ def runhook(
     vp: str,
     host: str,
     uname: str,
+    perms: str,
     mt: float,
     sz: int,
     ip: str,
@@ -3134,7 +3181,7 @@ def runhook(
     vp = vp.replace("\\", "/")
     for cmd in cmds:
         try:
-            if not _runhook(log, cmd, ap, vp, host, uname, mt, sz, ip, at, txt):
+            if not _runhook(log, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt):
                 return False
         except Exception as ex:
             (log or print)("hook: {}".format(ex))

copyparty/web/baguettebox.js

@@ -743,6 +743,8 @@ window.baguetteBox = (function () {
             image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
             image.setAttribute('controls', 'controls');
             image.onended = vidEnd;
+            image.onplay = function () { show_buttons(1); };
+            image.onpause = function () { show_buttons(); };
         }
         image.alt = thumbnailElement ? thumbnailElement.alt || '' : '';
         if (options.titleTag && imageCaption)
@@ -988,6 +990,12 @@ window.baguetteBox = (function () {
         }
     }
 
+    function show_buttons(v) {
+        clmod(ebi('bbox-btns'), 'off', v);
+        clmod(btnPrev, 'off', v);
+        clmod(btnNext, 'off', v);
+    }
+
     function bounceAnimation(direction) {
         slider.className = options.animation == 'slideIn' ? 'bounce-from-' + direction : 'eog';
         setTimeout(function () {
@@ -1051,10 +1059,8 @@ window.baguetteBox = (function () {
             if (fx > 0.7)
                 return showNextImage();
 
-            clmod(ebi('bbox-btns'), 'off', 't');
-            clmod(btnPrev, 'off', 't');
-            clmod(btnNext, 'off', 't');
-
+            show_buttons('t');
+            
             if (Date.now() - ctime <= 500 && !IPHONE)
                 tglfull();
 

copyparty/web/browser.css

@@ -269,6 +269,7 @@ html.bz {
 	--btn-bg: #202231;
 	--btn-h-bg: #2d2f45;
 	--btn-1-bg: #ba2959;
+	--btn-1-is: #f59;
 	--btn-1-fg: #fff;
 	--btn-1h-fg: #000;
 	--txt-sh: a;
@@ -3060,6 +3061,14 @@ html.b #ggrid>a {
 html.b .btn {
 	top: -.1em;
 }
+html.b .btn,
+html.b #u2conf a.b,
+html.b #u2conf input[type="checkbox"]:not(:checked)+label {
+	box-shadow: 0 .05em 0 var(--bg-d3) inset;
+}
+html.b .tgl.btn.on {
+	box-shadow: 0 .05em 0 var(--btn-1-is) inset;
+}
 html.b #op_up2k.srch sup {
 	color: #fc0;
 }

copyparty/web/browser.js

@@ -400,6 +400,7 @@ var Ls = {
 		"badreply": "Failed to parse reply from server",
 
 		"xhr403": "403: Access denied\n\ntry pressing F5, maybe you got logged out",
+		"xhr0": "unknown (probably lost connection to server, or server is offline)",
 		"cf_ok": "sorry about that -- DD" + wah + "oS protection kicked in\n\nthings should resume in about 30 sec\n\nif nothing happens, hit F5 to reload the page",
 		"tl_xe1": "could not list subfolders:\n\nerror ",
 		"tl_xe2": "404: Folder not found",
@@ -910,6 +911,7 @@ var Ls = {
 		"badreply": "Ugyldig svar ifra serveren",
 
 		"xhr403": "403: Tilgang nektet\n\nkanskje du ble logget ut? prøv å trykk F5",
+		"xhr0": "ukjent (enten nettverksproblemer eller serverkrasj)",
 		"cf_ok": "beklager -- liten tilfeldig kontroll, alt OK\n\nting skal fortsette om ca. 30 sekunder\n\nhvis ikkeno skjer, trykk F5 for å laste siden på nytt",
 		"tl_xe1": "kunne ikke hente undermapper:\n\nfeil ",
 		"tl_xe2": "404: Mappen finnes ikke",
@@ -1062,7 +1064,7 @@ ebi('ops').innerHTML = (
 	'<a href="#" data-perm="write" data-dest="bup" tt="' + L.ot_bup + '">🎈</a>' +
 	'<a href="#" data-perm="write" data-dest="mkdir" tt="' + L.ot_mkdir + '">📂</a>' +
 	'<a href="#" data-perm="read write" data-dest="new_md" tt="' + L.ot_md + '">📝</a>' +
-	'<a href="#" data-perm="write" data-dest="msg" tt="' + L.ot_msg + '">📟</a>' +
+	'<a href="#" data-dest="msg" tt="' + L.ot_msg + '">📟</a>' +
 	'<a href="#" data-dest="player" tt="' + L.ot_mp + '">🎺</a>' +
 	'<a href="#" data-dest="cfg" tt="' + L.ot_cfg + '">⚙️</a>' +
 	(IE ? '<span id="noie">' + L.ot_noie + '</span>' : '') +
@@ -2047,8 +2049,8 @@ var pbar = (function () {
 		r.buf = canvas_cfg(ebi('barbuf'));
 		r.pos = canvas_cfg(ebi('barpos'));
 		r.buf.ctx.font = '.5em sans-serif';
-		r.pos.ctx.font = '1em sans-serif';
-		r.pos.ctx.strokeStyle = 'rgba(24,56,0,0.4)';
+		r.pos.ctx.font = '.9em sans-serif';
+		r.pos.ctx.strokeStyle = 'rgba(24,56,0,0.5)';
 		r.drawbuf();
 		r.drawpos();
 		if (!r.pos.can.onmouseleave)
@@ -2214,7 +2216,7 @@ var pbar = (function () {
 		var m1 = pctx.measureText(t1),
 			m1b = pctx.measureText(t1 + ":88"),
 			m2 = pctx.measureText(t2),
-			yt = pc.h / 3 * 2.1,
+			yt = pc.h * 0.94,
 			xt1 = pc.w - (m1.width + 12),
 			xt2 = x < m1.width * 1.4 ? (x + 12) : (Math.min(pc.w - m1b.width, x - 12) - m2.width);
 
@@ -2431,7 +2433,7 @@ function dl_song() {
 		return toast.inf(10, L.f_dls);
 	}
 
-	var url = addq(mp.tracks[mp.au.tid], 'cache=987&_=' + ACB);
+	var url = addq(mp.au.osrc, 'cache=987&_=' + ACB);
 	dl_file(url);
 }
 
@@ -2493,7 +2495,7 @@ function mpause(e) {
 		seek_au_mul(x * 1.0 / rect.width);
 	};
 
-	if (!TOUCH)
+	if (!TOUCH) {
 		bar.onwheel = function (e) {
 			var dist = Math.sign(e.deltaY) * 10;
 			if (Math.abs(e.deltaY) < 30 && !e.deltaMode)
@@ -2505,6 +2507,19 @@ function mpause(e) {
 			seek_au_rel(dist);
 			ev(e);
 		};
+		ebi('pvol').onwheel = function (e) {
+			var dist = Math.sign(e.deltaY) * 10;
+			if (Math.abs(e.deltaY) < 30 && !e.deltaMode)
+				dist = e.deltaY;
+
+			if (!dist || !mp.au)
+				return true;
+
+			mp.setvol(mp.vol + dist / 500);
+			vbar.draw();
+			ev(e);
+		};
+	}
 })();
 
 
@@ -2589,6 +2604,12 @@ var mpui = (function () {
 						if (mpl.prescan_evp == evp)
 							throw "evp match";
 
+						if (mpl.traversals++ > 4) {
+							mpl.prescan_evp = null;
+							toast.inf(10, L.mm_nof);
+							throw L.mm_nof;
+						}
+
 						mpl.prescan_evp = evp;
 						toast.inf(10, L.mm_prescan);
 						treectl.ls_cb = repreload;
@@ -3040,6 +3061,7 @@ var afilt = (function () {
 
 // plays the tid'th audio file on the page
 function play(tid, is_ev, seek) {
+	clearTimeout(mpl.t_eplay);
 	if (mp.order.length == 0)
 		return console.log('no audio found wait what');
 
@@ -3116,6 +3138,7 @@ function play(tid, is_ev, seek) {
 	else
 		mp.au.src = mp.au.rsrc = url;
 
+	mp.au.osrc = mp.tracks[tid];
 	afilt.apply();
 
 	setTimeout(function () {
@@ -3172,7 +3195,7 @@ function play(tid, is_ev, seek) {
 		toast.err(0, esc(L.mm_playerr + basenames(ex)));
 	}
 	clmod(ebi(oid), 'act');
-	setTimeout(next_song, 5000);
+	mpl.t_eplay = setTimeout(next_song, 5000);
 }
 
 
@@ -3250,7 +3273,7 @@ function evau_error(e) {
 		return;
 	}
 
-	setTimeout(next_song, 15000);
+	mpl.t_eplay = setTimeout(next_song, 15000);
 }
 
 
@@ -5846,7 +5869,7 @@ var treectl = (function () {
 	bcfg_bind(r, 'ireadme', 'ireadme', true);
 	bcfg_bind(r, 'idxh', 'idxh', idxh, setidxh);
 	bcfg_bind(r, 'dyn', 'dyntree', true, onresize);
-	bcfg_bind(r, 'csel', 'csel', false);
+	bcfg_bind(r, 'csel', 'csel', dgsel);
 	bcfg_bind(r, 'dots', 'dotfiles', false, function (v) {
 		r.goto();
 		var xhr = new XHR();
@@ -6302,6 +6325,7 @@ var treectl = (function () {
 		r.nvis = r.lim;
 		r.sb_msg = false;
 		r.nextdir = xhr.top;
+		clearTimeout(mpl.t_eplay);
 		enspin('#tree');
 		enspin(thegrid.en ? '#gfiles' : '#files');
 		window.removeEventListener('scroll', r.tscroll);
@@ -6341,8 +6365,15 @@ var treectl = (function () {
 			var res = JSON.parse(this.responseText);
 		}
 		catch (ex) {
-			if (!this.hydrate)
+			if (r.ls_cb) {
+				r.ls_cb = null;
+				return toast.inf(10, L.mm_nof);
+			}
+
+			if (!this.hydrate) {
 				location = this.top;
+				return;
+			}
 
 			return toast.err(30, "bad <code>?ls</code> reply;\nexpected json, got this:\n\n" + esc(this.responseText + ''));
 		}

copyparty/web/md2.js

@@ -607,10 +607,10 @@ function md_newline() {
     var s = linebounds(true),
         ln = s.md.substring(s.n1, s.n2),
         m1 = /^( *)([0-9]+)(\. +)/.exec(ln),
-        m2 = /^[ \t>+-]*(\* )?/.exec(ln),
+        m2 = /^[ \t]*[>+*-]{0,2}[ \t]/.exec(ln),
         drop = dom_src.selectionEnd - dom_src.selectionStart;
 
-    var pre = m2[0];
+    var pre = m2 ? m2[0] : '';
     if (m1 !== null)
         pre = m1[1] + (parseInt(m1[2]) + 1) + m1[3];
 

copyparty/web/ui.css

@@ -184,6 +184,7 @@ html {
 	padding: 1.5em 2em;
 	border-width: .5em 0;
 }
+.logue code,
 #modalc code,
 #tt code {
 	color: #eee;

copyparty/web/util.js

@@ -2024,6 +2024,9 @@ function xhrchk(xhr, prefix, e404, lvl, tag) {
     if (xhr.status == 404)
         return toast.err(0, prefix + e404 + suf, tag);
 
+    if (!xhr.status && !errtxt)
+        return toast.err(0, prefix + L.xhr0);
+
     if (is_cf && (xhr.status == 403 || xhr.status == 503)) {
         var now = Date.now(), td = now - cf_cha_t;
         if (td < 15000)

docs/changelog.md

@@ -1,3 +1,85 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0601-2324  `v1.13.3`  700+
+
+## new features
+
+* keep tags when transcoding music to opus/mp3 07ea629c
+  * useful for batch-downloading folders with [on-the-fly transcoding](https://github.com/9001/copyparty#zip-downloads)
+  * excessively large tags will be individually dropped (traktor beatmaps, cover-art, xmp)
+
+## bugfixes
+
+* optimization for large amounts (700+) of tcp connections / clients 07b2bf11
+  * `select()` was used for non-https downloads and mdns/ssdp initialization, which would start spinning at more than 1024 FDs, so now they `poll()` when possible (so not on windows)
+  * default max number of connections on windows was lowered to 486 since windows maxes out at 512 FDs
+* the markdown editor autoindent would duplicate `<hr>` 692175f5
+
+## other changes
+
+* #83: more intuitive behavior for `--df` and the `df` volflag 5ad65450
+* print helpful warning if OS restrictions make it impossible to persist config b629d18d
+* censor filesystem paths in the download-as-zip error summary 5919607a
+* `u2c.exe`: explain that https is disabled bef96176
+* ux: 60c96f99
+  * hide lightbox buttons when a video is playing
+  * move audio seekbar text down a bit so it hides less of the waveform and minute-markers
+* updated dompurify to 3.1.5 f00b9394
+* updated docker images to alpine 3.20
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0510-1431  `v1.13.2`  s3xmodit.zip
+
+## new features
+
+* play [compressed](https://a.ocv.me/pub/demo/music/chiptunes/compressed/#af-99f0c0e4) s3xmodit chiptunes/modules c0466279
+  * can now read gz/xz/zip-compressed s3m/xm/mod/it songs
+  * new filetypes supported: mdz, mdgz, mdxz, s3z, s3gz, s3xz, xmz, xmgz, xmxz, itz, itgz, itxz
+  * and if you need to fit even more tracks on the mixtape, [try mo3](https://a.ocv.me/pub/demo/music/chiptunes/compressed/#af-0bc9b877)
+* option to batch-convert audio waveforms 38e4fdfe
+* volflag to improve audio waveform compression with pngquant 82ce6862
+* option to add or change mappings from file-extensions to mimetypes 560d7b66
+* export and publish the `--help` text for online viewing 560d7b66
+  * now available [as html](https://ocv.me/copyparty/helptext.html) and as [plaintext](https://ocv.me/copyparty/helptext.txt), includes many features not documented in the readme
+* another way to add your own UI translations 19d156ff
+
+## bugfixes
+
+* ensure OS signals are immediately received and processed 87c60a1e
+  * things like reload and shutdown signals from systemd could get lost/stuck
+* fix mimetype detection for uppercase file extensions 565daee9
+* when clicking a `.ts` file in the gridview, don't open it as text 925c7f0a
+  * ...as it's probably an mpeg transport-stream, not a typescript file
+* be less aggressive in dropping volume caches e396c5c2
+  * very minor performance gain, only really relevant if you're doing something like burning a copyparty volume onto a CD
+  * previously, adding or removing any volume at all was enough to drop covers cache for all volumes; now this only happens if an intersecting volume is added/removed
+
+## other changes
+
+* updated dompurify to 3.1.2 566cbb65
+* opengraph: add the full filename as url suffix 5c1e2390
+  * so discord picks a good filename when saving an image
+
+----
+
+# 💾 what to download?
+| download link | is it good? | description |
+| -- | -- | -- |
+| **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** | ✅ the best 👍 | runs anywhere! only needs python |
+| [a docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker/README.md) | it's ok | good if you prefer docker 🐋 |
+| [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) |  ⚠️ [acceptable](https://github.com/9001/copyparty#copypartyexe) | for [win8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png) or later; built-in thumbnailer |
+| [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.13.0/u2c.exe) | ⚠️ acceptable | [CLI uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/u2c.py) as a win7+ exe ([video](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm)) |
+| [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) | ⚠️ acceptable | similar to the regular sfx, [mostly worse](https://github.com/9001/copyparty#zipapp) |
+| [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) | ⛔️ [dangerous](https://github.com/9001/copyparty#copypartyexe) | for [win7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png) -- never expose to the internet! |
+| [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.10.1/copyparty-winpe64.exe) | ⛔️ dangerous | runs on [64bit WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png), otherwise useless |
+
+* except for [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.13.0/u2c.exe), all of the options above are mostly equivalent
+* the zip and tar.gz files below are just source code
+* python packages are available at [PyPI](https://pypi.org/project/copyparty/#files)
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2024-0506-0029  `v1.13.1`  ctrl-v
 
@@ -30,7 +112,7 @@
 
 ----
 
-this release introduces **[copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)**, yet another way to bring copyparty where it's needed -- very limited and with many drawbacks (see [readme](https://github.com/9001/copyparty#zipapp)) but may work when the others don't
+this release introduces **[copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)**, yet another way to bring copyparty where it's needed -- very limited and with many drawbacks (see [readme](https://github.com/9001/copyparty#zipapp)) but may work when the others don't
 
 
 

docs/devnotes.md

@@ -141,6 +141,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | GET | `?ups&filter=f` | ...where URL contains `f` |
 | GET | `?mime=foo` | specify return mimetype `foo` |
 | GET | `?v` | render markdown file at URL |
+| GET | `?v` | open image/video/audio in mediaplayer |
 | GET | `?txt` | get file at URL as plaintext |
 | GET | `?txt=iso-8859-1` | ...with specific charset |
 | GET | `?th` | get image/video at URL as thumbnail |
@@ -169,6 +170,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | mPOST | | `f=FILE` | upload `FILE` into the folder at URL |
 | mPOST | `?j` | `f=FILE` | ...and reply with json |
 | mPOST | `?replace` | `f=FILE` | ...and overwrite existing files |
+| mPOST | `?media` | `f=FILE` | ...and return medialink (not hotlink) |
 | mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
 | POST | `?delete` | | delete URL recursively |
 | jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |

pyproject.toml

@@ -49,7 +49,7 @@ thumbnails2 = ["pyvips"]
 audiotags = ["mutagen"]
 ftpd = ["pyftpdlib"]
 ftps = ["pyftpdlib", "pyopenssl"]
-tftpd = ["partftpy>=0.3.1"]
+tftpd = ["partftpy>=0.4.0"]
 pwhash = ["argon2-cffi"]
 
 [project.scripts]

scripts/deps-docker/Dockerfile

@@ -3,7 +3,7 @@ WORKDIR /z
 ENV     ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
         ver_hashwasm=4.10.0 \
         ver_marked=4.3.0 \
-        ver_dompf=3.1.2 \
+        ver_dompf=3.1.6 \
         ver_mde=2.18.0 \
         ver_codemirror=5.65.16 \
         ver_fontawesome=5.13.0 \

scripts/make-sfx.sh

@@ -219,9 +219,9 @@ necho() {
 	mv pyftpdlib ftp/
 
 	necho collecting partftpy
-	f="../build/partftpy-0.3.1.tar.gz"
+	f="../build/partftpy-0.4.0.tar.gz"
 	[ -e "$f" ] ||
-		(url=https://files.pythonhosted.org/packages/37/79/1a1de1d3fdf27ddc9c2d55fec6552e7b8ed115258fedac6120679898b83d/partftpy-0.3.1.tar.gz;
+		(url=https://files.pythonhosted.org/packages/8c/96/642bb3ddcb07a2c6764eb29aa562d1cf56877ad6c330c3c8921a5f05606d/partftpy-0.4.0.tar.gz;
 		wget -O$f "$url" || curl -L "$url" >$f)
 
 	tar -zxf $f
@@ -490,6 +490,11 @@ while IFS= read -r f; do
 	tmv "$f"
 done
 
+grep -rlE '^class [^(]+:' |
+while IFS= read -r f; do
+	ised 's/(^class [^(:]+):/\1(object):/' "$f"
+done
+
 # up2k goes from 28k to 22k laff
 awk 'BEGIN{gensub(//,"",1)}' </dev/null 2>/dev/null &&
 echo entabbening &&

scripts/pyinstaller/build.sh

@@ -16,7 +16,7 @@ uname -s | grep WOW64 && m=64 || m=32
 uname -s | grep NT-10 && w10=1 || w7=1
 [ $w7 ] && [ -e up2k.sh ] && [ ! "$1" ] && ./up2k.sh
 
-[ $w7 ] && pyv=37 || pyv=311
+[ $w7 ] && pyv=37 || pyv=312
 esuf=
 [ $w7 ] && [ $m = 32 ] && esuf=32
 [ $w7 ] && [ $m = 64 ] && esuf=-winpe64
@@ -127,3 +127,6 @@ grep -q $csum uplod.log && echo upload OK || {
     echo UPLOAD FAILED
     exit 1
 }
+
+echo; read -u1 -n1 -p 'shutdown? y/n: '
+[ "$REPLY" = y ] && shutdown -s -t 1

scripts/pyinstaller/deps.sha512

@@ -1,33 +1,39 @@
 f117016b1e6a7d7e745db30d3e67f1acf7957c443a0dd301b6c5e10b8368f2aa4db6be9782d2d3f84beadd139bfeef4982e40f21ca5d9065cb794eeb0e473e82  altgraph-0.17.4-py2.py3-none-any.whl
-e0d2e6183437af321a36944f04a501e85181243e5fa2da3254254305dd8119161f62048bc56bff8849b49f546ff175b02b4c999401f1c404f6b88e6f46a9c96e  Git-2.44.0-32-bit.exe
-9d2c31701a4d3fef553928c00528a48f9e1854ab5333528b50e358a214eba90029d687f039bcda5760b6fdf9f2de3bcf3784ae21a6374cf2a97a845d33b636c6  packaging-24.0-py3-none-any.whl
+6a624018f30da375581d5751eca0080edbbe37f102f643f856279fcfded3a4379fd1b6fb0661cdb2e72bbbbc726ca714a1f5990cc348df365db62bc53e4c4503  Git-2.45.2-32-bit.exe
 17ce52ba50692a9d964f57a23ac163fb74c77fdeb2ca988a6d439ae1fe91955ff43730c073af97a7b3223093ffea3479a996b9b50ee7fba0869247a56f74baa6  pefile-2023.2.7-py3-none-any.whl
-126ca016c00256f4ff13c88707ead21b3b98f3c665ae57a5bcbb80c8be3004bff36d9c7f9a1cc9d20551019708f2b195154f302d80a1e5a2026d6d0fe9f3d5f4  pyinstaller_hooks_contrib-2024.3-py2.py3-none-any.whl
 749a473646c6d4c7939989649733d4c7699fd1c359c27046bf5bc9c070d1a4b8b986bbc65f60d7da725baf16dbfdd75a4c2f5bb8335f2cb5685073f5fee5c2d1  pywin32_ctypes-0.2.2-py3-none-any.whl
-6e0d854040baff861e1647d2bece7d090bc793b2bd9819c56105b94090df54881a6a9b43ebd82578cd7c76d47181571b671e60672afd9def389d03c9dae84fcf  setuptools-68.2.2-py3-none-any.whl
+085d39ef4426aa5f097fbc484595becc16e61ca23fc7da4d2a8bba540a3b82e789e390b176c7151bdc67d01735cce22b1562cdb2e31273225a2d3e275851a4ad  setuptools-70.3.0-py3-none-any.whl
+360a141928f4a7ec18a994602cbb28bbf8b5cc7c077a06ac76b54b12fa769ed95ca0333a5cf728923a8e0baeb5cc4d5e73e5b3de2666beb05eb477d8ae719093  upx-4.2.4-win32.zip
 # u2c (win7)
 7a3bd4849f95e1715fe2e99613df70a0fedd944a9bfde71a0fadb837fe62c3431c30da4f0b75c74de6f1a459f1fdf7cb62eaf404fdbe45e2d121e0b1021f1580  certifi-2024.2.2-py3-none-any.whl
 9cc8acc5e269e6421bc32bb89261101da29d6ca337d39d60b9106de9ed7904e188716e4a48d78a2c4329026443fcab7acab013d2fe43778e30d6c4e4506a1b91  charset_normalizer-3.3.2-cp37-cp37m-win32.whl
 0ec1ae5c928b4a0001a254c8598b746049406e1eed720bfafa94d4474078eff76bf6e032124e2d4df4619052836523af36162443c6d746487b387d2e3476e691  idna-3.6-py3-none-any.whl
-b795abb26ba2f04f1afcfb196f21f638014b26c8186f8f488f1c2d91e8e0220962fbd259dbc9c3875222eb47fc95c73fc0606aaa6602b9ebc524809c9ba3501f  requests-2.31.0-py3-none-any.whl
-61ed4500b6361632030f05229705c5c5a52cb47e31c0e6b55151c8f3beed631cd752ca6c3d6393d56a2acf6a453cfcf801e877116123c550922249c3a976e0f4  urllib3-1.26.18-py2.py3-none-any.whl
+cc08d0d87d184401872a2f82266d589253979b4cd02f23b51290fbb2a20082848fc72acbed8aacb74ac4af068d575ef96e66196c5068bc38fb0bcafdc7626869  requests-2.29.0-py3-none-any.whl
+fe5fee6cb8a2c68800b32353a0015e5d2e1ad1cb6e0c9e6acf86e48e5cdb5606ad465dc4485ea5fbc8701d8716a8a7f7148c57724ef9da26b0c0a76f6dbbd698  urllib3-1.26.19-py2.py3-none-any.whl
 # win7
+3253e86471e6f9fa85bfdb7684cd2f964ed6e35c6a4db87f81cca157c049bef43e66dfcae1e037b2fb904567b1e028aaeefe8983ba3255105df787406d2aa71e  en_windows_7_professional_with_sp1_x86_dvd_u_677056.iso
+ab0db0283f61a5bbe44797d74546786bf41685175764a448d2e3bd629f292f1e7d829757b26be346b5044d78c9c1891736d93237cee4b1b6f5996a902c86d15f  en_windows_7_professional_with_sp1_x64_dvd_u_676939.iso
 d130bfa136bd171b9972b5c281c578545f2a84a909fdf18a6d2d71dd12fb3d512a7a1fa5cf7300433adece1d306eb2f22d7278f4c90e744e04dc67ba627a82c0  future-1.0.0-py3-none-any.whl
 0b4d07434bf8d314f42893d90bce005545b44a509e7353a73cad26dc9360b44e2824218a1a74f8174d02eba87fba91baffa82c8901279a32ebc6b8386b1b4275  importlib_metadata-6.7.0-py3-none-any.whl
+9d2c31701a4d3fef553928c00528a48f9e1854ab5333528b50e358a214eba90029d687f039bcda5760b6fdf9f2de3bcf3784ae21a6374cf2a97a845d33b636c6  packaging-24.0-py3-none-any.whl
 5d7462a584105bccaa9cf376f5a8c5827ead099c813c8af7392d478a4398f373d9e8cac7bbad2db51b335411ab966b21e119b1b1234c9a7ab70c6ddfc9306da6  pip-24.0-py3-none-any.whl
 f298e34356b5590dde7477d7b3a88ad39c622a2bcf3fcd7c53870ce8384dd510f690af81b8f42e121a22d3968a767d2e07595036b2ed7049c8ef4d112bcf3a61  pyinstaller-5.13.2-py3-none-win32.whl
+ea73aa54cc6d5db20dfb127e54562dabf890e4cd6171a91b10a51af2bcfc76e1d64cbdce4546df2dcfe42b624724c85b1cd05934be2413425b1f880222727b4f  pyinstaller-5.13.2-py3-none-win_amd64.whl
+2f4e3927a38cf7757bc9a1c06370d79209669a285a80f1b09cf9917137825c7022a50a56b351807e6e687e2c3a7bd7b2c5cc6daeb4d90e11920284c1a04a1cc3  pyinstaller_hooks_contrib-2023.8-py2.py3-none-any.whl
 6bb73cc2db795c59c92f2115727f5c173cacc9465af7710db9ff2f2aec2d73130d0992d0f16dcb3fac222dc15c0916562d0813b2337401022020673a4461df3d  python-3.7.9-amd64.exe
 500747651c87f59f2436c5ab91207b5b657856e43d10083f3ce27efb196a2580fadd199a4209519b409920c562aaaa7dcbdfb83ed2072a43eaccae6e2d056f31  python-3.7.9.exe
 03e50aecc85914567c114e38a1777e32628ee098756f37177bc23220eab33ac7d3ff591fd162db3b4d4e34d55cee93ef0dc67af68a69c38bb1435e0768dee57e  typing_extensions-4.7.1-py3-none-any.whl
-2e04acff170ca3bbceeeb18489c687126c951ec0bfd53cccfb389ba8d29a4576c1a9e8f2e5ea26c84dd21bfa2912f4e71fa72c1e2653b71e34afc0e65f1722d4  upx-4.2.2-win32.zip
 68e1b618d988be56aaae4e2eb92bc0093627a00441c1074ebe680c41aa98a6161e52733ad0c59888c643a33fe56884e4f935178b2557fbbdd105e92e0d993df6  windows6.1-kb2533623-x64.msu
 479a63e14586ab2f2228208116fc149ed8ee7b1e4ff360754f5bda4bf765c61af2e04b5ef123976623d04df4976b7886e0445647269da81436bd0a7b5671d361  windows6.1-kb2533623-x86.msu
 ac96786e5d35882e0c5b724794329c9125c2b86ae7847f17acfc49f0d294312c6afc1c3f248655de3f0ccb4ca426d7957d02ba702f4a15e9fcd7e2c314e72c19  zipp-3.15.0-py3-none-any.whl
 # win10
+0a2cd4cadf0395f0374974cd2bc2407e5cc65c111275acdffb6ecc5a2026eee9e1bb3da528b35c7f0ff4b64563a74857d5c2149051e281cc09ebd0d1968be9aa  en-us_windows_10_enterprise_ltsc_2021_x64_dvd_d289cf96.iso
+16cc0c58b5df6c7040893089f3eb29c074aed61d76dae6cd628d8a89a05f6223ac5d7f3f709a12417c147594a87a94cc808d1e04a6f1e407cc41f7c9f47790d1  virtio-win-0.1.248.iso
 d1420c8417fad7888766dd26b9706a87c63e8f33dceeb8e26d0056d5127b0b3ed9272e44b4b761132d4b3320327252eab1696520488ca64c25958896b41f547b  jinja2-3.1.4-py3-none-any.whl
-e21495f1d473d855103fb4a243095b498ec90eb68776b0f9b48e994990534f7286c0292448e129c507e5d70409f8a05cca58b98d59ce2a815993d0a873dfc480  MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
+8e6847bcde75a2736be0214731f834bc1b5854238d703351e68bc4e74d38404b212b8568565ae22c844189e466d3fbe6024836351cb69ffb1824131387644fef  MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl
 8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e  mutagen-1.47.0-py3-none-any.whl
-1dfe6f66bef5c9d62c9028a964196b902772ec9e19db215f3f41acb8d2d563586988d81b455fa6b895b434e9e1e9d57e4d271d1b1214483bdb3eadffcbba6a33  pillow-10.3.0-cp311-cp311-win_amd64.whl
-8760eab271e79256ae3bfb4af8ccc59010cb5d2eccdd74b325d1a533ae25eb127d51c2ec28ff90d449afed32dd7d6af62934fe9caaf1ae1f4d4831e948e912da  pyinstaller-6.5.0-py3-none-win_amd64.whl
-897a14d5ee5cbc6781a0f48beffc27807a4f789d58c4329d899233f615d168a5dcceddf7f8f2d5bb52212ddcf3eba4664590d9f1fdb25bb5201f44899e03b2f7  python-3.11.9-amd64.exe
-729dc52f1a02bc6274d012ce33f534102975a828cba11f6029600ea40e2d23aefeb07bf4ae19f9621d0565dd03eb2635bbb97d45fb692c1f756315e8c86c5255  upx-4.2.2-win64.zip
+0203ec2551c4836696cfab0b2c9fff603352f03fa36e7476e2e1ca7ec57a3a0c24bd791fcd92f342bf817f0887854d9f072e0271c643de4b313d8c9569ba8813  packaging-24.1-py3-none-any.whl
+2be320b4191f208cdd6af183c77ba2cf460ea52164ee45ac3ff17d6dfa57acd9deff016636c2dd42a21f4f6af977d5f72df7dacf599bebcf41757272354d14c1  pillow-10.4.0-cp312-cp312-win_amd64.whl
+776378f5414efd26ec8a1cb3228a7b5fdf6afca3fa335a0e9b071266d55d9d9e66ee157c25a468a05bfa70ccd33c48b101998523fc6ff6bcf5e82a1d81ed0af8  pyinstaller-6.9.0-py3-none-win_amd64.whl
+c0af77d2a57cb063ab038dc986ed3582bc5acc8c8bd91d726101935d6388f50854ddbca26bc846ed5d1022cdee4d96242938c66f0ddc4565c36b60d691064db8  pyinstaller_hooks_contrib-2024.7-py2.py3-none-any.whl
+2f9a11ffae6d9f1ed76bf816f28812fcba71f87080b0c92e52bfccb46243118c5803a7e25dd78003ca7d66501bfcdce8ff7c691c63c0038b0d409ca3842dcc89  python-3.12.4-amd64.exe

scripts/pyinstaller/deps.txt

@@ -34,6 +34,7 @@ https://support.microsoft.com/en-us/topic/microsoft-security-advisory-insecure-l
       see web.archive.org links below
 
 # direct links to version-frozen deps
+https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.248-1/virtio-win-0.1.248.iso
 https://www.python.org/ftp/python/3.7.9/python-3.7.9-amd64.exe
 https://www.python.org/ftp/python/3.7.9/python-3.7.9.exe
 https://web.archive.org/web/20200412130846if_/https://download.microsoft.com/download/2/D/7/2D78D0DD-2802-41F5-88D6-DC1D559F206D/Windows6.1-KB2533623-x86.msu

scripts/pyinstaller/notes.txt

@@ -1,14 +1,26 @@
+after performing all the initial setup in this file,
 run ./build.sh in git-bash to build + upload the exe
 
-
-## ============================================================
-## first-time setup on a stock win7x32sp1 and/or win10x64 vm:
-## 
-
 to obtain the files referenced below, see ./deps.txt
 
-download + install git (32bit OK on 64):
-http://192.168.123.1:3923/ro/pyi/Git-2.44.0-32-bit.exe
+and if you don't yet have a windows vm to build in, then the
+first step will be "creating the windows VM templates" below
+
+commands to start the VMs after initial setup:
+qemu-system-x86_64 -m 4096 -enable-kvm --machine q35 -cpu host -smp 4 -usb -device usb-tablet -net bridge,br=virhost0 -net nic,model=e1000e -drive file=win7-x32.qcow2,discard=unmap,detect-zeroes=unmap
+qemu-system-x86_64 -m 4096 -enable-kvm --machine q35 -cpu host -smp 4 -usb -device usb-tablet -net bridge,br=virhost0 -net nic,model=virtio -drive file=win10-e2021.qcow2,if=virtio,discard=unmap
+
+
+
+## ============================================================
+## first-time setup in a stock win7x32sp1 and/or win10x64 vm:
+## 
+
+grab & install from ftp2host:  Git-2.45.2-32-bit.exe
+
+...and do this on the host so you can grab these notes too:
+unix2dos <~/dev/copyparty/scripts/pyinstaller/notes.txt >~/dev/pyi/notes.txt
+
 
 ===[ copy-paste into git-bash ]================================
 uname -s | grep NT-10 && w10=1 || {
@@ -16,39 +28,40 @@ uname -s | grep NT-10 && w10=1 || {
 }
 fns=(
   altgraph-0.17.4-py2.py3-none-any.whl
-  packaging-24.0-py3-none-any.whl
   pefile-2023.2.7-py3-none-any.whl
-  pyinstaller_hooks_contrib-2024.3-py2.py3-none-any.whl
   pywin32_ctypes-0.2.2-py3-none-any.whl
-  setuptools-68.2.2-py3-none-any.whl
+  setuptools-70.3.0-py3-none-any.whl
+  upx-4.2.4-win32.zip
 )
 [ $w10 ] && fns+=(
-  pyinstaller-6.5.0-py3-none-win_amd64.whl
-  Jinja2-3.1.4-py3-none-any.whl
-  MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
+  jinja2-3.1.4-py3-none-any.whl
+  MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl
   mutagen-1.47.0-py3-none-any.whl
-  pillow-10.3.0-cp311-cp311-win_amd64.whl
-  python-3.11.9-amd64.exe
-  upx-4.2.2-win64.zip
+  packaging-24.1-py3-none-any.whl
+  pillow-10.4.0-cp312-cp312-win_amd64.whl
+  pyinstaller-6.9.0-py3-none-win_amd64.whl
+  pyinstaller_hooks_contrib-2024.7-py2.py3-none-any.whl
+  python-3.12.4-amd64.exe
 )
-[ $w7 ] && fns+=(
-  pyinstaller-5.13.2-py3-none-win32.whl
+[ $w7 ] && fns+=(  # u2c stuff
   certifi-2024.2.2-py3-none-any.whl
   charset_normalizer-3.3.2-cp37-cp37m-win32.whl
   idna-3.6-py3-none-any.whl
-  requests-2.31.0-py3-none-any.whl
-  urllib3-1.26.18-py2.py3-none-any.whl
-  upx-4.2.2-win32.zip
+  requests-2.29.0-py3-none-any.whl
+  urllib3-1.26.19-py2.py3-none-any.whl
 )
 [ $w7 ] && fns+=(
   future-1.0.0-py3-none-any.whl
   importlib_metadata-6.7.0-py3-none-any.whl
+  packaging-24.0-py3-none-any.whl
   pip-24.0-py3-none-any.whl
+  pyinstaller_hooks_contrib-2023.8-py2.py3-none-any.whl
   typing_extensions-4.7.1-py3-none-any.whl
   zipp-3.15.0-py3-none-any.whl
 )
 [ $w7x64 ] && fns+=(
   windows6.1-kb2533623-x64.msu
+  pyinstaller-5.13.2-py3-none-win64.whl
   python-3.7.9-amd64.exe
 )
 [ $w7x32 ] && fns+=(
@@ -57,20 +70,24 @@ fns=(
   python-3.7.9.exe
 )
 dl() { curl -fkLOC- "$1" && return 0; echo "$1"; return 1; }
-cd ~/Downloads &&
+cd ~/Downloads && rm -f Git-*.exe &&
 for fn in "${fns[@]}"; do
   dl "https://192.168.123.1:3923/ro/pyi/$fn" || {
     echo ERROR; ok=; break
   }
 done
 
-manually install:
-  windows6.1-kb2533623 + reboot
-  python-3.7.9
+
+WIN7-ONLY: manually install windows6.1-kb2533623 and reboot
+
+manually install python-3.99.99.exe and then delete it
+
+close and reopen git-bash so python is in PATH
+
 
 ===[ copy-paste into git-bash ]================================
 uname -s | grep NT-10 && w10=1 || w7=1
-[ $w7 ] && pyv=37 || pyv=311
+[ $w7 ] && pyv=37 || pyv=312
 appd=$(cygpath.exe "$APPDATA")
 cd ~/Downloads &&
 yes | unzip upx-*-win32.zip &&
@@ -78,7 +95,7 @@ mv upx-*/upx.exe . &&
 python -m ensurepip &&
 { [ $w10 ] || python -m pip install --user -U pip-*.whl; } &&
 python -m pip install --user -U packaging-*.whl &&
-{ [ $w7 ] || python -m pip install --user -U {setuptools,mutagen,Pillow,Jinja2,MarkupSafe}-*.whl; } &&
+{ [ $w7 ] || python -m pip install --user -U {setuptools,mutagen,pillow,jinja2,MarkupSafe}-*.whl; } &&
 { [ $w10 ] || python -m pip install --user -U {requests,urllib3,charset_normalizer,certifi,idna}-*.whl; } &&
 { [ $w10 ] || python -m pip install --user -U future-*.whl importlib_metadata-*.whl typing_extensions-*.whl zipp-*.whl; } &&
 python -m pip install --user -U pyinstaller-*.whl pefile-*.whl pywin32_ctypes-*.whl pyinstaller_hooks_contrib-*.whl altgraph-*.whl &&
@@ -90,9 +107,65 @@ rm -f build.sh &&
 curl -fkLO https://192.168.123.1:3923/cpp/scripts/pyinstaller/build.sh &&
 { [ $w10 ] || curl -fkLO https://192.168.123.1:3923/cpp/scripts/pyinstaller/up2k.sh; } &&
 echo ok
-# python -m pip install --user -U Pillow-9.2.0-cp37-cp37m-win32.whl
-# sed -ri 's/, bestopt, /]+bestopt+[/' $APPDATA/Python/Python37/site-packages/pyinstaller/building/utils.py
-# sed -ri 's/(^\s+bestopt = ).*/\1["--best","--lzma","--ultra-brute"]/' $APPDATA/Python/Python37/site-packages/pyinstaller/building/utils.py
+
+
+now is an excellent time to take another snapshot, but:
+* on win7: first do the 4g.nul thing again
+* on win10: first do a reboot so fstrim kicks in
+then shutdown and:  vmsnap the.qcow2 snap2
+
+
+
+
+
+## ============================================================
+## creating the windows VM templates
+##
+
+bash ~/dev/asm/doc/setup-virhost.sh  # github:9001/asm
+truncate -s 4G ~/dev/pyi/4g.nul  # win7 "trim"
+
+note: if you keep accidentally killing the vm with alt-f4 then remove "-device usb-tablet" in the qemu commands below
+
+# win7: don't bother with virtio stuff since win7 doesn't fstrim properly anyways (4g.nul takes care of that)
+rm -f win7-x32.qcow2
+qemu-img create -f qcow2 win7-x32.qcow2 64g
+qemu-system-x86_64 -m 4096 -enable-kvm --machine q35 -cpu host -smp 4 -usb -device usb-tablet -net bridge,br=virhost0 -net nic,model=e1000e -drive file=win7-x32.qcow2,discard=unmap,detect-zeroes=unmap \
+  -cdrom ~/iso/win7-X17-59183-english-32bit-professional.iso
+
+# win10: use virtio hdd and net (viostor+netkvm), but do NOT use qxl graphics (kills mouse cursor)
+rm -f win10-e2021.qcow2
+qemu-img create -f qcow2 win10-e2021.qcow2 64g
+qemu-system-x86_64 -m 4096 -enable-kvm --machine q35 -cpu host -smp 4 -usb -device usb-tablet -net bridge,br=virhost0 -net nic,model=virtio -drive file=win10-e2021.qcow2,if=virtio,discard=unmap \
+  -drive file=$HOME/iso/virtio-win-0.1.248.iso,media=cdrom -cdrom $HOME/iso/en-us_windows_10_enterprise_ltsc_2021_x64_dvd_d289cf96.iso
+
+tweak stuff to your preference, but also do these steps in order:
+* press ctrl-alt-g so you don't accidentally alt-f4 the vm
+* startmenu, type "sysdm.cpl" and hit Enter,
+  * system protection -> configure -> disable
+  * advanced > performance > advanced > virtual memory > no paging file
+* startmenu, type "cmd" and hit Ctrl-Shift-Enter, run command:  powercfg /h off
+* reboot
+* make screen resolution something comfy (1440x900 is always a winner)
+* cmd.exe window-width 176 (assuming 1440x900) and buffer-height 8191
+* fix explorer settings (show hidden files and file extensions)
+* WIN10-ONLY: startmenu, device manager, install netkvm driver for ethernet
+* create ftp2host.bat on desktop with following contents:
+    start explorer ftp://wark:k@192.168.123.1:3921/ro/pyi/
+* WIN7-ONLY: connect to ftp, download 4g.nul to desktop, then delete it (poor man's fstrim...)
+
+and finally take snapshots of the VMs by copypasting this stuff into your shell:
+vmsnap() { zstd --long=31 -vT0 -19 <$1 >$1.$2; };
+vmsnap win7-x32.qcow2 snap1
+vmsnap win10-e2021.qcow2 snap1
+
+note: vmsnap could have defragged the qcow2 as well, but
+  that makes it hard to do xdelta3 memes so it's not worth it --
+  but you can add this before "zstd" if you still want to:
+  qemu-img convert -f qcow2 -O qcow2 $1 a.qcow2 && mv a.qcow2 $1 &&
+
+
+
 
 
 ## ============================================================

scripts/pyinstaller/up2k.sh

@@ -37,6 +37,8 @@ grep -E '^from .ssl_ import' $APPDATA/python/python37/site-packages/urllib3/util
     echo golfed
 }
 
+sed -ri 's/(add_argument."-t[de]",.*help=")[^"]+/\1not applicable; HTTPS is disabled in this exe/; s/for some reason/in this exe for safety reasons/' u2c.py
+
 read a b _ < <(awk -F\" '/^S_VERSION =/{$0=$2;sub(/\./," ");print}' < u2c.py)
 sed -r 's/1,2,3,0/'$a,$b,0,0'/;s/1\.2\.3/'$a.$b.0/ <up2k.rc >up2k.rc2
 

scripts/strip_hints/a.py

@@ -47,6 +47,14 @@ def uh(top):
 
 
 def uh1(fp):
+    try:
+        uh2(fp)
+    except:
+        print("failed to process", fp)
+        raise
+
+
+def uh2(fp):
     pr(".")
     cs = strip_file_to_string(fp, no_ast=True, to_empty=True)
 

setup.py

@@ -141,7 +141,7 @@ args = {
         "audiotags": ["mutagen"],
         "ftpd": ["pyftpdlib"],
         "ftps": ["pyftpdlib", "pyopenssl"],
-        "tftpd": ["partftpy>=0.3.1"],
+        "tftpd": ["partftpy>=0.4.0"],
         "pwhash": ["argon2-cffi"],
     },
     "entry_points": {"console_scripts": ["copyparty = copyparty.__main__:main"]},

tests/test_dots.py

@@ -23,7 +23,7 @@ def hdr(query, uname):
     return (h % (query, uname)).encode("utf-8")
 
 
-class TestHttpCli(unittest.TestCase):
+class TestDots(unittest.TestCase):
     def setUp(self):
         self.td = tu.get_ramdisk()
 
@@ -31,7 +31,7 @@ class TestHttpCli(unittest.TestCase):
         os.chdir(tempfile.gettempdir())
         shutil.rmtree(self.td)
 
-    def test(self):
+    def test_dots(self):
         td = os.path.join(self.td, "vfs")
         os.mkdir(td)
         os.chdir(td)
@@ -118,6 +118,214 @@ class TestHttpCli(unittest.TestCase):
         url = "v?k=" + zj["dk"]
         self.assertEqual(self.tarsel(url, "u2", ["f1.txt", "a", ".b"]), "f1.txt")
 
+        shutil.rmtree("v")
+
+    def test_dk_fk(self):
+        # python3 -m unittest tests.test_dots.TestDots.test_dk_fk
+
+        td = os.path.join(self.td, "vfs")
+        os.mkdir(td)
+        os.chdir(td)
+
+        vcfg = []
+        for k in "dk dks dky fk fka dk,fk dks,fk".split():
+            vcfg += ["{0}:{0}:r.,u1:g,u2:c,{0}".format(k)]
+            zs = "%s/s1/s2" % (k,)
+            os.makedirs(zs)
+
+            with open("%s/f.t1" % (k,), "wb") as f:
+                f.write(b"f1")
+
+            with open("%s/s1/f.t2" % (k,), "wb") as f:
+                f.write(b"f2")
+
+            with open("%s/s1/s2/f.t3" % (k,), "wb") as f:
+                f.write(b"f3")
+
+        self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"])
+        self.asrv = AuthSrv(self.args, self.log)
+
+        dk = {}
+        for d in "dk dks dk,fk dks,fk".split():
+            zj = json.loads(self.curl("%s?ls" % (d,), "u1")[1])
+            dk[d] = zj["dk"]
+
+        ##
+        ## dk
+
+        # should not be able to access dk with wrong dirkey,
+        zs = self.curl("dk?ls&k=%s" % (dk["dks"]), "u2")[1]
+        self.assertEqual(zs, "\nJ2EOT")
+        # so use the right key
+        zs = self.curl("dk?ls&k=%s" % (dk["dk"]), "u2")[1]
+        zj = json.loads(zs)
+        self.assertEqual(len(zj["dirs"]), 0)
+        self.assertEqual(len(zj["files"]), 1)
+        self.assertEqual(zj["files"][0]["href"], "f.t1")
+
+        ##
+        ## dk thumbs
+
+        self.assertIn('">folder</text>', self.curl("dk?th=x", "u1")[1])
+        self.assertIn('">e403</text>', self.curl("dk?th=x", "u2")[1])
+
+        zs = "dk?th=x&k=%s" % (dk["dks"])
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        zs = "dk?th=x&k=%s" % (dk["dk"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # fk not enabled, so this should work
+        self.assertIn('">t1</text>', self.curl("dk/f.t1?th=x", "u2")[1])
+        self.assertIn('">t2</text>', self.curl("dk/s1/f.t2?th=x", "u2")[1])
+
+        ##
+        ## dks
+
+        # should not be able to access dks with wrong dirkey,
+        zs = self.curl("dks?ls&k=%s" % (dk["dk"]), "u2")[1]
+        self.assertEqual(zs, "\nJ2EOT")
+        # so use the right key
+        zs = self.curl("dks?ls&k=%s" % (dk["dks"]), "u2")[1]
+        zj = json.loads(zs)
+        self.assertEqual(len(zj["dirs"]), 1)
+        self.assertEqual(len(zj["files"]), 1)
+        self.assertEqual(zj["files"][0]["href"], "f.t1")
+        # dks should return correct dirkey of subfolders;
+        s1 = zj["dirs"][0]["href"]
+        self.assertEqual(s1.split("/")[0], "s1")
+        zs = self.curl("dks/%s&ls" % (s1), "u2")[1]
+        self.assertIn('"s2/?k=', zs)
+
+        ##
+        ## dks thumbs
+
+        self.assertIn('">folder</text>', self.curl("dks?th=x", "u1")[1])
+        self.assertIn('">e403</text>', self.curl("dks?th=x", "u2")[1])
+
+        zs = "dks?th=x&k=%s" % (dk["dk"])
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        zs = "dks?th=x&k=%s" % (dk["dks"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # fk not enabled, so this should work
+        self.assertIn('">t1</text>', self.curl("dks/f.t1?th=x", "u2")[1])
+        self.assertIn('">t2</text>', self.curl("dks/s1/f.t2?th=x", "u2")[1])
+
+        ##
+        ## dky
+
+        # doesn't care about keys
+        zs = self.curl("dky?ls&k=ok", "u2")[1]
+        self.assertEqual(zs, self.curl("dky?ls", "u2")[1])
+        zj = json.loads(zs)
+        self.assertEqual(len(zj["dirs"]), 0)
+        self.assertEqual(len(zj["files"]), 1)
+        self.assertEqual(zj["files"][0]["href"], "f.t1")
+
+        ##
+        ## dky thumbs
+
+        self.assertIn('">folder</text>', self.curl("dky?th=x", "u1")[1])
+        self.assertIn('">folder</text>', self.curl("dky?th=x", "u2")[1])
+
+        zs = "dky?th=x&k=%s" % (dk["dk"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # fk not enabled, so this should work
+        self.assertIn('">t1</text>', self.curl("dky/f.t1?th=x", "u2")[1])
+        self.assertIn('">t2</text>', self.curl("dky/s1/f.t2?th=x", "u2")[1])
+
+        ##
+        ## dk+fk
+
+        # should not be able to access dk with wrong dirkey,
+        zs = self.curl("dk,fk?ls&k=%s" % (dk["dk"]), "u2")[1]
+        self.assertEqual(zs, "\nJ2EOT")
+        # so use the right key
+        zs = self.curl("dk,fk?ls&k=%s" % (dk["dk,fk"]), "u2")[1]
+        zj = json.loads(zs)
+        self.assertEqual(len(zj["dirs"]), 0)
+        self.assertEqual(len(zj["files"]), 1)
+        self.assertEqual(zj["files"][0]["href"][:7], "f.t1?k=")
+
+        ##
+        ## dk+fk thumbs
+
+        self.assertIn('">folder</text>', self.curl("dk,fk?th=x", "u1")[1])
+        self.assertIn('">e403</text>', self.curl("dk,fk?th=x", "u2")[1])
+
+        zs = "dk,fk?th=x&k=%s" % (dk["dk"])
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        zs = "dk,fk?th=x&k=%s" % (dk["dk,fk"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # fk enabled, so this should fail
+        self.assertIn('">e404</text>', self.curl("dk,fk/f.t1?th=x", "u2")[1])
+        self.assertIn('">e404</text>', self.curl("dk,fk/s1/f.t2?th=x", "u2")[1])
+
+        # but dk should return correct filekeys, so try that
+        zs = "dk,fk/%s&th=x" % (zj["files"][0]["href"])
+        self.assertIn('">t1</text>', self.curl(zs, "u2")[1])
+
+        ##
+        ## dks+fk
+
+        # should not be able to access dk with wrong dirkey,
+        zs = self.curl("dks,fk?ls&k=%s" % (dk["dk"]), "u2")[1]
+        self.assertEqual(zs, "\nJ2EOT")
+        # so use the right key
+        zs = self.curl("dks,fk?ls&k=%s" % (dk["dks,fk"]), "u2")[1]
+        zj = json.loads(zs)
+        self.assertEqual(len(zj["dirs"]), 1)
+        self.assertEqual(len(zj["files"]), 1)
+        self.assertEqual(zj["dirs"][0]["href"][:6], "s1/?k=")
+        self.assertEqual(zj["files"][0]["href"][:7], "f.t1?k=")
+
+        ##
+        ## dks+fk thumbs
+
+        self.assertIn('">folder</text>', self.curl("dks,fk?th=x", "u1")[1])
+        self.assertIn('">e403</text>', self.curl("dks,fk?th=x", "u2")[1])
+
+        zs = "dks,fk?th=x&k=%s" % (dk["dk"])
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        zs = "dks,fk?th=x&k=%s" % (dk["dks,fk"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # subdir s1 without key
+        zs = "dks,fk/s1/?th=x"
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        # subdir s1 with bad key
+        zs = "dks,fk/s1/?th=x&k=no"
+        self.assertIn('">e403</text>', self.curl(zs, "u2")[1])
+
+        # subdir s1 with correct key
+        zs = "dks,fk/%s&th=x" % (zj["dirs"][0]["href"])
+        self.assertIn('">folder</text>', self.curl(zs, "u2")[1])
+
+        # fk enabled, so this should fail
+        self.assertIn('">e404</text>', self.curl("dks,fk/f.t1?th=x", "u2")[1])
+        self.assertIn('">e404</text>', self.curl("dks,fk/s1/f.t2?th=x", "u2")[1])
+
+        # but dk should return correct filekeys, so try that
+        zs = "dks,fk/%s&th=x" % (zj["files"][0]["href"])
+        self.assertIn('">t1</text>', self.curl(zs, "u2")[1])
+
+        # subdir
+        self.assertIn('">e403</text>', self.curl("dks,fk/s1/?th=x", "u2")[1])
+        self.assertEqual("\nJ2EOT", self.curl("dks,fk/s1/?ls", "u2")[1])
+        zs = "dks,fk/s1%s&th=x" % (zj["files"][0]["href"])
+        zs = self.curl("dks,fk?ls&k=%s" % (dk["dks,fk"]), "u2")[1]
+        zj = json.loads(zs)
+        url = "dks,fk/%s" % zj["dirs"][0]["href"]
+        self.assertIn('"files"', self.curl(url + "&ls", "u2")[1])
+        self.assertEqual("\nJ2EOT", self.curl(url + "x&ls", "u2")[1])
+
     def tardir(self, url, uname):
         top = url.split("?")[0]
         top = ("top" if not top else top.lstrip(".").split("/")[0]) + "/"

tests/util.py

@@ -43,6 +43,7 @@ if MACOS:
 
 from copyparty.__init__ import E
 from copyparty.__main__ import init_E
+from copyparty.ico import Ico
 from copyparty.u2idx import U2idx
 from copyparty.util import FHC, CachedDict, Garda, Unrecv
 
@@ -110,7 +111,7 @@ class Cfg(Namespace):
     def __init__(self, a=None, v=None, c=None, **ka0):
         ka = {}
 
-        ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_pipe no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw og og_no_head og_s_title q rand smb srch_dbg stats uqe vague_403 vc ver xdev xlink xvol"
+        ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid gsel hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_pipe no_poll no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw og og_no_head og_s_title q rand smb srch_dbg stats uqe vague_403 vc ver xdev xlink xvol"
         ka.update(**{k: False for k in ex.split()})
 
         ex = "dotpart dotsrch no_dhash no_fastboot no_rescan no_sendfile no_snap no_voldump re_dhash plain_ip"
@@ -125,16 +126,16 @@ class Cfg(Namespace):
         ex = "au_vol mtab_age reg_cap s_thead s_tbody th_convt"
         ka.update(**{k: 9 for k in ex.split()})
 
-        ex = "db_act df k304 loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
+        ex = "db_act k304 loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
         ka.update(**{k: 0 for k in ex.split()})
 
-        ex = "ah_alg bname doctitle exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i tcolor textfiles unlist vname R RS SR"
+        ex = "ah_alg bname doctitle df exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i tcolor textfiles unlist vname R RS SR"
         ka.update(**{k: "" for k in ex.split()})
 
         ex = "grp on403 on404 xad xar xau xban xbd xbr xbu xiu xm"
         ka.update(**{k: [] for k in ex.split()})
 
-        ex = "exp_lg exp_md th_coversd"
+        ex = "exp_lg exp_md"
         ka.update(**{k: {} for k in ex.split()})
 
         ka.update(ka0)
@@ -161,6 +162,10 @@ class Cfg(Namespace):
             s_wr_sz=256 * 1024,
             sort="href",
             srch_hits=99999,
+            th_covers=["folder.png"],
+            th_coversd=["folder.png"],
+            th_covers_set=set(["folder.png"]),
+            th_coversd_set=set(["folder.png"]),
             th_crop="y",
             th_size="320x256",
             th_x3="n",
@@ -245,7 +250,7 @@ class VHttpConn(object):
         self.bans = {}
         self.freshen_pwd = 0.0
         self.hsrv = VHttpSrv(args, asrv, log)
-        self.ico = None
+        self.ico = Ico(args)
         self.ipa_nm = None
         self.lf_url = None
         self.log_func = log