v1.12.0 (closes #64 )

update deps
apply audio equalizer to videos too
2024-04-06 20:11:49 +00:00 · 2024-04-06 19:51:38 +00:00 · 2024-04-06 18:44:08 +00:00 · 2024-04-06 18:43:45 +00:00 · 2024-04-06 00:27:06 +00:00 · 2024-04-06 00:25:28 +00:00
41 changed files with 953 additions and 382 deletions
--- a/README.md
+++ b/README.md
@@ -94,6 +94,7 @@ turn almost any device into a file server with resumable uploads/downloads using
    * [gotchas](#gotchas) - behavior that might be unexpected
    * [cors](#cors) - cross-site request config
    * [filekeys](#filekeys) - prevent filename bruteforcing
+        * [dirkeys](#dirkeys) - share specific folders in a volume
    * [password hashing](#password-hashing) - you can hash passwords
    * [https](#https) - both HTTP and HTTPS are accepted
 * [recovering from crashes](#recovering-from-crashes)
@@ -199,7 +200,7 @@ firewall-cmd --reload
 * browser
  * ☑ [navpane](#navpane) (directory tree sidebar)
  * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
-  * ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus transcoding)
+  * ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus/mp3 transcoding)
  * ☑ image gallery with webm player
  * ☑ textfile browser with syntax hilighting
  * ☑ [thumbnails](#thumbnails)
@@ -587,7 +588,7 @@ you can also zip a selection of files or folders by clicking them in the browser

 ![copyparty-zipsel-fs8](https://user-images.githubusercontent.com/241032/129635374-e5136e01-470a-49b1-a762-848e8a4c9cdc.png)

-cool trick: download a folder by appending url-params `?tar&opus` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus before they're added to the archive
+cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
 * super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
 * and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images
  * can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
@@ -778,9 +779,9 @@ open the `[🎺]` media-player-settings tab to configure it,
  * `[loop]` keeps looping the folder
  * `[next]` plays into the next folder
 * "transcode":
-  * `[flac]` converts `flac` and `wav` files into opus
-  * `[aac]` converts `aac` and `m4a` files into opus
-  * `[oth]` converts all other known formats into opus
+  * `[flac]` converts `flac` and `wav` files into opus (if supported by browser) or mp3
+  * `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
+  * `[oth]` converts all other known formats into opus (if supported by browser) or mp3
    * `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
 * "tint" reduces the contrast of the playback bar

@@ -1291,6 +1292,8 @@ you may experience poor upload performance this way, but that can sometimes be f

 someone has also tested geesefs in combination with [gocryptfs](https://nuetzlich.net/gocryptfs/) with surprisingly good results, getting 60 MiB/s upload speeds on a gbit line, but JuiceFS won with 80 MiB/s using its built-in encryption

+you may improve performance by specifying larger values for `--iobuf` / `--s-rd-sz` / `--s-wr-sz`
+

 ## hiding from google

@@ -1740,6 +1743,7 @@ below are some tweaks roughly ordered by usefulness:
 * `--hist` pointing to a fast location (ssd) will make directory listings and searches faster when `-e2d` or `-e2t` is set
  * and also makes thumbnails load faster, regardless of e2d/e2t
 * `--no-hash .` when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable
+* if your volumes are on a network-disk such as NFS / SMB / s3, specifying larger values for `--iobuf` and/or `--s-rd-sz` and/or `--s-wr-sz` may help; try setting all of them to `524288` or `1048576` or `4194304`
 * `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
 * `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
  * lots of connections (many users or heavy clients)
@@ -1836,12 +1840,29 @@ cors can be configured with `--acao` and `--acam`, or the protections entirely d

 prevent filename bruteforcing

-volflag `c,fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
+volflag `fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404

 by default, filekeys are generated based on salt (`--fk-salt`) + filesystem-path + file-size + inode (if not windows); add volflag `fka` to generate slightly weaker filekeys which will not be invalidated if the file is edited (only salt + path)

 permissions `wG` (write + upget) lets users upload files and receive their own filekeys, still without being able to see other uploads

+### dirkeys
+
+share specific folders in a volume  without giving away full read-access to the rest -- the visitor only needs the `g` (get) permission to view the link
+
+volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, granting read-access to that folder; by default only that folder itself, no subfolders
+
+volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
+
+* `dk` + `dky` gives the same behavior as if all users with `g` access have full read-access, but subfolders are hidden files (their names start with a dot), so `dky` is an alternative to renaming all the folders for that purpose, maybe just for some users
+
+volflag `dks` lets people enter subfolders as well, and also enables download-as-zip/tar
+
+dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and have a few limitations:
+* the key does not change if the contents of the folder is modified
+  * if you need a new dirkey, either change the salt or rename the folder
+* linking to a textfile (so it opens in the textfile viewer) is not possible if recipient doesn't have read-access
+

 ## password hashing

--- a/contrib/package/arch/PKGBUILD
+++ b/contrib/package/arch/PKGBUILD
@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.11.0"
+pkgver="1.11.2"
 pkgrel=1
 pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("95f39a239dc38844fc27c5a1473635d07d8907bc98679dc79eb1de475e36fe42")
+sha256sums=("0b37641746d698681691ea9e7070096404afc64a42d3d4e96cc4e036074fded9")

 build() {
    cd "${srcdir}/${pkgname}-${pkgver}"
--- a/contrib/package/nix/copyparty/pin.json
+++ b/contrib/package/nix/copyparty/pin.json
@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.11.0/copyparty-sfx.py",
-    "version": "1.11.0",
-    "hash": "sha256-MkNp+tI/Pl5QB4FMdJNOePbSUPO1MHWJLLC7gNh9K+c="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.11.2/copyparty-sfx.py",
+    "version": "1.11.2",
+    "hash": "sha256-3nIHLM4xJ9RQH3ExSGvBckHuS40IdzyREAtMfpJmfug="
 }
--- a/copyparty/init.py
+++ b/copyparty/init.py
@@ -56,7 +56,6 @@ class EnvParams(object):
        self.t0 = time.time()
        self.mod = ""
        self.cfg = ""
-        self.ox = getattr(sys, "oxidized", None)


 E = EnvParams()
--- a/copyparty/main.py
+++ b/copyparty/main.py
@@ -157,7 +157,8 @@ def warn(msg: str) -> None:


 def init_E(EE: EnvParams) -> None:
-    # __init__ runs 18 times when oxidized; do expensive stuff here
+    # some cpython alternatives (such as pyoxidizer) can
+    # __init__ several times, so do expensive stuff here

    E = EE  # pylint: disable=redefined-outer-name

@@ -190,34 +191,9 @@ def init_E(EE: EnvParams) -> None:

        raise Exception("could not find a writable path for config")

-    def _unpack() -> str:
-        import atexit
-        import tarfile
-        import tempfile
-        from importlib.resources import open_binary
-
-        td = tempfile.TemporaryDirectory(prefix="")
-        atexit.register(td.cleanup)
-        tdn = td.name
-
-        with open_binary("copyparty", "z.tar") as tgz:
-            with tarfile.open(fileobj=tgz) as tf:
-                try:
-                    tf.extractall(tdn, filter="tar")
-                except TypeError:
-                    tf.extractall(tdn)  # nosec (archive is safe)
-
-        return tdn
-
-    try:
-        E.mod = os.path.dirname(os.path.realpath(__file__))
-        if E.mod.endswith("__init__"):
-            E.mod = os.path.dirname(E.mod)
-    except:
-        if not E.ox:
-            raise
-
-        E.mod = _unpack()
+    E.mod = os.path.dirname(os.path.realpath(__file__))
+    if E.mod.endswith("__init__"):
+        E.mod = os.path.dirname(E.mod)

    if sys.platform == "win32":
        bdir = os.environ.get("APPDATA") or os.environ.get("TEMP") or "."
@@ -274,6 +250,19 @@ def get_fk_salt() -> str:
    return ret.decode("utf-8")


+def get_dk_salt() -> str:
+    fp = os.path.join(E.cfg, "dk-salt.txt")
+    try:
+        with open(fp, "rb") as f:
+            ret = f.read().strip()
+    except:
+        ret = base64.b64encode(os.urandom(30))
+        with open(fp, "wb") as f:
+            f.write(ret + b"\n")
+
+    return ret.decode("utf-8")
+
+
 def get_ah_salt() -> str:
    fp = os.path.join(E.cfg, "ah-salt.txt")
    try:
@@ -869,6 +858,7 @@ def add_fs(ap):
    ap2 = ap.add_argument_group("filesystem options")
    rm_re_def = "5/0.1" if ANYWIN else "0/0"
    ap2.add_argument("--rm-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be deleted because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=rm_retry)")
+    ap2.add_argument("--iobuf", metavar="BYTES", type=int, default=256*1024, help="file I/O buffer-size; if your volumes are on a network drive, try increasing to \033[32m524288\033[0m or even \033[32m4194304\033[0m (and let me know if that improves your performance)")


 def add_upload(ap):
@@ -916,6 +906,7 @@ def add_network(ap):
        ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
    ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
    ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
+    ap2.add_argument("--s-rd-sz", metavar="B", type=int, default=256*1024, help="socket read size in bytes (indirectly affects filesystem writes; recommendation: keep equal-to or lower-than \033[33m--iobuf\033[0m)")
    ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
    ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0, help="debug: socket write delay in seconds")
    ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0, help="debug: response delay in seconds")
@@ -1129,13 +1120,14 @@ def add_safety(ap):
    ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like \033[33m--acao\033[0m's description)")


-def add_salt(ap, fk_salt, ah_salt):
+def add_salt(ap, fk_salt, dk_salt, ah_salt):
    ap2 = ap.add_argument_group('salting options')
    ap2.add_argument("--ah-alg", metavar="ALG", type=u, default="none", help="account-pw hashing algorithm; one of these, best to worst: \033[32margon2 scrypt sha2 none\033[0m (each optionally followed by alg-specific comma-sep. config)")
    ap2.add_argument("--ah-salt", metavar="SALT", type=u, default=ah_salt, help="account-pw salt; ignored if \033[33m--ah-alg\033[0m is none (default)")
    ap2.add_argument("--ah-gen", metavar="PW", type=u, default="", help="generate hashed password for \033[33mPW\033[0m, or read passwords from STDIN if \033[33mPW\033[0m is [\033[32m-\033[0m]")
    ap2.add_argument("--ah-cli", action="store_true", help="launch an interactive shell which hashes passwords without ever storing or displaying the original passwords")
    ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files")
+    ap2.add_argument("--dk-salt", metavar="SALT", type=u, default=dk_salt, help="per-directory accesskey salt; used to generate unpredictable URLs to share folders with users who only have the 'get' permission")
    ap2.add_argument("--warksalt", metavar="SALT", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")


@@ -1201,6 +1193,8 @@ def add_thumbnail(ap):

 def add_transcoding(ap):
    ap2 = ap.add_argument_group('transcoding options')
+    ap2.add_argument("--q-opus", metavar="KBPS", type=int, default=128, help="target bitrate for transcoding to opus; set 0 to disable")
+    ap2.add_argument("--q-mp3", metavar="QUALITY", type=u, default="q2", help="target quality for transcoding to mp3, for example [\033[32m192k\033[0m] (CBR) or [\033[32mq0\033[0m] (CQ/CRF, q0=maxquality, q9=smallest); set 0 to disable")
    ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
    ap2.add_argument("--no-bacode", action="store_true", help="disable batch audio transcoding by folder download (zip/tar)")
    ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete cached transcode output after \033[33mSEC\033[0m seconds")
@@ -1317,6 +1311,7 @@ def run_argparse(
    cert_path = os.path.join(E.cfg, "cert.pem")

    fk_salt = get_fk_salt()
+    dk_salt = get_dk_salt()
    ah_salt = get_ah_salt()

    # alpine peaks at 5 threads for some reason,
@@ -1348,7 +1343,7 @@ def run_argparse(
    add_tftp(ap)
    add_smb(ap)
    add_safety(ap)
-    add_salt(ap, fk_salt, ah_salt)
+    add_salt(ap, fk_salt, dk_salt, ah_salt)
    add_optouts(ap)
    add_shutdown(ap)
    add_yolo(ap)
--- a/copyparty/version.py
+++ b/copyparty/version.py
@@ -1,8 +1,8 @@
 # coding: utf-8

-VERSION = (1, 11, 1)
-CODENAME = "You Can (Not) Proceed"
-BUILD_DT = (2024, 3, 18)
+VERSION = (1, 12, 0)
+CODENAME = "locksmith"
+BUILD_DT = (2024, 4, 6)

 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
--- a/copyparty/authsrv.py
+++ b/copyparty/authsrv.py
@@ -555,7 +555,12 @@ class VFS(object):
            # no vfs nodes in the list of real inodes
            real = [x for x in real if x[0] not in self.nodes]

+            dbv = self.dbv or self
            for name, vn2 in sorted(self.nodes.items()):
+                if vn2.dbv == dbv and self.flags.get("dk"):
+                    virt_vis[name] = vn2
+                    continue
+
                ok = False
                zx = vn2.axs
                axs = [zx.uread, zx.uwrite, zx.umove, zx.udel, zx.uget]
@@ -1224,7 +1229,9 @@ class AuthSrv(object):
            if un.startswith("@"):
                grp = un[1:]
                uns = [x[0] for x in un_gns.items() if grp in x[1]]
-                if not uns and grp != "${g}" and not self.args.idp_h_grp:
+                if grp == "${g}":
+                    unames.append(un)
+                elif not uns and not self.args.idp_h_grp:
                    t = "group [%s] must be defined with --grp argument (or in a [groups] config section)"
                    raise CfgEx(t % (grp,))

@@ -1234,31 +1241,28 @@ class AuthSrv(object):

        # unames may still contain ${u} and ${g} so now expand those;
        un_gn = [(un, gn) for un, gns in un_gns.items() for gn in gns]
-        if "*" not in un_gns:
-            # need ("*","") to match "*" in unames
-            un_gn.append(("*", ""))

-        for _, dst, vu, vg in vols:
-            unames2 = set()
-            for un, gn in un_gn:
-                # if vu/vg (volume user/group) is non-null,
-                # then each non-null value corresponds to
-                # ${u}/${g}; consider this a filter to
-                # apply to unames, as well as un_gn
-                if (vu and vu != un) or (vg and vg != gn):
-                    continue
+        for src, dst, vu, vg in vols:
+            unames2 = set(unames)

-                for uname in unames + ([un] if vu or vg else []):
-                    if uname == "${u}":
-                        uname = vu or un
-                    elif uname in ("${g}", "@${g}"):
-                        uname = vg or gn
+            if "${u}" in unames:
+                if not vu:
+                    t = "cannot use ${u} in accs of volume [%s] because the volume url does not contain ${u}"
+                    raise CfgEx(t % (src,))
+                unames2.add(vu)

-                    if vu and vu != uname:
-                        continue
+            if "@${g}" in unames:
+                if not vg:
+                    t = "cannot use @${g} in accs of volume [%s] because the volume url does not contain @${g}"
+                    raise CfgEx(t % (src,))
+                unames2.update([un for un, gn in un_gn if gn == vg])

-                    if uname:
-                        unames2.add(uname)
+            if "${g}" in unames:
+                t = 'the accs of volume [%s] contains "${g}" but the only supported way of specifying that is "@${g}"'
+                raise CfgEx(t % (src,))
+
+            unames2.discard("${u}")
+            unames2.discard("@${g}")

            self._read_vol_str(lvl, list(unames2), axs[dst])

@@ -1682,6 +1686,20 @@ class AuthSrv(object):
                vol.flags["fk"] = int(fk) if fk is not True else 8
                have_fk = True

+            dk = vol.flags.get("dk")
+            dks = vol.flags.get("dks")
+            dky = vol.flags.get("dky")
+            if dks is not None and dky is not None:
+                t = "WARNING: volume /%s has both dks and dky enabled; this is too yolo and not permitted"
+                raise Exception(t % (vol.vpath,))
+
+            if dks and not dk:
+                dk = dks
+            if dky and not dk:
+                dk = dky
+            if dk:
+                vol.flags["dk"] = int(dk) if dk is not True else 8
+
        if have_fk and re.match(r"^[0-9\.]+$", self.args.fk_salt):
            self.log("filekey salt: {}".format(self.args.fk_salt))

--- a/copyparty/ftpd.py
+++ b/copyparty/ftpd.py
@@ -218,7 +218,7 @@ class FtpFs(AbstractedFS):
                raise FSE("Cannot open existing file for writing")

        self.validpath(ap)
-        return open(fsenc(ap), mode)
+        return open(fsenc(ap), mode, self.args.iobuf)

    def chdir(self, path: str) -> None:
        nwd = join(self.cwd, path)
--- a/copyparty/httpcli.py
+++ b/copyparty/httpcli.py
@@ -36,6 +36,7 @@ from .bos import bos
 from .star import StreamTar
 from .sutil import StreamArc, gfilter
 from .szip import StreamZip
+from .util import unquote  # type: ignore
 from .util import (
    APPLESAN_RE,
    BITNESS,
@@ -84,7 +85,6 @@ from .util import (
    sendfile_py,
    undot,
    unescape_cookie,
-    unquote,  # type: ignore
    unquotep,
    vjoin,
    vol_san,
@@ -174,7 +174,6 @@ class HttpCli(object):
        self.parser: Optional[MultipartParser] = None
        # end placeholders

-        self.bufsz = 1024 * 32
        self.html_head = ""

    def log(self, msg: str, c: Union[int, str] = 0) -> None:
@@ -1611,15 +1610,16 @@ class HttpCli(object):
        return enc or "utf-8"

    def get_body_reader(self) -> tuple[Generator[bytes, None, None], int]:
+        bufsz = self.args.s_rd_sz
        if "chunked" in self.headers.get("transfer-encoding", "").lower():
-            return read_socket_chunked(self.sr), -1
+            return read_socket_chunked(self.sr, bufsz), -1

        remains = int(self.headers.get("content-length", -1))
        if remains == -1:
            self.keepalive = False
-            return read_socket_unbounded(self.sr), remains
+            return read_socket_unbounded(self.sr, bufsz), remains
        else:
-            return read_socket(self.sr, remains), remains
+            return read_socket(self.sr, bufsz, remains), remains

    def dump_to_file(self, is_put: bool) -> tuple[int, str, str, int, str, str]:
        # post_sz, sha_hex, sha_b64, remains, path, url
@@ -1641,7 +1641,7 @@ class HttpCli(object):
        bos.makedirs(fdir)

        open_ka: dict[str, Any] = {"fun": open}
-        open_a = ["wb", 512 * 1024]
+        open_a = ["wb", self.args.iobuf]

        # user-request || config-force
        if ("gz" in vfs.flags or "xz" in vfs.flags) and (
@@ -1900,7 +1900,7 @@ class HttpCli(object):
        f.seek(ofs)
        with open(fp, "wb") as fo:
            while nrem:
-                buf = f.read(min(nrem, 512 * 1024))
+                buf = f.read(min(nrem, self.args.iobuf))
                if not buf:
                    break

@@ -1922,7 +1922,7 @@ class HttpCli(object):
        return "%s %s n%s" % (spd1, spd2, self.conn.nreq)

    def handle_post_multipart(self) -> bool:
-        self.parser = MultipartParser(self.log, self.sr, self.headers)
+        self.parser = MultipartParser(self.log, self.args, self.sr, self.headers)
        self.parser.parse()

        file0: list[tuple[str, Optional[str], Generator[bytes, None, None]]] = []
@@ -1966,7 +1966,12 @@ class HttpCli(object):

        v = self.uparam[k]

-        vn, rem = self.asrv.vfs.get(self.vpath, self.uname, True, False)
+        if self._use_dirkey():
+            vn = self.vn
+            rem = self.rem
+        else:
+            vn, rem = self.asrv.vfs.get(self.vpath, self.uname, True, False)
+
        zs = self.parser.require("files", 1024 * 1024)
        if not zs:
            raise Pebkac(422, "need files list")
@@ -2151,7 +2156,7 @@ class HttpCli(object):

            self.log("writing {} #{} @{} len {}".format(path, chash, cstart, remains))

-            reader = read_socket(self.sr, remains)
+            reader = read_socket(self.sr, self.args.s_rd_sz, remains)

            f = None
            fpool = not self.args.no_fpool and sprs
@@ -2162,7 +2167,7 @@ class HttpCli(object):
                    except:
                        pass

-            f = f or open(fsenc(path), "rb+", 512 * 1024)
+            f = f or open(fsenc(path), "rb+", self.args.iobuf)

            try:
                f.seek(cstart[0])
@@ -2185,7 +2190,8 @@ class HttpCli(object):
                    )
                    ofs = 0
                    while ofs < chunksize:
-                        bufsz = min(chunksize - ofs, 4 * 1024 * 1024)
+                        bufsz = max(4 * 1024 * 1024, self.args.iobuf)
+                        bufsz = min(chunksize - ofs, bufsz)
                        f.seek(cstart[0] + ofs)
                        buf = f.read(bufsz)
                        for wofs in cstart[1:]:
@@ -2438,6 +2444,18 @@ class HttpCli(object):
                    suffix = "-{:.6f}-{}".format(time.time(), dip)
                    open_args = {"fdir": fdir, "suffix": suffix}

+                    if "replace" in self.uparam:
+                        abspath = os.path.join(fdir, fname)
+                        if not self.can_delete:
+                            self.log("user not allowed to overwrite with ?replace")
+                        elif bos.path.exists(abspath):
+                            try:
+                                wunlink(self.log, abspath, vfs.flags)
+                                t = "overwriting file with new upload: %s"
+                            except:
+                                t = "toctou while deleting for ?replace: %s"
+                            self.log(t % (abspath,))
+
                    # reserve destination filename
                    with ren_open(fname, "wb", fdir=fdir, suffix=suffix) as zfw:
                        fname = zfw["orz"][1]
@@ -2482,7 +2500,7 @@ class HttpCli(object):
                        v2 = lim.dfv - lim.dfl
                        max_sz = min(v1, v2) if v1 and v2 else v1 or v2

-                    with ren_open(tnam, "wb", 512 * 1024, **open_args) as zfw:
+                    with ren_open(tnam, "wb", self.args.iobuf, **open_args) as zfw:
                        f, tnam = zfw["orz"]
                        tabspath = os.path.join(fdir, tnam)
                        self.log("writing to {}".format(tabspath))
@@ -2778,7 +2796,7 @@ class HttpCli(object):
        if bos.path.exists(fp):
            wunlink(self.log, fp, vfs.flags)

-        with open(fsenc(fp), "wb", 512 * 1024) as f:
+        with open(fsenc(fp), "wb", self.args.iobuf) as f:
            sz, sha512, _ = hashcopy(p_data, f, self.args.s_wr_slp)

        if lim:
@@ -2857,6 +2875,30 @@ class HttpCli(object):

        return file_lastmod, True

+    def _use_dirkey(self, ap: str = "") -> bool:
+        if self.can_read or not self.can_get:
+            return False
+
+        if self.vn.flags.get("dky"):
+            return True
+
+        req = self.uparam.get("k") or ""
+        if not req:
+            return False
+
+        dk_len = self.vn.flags.get("dk")
+        if not dk_len:
+            return False
+
+        ap = ap or self.vn.canonical(self.rem)
+        zs = self.gen_fk(2, self.args.dk_salt, ap, 0, 0)[:dk_len]
+        if req == zs:
+            return True
+
+        t = "wrong dirkey, want %s, got %s\n  vp: %s\n  ap: %s"
+        self.log(t % (zs, req, self.req, ap), 6)
+        return False
+
    def _expand(self, txt: str, phs: list[str]) -> str:
        for ph in phs:
            if ph.startswith("hdr."):
@@ -3010,8 +3052,7 @@ class HttpCli(object):
            upper = gzip_orig_sz(fs_path)
        else:
            open_func = open
-            # 512 kB is optimal for huge files, use 64k
-            open_args = [fsenc(fs_path), "rb", 64 * 1024]
+            open_args = [fsenc(fs_path), "rb", self.args.iobuf]
            use_sendfile = (
                # fmt: off
                not self.tls
@@ -3136,7 +3177,7 @@ class HttpCli(object):
        # for f in fgen: print(repr({k: f[k] for k in ["vp", "ap"]}))
        cfmt = ""
        if self.thumbcli and not self.args.no_bacode:
-            for zs in ("opus", "w", "j"):
+            for zs in ("opus", "mp3", "w", "j"):
                if zs in self.ouparam or uarg == zs:
                    cfmt = zs

@@ -3146,6 +3187,7 @@ class HttpCli(object):

        bgen = packer(
            self.log,
+            self.args,
            fgen,
            utf8="utf" in uarg,
            pre_crc="crc" in uarg,
@@ -3223,7 +3265,7 @@ class HttpCli(object):
        sz_md = 0
        lead = b""
        fullfile = b""
-        for buf in yieldfile(fs_path):
+        for buf in yieldfile(fs_path, self.args.iobuf):
            if sz_md < max_sz:
                fullfile += buf
            else:
@@ -3296,7 +3338,7 @@ class HttpCli(object):
            if fullfile:
                self.s.sendall(fullfile)
            else:
-                for buf in yieldfile(fs_path):
+                for buf in yieldfile(fs_path, self.args.iobuf):
                    self.s.sendall(html_bescape(buf))

            self.s.sendall(html[1])
@@ -3544,7 +3586,7 @@ class HttpCli(object):

            dst = dst[len(top) + 1 :]

-        ret = self.gen_tree(top, dst)
+        ret = self.gen_tree(top, dst, self.uparam.get("k", ""))
        if self.is_vproxied:
            parents = self.args.R.split("/")
            for parent in reversed(parents):
@@ -3554,18 +3596,25 @@ class HttpCli(object):
        self.reply(zs.encode("utf-8"), mime="application/json")
        return True

-    def gen_tree(self, top: str, target: str) -> dict[str, Any]:
+    def gen_tree(self, top: str, target: str, dk: str) -> dict[str, Any]:
        ret: dict[str, Any] = {}
        excl = None
        if target:
            excl, target = (target.split("/", 1) + [""])[:2]
-            sub = self.gen_tree("/".join([top, excl]).strip("/"), target)
+            sub = self.gen_tree("/".join([top, excl]).strip("/"), target, dk)
            ret["k" + quotep(excl)] = sub

        vfs = self.asrv.vfs
+        dk_sz = False
+        if dk:
+            vn, rem = vfs.get(top, self.uname, False, False)
+            if vn.flags.get("dks") and self._use_dirkey(vn.canonical(rem)):
+                dk_sz = vn.flags.get("dk")
+
        dots = False
+        fsroot = ""
        try:
-            vn, rem = vfs.get(top, self.uname, True, False)
+            vn, rem = vfs.get(top, self.uname, not dk_sz, False)
            fsroot, vfs_ls, vfs_virt = vn.ls(
                rem,
                self.uname,
@@ -3573,7 +3622,9 @@ class HttpCli(object):
                [[True, False], [False, True]],
            )
            dots = self.uname in vn.axs.udot
+            dk_sz = vn.flags.get("dk")
        except:
+            dk_sz = None
            vfs_ls = []
            vfs_virt = {}
            for v in self.rvol:
@@ -3588,6 +3639,14 @@ class HttpCli(object):

        dirs = [quotep(x) for x in dirs if x != excl]

+        if dk_sz and fsroot:
+            kdirs = []
+            for dn in dirs:
+                ap = os.path.join(fsroot, dn)
+                zs = self.gen_fk(2, self.args.dk_salt, ap, 0, 0)[:dk_sz]
+                kdirs.append(dn + "?k=" + zs)
+            dirs = kdirs
+
        for x in vfs_virt:
            if x != excl:
                try:
@@ -3852,6 +3911,7 @@ class HttpCli(object):
            self.out_headers["X-Robots-Tag"] = "noindex, nofollow"

        is_dir = stat.S_ISDIR(st.st_mode)
+        is_dk = False
        fk_pass = False
        icur = None
        if is_dir and (e2t or e2d):
@@ -3860,7 +3920,7 @@ class HttpCli(object):
                icur = idx.get_cur(dbv.realpath)

        th_fmt = self.uparam.get("th")
-        if self.can_read:
+        if self.can_read or (self.can_get and vn.flags.get("dk")):
            if th_fmt is not None:
                nothumb = "dthumb" in dbv.flags
                if is_dir:
@@ -3966,8 +4026,11 @@ class HttpCli(object):

            return self.tx_file(abspath)

-        elif is_dir and not self.can_read and not self.can_write:
-            return self.tx_404(True)
+        elif is_dir and not self.can_read:
+            if self._use_dirkey(abspath):
+                is_dk = True
+            elif not self.can_write:
+                return self.tx_404(True)

        srv_info = []

@@ -3989,7 +4052,7 @@ class HttpCli(object):
        srv_infot = "</span> // <span>".join(srv_info)

        perms = []
-        if self.can_read:
+        if self.can_read or is_dk:
            perms.append("read")
        if self.can_write:
            perms.append("write")
@@ -4117,7 +4180,7 @@ class HttpCli(object):
        if not self.conn.hsrv.prism:
            j2a["no_prism"] = True

-        if not self.can_read:
+        if not self.can_read and not is_dk:
            if is_ls:
                return self.tx_ls(ls_ret)

@@ -4170,8 +4233,15 @@ class HttpCli(object):
        ):
            ls_names = exclude_dotfiles(ls_names)

+        add_dk = vf.get("dk")
        add_fk = vf.get("fk")
        fk_alg = 2 if "fka" in vf else 1
+        if add_dk:
+            if vf.get("dky"):
+                add_dk = False
+            else:
+                zs = self.gen_fk(2, self.args.dk_salt, abspath, 0, 0)[:add_dk]
+                ls_ret["dk"] = cgv["dk"] = zs

        dirs = []
        files = []
@@ -4199,6 +4269,12 @@ class HttpCli(object):
                href += "/"
                if self.args.no_zip:
                    margin = "DIR"
+                elif add_dk:
+                    zs = absreal(fspath)
+                    margin = '<a href="%s?k=%s&zip" rel="nofollow">zip</a>' % (
+                        quotep(href),
+                        self.gen_fk(2, self.args.dk_salt, zs, 0, 0)[:add_dk],
+                    )
                else:
                    margin = '<a href="%s?zip" rel="nofollow">zip</a>' % (quotep(href),)
            elif fn in hist:
@@ -4239,6 +4315,11 @@ class HttpCli(object):
                        0 if ANYWIN else inf.st_ino,
                    )[:add_fk],
                )
+            elif add_dk and is_dir:
+                href = "%s?k=%s" % (
+                    quotep(href),
+                    self.gen_fk(2, self.args.dk_salt, fspath, 0, 0)[:add_dk],
+                )
            else:
                href = quotep(href)

@@ -4257,6 +4338,9 @@ class HttpCli(object):
                files.append(item)
                item["rd"] = rem

+        if is_dk and not vf.get("dks"):
+            dirs = []
+
        if (
            self.cookies.get("idxh") == "y"
            and "ls" not in self.uparam
--- a/copyparty/mtag.py
+++ b/copyparty/mtag.py
@@ -551,8 +551,7 @@ class MTag(object):
            pypath = str(os.pathsep.join(zsl))
            env["PYTHONPATH"] = pypath
        except:
-            if not E.ox and not EXE:
-                raise
+            raise  # might be expected outside cpython

        ret: dict[str, Any] = {}
        for tagname, parser in sorted(parsers.items(), key=lambda x: (x[1].pri, x[0])):
--- a/copyparty/star.py
+++ b/copyparty/star.py
@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals

+import argparse
 import re
 import stat
 import tarfile
@@ -44,11 +45,12 @@ class StreamTar(StreamArc):
    def __init__(
        self,
        log: "NamedLogger",
+        args: argparse.Namespace,
        fgen: Generator[dict[str, Any], None, None],
        cmp: str = "",
        **kwargs: Any
    ):
-        super(StreamTar, self).__init__(log, fgen)
+        super(StreamTar, self).__init__(log, args, fgen)

        self.ci = 0
        self.co = 0
@@ -126,7 +128,7 @@ class StreamTar(StreamArc):
        inf.gid = 0

        self.ci += inf.size
-        with open(fsenc(src), "rb", 512 * 1024) as fo:
+        with open(fsenc(src), "rb", self.args.iobuf) as fo:
            self.tar.addfile(inf, fo)

    def _gen(self) -> None:
--- a/copyparty/sutil.py
+++ b/copyparty/sutil.py
@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals

+import argparse
 import os
 import tempfile
 from datetime import datetime
@@ -20,10 +21,12 @@ class StreamArc(object):
    def __init__(
        self,
        log: "NamedLogger",
+        args: argparse.Namespace,
        fgen: Generator[dict[str, Any], None, None],
        **kwargs: Any
    ):
        self.log = log
+        self.args = args
        self.fgen = fgen
        self.stopped = False

@@ -78,7 +81,9 @@ def enthumb(
 ) -> dict[str, Any]:
    rem = f["vp"]
    ext = rem.rsplit(".", 1)[-1].lower()
-    if fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|"):
+    if (fmt == "mp3" and ext == "mp3") or (
+        fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|")
+    ):
        raise Exception()

    vp = vjoin(vtop, rem.split("/", 1)[1])
--- a/copyparty/svchub.py
+++ b/copyparty/svchub.py
@@ -28,7 +28,7 @@ if True:  # pylint: disable=using-constant-test
    import typing
    from typing import Any, Optional, Union

-from .__init__ import ANYWIN, E, EXE, MACOS, TYPE_CHECKING, EnvParams, unicode
+from .__init__ import ANYWIN, EXE, MACOS, TYPE_CHECKING, E, EnvParams, unicode
 from .authsrv import BAD_CFG, AuthSrv
 from .cert import ensure_cert
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
@@ -173,6 +173,26 @@ class SvcHub(object):
            self.log("root", t.format(args.j), c=3)
            args.no_fpool = True

+        for name, arg in (
+            ("iobuf", "iobuf"),
+            ("s-rd-sz", "s_rd_sz"),
+            ("s-wr-sz", "s_wr_sz"),
+        ):
+            zi = getattr(args, arg)
+            if zi < 32768:
+                t = "WARNING: expect very poor performance because you specified a very low value (%d) for --%s"
+                self.log("root", t % (zi, name), 3)
+                zi = 2
+            zi2 = 2 ** (zi - 1).bit_length()
+            if zi != zi2:
+                zi3 = 2 ** ((zi - 1).bit_length() - 1)
+                t = "WARNING: expect poor performance because --%s is not a power-of-two; consider using %d or %d instead of %d"
+                self.log("root", t % (name, zi2, zi3, zi), 3)
+
+        if args.s_rd_sz > args.iobuf:
+            t = "WARNING: --s-rd-sz (%d) is larger than --iobuf (%d); this may lead to reduced performance"
+            self.log("root", t % (args.s_rd_sz, args.iobuf), 3)
+
        bri = "zy"[args.theme % 2 :][:1]
        ch = "abcdefghijklmnopqrstuvwx"[int(args.theme / 2)]
        args.theme = "{0}{1} {0} {1}".format(ch, bri)
@@ -256,6 +276,11 @@ class SvcHub(object):
        if want_ff and ANYWIN:
            self.log("thumb", "download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)

+        if not args.no_acode:
+            if not re.match("^(0|[qv][0-9]|[0-9]{2,3}k)$", args.q_mp3.lower()):
+                t = "invalid mp3 transcoding quality [%s] specified; only supports [0] to disable, a CBR value such as [192k], or a CQ/CRF value such as [v2]"
+                raise Exception(t % (args.q_mp3,))
+
        args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)

        zms = ""
--- a/copyparty/szip.py
+++ b/copyparty/szip.py
@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals

+import argparse
 import calendar
 import stat
 import time
@@ -218,12 +219,13 @@ class StreamZip(StreamArc):
    def __init__(
        self,
        log: "NamedLogger",
+        args: argparse.Namespace,
        fgen: Generator[dict[str, Any], None, None],
        utf8: bool = False,
        pre_crc: bool = False,
        **kwargs: Any
    ) -> None:
-        super(StreamZip, self).__init__(log, fgen)
+        super(StreamZip, self).__init__(log, args, fgen)

        self.utf8 = utf8
        self.pre_crc = pre_crc
@@ -248,7 +250,7 @@ class StreamZip(StreamArc):

        crc = 0
        if self.pre_crc:
-            for buf in yieldfile(src):
+            for buf in yieldfile(src, self.args.iobuf):
                crc = zlib.crc32(buf, crc)

            crc &= 0xFFFFFFFF
@@ -257,7 +259,7 @@ class StreamZip(StreamArc):
        buf = gen_hdr(None, name, sz, ts, self.utf8, crc, self.pre_crc)
        yield self._ct(buf)

-        for buf in yieldfile(src):
+        for buf in yieldfile(src, self.args.iobuf):
            if not self.pre_crc:
                crc = zlib.crc32(buf, crc)

--- a/copyparty/tftpd.py
+++ b/copyparty/tftpd.py
@@ -340,6 +340,9 @@ class Tftpd(object):
        if not self.args.tftp_nols and bos.path.isdir(ap):
            return self._ls(vpath, "", 0, True)

+        if not a:
+            a = [self.args.iobuf]
+
        return open(ap, mode, *a, **ka)

    def _mkdir(self, vpath: str, *a) -> None:
--- a/copyparty/th_cli.py
+++ b/copyparty/th_cli.py
@@ -57,7 +57,7 @@ class ThumbCli(object):
        if is_vid and "dvthumb" in dbv.flags:
            return None

-        want_opus = fmt in ("opus", "caf")
+        want_opus = fmt in ("opus", "caf", "mp3")
        is_au = ext in self.fmt_ffa
        if is_au:
            if want_opus:
--- a/copyparty/th_srv.py
+++ b/copyparty/th_srv.py
@@ -16,9 +16,9 @@ from .__init__ import ANYWIN, TYPE_CHECKING
 from .authsrv import VFS
 from .bos import bos
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
+from .util import BytesIO  # type: ignore
 from .util import (
    FFMPEG_URL,
-    BytesIO,  # type: ignore
    Cooldown,
    Daemon,
    Pebkac,
@@ -109,7 +109,7 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
    h = hashlib.sha512(afsenc(fn)).digest()
    fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]

-    if fmt in ("opus", "caf"):
+    if fmt in ("opus", "caf", "mp3"):
        cat = "ac"
    else:
        fc = fmt[:1]
@@ -307,6 +307,8 @@ class ThumbSrv(object):
                    elif lib == "ff" and ext in self.fmt_ffa:
                        if tpath.endswith(".opus") or tpath.endswith(".caf"):
                            funs.append(self.conv_opus)
+                        elif tpath.endswith(".mp3"):
+                            funs.append(self.conv_mp3)
                        elif tpath.endswith(".png"):
                            funs.append(self.conv_waves)
                            png_ok = True
@@ -637,8 +639,47 @@ class ThumbSrv(object):
        cmd += [fsenc(tpath)]
        self._run_ff(cmd, vn)

+    def conv_mp3(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
+        quality = self.args.q_mp3.lower()
+        if self.args.no_acode or not quality:
+            raise Exception("disabled in server config")
+
+        self.wait4ram(0.2, tpath)
+        ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
+        if "ac" not in ret:
+            raise Exception("not audio")
+
+        if quality.endswith("k"):
+            qk = b"-b:a"
+            qv = quality.encode("ascii")
+        else:
+            qk = b"-q:a"
+            qv = quality[1:].encode("ascii")
+
+        # extremely conservative choices for output format
+        # (always 2ch 44k1) because if a device is old enough
+        # to not support opus then it's probably also super picky
+
+        # fmt: off
+        cmd = [
+            b"ffmpeg",
+            b"-nostdin",
+            b"-v", b"error",
+            b"-hide_banner",
+            b"-i", fsenc(abspath),
+            b"-map_metadata", b"-1",
+            b"-map", b"0:a:0",
+            b"-ar", b"44100",
+            b"-ac", b"2",
+            b"-c:a", b"libmp3lame",
+            qk, qv,
+            fsenc(tpath)
+        ]
+        # fmt: on
+        self._run_ff(cmd, vn, oom=300)
+
    def conv_opus(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
-        if self.args.no_acode:
+        if self.args.no_acode or not self.args.q_opus:
            raise Exception("disabled in server config")

        self.wait4ram(0.2, tpath)
@@ -662,6 +703,7 @@ class ThumbSrv(object):
                pass

        caf_src = abspath if src_opus else tmp_opus
+        bq = ("%dk" % (self.args.q_opus,)).encode("ascii")

        if not want_caf or not src_opus:
            # fmt: off
@@ -674,7 +716,7 @@ class ThumbSrv(object):
                b"-map_metadata", b"-1",
                b"-map", b"0:a:0",
                b"-c:a", b"libopus",
-                b"-b:a", b"128k",
+                b"-b:a", bq,
                fsenc(tmp_opus)
            ]
            # fmt: on
@@ -697,7 +739,7 @@ class ThumbSrv(object):
                b"-map_metadata", b"-1",
                b"-ac", b"2",
                b"-c:a", b"libopus",
-                b"-b:a", b"128k",
+                b"-b:a", bq,
                b"-f", b"caf",
                fsenc(tpath)
            ]
@@ -771,7 +813,7 @@ class ThumbSrv(object):

    def _clean(self, cat: str, thumbpath: str) -> int:
        # self.log("cln {}".format(thumbpath))
-        exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf"]
+        exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf", "mp3"]
        maxage = getattr(self.args, cat + "_maxage")
        now = time.time()
        prev_b64 = None
--- a/copyparty/up2k.py
+++ b/copyparty/up2k.py
@@ -3920,7 +3920,7 @@ class Up2k(object):
        csz = up2k_chunksize(fsz)
        ret = []
        suffix = " MB, {}".format(path)
-        with open(fsenc(path), "rb", 512 * 1024) as f:
+        with open(fsenc(path), "rb", self.args.iobuf) as f:
            if self.mth and fsz >= 1024 * 512:
                tlt = self.mth.hash(f, fsz, csz, self.pp, prefix, suffix)
                ret = [x[0] for x in tlt]
--- a/copyparty/util.py
+++ b/copyparty/util.py
@@ -1400,10 +1400,15 @@ def ren_open(

 class MultipartParser(object):
    def __init__(
-        self, log_func: "NamedLogger", sr: Unrecv, http_headers: dict[str, str]
+        self,
+        log_func: "NamedLogger",
+        args: argparse.Namespace,
+        sr: Unrecv,
+        http_headers: dict[str, str],
    ):
        self.sr = sr
        self.log = log_func
+        self.args = args
        self.headers = http_headers

        self.re_ctype = re.compile(r"^content-type: *([^; ]+)", re.IGNORECASE)
@@ -1502,7 +1507,7 @@ class MultipartParser(object):

    def _read_data(self) -> Generator[bytes, None, None]:
        blen = len(self.boundary)
-        bufsz = 32 * 1024
+        bufsz = self.args.s_rd_sz
        while True:
            try:
                buf = self.sr.recv(bufsz)
@@ -2243,10 +2248,11 @@ def shut_socket(log: "NamedLogger", sck: socket.socket, timeout: int = 3) -> Non
        sck.close()


-def read_socket(sr: Unrecv, total_size: int) -> Generator[bytes, None, None]:
+def read_socket(
+    sr: Unrecv, bufsz: int, total_size: int
+) -> Generator[bytes, None, None]:
    remains = total_size
    while remains > 0:
-        bufsz = 32 * 1024
        if bufsz > remains:
            bufsz = remains

@@ -2260,16 +2266,16 @@ def read_socket(sr: Unrecv, total_size: int) -> Generator[bytes, None, None]:
        yield buf


-def read_socket_unbounded(sr: Unrecv) -> Generator[bytes, None, None]:
+def read_socket_unbounded(sr: Unrecv, bufsz: int) -> Generator[bytes, None, None]:
    try:
        while True:
-            yield sr.recv(32 * 1024)
+            yield sr.recv(bufsz)
    except:
        return


 def read_socket_chunked(
-    sr: Unrecv, log: Optional["NamedLogger"] = None
+    sr: Unrecv, bufsz: int, log: Optional["NamedLogger"] = None
 ) -> Generator[bytes, None, None]:
    err = "upload aborted: expected chunk length, got [{}] |{}| instead"
    while True:
@@ -2303,7 +2309,7 @@ def read_socket_chunked(
        if log:
            log("receiving %d byte chunk" % (chunklen,))

-        for chunk in read_socket(sr, chunklen):
+        for chunk in read_socket(sr, bufsz, chunklen):
            yield chunk

        x = sr.recv_ex(2, False)
@@ -2361,10 +2367,11 @@ def build_netmap(csv: str):
    return NetMap(ips, cidrs, True)


-def yieldfile(fn: str) -> Generator[bytes, None, None]:
-    with open(fsenc(fn), "rb", 512 * 1024) as f:
+def yieldfile(fn: str, bufsz: int) -> Generator[bytes, None, None]:
+    readsz = min(bufsz, 128 * 1024)
+    with open(fsenc(fn), "rb", bufsz) as f:
        while True:
-            buf = f.read(128 * 1024)
+            buf = f.read(readsz)
            if not buf:
                break

--- a/copyparty/web/baguettebox.js
+++ b/copyparty/web/baguettebox.js
@@ -394,8 +394,7 @@ window.baguetteBox = (function () {
    }

    function dlpic() {
-        var url = findfile()[3].href;
-        url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache';
+        var url = addq(findfile()[3].href, 'cache');
        dl_file(url);
    }

@@ -592,12 +591,13 @@ window.baguetteBox = (function () {
            document.documentElement.style.overflowY = 'auto';
            document.body.style.overflowY = 'auto';
        }
-        if (overlay.style.display === 'none')
-            return;

        if (options.duringHide)
            options.duringHide();

+        if (overlay.style.display === 'none')
+            return;
+
        sethash('');
        unbindEvents();
        try {
@@ -682,7 +682,7 @@ window.baguetteBox = (function () {
                options.captions.call(currentGallery, imageElement) :
                imageElement.getAttribute('data-caption') || imageElement.title;

-        imageSrc += imageSrc.indexOf('?') < 0 ? '?cache' : '&cache';
+        imageSrc = addq(imageSrc, 'cache');

        if (is_vid && index != currentIndex)
            return;  // no preload
@@ -720,6 +720,9 @@ window.baguetteBox = (function () {

        figure.appendChild(image);

+        if (is_vid && window.afilt)
+            afilt.apply(undefined, image);
+
        if (options.async && callback)
            callback();
    }
--- a/copyparty/web/browser.js
+++ b/copyparty/web/browser.js
@@ -244,6 +244,7 @@ var Ls = {
 		"mt_preload": "start loading the next song near the end for gapless playback\">preload",
 		"mt_prescan": "go to the next folder before the last song$Nends, keeping the webbrowser happy$Nso it doesn't stop the playback\">nav",
 		"mt_fullpre": "try to preload the entire song;$N✅ enable on <b>unreliable</b> connections,$N❌ <b>disable</b> on slow connections probably\">full",
+		"mt_fau": "on phones, prevent music from stopping if the next song doesn't preload fast enough (can make tags display glitchy)\">☕️",
 		"mt_waves": "waveform seekbar:$Nshow audio amplitude in the scrubber\">~s",
 		"mt_npclip": "show buttons for clipboarding the currently playing song\">/np",
 		"mt_octl": "os integration (media hotkeys / osd)\">os-ctl",
@@ -745,6 +746,7 @@ var Ls = {
 		"mt_preload": "hent ned litt av neste sang i forkant,$Nslik at pausen i overgangen blir mindre\">forles",
 		"mt_prescan": "ved behov, bla til neste mappe$Nslik at nettleseren lar oss$Nfortsette å spille musikk\">bla",
 		"mt_fullpre": "hent ned hele neste sang, ikke bare litt:$N✅ skru på hvis nettet ditt er <b>ustabilt</b>,$N❌ skru av hvis nettet ditt er <b>tregt</b>\">full",
+		"mt_fau": "for telefoner: forhindre at avspilling stopper hvis nettet er for tregt til å laste neste sang i tide. Hvis påskrudd, kan forårsake at sang-info ikke vises korrekt i OS'et\">☕️",
 		"mt_waves": "waveform seekbar:$Nvis volumkurve i avspillingsfeltet\">~s",
 		"mt_npclip": "vis knapper for å kopiere info om sangen du hører på\">/np",
 		"mt_octl": "integrering med operativsystemet (fjernkontroll, info-skjerm)\">os-ctl",
@@ -1400,7 +1402,9 @@ var ACtx = !IPHONE && (window.AudioContext || window.webkitAudioContext),
 	ACB = sread('au_cbv') || 1,
 	noih = /[?&]v\b/.exec('' + location),
 	hash0 = location.hash,
-	mp;
+	ldks = [],
+	dks = {},
+	dk, mp;


 var mpl = (function () {
@@ -1413,6 +1417,7 @@ var mpl = (function () {
 		'<a href="#" class="tgl btn" id="au_preload" tt="' + L.mt_preload + '</a>' +
 		'<a href="#" class="tgl btn" id="au_prescan" tt="' + L.mt_prescan + '</a>' +
 		'<a href="#" class="tgl btn" id="au_fullpre" tt="' + L.mt_fullpre + '</a>' +
+		'<a href="#" class="tgl btn" id="au_fau" tt="' + L.mt_fau + '</a>' +
 		'<a href="#" class="tgl btn" id="au_waves" tt="' + L.mt_waves + '</a>' +
 		'<a href="#" class="tgl btn" id="au_npclip" tt="' + L.mt_npclip + '</a>' +
 		'<a href="#" class="tgl btn" id="au_os_ctl" tt="' + L.mt_octl + '</a>' +
@@ -1459,6 +1464,15 @@ var mpl = (function () {
 	bcfg_bind(r, 'preload', 'au_preload', true);
 	bcfg_bind(r, 'prescan', 'au_prescan', true);
 	bcfg_bind(r, 'fullpre', 'au_fullpre', false);
+	bcfg_bind(r, 'fau', 'au_fau', MOBILE && !IPHONE, function (v) {
+		mp.nopause();
+		if (mp.fau) {
+			mp.fau.pause();
+			mp.fau = mpo.fau = null;
+			console.log('stop fau');
+		}
+		mp.init_fau();
+	});
 	bcfg_bind(r, 'waves', 'au_waves', true, function (v) {
 		if (!v) pbar.unwave();
 	});
@@ -1532,7 +1546,7 @@ var mpl = (function () {
 			c = r.ac_flac;
 		else if (/\.(aac|m4a)$/i.exec(url))
 			c = r.ac_aac;
-		else if (/\.opus$/i.exec(url) && !can_ogg)
+		else if (/\.(ogg|opus)$/i.exec(url) && !can_ogg)
 			c = true;
 		else if (re_au_native.exec(url))
 			c = false;
@@ -1540,7 +1554,7 @@ var mpl = (function () {
 		if (!c)
 			return url;

-		return url + (url.indexOf('?') < 0 ? '?' : '&') + 'th=' + (can_ogg ? 'opus' : 'caf');
+		return addq(url, 'th=') + (can_ogg ? 'opus' : (IPHONE || MACOS) ? 'caf' : 'mp3');
 	};

 	r.pp = function () {
@@ -1591,7 +1605,7 @@ var mpl = (function () {
 			}

 			if (cover) {
-				cover += (cover.indexOf('?') === -1 ? '?' : '&') + 'th=j';
+				cover = addq(cover, 'th=j');
 				tags.artwork = [{ "src": cover, type: "image/jpeg" }];
 			}
 		}
@@ -1650,26 +1664,23 @@ var mpl = (function () {
 var can_ogg = true;
 try {
 	can_ogg = new Audio().canPlayType('audio/ogg; codecs=opus') === 'probably';
-
-	if (document.documentMode)
-		can_ogg = true;  // ie8-11
 }
 catch (ex) { }


-var re_au_native = can_ogg ? /\.(aac|flac|m4a|mp3|ogg|opus|wav)$/i :
-	have_acode ? /\.(aac|flac|m4a|mp3|opus|wav)$/i : /\.(aac|flac|m4a|mp3|wav)$/i,
+var re_au_native = (can_ogg || have_acode) ? /\.(aac|flac|m4a|mp3|ogg|opus|wav)$/i : /\.(aac|flac|m4a|mp3|wav)$/i,
 	re_au_all = /\.(aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk)$/i;


 // extract songs + add play column
-var mpo = { "au": null, "au2": null, "acs": null };
+var mpo = { "au": null, "au2": null, "acs": null, "fau": null };
 function MPlayer() {
 	var r = this;
 	r.id = Date.now();
 	r.au = mpo.au;
 	r.au2 = mpo.au2;
 	r.acs = mpo.acs;
+	r.fau = mpo.fau;
 	r.tracks = {};
 	r.order = [];
 	r.cd_pause = 0;
@@ -1682,8 +1693,8 @@ function MPlayer() {
 			link = tds[1].getElementsByTagName('a');

 		link = link[link.length - 1];
-		var url = noq_href(link),
-			m = re_audio.exec(url);
+		var url = link.getAttribute('href'),
+			m = re_audio.exec(url.split('?')[0]);

 		if (m) {
 			var tid = link.getAttribute('id');
@@ -1795,12 +1806,11 @@ function MPlayer() {
 		var t0 = Date.now(),
 			fname = uricom_dec(url.split('/').pop());

-		url = mpl.acode(url);
-		url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=987&_=' + ACB;
+		url = addq(mpl.acode(url), 'cache=987&_=' + ACB);
 		mpl.preload_url = full ? url : null;

 		if (mpl.waves)
-			fetch(url.replace(/\bth=opus&/, '') + '&th=p').then(function (x) {
+			fetch(url.replace(/\bth=(opus|mp3)&/, '') + '&th=p').then(function (x) {
 				x.body.getReader().read();
 			});

@@ -1842,6 +1852,17 @@ function MPlayer() {
 	r.nopause = function () {
 		r.cd_pause = Date.now();
 	};
+
+	r.init_fau = function () {
+		if (r.fau || !mpl.fau)
+			return;
+
+		// breaks touchbar-macs
+		console.log('init fau');
+		r.fau = new Audio(SR + '/.cpr/deps/busy.mp3?_=' + TS);
+		r.fau.loop = true;
+		r.fau.play();
+	};
 }


@@ -2379,8 +2400,7 @@ function dl_song() {
 		return toast.inf(10, L.f_dls);
 	}

-	var url = mp.tracks[mp.au.tid];
-	url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=987&_=' + ACB;
+	var url = addq(mp.tracks[mp.au.tid], 'cache=987&_=' + ACB);
 	dl_file(url);
 }

@@ -2514,11 +2534,11 @@ var mpui = (function () {
 				rem = pos > 1 ? len - pos : 999,
 				full = null;

-			if (rem < (mpl.fullpre ? 7 : 20)) {
+			if (rem < 7 || (!mpl.fullpre && (rem < 40 || pos > 10))) {
 				preloaded = fpreloaded = mp.au.rsrc;
 				full = false;
 			}
-			else if (rem < 40 && mpl.fullpre && fpreloaded != mp.au.rsrc) {
+			else if (rem < 60 && mpl.fullpre && fpreloaded != mp.au.rsrc) {
 				fpreloaded = mp.au.rsrc;
 				full = true;
 			}
@@ -2714,7 +2734,7 @@ var afilt = (function () {
 		mp.acs = mpo.acs = null;
 	};

-	r.apply = function (v) {
+	r.apply = function (v, au) {
 		r.init();
 		r.draw();

@@ -2734,12 +2754,13 @@ var afilt = (function () {
 			if (r.plugs[a].en)
 				plug = true;

-		if (!actx || !mp.au || (!r.eqen && !plug && !mp.acs))
+		au = au || (mp && mp.au);
+		if (!actx || !au || (!r.eqen && !plug && !mp.acs))
 			return;

 		r.stop();
-		mp.au.id = mp.au.id || Date.now();
-		mp.acs = r.acst[mp.au.id] = r.acst[mp.au.id] || actx.createMediaElementSource(mp.au);
+		au.id = au.id || Date.now();
+		mp.acs = r.acst[au.id] = r.acst[au.id] || actx.createMediaElementSource(au);

 		if (r.eqen)
 			add_eq();
@@ -2995,6 +3016,7 @@ function play(tid, is_ev, seek) {
 		return;

 	mpl.preload_url = null;
+	mp.nopause();
 	mp.stopfade(true);

 	var tn = tid;
@@ -3041,9 +3063,9 @@ function play(tid, is_ev, seek) {
 		mp.au.onended = next_song;
 		widget.open();
 	}
+	mp.init_fau();

-	var url = mpl.acode(mp.tracks[tid]);
-	url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=987&_=' + ACB;
+	var url = addq(mpl.acode(mp.tracks[tid]), 'cache=987&_=' + ACB);

 	if (mp.au.rsrc == url)
 		mp.au.currentTime = 0;
@@ -3107,7 +3129,7 @@ function play(tid, is_ev, seek) {

 		pbar.unwave();
 		if (mpl.waves)
-			pbar.loadwaves(url.replace(/\bth=opus&/, '') + '&th=p');
+			pbar.loadwaves(url.replace(/\bth=(opus|mp3)&/, '') + '&th=p');

 		mpui.progress_updater();
 		pbar.onresize();
@@ -3194,7 +3216,10 @@ function evau_error(e) {
 			toast.warn(15, esc(basenames(err + mfile)));
 		};
 		xhr.send();
+		return;
 	}
+
+	setTimeout(next_song, 15000);
 }


@@ -4214,12 +4239,12 @@ var showfile = (function () {
 		qsr('#prism_css');
 		var el = mknod('link', 'prism_css');
 		el.rel = 'stylesheet';
-		el.href = SR + '/.cpr/deps/prism' + (light ? '' : 'd') + '.css';
+		el.href = SR + '/.cpr/deps/prism' + (light ? '' : 'd') + '.css?_=' + TS;
 		document.head.appendChild(el);
 	};

 	r.active = function () {
-		return location.search.indexOf('doc=') + 1;
+		return !!/[?&]doc=/.exec(location.search);
 	};

 	r.getlang = function (fn) {
@@ -4260,12 +4285,15 @@ var showfile = (function () {
 	};

 	r.show = function (url, no_push) {
-		var xhr = new XHR();
+		var xhr = new XHR(),
+			m = /[?&](k=[^&]+)/.exec(url);
+
+		url = url.split('?')[0] + (m ? '?' + m[1] : '');
 		xhr.url = url;
 		xhr.fname = uricom_dec(url.split('/').pop());
 		xhr.no_push = no_push;
 		xhr.ts = Date.now();
-		xhr.open('GET', url.split('?')[0], true);
+		xhr.open('GET', url, true);
 		xhr.onprogress = loading;
 		xhr.onload = xhr.onerror = load_cb;
 		xhr.send();
@@ -4303,14 +4331,14 @@ var showfile = (function () {
 		var url = doc[0],
 			lnh = doc[1],
 			txt = doc[2],
-			name = url.split('/').pop(),
+			name = url.split('?')[0].split('/').pop(),
 			tname = uricom_dec(name),
 			lang = r.getlang(name),
 			is_md = lang == 'md';

 		ebi('files').style.display = ebi('gfiles').style.display = ebi('lazy').style.display = ebi('pro').style.display = ebi('epi').style.display = 'none';
 		ebi('dldoc').setAttribute('href', url);
-		ebi('editdoc').setAttribute('href', url + (url.indexOf('?') > 0 ? '&' : '?') + 'edit');
+		ebi('editdoc').setAttribute('href', addq(url, 'edit'));
 		ebi('editdoc').style.display = (has(perms, 'write') && (is_md || has(perms, 'delete'))) ? '' : 'none';

 		var wr = ebi('bdoc'),
@@ -4361,7 +4389,7 @@ var showfile = (function () {
 		wintitle(tname + ' \u2014 ');
 		document.documentElement.scrollTop = 0;
 		var hfun = no_push ? hist_replace : hist_push;
-		hfun(get_evpath() + '?doc=' + url.split('/').pop());
+		hfun(get_evpath() + '?doc=' + name);  // can't dk: server wants dk and js needs fk

 		qsr('#docname');
 		el = mknod('span', 'docname');
@@ -4563,7 +4591,7 @@ var thegrid = (function () {
 			if (!force)
 				return;

-			hist_push(get_evpath());
+			hist_push(get_evpath() + (dk ? '?k=' + dk : ''));
 			wintitle();
 		}

@@ -4795,10 +4823,10 @@ var thegrid = (function () {
 				ref = ao.getAttribute('id'),
 				isdir = href.endsWith('/'),
 				ac = isdir ? ' class="dir"' : '',
-				ihref = href;
+				ihref = ohref;

 			if (r.thumbs) {
-				ihref += '?th=' + (have_webp ? 'w' : 'j');
+				ihref = addq(ihref, 'th=') + (have_webp ? 'w' : 'j');
 				if (!r.crop)
 					ihref += 'f';
 				if (r.x3)
@@ -4834,7 +4862,7 @@ var thegrid = (function () {
 				}
 				ihref = SR + '/.cpr/ico/' + ext;
 			}
-			ihref += (ihref.indexOf('?') > 0 ? '&' : '?') + 'cache=i&_=' + ACB;
+			ihref = addq(ihref, 'cache=i&_=' + ACB + TS);
 			if (CHROME)
 				ihref += "&raster";

@@ -4875,10 +4903,10 @@ var thegrid = (function () {
 			baguetteBox.destroy();

 		var br = baguetteBox.run(isrc, {
+			noScrollbars: true,
 			duringHide: r.onhide,
 			afterShow: function () {
 				r.bbox_opts.refocus = true;
-				document.body.style.overflow = 'hidden';
 			},
 			captions: function (g) {
 				var idx = -1,
@@ -4901,7 +4929,8 @@ var thegrid = (function () {
 	};

 	r.onhide = function () {
-		document.body.style.overflow = '';
+		afilt.apply();
+
 		if (!thegrid.ihop)
 			return;

@@ -5723,6 +5752,14 @@ var treectl = (function () {
 	};
 	r.nvis = r.lim;

+	ldks = jread('dks', []);
+	for (var a = ldks.length - 1; a >= 0; a--) {
+		var s = ldks[a],
+			o = s.lastIndexOf('?');
+
+		dks[s.slice(0, o)] = s.slice(o + 1);
+	}
+
 	function setwrap(v) {
 		clmod(ebi('tree'), 'nowrap', !v);
 		reload_tree();
@@ -5937,12 +5974,15 @@ var treectl = (function () {
 	};

 	function get_tree(top, dst, rst) {
-		var xhr = new XHR();
+		var xhr = new XHR(),
+			m = /[?&](k=[^&]+)/.exec(dst),
+			k = m ? '&' + m[1] : dk ? '&k=' + dk : '';
+
 		xhr.top = top;
 		xhr.dst = dst;
 		xhr.rst = rst;
 		xhr.ts = Date.now();
-		xhr.open('GET', dst + '?tree=' + top + (r.dots ? '&dots' : ''), true);
+		xhr.open('GET', addq(dst, 'tree=') + top + (r.dots ? '&dots' : '') + k, true);
 		xhr.onload = xhr.onerror = recvtree;
 		xhr.send();
 		enspin('#tree');
@@ -5969,7 +6009,7 @@ var treectl = (function () {
 		}
 		ebi('treeul').setAttribute('ts', ts);

-		var top = top0 == '.' ? dst : top0,
+		var top = (top0 == '.' ? dst : top0).split('?')[0],
 			name = uricom_dec(top.split('/').slice(-2)[0]),
 			rtop = top.replace(/^\/+/, ""),
 			html = parsetree(res, rtop);
@@ -5986,7 +6026,7 @@ var treectl = (function () {

 			var links = QSA('#treeul a+a');
 			for (var a = 0, aa = links.length; a < aa; a++) {
-				if (links[a].getAttribute('href') == top) {
+				if (links[a].getAttribute('href').split('?')[0] == top) {
 					var o = links[a].parentNode;
 					if (!o.getElementsByTagName('li').length)
 						o.innerHTML = html;
@@ -6019,14 +6059,20 @@ var treectl = (function () {

 	function reload_tree() {
 		var cdir = r.nextdir || get_vpath(),
+			cevp = get_evpath(),
 			links = QSA('#treeul a+a'),
 			nowrap = QS('#tree.nowrap') && QS('#hovertree.on'),
 			act = null;

 		for (var a = 0, aa = links.length; a < aa; a++) {
-			var href = uricom_dec(links[a].getAttribute('href')),
+			var qhref = links[a].getAttribute('href'),
+				ehref = qhref.split('?')[0],
+				href = uricom_dec(ehref),
 				cl = '';

+			if (dk && ehref == cevp && !/[?&]k=/.exec(qhref))
+				links[a].setAttribute('href', addq(qhref, 'k=') + dk);
+
 			if (href == cdir) {
 				act = links[a];
 				cl = 'hl';
@@ -6125,13 +6171,16 @@ var treectl = (function () {
 		if (IE && !history.pushState)
 			return window.location = url;

-		var xhr = new XHR();
+		var xhr = new XHR(),
+			m = /[?&](k=[^&]+)/.exec(url),
+			k = m ? '&' + m[1] : dk ? '&k=' + dk : '';
+
 		xhr.top = url.split('?')[0];
 		xhr.back = back
 		xhr.hpush = hpush;
 		xhr.hydrate = hydrate;
 		xhr.ts = Date.now();
-		xhr.open('GET', xhr.top + '?ls' + (r.dots ? '&dots' : ''), true);
+		xhr.open('GET', xhr.top + '?ls' + (r.dots ? '&dots' : '') + k, true);
 		xhr.onload = xhr.onerror = recvls;
 		xhr.send();

@@ -6193,6 +6242,7 @@ var treectl = (function () {
 		read_dsort(res.dsort);
 		dcrop = res.dcrop;
 		dth3x = res.dth3x;
+		dk = res.dk;

 		srvinf = res.srvinf;
 		try {
@@ -6201,14 +6251,22 @@ var treectl = (function () {
 		catch (ex) { }

 		if (this.hpush && !showfile.active())
-			hist_push(this.top);
+			hist_push(this.top + (dk ? '?k=' + dk : ''));

 		if (!this.back) {
 			var dirs = [];
-			for (var a = 0; a < res.dirs.length; a++)
-				dirs.push(res.dirs[a].href.split('/')[0].split('?')[0]);
+			for (var a = 0; a < res.dirs.length; a++) {
+				var dh = res.dirs[a].href,
+					dn = dh.split('/')[0].split('?')[0],
+					m = /[?&](k=[^&]+)/.exec(dh);

-			rendertree({ "a": dirs }, this.ts, ".", get_evpath());
+				if (m)
+					dn += '?' + m[1];
+
+				dirs.push(dn);
+			}
+
+			rendertree({ "a": dirs }, this.ts, ".", get_evpath() + (dk ? '?k=' + dk : ''));
 		}

 		r.gentab(this.top, res);
@@ -6268,6 +6326,10 @@ var treectl = (function () {
 			if (ae = ae.querySelector('a[id]'))
 				cid = ae.getAttribute('id');

+		var m = /[?&]k=([^&]+)/.exec(location.search);
+		if (m)
+			memo_dk(top, m[1]);
+
 		r.lsc = res;
 		if (res.unlist) {
 			var ptn = new RegExp(res.unlist);
@@ -6309,7 +6371,7 @@ var treectl = (function () {
 			if (lang) {
 				showfile.files.push({ 'id': id, 'name': fname });
 				if (lang == 'md')
-					tn.href += tn.href.indexOf('?') < 0 ? '?v' : '&v';
+					tn.href = addq(tn.href, 'v');
 			}

 			if (tn.lead == '-')
@@ -6385,7 +6447,7 @@ var treectl = (function () {
 				url = url.href;
 				var mt = m[0] == 'a' ? 'audio' : /\.(webm|mkv)($|\?)/i.exec(url) ? 'video' : 'image'
 				if (mt == 'image') {
-					url += url.indexOf('?') < 0 ? '?cache' : '&cache';
+					url = addq(url, 'cache');
 					console.log(url);
 					new Image().src = url;
 				}
@@ -6417,6 +6479,30 @@ var treectl = (function () {
 		setTimeout(eval_hash, 1);
 	};

+	function memo_dk(vp, k) {
+		dks[vp] = k;
+		var lv = vp + "?" + k;
+		if (has(ldks, lv))
+			return;
+
+		ldks.unshift(lv);
+		if (ldks.length > 32) {
+			var keep = [], evp = get_evpath();
+			for (var a = 0; a < ldks.length; a++) {
+				var s = ldks[a];
+				if (evp.startsWith(s.replace(/\?[^?]+$/, '')))
+					keep.push(s);
+			}
+			var lim = 32 - keep.length;
+			for (var a = 0; a < lim; a++) {
+				if (!has(keep, ldks[a]))
+					keep.push(ldks[a])
+			}
+			ldks = keep;
+		}
+		jwrite('dks', ldks);
+	}
+
 	r.setlazy = function (plain) {
 		var html = ['<div id="plazy">', esc(plain.join(' ')), '</div>'],
 			all = r.lsc.files.length + r.lsc.dirs.length,
@@ -6488,7 +6574,9 @@ var treectl = (function () {
 		keys.sort(function (a, b) { return a.localeCompare(b); });
 		for (var a = 0; a < keys.length; a++) {
 			var kk = keys[a],
-				ks = kk.slice(1),
+				m = /(\?k=[^\n]+)/.exec(kk),
+				kdk = m ? m[1] : '',
+				ks = kk.replace(kdk, '').slice(1),
 				ded = ks.endsWith('\n'),
 				k = uricom_sdec(ded ? ks.replace(/\n$/, '') : ks),
 				hek = esc(k[0]),
@@ -6496,7 +6584,7 @@ var treectl = (function () {
 				url = '/' + (top ? top + uek : uek) + '/',
 				sym = res[kk] ? '-' : '+',
 				link = '<a href="#">' + sym + '</a><a href="' +
-					url + '">' + hek + '</a>';
+					url + kdk + '">' + hek + '</a>';

 			if (res[kk]) {
 				var subtree = parsetree(res[kk], url.slice(1));
@@ -6537,16 +6625,24 @@ var treectl = (function () {
 		if (!e.state)
 			return;

-		var url = new URL(e.state, "https://" + document.location.host);
-		var hbase = url.pathname;
-		var cbase = document.location.pathname;
-		if (url.search.indexOf('doc=') + 1 && hbase == cbase)
+		var url = new URL(e.state, "https://" + location.host),
+			req = url.pathname,
+			hbase = req,
+			cbase = location.pathname,
+			mdoc = /[?&]doc=/.exec('' + url),
+			mdk = /[?&](k=[^&]+)/.exec('' + url);
+
+		if (mdoc && hbase == cbase)
 			return showfile.show(hbase + showfile.sname(url.search), true);

-		r.goto(url.pathname, false, true);
+		if (mdk)
+			req += '?' + mdk[1];
+
+		r.goto(req, false, true);
 	};

-	hist_replace(get_evpath() + location.hash);
+	var evp = get_evpath() + (dk ? '?k=' + dk : '');
+	hist_replace(evp + location.hash);
 	r.onscroll = onscroll;
 	return r;
 })();
@@ -7171,11 +7267,11 @@ var arcfmt = (function () {
 			if (!/^(zip|tar|pax|tgz|txz)$/.exec(txt))
 				continue;

-			var ofs = href.lastIndexOf('?');
-			if (ofs < 0)
+			var m = /(.*[?&])(tar|zip)([^&]*)(.*)$/.exec(href);
+			if (!m)
 				throw new Error('missing arg in url');

-			o.setAttribute("href", href.slice(0, ofs + 1) + arg);
+			o.setAttribute("href", m[1] + arg + m[4]);
 			o.textContent = fmt.split('_')[0];
 		}
 		ebi('selzip').textContent = fmt.split('_')[0];
@@ -7238,13 +7334,20 @@ var msel = (function () {
 			vbase = get_evpath();

 		for (var a = 0, aa = links.length; a < aa; a++) {
-			var href = noq_href(links[a]).replace(/\/$/, ""),
+			var qhref = links[a].getAttribute('href'),
+				href = qhref.split('?')[0].replace(/\/$/, ""),
 				item = {};

 			item.id = links[a].getAttribute('id');
 			item.sel = clgot(links[a].closest('tr'), 'sel');
 			item.vp = href.indexOf('/') !== -1 ? href : vbase + href;

+			if (dk) {
+				var m = /[?&](k=[^&]+)/.exec(qhref);
+				item.q = m ? '?' + m[1] : '';
+			}
+			else item.q = '';
+
 			r.all.push(item);
 			if (item.sel)
 				r.sel.push(item);
@@ -7361,6 +7464,9 @@ var msel = (function () {
 			frm = mknod('form'),
 			txt = [];

+		if (dk)
+			arg += '&k=' + dk;
+
 		for (var a = 0; a < sel.length; a++)
 			txt.push(vsplit(sel[a].vp)[1]);

@@ -7385,7 +7491,7 @@ var msel = (function () {
 		ev(e);
 		var sel = r.getsel();
 		for (var a = 0; a < sel.length; a++)
-			dl_file(sel[a].vp);
+			dl_file(sel[a].vp + sel[a].q);
 	};
 	r.render = function () {
 		var tds = QSA('#files tbody td+td+td'),
@@ -7405,7 +7511,7 @@ var msel = (function () {


 (function () {
-	if (!window.FormData)
+	if (!FormData)
 		return;

 	var form = QS('#op_new_md>form'),
@@ -7427,7 +7533,7 @@ var msel = (function () {


 (function () {
-	if (!window.FormData)
+	if (!FormData)
 		return;

 	var form = QS('#op_mkdir>form'),
@@ -7621,7 +7727,7 @@ function show_md(md, name, div, url, depth) {
 		if (depth) {
 			clmod(div, 'raw', 1);
 			div.textContent = "--[ " + name + " ]---------\r\n" + md;
-			return toast.warn(10, errmsg + (window.WebAssembly ? 'failed to load marked.js' : 'your browser is too old'));
+			return toast.warn(10, errmsg + (WebAssembly ? 'failed to load marked.js' : 'your browser is too old'));
 		}

 		wfp_debounce.n--;
@@ -7934,7 +8040,7 @@ var unpost = (function () {

 	function linklist() {
 		var ret = [],
-			base = document.location.origin.replace(/\/$/, '');
+			base = location.origin.replace(/\/$/, '');

 		for (var a = 0; a < r.files.length; a++)
 			ret.push(base + r.files[a].vp);
@@ -8111,8 +8217,9 @@ ebi('files').onclick = ebi('docul').onclick = function (e) {
 	tgt = e.target.closest('a[hl]');
 	if (tgt) {
 		var a = ebi(tgt.getAttribute('hl')),
+			href = a.getAttribute('href'),
 			fun = function () {
-				showfile.show(noq_href(a), tgt.getAttribute('lang'));
+				showfile.show(href, tgt.getAttribute('lang'));
 			},
 			szs = ft2dict(a.closest('tr'))[0].sz,
 			sz = parseInt(szs.replace(/[, ]/g, ''));
@@ -8139,6 +8246,7 @@ function reload_mp() {
 		mpo.au = mp.au;
 		mpo.au2 = mp.au2;
 		mpo.acs = mp.acs;
+		mpo.fau = mp.fau;
 		mpl.unbuffer();
 	}
 	var plays = QSA('tr>td:first-child>a.play');
@@ -8173,8 +8281,10 @@ function reload_browser() {

 	for (var a = 0; a < parts.length - 1; a++) {
 		link += parts[a] + '/';
+		var link2 = dks[link] ? addq(link, 'k=') + dks[link] : link;
+
 		o = mknod('a');
-		o.setAttribute('href', link);
+		o.setAttribute('href', link2);
 		o.textContent = uricom_dec(parts[a]) || '/';
 		ebi('path').appendChild(mknod('i'));
 		ebi('path').appendChild(o);
--- a/copyparty/web/up2k.js
+++ b/copyparty/web/up2k.js
@@ -17,7 +17,7 @@ function goto_up2k() {
 var up2k = null,
    up2k_hooks = [],
    hws = [],
-    sha_js = window.WebAssembly ? 'hw' : 'ac',  // ff53,c57,sa11
+    sha_js = WebAssembly ? 'hw' : 'ac',  // ff53,c57,sa11
    m = 'will use ' + sha_js + ' instead of native sha512 due to';

 try {
@@ -717,7 +717,7 @@ function Donut(uc, st) {
            sfx();

        // firefox may forget that filedrops are user-gestures so it can skip this:
-        if (uc.upnag && window.Notification && Notification.permission == 'granted')
+        if (uc.upnag && Notification && Notification.permission == 'granted')
            new Notification(uc.nagtxt);
    }

@@ -779,8 +779,8 @@ function up2k_init(subtle) {
    };

    setTimeout(function () {
-        if (window.WebAssembly && !hws.length)
-            fetch(SR + '/.cpr/w.hash.js' + CB);
+        if (WebAssembly && !hws.length)
+            fetch(SR + '/.cpr/w.hash.js?_=' + TS);
    }, 1000);

    function showmodal(msg) {
@@ -869,7 +869,7 @@ function up2k_init(subtle) {
    bcfg_bind(uc, 'turbo', 'u2turbo', turbolvl > 1, draw_turbo);
    bcfg_bind(uc, 'datechk', 'u2tdate', turbolvl < 3, null);
    bcfg_bind(uc, 'az', 'u2sort', u2sort.indexOf('n') + 1, set_u2sort);
-    bcfg_bind(uc, 'hashw', 'hashw', !!window.WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
+    bcfg_bind(uc, 'hashw', 'hashw', !!WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
    bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
    bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);

@@ -1347,9 +1347,9 @@ function up2k_init(subtle) {
        var evpath = get_evpath(),
            draw_each = good_files.length < 50;

-        if (window.WebAssembly && !hws.length) {
+        if (WebAssembly && !hws.length) {
            for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
-                hws.push(new Worker(SR + '/.cpr/w.hash.js' + CB));
+                hws.push(new Worker(SR + '/.cpr/w.hash.js?_=' + TS));

            console.log(hws.length + " hashers");
        }
@@ -2950,7 +2950,7 @@ function up2k_init(subtle) {
    }

    function set_hashw() {
-        if (!window.WebAssembly) {
+        if (!WebAssembly) {
            bcfg_set('hashw', uc.hashw = false);
            toast.err(10, L.u_nowork);
        }
@@ -2967,7 +2967,7 @@ function up2k_init(subtle) {
                nopenag();
        }

-        if (!window.Notification || !HTTPS)
+        if (!Notification || !HTTPS)
            return nopenag();

        if (en && Notification.permission == 'default')
@@ -2989,7 +2989,7 @@ function up2k_init(subtle) {
        };
    }

-    if (uc.upnag && (!window.Notification || Notification.permission != 'granted'))
+    if (uc.upnag && (!Notification || Notification.permission != 'granted'))
        bcfg_set('upnag', uc.upnag = false);

    ebi('nthread_add').onclick = function (e) {
--- a/copyparty/web/util.js
+++ b/copyparty/web/util.js
@@ -16,7 +16,6 @@ var wah = '',
    NOAC = 'autocorrect="off" autocapitalize="off"',
    L, tt, treectl, thegrid, up2k, asmCrypto, hashwasm, vbar, marked,
    T0 = Date.now(),
-    CB = '?_=' + Math.floor(T0 / 1000).toString(36),
    R = SR.slice(1),
    RS = R ? "/" + R : "",
    HALFMAX = 8192 * 8192 * 8192 * 8192,
@@ -52,8 +51,6 @@ catch (ex) {
 }

 try {
-    CB = '?' + document.currentScript.src.split('?').pop();
-
    if (navigator.userAgentData.mobile)
        MOBILE = true;

@@ -130,7 +127,7 @@ if ((document.location + '').indexOf(',rej,') + 1)

 try {
    console.hist = [];
-    var CMAXHIST = 100;
+    var CMAXHIST = 1000;
    var hook = function (t) {
        var orig = console[t].bind(console),
            cfun = function () {
@@ -151,8 +148,6 @@ try {
    hook('error');
 }
 catch (ex) {
-    if (console.stdlog)
-        console.log = console.stdlog;
    console.log('console capture failed', ex);
 }
 var crashed = false, ignexd = {}, evalex_fatal = false;
@@ -182,7 +177,7 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
    if (url.indexOf('easymde.js') + 1)
        return;  // clicking the preview pane

-    if (url.indexOf('deps/marked.js') + 1 && !window.WebAssembly)
+    if (url.indexOf('deps/marked.js') + 1 && !WebAssembly)
        return; // ff<52

    crashed = true;
@@ -740,6 +735,11 @@ function vjoin(p1, p2) {
 }


+function addq(url, q) {
+    return url + (url.indexOf('?') < 0 ? '?' : '&') + (q === undefined ? '' : q);
+}
+
+
 function uricom_enc(txt, do_fb_enc) {
    try {
        return encodeURIComponent(txt);
@@ -1469,7 +1469,7 @@ var toast = (function () {
        clmod(obj, 'vis');
        r.visible = false;
        r.tag = obj;
-        if (!window.WebAssembly)
+        if (!WebAssembly)
            te = setTimeout(function () {
                obj.className = 'hide';
            }, 500);
@@ -1885,7 +1885,7 @@ function md_thumbs(md) {
            float = has(flags, 'l') ? 'left' : has(flags, 'r') ? 'right' : '';

        if (!/[?&]cache/.exec(url))
-            url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=i';
+            url = addq(url, 'cache=i');

        md[a] = '<a href="' + url + '" class="mdth mdth' + float.slice(0, 1) + '"><img src="' + url + '&th=w" alt="' + alt + '" /></a>' + md[a].slice(o2 + 1);
    }
--- a/docs/TODO.md
+++ b/docs/TODO.md
@@ -5,9 +5,6 @@ a living list of upcoming features / fixes / changes, very roughly in order of p
  * maybe resumable downloads (chrome-only, jank api)
  * maybe checksum validation (return sha512 of requested range in responses, and probably also warks)

-* [github issue #64](https://github.com/9001/copyparty/issues/64) - dirkeys 2nd season
-  * popular feature request, finally time to refactor browser.js i suppose...
-
 * [github issue #37](https://github.com/9001/copyparty/issues/37) - upload PWA
  * or [maybe not](https://arstechnica.com/tech-policy/2024/02/apple-under-fire-for-disabling-iphone-web-apps-eu-asks-developers-to-weigh-in/), or [maybe](https://arstechnica.com/gadgets/2024/03/apple-changes-course-will-keep-iphone-eu-web-apps-how-they-are-in-ios-17-4/)

--- a/docs/bufsize.txt
+++ b/docs/bufsize.txt
@@ -0,0 +1,96 @@
+notes from testing various buffer sizes of files and sockets
+
+summary:
+
+download-folder-as-tar:   would be 7% faster with --iobuf 65536 (but got 20% faster in v1.11.2)
+
+download-folder-as-zip:   optimal with default --iobuf 262144
+
+download-file-over-https: optimal with default --iobuf 262144
+
+put-large-file:           optimal with default --iobuf 262144, --s-rd-sz 262144 (and got 14% faster in v1.11.2)
+
+post-large-file:          optimal with default --iobuf 262144, --s-rd-sz 262144 (and got 18% faster in v1.11.2)
+
+----
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/?tar
+ 3.3            req/s 1.11.1
+ 4.3  4.0  3.3  req/s 1.12.2
+  64  256  512  --iobuf        256 (prefer smaller)
+  32   32   32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/?zip
+ 2.9            req/s 1.11.1
+ 2.5  2.9  2.9  req/s 1.12.2
+  64  256  512  --iobuf        256 (prefer bigger)
+  32   32   32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/?tar
+ 8.3            req/s 1.11.1
+ 8.4  8.4  8.5  req/s 1.12.2
+  64  256  512  --iobuf        256 (prefer bigger)
+  32   32   32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/?zip
+ 13.9              req/s 1.11.1
+ 14.1  14.0  13.8  req/s 1.12.2
+   64   256   512  --iobuf     256 (prefer smaller)
+   32    32    32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/987a
+5260                    req/s 1.11.1
+5246  5246  5280  5268  req/s 1.12.2
+  64   256   512   256  --iobuf      dontcare
+  32    32    32   512  --s-rd-sz    dontcare
+
+oha -z10s -c1 --ipv4 --insecure https://127.0.0.1:3923/pairdupes/987a
+4445              req/s 1.11.1
+4462  4494  4444  req/s 1.12.2
+  64   256   512  --iobuf      dontcare
+  32    32    32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/gssc-02-cannonball-skydrift/track10.cdda.flac
+ 95       req/s 1.11.1
+ 95   97  req/s 1.12.2
+ 64  512  --iobuf              dontcare
+ 32   32  --s-rd-sz
+
+oha -z10s -c1 --ipv4 --insecure https://127.0.0.1:3923/bigs/gssc-02-cannonball-skydrift/track10.cdda.flac
+ 15.4                    req/s 1.11.1
+ 15.4  15.3  14.9  15.4  req/s 1.12.2
+   64   256   512   512  --iobuf        256 (prefer smaller, and smaller than s-wr-sz)
+   32    32    32    32  --s-rd-sz
+  256   256   256   512  --s-wr-sz
+
+----
+
+python3 ~/dev/old/copyparty\ v1.11.1\ dont\ ban\ the\ pipes.py -q -i 127.0.0.1 -v .::A --daw 
+python3 ~/dev/copyparty/dist/copyparty-sfx.py -q -i 127.0.0.1 -v .::A --daw --iobuf $((1024*512)) 
+
+oha -z10s -c1 --ipv4 --insecure -mPUT -r0 -D ~/Music/gssc-02-cannonball-skydrift/track10.cdda.flac http://127.0.0.1:3923/a.bin 
+10.8                                req/s 1.11.1
+10.8  11.5  11.8  12.1  12.2  12.3  req/s new
+ 512   512   512   512   512   256  --iobuf         256
+  32    64   128   256   512   256  --s-rd-sz       256 (prefer bigger)
+
+----
+
+buildpost() {
+b=--jeg-er-grensestaven;
+printf -- "$b\r\nContent-Disposition: form-data; name=\"act\"\r\n\r\nbput\r\n$b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"a.bin\"\r\nContent-Type: audio/mpeg\r\n\r\n"
+cat "$1"
+printf -- "\r\n${b}--\r\n"
+}
+buildpost ~/Music/gssc-02-cannonball-skydrift/track10.cdda.flac >big.post
+buildpost ~/Music/bottomtext.txt >smol.post
+
+oha -z10s -c1 --ipv4 --insecure -mPOST -r0 -T 'multipart/form-data; boundary=jeg-er-grensestaven' -D big.post http://127.0.0.1:3923/?replace
+9.6  11.2  11.3  11.1  10.9  req/s v1.11.2
+512   512   256   128   256  --iobuf         256
+ 32   512   256   128   128  --s-rd-sz       256
+
+oha -z10s -c1 --ipv4 --insecure -mPOST -r0 -T 'multipart/form-data; boundary=jeg-er-grensestaven' -D smol.post http://127.0.0.1:3923/?replace
+2445  2414  2401  2437  
+ 256   128   256   256  --iobuf           256
+ 128   128   256    64  --s-rd-sz         128 (but use 256 since big posts are more important)
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -1,3 +1,65 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0323-1724  `v1.11.2`  public idp volumes
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+there is a [discord server](https://discord.gg/25J8CdTT6G) with an `@everyone` in case of future important updates, such as [vulnerabilities](https://github.com/9001/copyparty/security) (most recently 2023-07-23)
+
+## new features
+
+* global-option `--iobuf` to set a custom I/O buffersize 2b24c50e
+  * changes the default buffersize to 256 KiB everywhere (was a mix of 64 and 512)
+  * may improve performance of networked volumes (s3 etc.) if increased
+  * on gbit networks: download-as-tar is now up to 20% faster
+  * slightly faster FTP and TFTP too
+
+* global-option `--s-rd-sz` to set a custom read-size for sockets c6acd3a9
+  * changes the default from 32 to 256 KiB
+  * may improve performance of networked volumes (s3 etc.) if increased
+  * on 10gbit networks: uploading large files is now up to 17% faster
+
+* add url parameter `?replace` to overwrite any existing files with a multipart-post c6acd3a9
+
+## bugfixes
+
+* #79 idp volumes (introduced in [v1.11.0](https://github.com/9001/copyparty/releases/tag/v1.11.0)) would only accept permissions for the user that owned the volume; was impossible to grant read/write-access to other users d30ae845
+
+## other changes
+
+* mention the [lack of persistence for idp volumes](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#important-notes) in the IdP docs 2f20d29e
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0318-1709  `v1.11.1`  dont ban the pipes
+
+the [previous release](https://github.com/9001/copyparty/releases/tag/v1.11.0) had all the fun new features... this one's just bugfixes
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [docker image](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) ╱ [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) ╱ [client testbed](https://cd.ocv.me/b/)
+
+### no vulnerabilities since 2023-07-23
+* there is a [discord server](https://discord.gg/25J8CdTT6G) with an `@everyone` in case of future important updates
+* [v1.8.7](https://github.com/9001/copyparty/releases/tag/v1.8.7) (2023-07-23) - [CVE-2023-38501](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-38501) - reflected XSS
+* [v1.8.2](https://github.com/9001/copyparty/releases/tag/v1.8.2) (2023-07-14) - [CVE-2023-37474](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-37474) - path traversal (first CVE)
+
+## bugfixes
+
+* less aggressive rejection of requests from banned IPs 51d31588
+  * clients would get kicked before the header was parsed (which contains the xff header), meaning the server could become inaccessible to everyone if the reverse-proxy itself were to "somehow" get banned
+    * ...which can happen if a server behind cloudflare also accepts non-cloudflare connections, meaning the client IP would not be resolved, and it'll ban the LAN IP instead heh
+      * that part still happens, but now it won't affect legit clients through the intended route
+  * the old behavior can be restored with `--early-ban` to save some cycles, and/or avoid slowloris somewhat
+* the unpost feature could appear to be disabled on servers where no volume was mapped to `/` 0287c7ba
+* python 3.12 support for [compiling the dependencies](https://github.com/9001/copyparty/tree/hovudstraum/bin/mtag#dependencies) necessary to detect bpm/key in audio files 32553e45
+
+## other changes
+
+* mention [real-ip configuration](https://github.com/9001/copyparty?tab=readme-ov-file#real-ip) in the readme ee80cdb9
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2024-0315-2047  `v1.11.0`  You Can (Not) Proceed

--- a/docs/devnotes.md
+++ b/docs/devnotes.md
@@ -20,6 +20,8 @@
    * [just the sfx](#just-the-sfx)
    * [build from release tarball](#build-from-release-tarball) - uses the included prebuilt webdeps
    * [complete release](#complete-release)
+* [debugging](#debugging)
+    * [music playback halting on phones](#music-playback-halting-on-phones) - mostly fine on android
 * [todo](#todo) - roughly sorted by priority
    * [discarded ideas](#discarded-ideas)

@@ -164,6 +166,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | PUT | `?xz` | (binary data) | compress with xz and write into file at URL |
 | mPOST | | `f=FILE` | upload `FILE` into the folder at URL |
 | mPOST | `?j` | `f=FILE` | ...and reply with json |
+| mPOST | `?replace` | `f=FILE` | ...and overwrite existing files |
 | mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
 | POST | `?delete` | | delete URL recursively |
 | jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
@@ -300,6 +303,19 @@ in the `scripts` folder:
 * run `./rls.sh 1.2.3` which uploads to pypi + creates github release + sfx


+# debugging
+
+## music playback halting on phones
+
+mostly fine on android,  but still haven't find a way to massage iphones into behaving well
+
+* conditionally starting/stopping mp.fau according to mp.au.readyState <3 or <4 doesn't help
+* loop=true doesn't work, and manually looping mp.fau from an onended also doesn't work (it does nothing)
+* assigning fau.currentTime in a timer doesn't work, as safari merely pretends to assign it
+
+can be reproduced with `--no-sendfile --s-wr-sz 8192 --s-wr-slp 0.3 --rsp-slp 6` and then play a collection of small audio files with the screen off, `ffmpeg -i track01.cdda.flac -c:a libopus -b:a 128k -segment_time 12 -f segment smol-%02d.opus`
+
+
 # todo

 roughly sorted by priority
@@ -309,6 +325,10 @@ roughly sorted by priority

 ## discarded ideas

+* optimization attempts which didn't improve performance
+  * remove brokers / multiprocessing stuff; https://github.com/9001/copyparty/tree/no-broker
+  * reduce the nesting / indirections in `HttpCli` / `httpcli.py`
+    * nearly zero benefit from stuff like replacing all the `self.conn.hsrv` with a local `hsrv` variable
 * reduce up2k roundtrips
  * start from a chunk index and just go
  * terminate client on bad data
--- a/docs/idp.md
+++ b/docs/idp.md
@@ -5,3 +5,18 @@ to configure IdP from scratch, you must place copyparty behind a reverse-proxy w
 in the copyparty `[global]` config, specify which headers to read client info from; username is required (`idp-h-usr: X-Authooley-User`), group(s) are optional (`idp-h-grp: X-Authooley-Groups`)

 * it is also required to specify the subnet that legit requests will be coming from, for example `--xff-src=10.88.0.0/24` to allow 10.88.x.x (or `--xff-src=lan` for all private IPs), and it is recommended to configure the reverseproxy to include a secret header as proof that the other headers are also legit (and not smuggled in by a malicious client), telling copyparty the headername to expect with `idp-h-key: shangala-bangala`
+
+
+# important notes
+
+## IdP volumes are forgotten on shutdown
+
+IdP volumes, meaning dynamically-created volumes, meaning volumes that contain `${u}` or `${g}` in their URL, will be forgotten during a server restart and then "revived" when the volume's owner sends their first request after the restart
+
+until each IdP volume is revived, it will inherit the permissions of its parent volume (if any)
+
+this means that, if an IdP volume is located inside a folder that is readable by anyone, then each of those IdP volumes will **also become readable by anyone** until the volume is revived
+
+and likewise -- if the IdP volume is inside a folder that is only accessible by certain users, but the IdP volume is configured to allow access from unauthenticated users, then the contents of the volume will NOT be accessible until it is revived
+
+until this limitation is fixed (if ever), it is recommended to place IdP volumes inside an appropriate parent volume, so they can inherit acceptable permissions until their revival; see the "strategic volumes" at the bottom of [./examples/docker/idp/copyparty.conf](./examples/docker/idp/copyparty.conf)
--- a/docs/pyoxidizer.txt
+++ b/docs/pyoxidizer.txt
@@ -1,52 +0,0 @@
-pyoxidizer doesn't crosscompile yet so need to build in a windows vm,
-luckily possible to do mostly airgapped (https-proxy for crates)
-
-none of this is version-specific but doing absolute links just in case
-(only exception is py3.8 which is the final win7 ver)
-
-# deps (download on linux host):
-https://www.python.org/ftp/python/3.10.7/python-3.10.7-amd64.exe
-https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.22.0/pyoxidizer-0.22.0-x86_64-pc-windows-msvc.zip
-https://github.com/upx/upx/releases/download/v3.96/upx-3.96-win64.zip
-https://static.rust-lang.org/dist/rust-1.61.0-x86_64-pc-windows-msvc.msi
-https://github.com/indygreg/python-build-standalone/releases/download/20220528/cpython-3.8.13%2B20220528-i686-pc-windows-msvc-static-noopt-full.tar.zst
-
-# need cl.exe, prefer 2017 -- download on linux host:
-https://visualstudio.microsoft.com/downloads/?q=build+tools
-https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-history#release-dates-and-build-numbers
-https://aka.ms/vs/15/release/vs_buildtools.exe  # 2017
-https://aka.ms/vs/16/release/vs_buildtools.exe  # 2019
-https://aka.ms/vs/17/release/vs_buildtools.exe  # 2022
-https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2017
-
-# use disposable w10 vm to prep offline installer; xfer to linux host with firefox to copyparty
-vs_buildtools-2017.exe --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools --add Microsoft.VisualStudio.Component.Windows10SDK.17763 --layout c:\msbt2017 --lang en-us
-
-# need two proxies on host; s5s or ssh for msys2(socks5), and tinyproxy for rust(http)
-UP=- python3 socks5server.py 192.168.123.1 4321
-ssh -vND 192.168.123.1:4321 localhost
-git clone https://github.com/tinyproxy/tinyproxy.git
-  ./autogen.sh
-  ./configure --prefix=/home/ed/pe/tinyproxy
-  make -j24 install
-  printf '%s\n' >cfg "Port 4380" "Listen 192.168.123.1"
-  ./tinyproxy -dccfg
-
-https://github.com/msys2/msys2-installer/releases/download/2022-09-04/msys2-x86_64-20220904.exe
-export all_proxy=socks5h://192.168.123.1:4321
-# if chat dies after auth (2 messages) it probably failed dns, note the h in socks5h to tunnel dns
-pacman -Syuu
-pacman -S git patch mingw64/mingw-w64-x86_64-zopfli
-cd /c && curl -k https://192.168.123.1:3923/ro/ox/msbt2017/?tar | tar -xv
-
-first install certs from msbt/certificates then admin-cmd `vs_buildtools.exe --noweb`,
-default selection (vc++2017-v15.9-v14.16, vc++redist, vc++bt-core) += win10sdk (for io.h)
-
-install rust without documentation, python 3.10, put upx and pyoxidizer into ~/bin,
-[cmd.exe] python -m pip install --user -U wheel-0.37.1.tar.gz strip-hints-0.1.10.tar.gz
-p=192.168.123.1:4380; export https_proxy=$p; export http_proxy=$p
-
-# and with all of the one-time-setup out of the way,
-mkdir /c/d; cd /c/d && curl -k https://192.168.123.1:3923/cpp/gb?pw=wark > gb && git clone gb copyparty
-cd /c/d/copyparty/ && curl -k https://192.168.123.1:3923/cpp/patch?pw=wark | patch -p1
-cd /c/d/copyparty/scripts && CARGO_HTTP_CHECK_REVOKE=false PATH=/c/Users/$USER/AppData/Local/Programs/Python/Python310:/c/Users/$USER/bin:"$(cygpath "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86"):$PATH"  ./make-sfx.sh ox ultra
--- a/pyoxidizer.bzl
+++ b/pyoxidizer.bzl
@@ -1,48 +0,0 @@
-# builds win7-i386 exe on win10-ltsc-1809(17763.316)
-# see docs/pyoxidizer.txt
-
-def make_exe():
-    dist = default_python_distribution(flavor="standalone_static", python_version="3.8")
-    policy = dist.make_python_packaging_policy()
-    policy.allow_files = True
-    policy.allow_in_memory_shared_library_loading = True
-    #policy.bytecode_optimize_level_zero = True
-    #policy.include_distribution_sources = False  # error instantiating embedded Python interpreter: during initializing Python main: init_fs_encoding: failed to get the Python codec of the filesystem encoding
-    policy.include_distribution_resources = False
-    policy.include_non_distribution_sources = False
-    policy.include_test = False
-    python_config = dist.make_python_interpreter_config()
-    #python_config.module_search_paths = ["$ORIGIN/lib"]
-
-    python_config.run_module = "copyparty"
-    exe = dist.to_python_executable(
-        name="copyparty",
-        config=python_config,
-        packaging_policy=policy,
-    )
-    exe.windows_runtime_dlls_mode = "never"
-    exe.windows_subsystem = "console"
-    exe.add_python_resources(exe.read_package_root(
-        path="sfx",
-        packages=[
-            "copyparty",
-            "jinja2",
-            "markupsafe",
-            "pyftpdlib",
-            "python-magic",
-        ]
-    ))
-    return exe
-
-def make_embedded_resources(exe):
-    return exe.to_embedded_resources()
-
-def make_install(exe):
-    files = FileManifest()
-    files.add_python_resource("copyparty", exe)
-    return files
-
-register_target("exe", make_exe)
-register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
-register_target("install", make_install, depends=["exe"], default=True)
-resolve_targets()
--- a/scripts/deps-docker/Dockerfile
+++ b/scripts/deps-docker/Dockerfile
@@ -3,7 +3,7 @@ WORKDIR /z
 ENV     ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
        ver_hashwasm=4.10.0 \
        ver_marked=4.3.0 \
-        ver_dompf=3.0.9 \
+        ver_dompf=3.0.11 \
        ver_mde=2.18.0 \
        ver_codemirror=5.65.16 \
        ver_fontawesome=5.13.0 \
@@ -24,7 +24,9 @@ ENV     ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
 # the scp url is regular latin from https://fonts.googleapis.com/css2?family=Source+Code+Pro&display=swap
 RUN     mkdir -p /z/dist/no-pk \
        && wget https://fonts.gstatic.com/s/sourcecodepro/v11/HI_SiYsKILxRpg3hIP6sJ7fM7PqlPevW.woff2 -O scp.woff2 \
-        && apk add cmake make g++ git bash npm patch wget tar pigz brotli gzip unzip python3 python3-dev py3-brotli \
+        && apk add \
+                bash brotli cmake make g++ git gzip lame npm patch pigz \
+                python3 python3-dev py3-brotli sox tar unzip wget \
        && rm -f /usr/lib/python3*/EXTERNALLY-MANAGED \
        && wget https://github.com/openpgpjs/asmcrypto.js/archive/$ver_asmcrypto.tar.gz -O asmcrypto.tgz \
        && wget https://github.com/markedjs/marked/archive/v$ver_marked.tar.gz -O marked.tgz \
@@ -58,6 +60,11 @@ RUN     mkdir -p /z/dist/no-pk \
        && tar -xf zopfli.tgz


+COPY    busy-mp3.sh /z/
+RUN     /z/busy-mp3.sh \
+        && mv -v /dev/shm/busy.mp3.gz /z/dist
+
+
 # build fonttools (which needs zopfli)
 RUN     tar -xf zopfli.tgz \
        && cd zopfli* \
--- a/scripts/deps-docker/busy-mp3.sh
+++ b/scripts/deps-docker/busy-mp3.sh
@@ -0,0 +1,61 @@
+#!/bin/bash
+set -e
+
+cat >/dev/null <<EOF
+a frame is 1152 samples
+1sec @ 48000 = 41.66 frames
+11 frames = 12672 samples = 0.264 sec
+22 frames = 25344 samples = 0.528 sec
+EOF
+
+fast=1
+fast=
+
+echo
+mkdir -p /dev/shm/$1
+cd /dev/shm/$1
+find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete
+min=99999999
+
+for freq in 425; do  # {400..500}
+for vol in 0; do  # {10..30}
+for kbps in 32; do
+for fdur in 1124; do  # {800..1200}
+for fdu2 in 1042; do  # {800..1200}
+for ftyp in h; do  # q h t l p
+for ofs1 in 9214; do  # {0..32768}
+for ofs2 in 0; do  # {0..4096}
+for ofs3 in 0; do  # {0..4096}
+for nores in --nores; do  # '' --nores
+
+f=a.b$kbps$nores-f$freq-v$vol-$ftyp$fdur-$fdu2-o$ofs1-$ofs2-$ofs3
+sox -r48000 -Dn -r48000 -b16 -c2 -t raw s1.pcm synth 25344s sin $freq vol 0.$vol fade $ftyp ${fdur}s 25344s ${fdu2}s
+sox -r48000 -Dn -r48000 -b16 -c2 -t raw s0.pcm synth 12672s sin $freq vol 0
+tail -c +$ofs1 s0.pcm >s0a.pcm
+tail -c +$ofs2 s0.pcm >s0b.pcm
+tail -c +$ofs3 s0.pcm >s0c.pcm
+cat s{0a,1,0,0b,1,0c}.pcm > a.pcm
+lame --silent -r -s 48 --bitwidth 16 --signed a.pcm -m j --resample 48 -b $kbps -q 0 $nores $f.mp3
+if [ $fast ]
+then gzip -c9 <$f.mp3 >$f.mp3.gz
+else pigz -c11 -I1 <$f.mp3 >$f.mp3.gz
+fi
+sz=$(wc -c <$f.mp3.gz)
+printf '\033[A%d %s\033[K\n' $sz $f
+[ $sz -le $((min+10)) ] && echo
+[ $sz -le $min ] && echo && min=$sz
+
+done;done;done;done;done;done;done;done;done;done
+true
+
+f=a.b32--nores-f425-v0-h1124-1042-o9214-0-0.mp3
+[ $fast ] &&
+    pigz -c11 -I1 <$f >busy.mp3.gz ||
+    mv $f.gz busy.mp3.gz
+
+sz=$(wc -c <busy.mp3.gz)
+[ "$sz" -eq 106 ] &&
+    echo busy.mp3 built successfully ||
+    echo WARNING: unexpected size of busy.mp3
+
+find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete
--- a/scripts/make-sfx.sh
+++ b/scripts/make-sfx.sh
@@ -16,8 +16,6 @@ help() { exec cat <<'EOF'
 # `re` does a repack of an sfx which you already executed once
 #   (grabs files from the sfx-created tempdir), overrides `clean`
 #
-# `ox` builds a pyoxidizer exe instead of py
-#
 # `gz` creates a gzip-compressed python sfx instead of bzip2
 #
 # `lang` limits which languages/translations to include,
@@ -111,7 +109,6 @@ while [ ! -z "$1" ]; do
 	case $1 in
 		clean)  clean=1  ; ;;
 		re)     repack=1 ; ;;
-		ox)     use_ox=1 ; ;;
 		gz)     use_gz=1 ; ;;
 		gzz)    shift;use_gzz=$1;use_gz=1; ;;
 		no-ftp) no_ftp=1 ; ;;
@@ -461,8 +458,8 @@ rm -f ftp/pyftpdlib/{__main__,prefork}.py
 		iawk '/^\}/{l=0} !l; /^var Ls =/{l=1;next} o; /^\t["}]/{o=0} /^\t"'"$langs"'"/{o=1;print}' $f
 	done

-[ ! $repack ] && [ ! $use_ox ] && {
-	# uncomment; oxidized drops 45 KiB but becomes undebuggable
+[ ! $repack ] && {
+	# uncomment
 	find | grep -E '\.py$' |
 		grep -vE '__version__' |
 		tr '\n' '\0' |
@@ -570,33 +567,6 @@ nf=$(ls -1 "$zdir"/arc.* 2>/dev/null | wc -l)
 }


-[ $use_ox ] && {
-	tgt=x86_64-pc-windows-msvc
-	tgt=i686-pc-windows-msvc  # 2M smaller (770k after upx)
-	bdir=build/$tgt/release/install/copyparty
-
-	t="res web"
-	(printf "\n\n\nBUT WAIT! THERE'S MORE!!\n\n";
-	cat ../$bdir/COPYING.txt) >> copyparty/res/COPYING.txt ||
-		echo "copying.txt 404 pls rebuild"
-
-	mv ftp/* j2/* .
-	rm -rf ftp j2 py2 py37
-	(cd copyparty; tar -cvf z.tar $t; rm -rf $t)
-	cd ..
-	pyoxidizer build --release --target-triple $tgt
-	mv $bdir/copyparty.exe dist/
-	cp -pv "$(for d in '/c/Program Files (x86)/Microsoft Visual Studio/'*'/BuildTools/VC/Redist/MSVC'; do
-		find "$d" -name vcruntime140.dll; done | sort | grep -vE '/x64/|/onecore/' | head -n 1)" dist/
-	dist/copyparty.exe --version
-	cp -pv dist/copyparty{,.orig}.exe
-	[ $ultra ] && a="--best --lzma" || a=-1
-	/bin/time -f %es upx $a dist/copyparty.exe >/dev/null
-	ls -al dist/copyparty{,.orig}.exe
-	exit 0
-}
-
-
 echo gen tarlist
 for d in copyparty partftpy magic j2 py2 py37 ftp; do find $d -type f || true; done |  # strip_hints
 sed -r 's/(.*)\.(.*)/\2 \1/' | LC_ALL=C sort |
--- a/scripts/pyinstaller/deps.sha512
+++ b/scripts/pyinstaller/deps.sha512
@@ -27,7 +27,7 @@ e3e2e6bd511dec484dd0292f4c46c55c88a885eabf15413d53edea2dd4a4dbae1571735b9424f78c
 e21495f1d473d855103fb4a243095b498ec90eb68776b0f9b48e994990534f7286c0292448e129c507e5d70409f8a05cca58b98d59ce2a815993d0a873dfc480  MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
 8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e  mutagen-1.47.0-py3-none-any.whl
 656015f5cc2c04aa0653ee5609c39a7e5f0b6a58c84fe26b20bd070c52d20b4effb810132f7fb771168483e9fd975cc3302837dd7a1a687ee058b0460c857cc4  packaging-23.2-py3-none-any.whl
-424e20dc7263a31d524307bc39ed755a9dd82f538086fff68d98dd97e236c9b00777a8ac2e3853081b532b0e93cef44983e74d0ab274877440e8b7341b19358a  pillow-10.2.0-cp311-cp311-win_amd64.whl
+1dfe6f66bef5c9d62c9028a964196b902772ec9e19db215f3f41acb8d2d563586988d81b455fa6b895b434e9e1e9d57e4d271d1b1214483bdb3eadffcbba6a33  pillow-10.3.0-cp311-cp311-win_amd64.whl
 8760eab271e79256ae3bfb4af8ccc59010cb5d2eccdd74b325d1a533ae25eb127d51c2ec28ff90d449afed32dd7d6af62934fe9caaf1ae1f4d4831e948e912da  pyinstaller-6.5.0-py3-none-win_amd64.whl
-e6bdbae1affd161e62fc87407c912462dfe875f535ba9f344d0c4ade13715c947cd3ae832eff60f1bad4161938311d06ac8bc9b52ef203f7b0d9de1409f052a5  python-3.11.8-amd64.exe
+897a14d5ee5cbc6781a0f48beffc27807a4f789d58c4329d899233f615d168a5dcceddf7f8f2d5bb52212ddcf3eba4664590d9f1fdb25bb5201f44899e03b2f7  python-3.11.9-amd64.exe
 729dc52f1a02bc6274d012ce33f534102975a828cba11f6029600ea40e2d23aefeb07bf4ae19f9621d0565dd03eb2635bbb97d45fb692c1f756315e8c86c5255  upx-4.2.2-win64.zip
--- a/scripts/pyinstaller/notes.txt
+++ b/scripts/pyinstaller/notes.txt
@@ -27,8 +27,8 @@ fns=(
  MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
  mutagen-1.47.0-py3-none-any.whl
  packaging-23.2-py3-none-any.whl
-  pillow-10.2.0-cp311-cp311-win_amd64.whl
-  python-3.11.8-amd64.exe
+  pillow-10.3.0-cp311-cp311-win_amd64.whl
+  python-3.11.9-amd64.exe
  upx-4.2.2-win64.zip
 )
 [ $w7 ] && fns+=(
--- a/scripts/sfx.ls
+++ b/scripts/sfx.ls
@@ -81,6 +81,7 @@ copyparty/web/dd/5.png,
 copyparty/web/dd/__init__.py,
 copyparty/web/deps,
 copyparty/web/deps/__init__.py,
+copyparty/web/deps/busy.mp3,
 copyparty/web/deps/easymde.css,
 copyparty/web/deps/easymde.js,
 copyparty/web/deps/marked.js,
--- a/scripts/sfx.py
+++ b/scripts/sfx.py
@@ -234,8 +234,9 @@ def u8(gen):


 def yieldfile(fn):
-    with open(fn, "rb") as f:
-        for block in iter(lambda: f.read(64 * 1024), b""):
+    s = 64 * 1024
+    with open(fn, "rb", s * 4) as f:
+        for block in iter(lambda: f.read(s), b""):
            yield block


--- a/tests/res/idp/6.conf
+++ b/tests/res/idp/6.conf
@@ -0,0 +1,24 @@
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
+[global]
+  idp-h-usr: x-idp-user
+  idp-h-grp: x-idp-group
+
+[/get/${u}]
+  /get/${u}
+  accs:
+    g: *
+    r: ${u}, @su
+    m: @su
+
+[/priv/${u}]
+  /priv/${u}
+  accs:
+    r: ${u}, @su
+    m: @su
+
+[/team/${g}/${u}]
+  /team/${g}/${u}
+  accs:
+    r: @${g}
--- a/tests/test_dots.py
+++ b/tests/test_dots.py
@@ -4,6 +4,9 @@ from __future__ import print_function, unicode_literals

 import io
 import os
+import time
+import json
+import pprint
 import shutil
 import tarfile
 import tempfile
@@ -49,11 +52,7 @@ class TestHttpCli(unittest.TestCase):
                with open(filepath, "wb") as f:
                    f.write(filepath.encode("utf-8"))

-        vcfg = [
-            ".::r,u1:r.,u2",
-            "a:a:r,u1:r,u2",
-            ".b:.b:r.,u1:r,u2"
-        ]
+        vcfg = [".::r,u1:r.,u2", "a:a:r,u1:r,u2", ".b:.b:r.,u1:r,u2"]
        self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"], e2dsa=True)
        self.asrv = AuthSrv(self.args, self.log)

@@ -66,7 +65,9 @@ class TestHttpCli(unittest.TestCase):

        self.assertEqual(self.curl("?tar", "x")[1][:17], "\nJ2EOT")

-        # search
+        ##
+        ## search
+
        up2k = Up2k(self)
        u2idx = U2idx(self)
        allvols = list(self.asrv.vfs.all_vols.values())
@@ -91,15 +92,55 @@ class TestHttpCli(unittest.TestCase):
        xe = "a/da/f4 a/f3 f0 t/f1"
        self.assertEqual(x, xe)

-    def tardir(self, url, uname):
-        h, b = self.curl("/" + url + "?tar", uname, True)
-        tar = tarfile.open(fileobj=io.BytesIO(b), mode="r|").getnames()
-        top = ("top" if not url else url.lstrip(".").split("/")[0]) + "/"
-        assert len(tar) == len([x for x in tar if x.startswith(top)])
-        return " ".join([x[len(top):] for x in tar])
+        ##
+        ## dirkeys

-    def curl(self, url, uname, binary=False):
-        conn = tu.VHttpConn(self.args, self.asrv, self.log, hdr(url, uname))
+        os.mkdir("v")
+        with open("v/f1.txt", "wb") as f:
+            f.write(b"a")
+        os.rename("a", "v/a")
+        os.rename(".b", "v/.b")
+
+        vcfg = [
+            ".::r.,u1:g,u2:c,dk",
+            "v/a:v/a:r.,u1:g,u2:c,dk",
+            "v/.b:v/.b:r.,u1:g,u2:c,dk",
+        ]
+        self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"])
+        self.asrv = AuthSrv(self.args, self.log)
+        zj = json.loads(self.curl("?ls", "u1")[1])
+        url = "?k=" + zj["dk"]
+        # should descend into folders, but not other volumes:
+        self.assertEqual(self.tardir(url, "u2"), "f0 t/f1 v/f1.txt")
+
+        zj = json.loads(self.curl("v?ls", "u1")[1])
+        url = "v?k=" + zj["dk"]
+        self.assertEqual(self.tarsel(url, "u2", ["f1.txt", "a", ".b"]), "f1.txt")
+
+    def tardir(self, url, uname):
+        top = url.split("?")[0]
+        top = ("top" if not top else top.lstrip(".").split("/")[0]) + "/"
+        url += ("&" if "?" in url else "?") + "tar"
+        h, b = self.curl(url, uname, True)
+        tar = tarfile.open(fileobj=io.BytesIO(b), mode="r|").getnames()
+        if len(tar) != len([x for x in tar if x.startswith(top)]):
+            raise Exception("bad-prefix:", tar)
+        return " ".join([x[len(top) :] for x in tar])
+
+    def tarsel(self, url, uname, sel):
+        url += ("&" if "?" in url else "?") + "tar"
+        zs = '--XD\r\nContent-Disposition: form-data; name="act"\r\n\r\nzip\r\n--XD\r\nContent-Disposition: form-data; name="files"\r\n\r\n'
+        zs += "\r\n".join(sel) + "\r\n--XD--\r\n"
+        zb = zs.encode("utf-8")
+        hdr = "POST /%s HTTP/1.1\r\nPW: %s\r\nConnection: close\r\nContent-Type: multipart/form-data; boundary=XD\r\nContent-Length: %d\r\n\r\n"
+        req = (hdr % (url, uname, len(zb))).encode("utf-8") + zb
+        h, b = self.curl("/" + url, uname, True, req)
+        tar = tarfile.open(fileobj=io.BytesIO(b), mode="r|").getnames()
+        return " ".join(tar)
+
+    def curl(self, url, uname, binary=False, req=b""):
+        req = req or hdr(url, uname)
+        conn = tu.VHttpConn(self.args, self.asrv, self.log, req)
        HttpCli(conn).run()
        if binary:
            h, b = conn.s._reply.split(b"\r\n\r\n", 1)
--- a/tests/test_idp.py
+++ b/tests/test_idp.py
@@ -15,6 +15,16 @@ class TestVFS(unittest.TestCase):
        print(json.dumps(vfs, indent=4, sort_keys=True, default=lambda o: o.__dict__))

    def log(self, src, msg, c=0):
+        m = "%s" % (msg,)
+        if (
+            "warning: filesystem-path does not exist:" in m
+            or "you are sharing a system directory:" in m
+            or "reinitializing due to new user from IdP:" in m
+            or m.startswith("hint: argument")
+            or (m.startswith("loaded ") and " config files:" in m)
+        ):
+            return
+
        print(("[%s] %s" % (src, msg)).encode("ascii", "replace").decode("ascii"))

    def nav(self, au, vp):
@@ -30,21 +40,23 @@ class TestVFS(unittest.TestCase):
        self.assertEqual(unpacked, expected + [[]] * pad)

    def assertAxsAt(self, au, vp, expected):
-        self.assertAxs(self.nav(au, vp).axs, expected)
+        vn = self.nav(au, vp)
+        self.assertAxs(vn.axs, expected)

    def assertNodes(self, vfs, expected):
        got = list(sorted(vfs.nodes.keys()))
        self.assertEqual(got, expected)

    def assertNodesAt(self, au, vp, expected):
-        self.assertNodes(self.nav(au, vp), expected)
+        vn = self.nav(au, vp)
+        self.assertNodes(vn, expected)

    def prep(self):
        here = os.path.abspath(os.path.dirname(__file__))
        cfgdir = os.path.join(here, "res", "idp")

        # globals are applied by main so need to cheat a little
-        xcfg = { "idp_h_usr": "x-idp-user", "idp_h_grp": "x-idp-group" }
+        xcfg = {"idp_h_usr": "x-idp-user", "idp_h_grp": "x-idp-group"}

        return here, cfgdir, xcfg

@@ -140,6 +152,11 @@ class TestVFS(unittest.TestCase):
        self.assertEqual(self.nav(au, "vg/iga1").realpath, "/g1-iga")
        self.assertEqual(self.nav(au, "vg/iga2").realpath, "/g2-iga")

+        au.idp_checkin(None, "iub", "iga")
+        self.assertAxsAt(au, "vu/iua", [["iua"]])
+        self.assertAxsAt(au, "vg/iga1", [["iua", "iub"]])
+        self.assertAxsAt(au, "vg/iga2", [["iua", "iub", "ua"]])
+
    def test_5(self):
        """
        one IdP user in multiple groups
@@ -169,3 +186,44 @@ class TestVFS(unittest.TestCase):
        self.assertAxsAt(au, "g", [["iua"]])
        self.assertAxsAt(au, "ga", [["iua"]])
        self.assertAxsAt(au, "gb", [["iua"]])
+
+    def test_6(self):
+        """
+        IdP volumes with anon-get and other users/groups (github#79)
+        """
+        _, cfgdir, xcfg = self.prep()
+        au = AuthSrv(Cfg(c=[cfgdir + "/6.conf"], **xcfg), self.log)
+
+        self.assertAxs(au.vfs.axs, [])
+        self.assertEqual(au.vfs.vpath, "")
+        self.assertEqual(au.vfs.realpath, "")
+        self.assertNodes(au.vfs, [])
+
+        au.idp_checkin(None, "iua", "")
+        star = ["*", "iua"]
+        self.assertNodes(au.vfs, ["get", "priv"])
+        self.assertAxsAt(au, "get/iua", [["iua"], [], [], [], star])
+        self.assertAxsAt(au, "priv/iua", [["iua"], [], []])
+
+        au.idp_checkin(None, "iub", "")
+        star = ["*", "iua", "iub"]
+        self.assertNodes(au.vfs, ["get", "priv"])
+        self.assertAxsAt(au, "get/iua", [["iua"], [], [], [], star])
+        self.assertAxsAt(au, "get/iub", [["iub"], [], [], [], star])
+        self.assertAxsAt(au, "priv/iua", [["iua"], [], []])
+        self.assertAxsAt(au, "priv/iub", [["iub"], [], []])
+
+        au.idp_checkin(None, "iuc", "su")
+        star = ["*", "iua", "iub", "iuc"]
+        self.assertNodes(au.vfs, ["get", "priv", "team"])
+        self.assertAxsAt(au, "get/iua", [["iua", "iuc"], [], ["iuc"], [], star])
+        self.assertAxsAt(au, "get/iub", [["iub", "iuc"], [], ["iuc"], [], star])
+        self.assertAxsAt(au, "get/iuc", [["iuc"], [], ["iuc"], [], star])
+        self.assertAxsAt(au, "priv/iua", [["iua", "iuc"], [], ["iuc"]])
+        self.assertAxsAt(au, "priv/iub", [["iub", "iuc"], [], ["iuc"]])
+        self.assertAxsAt(au, "priv/iuc", [["iuc"], [], ["iuc"]])
+        self.assertAxsAt(au, "team/su/iuc", [["iuc"]])
+
+        au.idp_checkin(None, "iud", "su")
+        self.assertAxsAt(au, "team/su/iuc", [["iuc", "iud"]])
+        self.assertAxsAt(au, "team/su/iud", [["iuc", "iud"]])
--- a/tests/util.py
+++ b/tests/util.py
@@ -145,8 +145,10 @@ class Cfg(Namespace):
            c=c,
            E=E,
            dbd="wal",
+            dk_salt="b" * 16,
            fk_salt="a" * 16,
            idp_gsep=re.compile("[|:;+,]"),
+            iobuf=256 * 1024,
            lang="eng",
            log_badpwd=1,
            logout=573,
@@ -154,7 +156,8 @@ class Cfg(Namespace):
            mth={},
            mtp=[],
            rm_retry="0/0",
-            s_wr_sz=512 * 1024,
+            s_rd_sz=256 * 1024,
+            s_wr_sz=256 * 1024,
            sort="href",
            srch_hits=99999,
            th_crop="y",
@@ -254,4 +257,4 @@ class VHttpConn(object):
        self.thumbcli = None
        self.u2fh = FHC()

-        self.get_u2idx = self.hsrv.get_u2idx
+        self.get_u2idx = self.hsrv.get_u2idx
Author	SHA1	Message	Date
ed	a0022805d1	v1.12.0 (closes #64 )	2024-04-06 20:11:49 +00:00
ed	853adb5d04	update deps	2024-04-06 19:51:38 +00:00
ed	7744226b5c	apply audio equalizer to videos too	2024-04-06 18:44:08 +00:00
ed	d94b5b3fc9	fau doesn't work on iphones; compensate by preloading much earlier	2024-04-06 18:43:45 +00:00
ed	e6ba065bc2	improve cachebusters	2024-04-06 00:27:06 +00:00
ed	59a53ba9ac	on phones, fix playback halting if next song didn't buffer in time	2024-04-06 00:25:28 +00:00
ed	b88cc7b5ce	turns out it doesn't need to be audible...	2024-04-05 23:06:26 +00:00
ed	5ab54763c6	remove pyoxidizer (unmaintained) partially reverts `e430b2567a` the remaining stuff might be useful for other cpython alternatives	2024-04-05 17:51:26 +00:00
ed	59f815ff8c	deps: add busy.mp3	2024-04-04 09:27:01 +00:00
ed	9c42cbec6f	maybe fix #81	2024-04-03 00:28:15 +00:00
ed	f471b05aa4	todo: fix playback stopping on phones if slow preload	2024-04-02 23:20:58 +00:00
ed	34c32e3e89	golf: util.js ensures `WebAssembly`, `Notification`, and `FormData` are always declared, setting them false when not available	2024-04-02 20:25:06 +00:00
ed	a080759a03	add transcoding to mp3 because CU's car stereo can't do opus... incidentally adds support for playing any audio format in ie11	2024-03-29 16:36:56 +00:00
ed	0ae12868e5	dirkeys: add volflag dky (skip keycheck)	2024-03-27 21:03:58 +00:00
ed	ef52e2c06c	dirkeys: fix 403 in dks volumes	2024-03-27 20:34:34 +00:00
ed	32c912bb16	fix a bunch of dirkey stuff: * breadcrumb navigation * tree generation in `recvls` * dirkeys in initial tree	2024-03-27 16:05:05 +00:00
ed	20870fda79	Merge branch 'dirkeys' into hovudstraum	2024-03-25 10:34:08 +00:00
ed	bdfe2c1a5f	mention unproductive optimizations	2024-03-24 22:07:23 +00:00
ed	cb99fbf442	update pkgs to 1.11.2	2024-03-23 17:53:19 +00:00
ed	bccc44dc21	v1.11.2	2024-03-23 17:24:36 +00:00
ed	2f20d29edd	idp: mention lack of volume persistence	2024-03-23 16:35:45 +00:00
ed	c6acd3a904	add option `--s-rd-sz` (socket read size): counterpart of `--s-wr-sz` which existed already the default (256 KiB) appears optimal in the most popular scenario (linux host with storage on local physical disk, usually NVMe) was previously 32 KiB, so large uploads should now use 17% less CPU also adds sanchecks for values of `--iobuf`, `--s-rd-sz`, `--s-wr-sz` also adds file-overwrite feature for multipart posts	2024-03-23 16:35:14 +00:00
ed	2b24c50eb7	add option `--iobuf` (file r/w buffersize): the default (256 KiB) appears optimal in the most popular scenario (linux host with storage on local physical disk, usually NVMe) was previously a mix of 64 and 512 KiB; now the same value is enforced everywhere download-as-tar is now 20% faster with the default value	2024-03-23 16:17:40 +00:00
ed	d30ae8453d	idp: precise expansion of `${u}` (fixes #79 ); it is now possible to grant access to users other than `${u}` (the user which the volume belongs to) previously, permissions did not apply correctly to IdP volumes due to the way `${u}` and `${g}` was expanded, which was a funky iteration over all known users/groups instead of... just expanding them? also adds another sanchk that a volume's URL must contain a `${u}` to be allowed to mention `${u}` in the accs list, and similarly for `${g}` / `@${g}` since users can be in multiple groups	2024-03-21 20:10:27 +00:00
ed	8e5c436bef	black + isort	2024-03-21 18:51:23 +00:00
ed	f500e55e68	update pkgs to 1.11.1	2024-03-18 17:41:43 +00:00
ed	10bc2d9205	unsuccessful attempt at dirkeys (#64 )	2023-12-17 22:30:22 +00:00