v1.9.5

smb: upgrade to impacket 0.11, full user account support,
permissions are now per-account instead of coalescing also stops windows from freaking out if there's an offline volume
2023-09-09 13:36:56 +00:00 · 2023-09-09 12:46:37 +00:00 · 2023-09-08 21:42:05 +00:00 · 2023-09-08 21:13:23 +00:00 · 2023-09-08 19:49:29 +00:00 · 2023-09-08 19:35:12 +00:00
26 changed files with 519 additions and 137 deletions
--- a/README.md
+++ b/README.md
@@ -285,8 +285,11 @@ server notes:
 * Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)

 * iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
-  * *future workaround:* enable the equalizer, make it all-zero, and set a negative boost to reduce the volume
-    * "future" because `AudioContext` can't maintain a stable playback speed in the current iOS version (15.7), maybe one day...
+  * `AudioContext` will probably never be a viable workaround as apple introduces new issues faster than they fix current ones
+
+* iPhones: the preload feature (in the media-player-options tab) can cause a tiny audio glitch 20sec before the end of each song, but disabling it may cause worse iOS bugs to appear instead
+  * just a hunch, but disabling preloading may cause playback to stop entirely, or possibly mess with bluetooth speakers
+  * tried to add a tooltip regarding this but looks like apple broke my tooltips

 * Windows: folders cannot be accessed if the name ends with `.`
  * python or windows bug
@@ -348,6 +351,7 @@ permissions:
 * `d` (delete): delete files/folders
 * `g` (get): only download files, cannot see folder contents or zip/tar
 * `G` (upget): same as `g` except uploaders get to see their own filekeys (see `fk` in examples below)
+* `h` (html): same as `g` except folders return their index.html, and filekeys are not necessary for index.html
 * `a` (admin): can see uploader IPs, config-reload

 examples:
@@ -506,10 +510,16 @@ select which type of archive you want in the `[⚙️] config` tab:
 | name | url-suffix | description |
 |--|--|--|
 | `tar` | `?tar` | plain gnutar, works great with `curl \| tar -xv` |
+| `pax` | `?tar=pax` | pax-format tar, futureproof, not as fast |
+| `tgz` | `?tar=gz` | gzip compressed gnu-tar (slow), for `curl \| tar -xvz` |
+| `txz` | `?tar=xz` | gnu-tar with xz / lzma compression (v.slow) |
 | `zip` | `?zip=utf8` | works everywhere, glitchy filenames on win7 and older |
 | `zip_dos` | `?zip` | traditional cp437 (no unicode) to fix glitchy filenames |
 | `zip_crc` | `?zip=crc` | cp437 with crc32 computed early for truly ancient software |

+* gzip default level is `3` (0=fast, 9=best), change with `?tar=gz:9`
+* xz default level is `1` (0=fast, 9=best), change with `?tar=xz:9`
+* bz2 default level is `2` (1=fast, 9=best), change with `?tar=bz2:9`
 * hidden files (dotfiles) are excluded unless `-ed`
  * `up2k.db` and `dir.txt` is always excluded
 * `zip_crc` will take longer to download since the server has to read each file twice
@@ -721,6 +731,8 @@ can also boost the volume in general, or increase/decrease stereo width (like [c

 has the convenient side-effect of reducing the pause between songs, so gapless albums play better with the eq enabled (just make it flat)

+not available on iPhones / iPads because AudioContext currently breaks background audio playback on iOS (15.7.8)
+

 ### fix unreliable playback on android

@@ -892,15 +904,13 @@ unsafe, slow, not recommended for wan,  enable with `--smb` for read-only or `--

 click the [connect](http://127.0.0.1:3923/?hc) button in the control-panel to see connection instructions for windows, linux, macos

-dependencies: `python3 -m pip install --user -U impacket==0.10.0`
+dependencies: `python3 -m pip install --user -U impacket==0.11.0`
 * newer versions of impacket will hopefully work just fine but there is monkeypatching so maybe not

 some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
 * not entirely confident that read-only is read-only
 * the smb backend is not fully integrated with vfs, meaning there could be security issues (path traversal). Please use `--smb-port` (see below) and [prisonparty](./bin/prisonparty.sh)
-  * account passwords work per-volume as expected, but account permissions are coalesced; all accounts have read-access to all volumes, and if a single account has write-access to some volume then all other accounts also do
-    * if no accounts have write-access to a specific volume, or if `--smbw` is not set, then writing to that volume from smb *should* be impossible
-    * will be fixed once [impacket v0.11.0](https://github.com/SecureAuthCorp/impacket/commit/d923c00f75d54b972bca573a211a82f09b55261a) is released
+  * account passwords work per-volume as expected, and so does account permissions (read/write/move/delete), but `--smbw` must be given to allow write-access from smb
  * [shadowing](#shadowing) probably works as expected but no guarantees

 and some minor issues,
@@ -911,7 +921,7 @@ and some minor issues,
  * win10 onwards does not allow connecting anonymously / without accounts
 * on windows, creating a new file through rightclick --> new --> textfile throws an error due to impacket limitations -- hit OK and F5 to get your file
 * python3 only
-* slow
+* slow (the builtin webdav support in windows is 5x faster, and rclone-webdav is 30x faster)

 known client bugs:
 * on win7 only, `--smb1` is much faster than smb2 (default) because it keeps rescanning folders on smb2
@@ -1630,6 +1640,8 @@ other misc notes:
  * combine this with volflag `c,fk` to generate filekeys (per-file accesskeys); users which have full read-access will then see URLs with `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
    * the default filekey entropy is fairly small so give `--fk-salt` around 30 characters if you want filekeys longer than 16 chars
  * permissions `wG` lets users upload files and receive their own filekeys, still without being able to see other uploads
+  * permission `h` instead of `r` makes copyparty behave like a traditional webserver with directory listing/index disabled, returning index.html instead
+    * compatibility with filekeys: index.html itself can be retrieved without the correct filekey, but all other files are protected


 ## gotchas
@@ -1733,7 +1745,7 @@ enable [thumbnails](#thumbnails) of...
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`

 enable [smb](#smb-server) support (**not** recommended):
-* `impacket==0.10.0`
+* `impacket==0.11.0`

 `pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`

--- a/contrib/nixos/modules/copyparty.nix
+++ b/contrib/nixos/modules/copyparty.nix
@@ -138,7 +138,8 @@ in {
                "d" (delete): permanently delete files and folders
                "g" (get):    download files, but cannot see folder contents
                "G" (upget):  "get", but can see filekeys of their own uploads
-                "a" (upget):  can see uploader IPs, config-reload
+                "h" (html):   "get", but folders return their index.html
+                "a" (admin):  can see uploader IPs, config-reload

              For example: "rwmd"

--- a/contrib/package/arch/PKGBUILD
+++ b/contrib/package/arch/PKGBUILD
@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.9.1"
+pkgver="1.9.4"
 pkgrel=1
 pkgdesc="Portable file sharing hub"
 arch=("any")
@@ -20,7 +20,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("95466e2fdc172ce4bc98cbbaec44ce32a9851d15b4eb0f32ad6bf98ed89fbf14")
+sha256sums=("c327ac35deaa5e6cc86b3b1a251cc78517be5578c37c4dff90e98465ced82abc")

 build() {
    cd "${srcdir}/${pkgname}-${pkgver}"
--- a/contrib/package/nix/copyparty/pin.json
+++ b/contrib/package/nix/copyparty/pin.json
@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.9.1/copyparty-sfx.py",
-    "version": "1.9.1",
-    "hash": "sha256-dhFc3sjRIF0ZEmQO09OEvLpDyb5ugwaaIvVKJYHfQFQ="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.9.4/copyparty-sfx.py",
+    "version": "1.9.4",
+    "hash": "sha256-17dBvr6uUzaQtzunZUX84BLTWCGe1Hucz9b48BKowHs="
 }
--- a/copyparty/main.py
+++ b/copyparty/main.py
@@ -492,6 +492,7 @@ def get_sects():
              "d" (delete): permanently delete files and folders
              "g" (get):    download files, but cannot see folder contents
              "G" (upget):  "get", but can see filekeys of their own uploads
+              "h" (html):   "get", but folders return their index.html
              "a" (admin):  can see uploader IPs, config-reload

            too many volflags to list here, see --help-flags
@@ -931,12 +932,13 @@ def add_webdav(ap):

 def add_smb(ap):
    ap2 = ap.add_argument_group('SMB/CIFS options')
-    ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless --smb-port is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is dangerous! Never expose to the internet. Account permissions are coalesced; if one account has write-access to a volume, then all accounts do.")
+    ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless --smb-port is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is dangerous! Never expose to the internet!")
    ap2.add_argument("--smbw", action="store_true", help="enable write support (please dont)")
    ap2.add_argument("--smb1", action="store_true", help="disable SMBv2, only enable SMBv1 (CIFS)")
    ap2.add_argument("--smb-port", metavar="PORT", type=int, default=445, help="port to listen on -- if you change this value, you must NAT from TCP:445 to this port using iptables or similar")
    ap2.add_argument("--smb-nwa-1", action="store_true", help="disable impacket#1433 workaround (truncate directory listings to 64kB)")
    ap2.add_argument("--smb-nwa-2", action="store_true", help="disable impacket workaround for filecopy globs")
+    ap2.add_argument("--smba", action="store_true", help="small performance boost: disable per-account permissions, enables account coalescing instead (if one user has write/delete-access, then everyone does)")
    ap2.add_argument("--smbv", action="store_true", help="verbose")
    ap2.add_argument("--smbvv", action="store_true", help="verboser")
    ap2.add_argument("--smbvvv", action="store_true", help="verbosest")
@@ -989,6 +991,7 @@ def add_optouts(ap):
    ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI")
    ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
    ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
+    ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
    ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (as specified by the 'lifetime' volflag)")


@@ -1149,7 +1152,7 @@ def add_ui(ap, retry):
    ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
    ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
    ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
-    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI)")
+    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
    ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
    ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
    ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty @ --name", help="title / service-name to show in html documents")
@@ -1397,7 +1400,7 @@ def main(argv: Optional[list[str]] = None) -> None:
            if re.match("c[^,]", opt):
                mod = True
                na.append("c," + opt[1:])
-            elif re.sub("^[rwmdgGa]*", "", opt) and "," not in opt:
+            elif re.sub("^[rwmdgGha]*", "", opt) and "," not in opt:
                mod = True
                perm = opt[0]
                na.append(perm + "," + opt[1:])
--- a/copyparty/version.py
+++ b/copyparty/version.py
@@ -1,8 +1,8 @@
 # coding: utf-8

-VERSION = (1, 9, 2)
+VERSION = (1, 9, 5)
 CODENAME = "prometheable"
-BUILD_DT = (2023, 8, 26)
+BUILD_DT = (2023, 9, 9)

 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
--- a/copyparty/authsrv.py
+++ b/copyparty/authsrv.py
@@ -67,6 +67,7 @@ class AXS(object):
        udel: Optional[Union[list[str], set[str]]] = None,
        uget: Optional[Union[list[str], set[str]]] = None,
        upget: Optional[Union[list[str], set[str]]] = None,
+        uhtml: Optional[Union[list[str], set[str]]] = None,
        uadmin: Optional[Union[list[str], set[str]]] = None,
    ) -> None:
        self.uread: set[str] = set(uread or [])
@@ -75,10 +76,11 @@ class AXS(object):
        self.udel: set[str] = set(udel or [])
        self.uget: set[str] = set(uget or [])
        self.upget: set[str] = set(upget or [])
+        self.uhtml: set[str] = set(uhtml or [])
        self.uadmin: set[str] = set(uadmin or [])

    def __repr__(self) -> str:
-        ks = "uread uwrite umove udel uget upget uadmin".split()
+        ks = "uread uwrite umove udel uget upget uhtml uadmin".split()
        return "AXS(%s)" % (", ".join("%s=%r" % (k, self.__dict__[k]) for k in ks),)


@@ -329,6 +331,7 @@ class VFS(object):
        self.adel: dict[str, list[str]] = {}
        self.aget: dict[str, list[str]] = {}
        self.apget: dict[str, list[str]] = {}
+        self.ahtml: dict[str, list[str]] = {}
        self.aadmin: dict[str, list[str]] = {}

        if realpath:
@@ -456,6 +459,7 @@ class VFS(object):
            uname in c.upget or "*" in c.upget,
            uname in c.uadmin or "*" in c.uadmin,
        )
+        # skip uhtml because it's rarely needed

    def get(
        self,
@@ -476,7 +480,8 @@ class VFS(object):
                    self.log("vfs", "invalid relpath [{}]".format(vpath))
                raise Pebkac(404)

-        vn, rem = self._find(undot(vpath))
+        cvpath = undot(vpath)
+        vn, rem = self._find(cvpath)
        c: AXS = vn.axs

        for req, d, msg in [
@@ -487,6 +492,11 @@ class VFS(object):
            (will_get, c.uget, "get"),
        ]:
            if req and (uname not in d and "*" not in d) and uname != LEELOO_DALLAS:
+                if vpath != cvpath and vpath != "." and self.log:
+                    ap = vn.canonical(rem)
+                    t = "{} has no {} in [{}] => [{}] => [{}]"
+                    self.log("vfs", t.format(uname, msg, vpath, cvpath, ap), 6)
+
                t = "you don't have {}-access for this location"
                raise Pebkac(err, t.format(msg))

@@ -949,7 +959,7 @@ class AuthSrv(object):
                try:
                    self._l(ln, 5, "volume access config:")
                    sk, sv = ln.split(":")
-                    if re.sub("[rwmdgGa]", "", sk) or not sk:
+                    if re.sub("[rwmdgGha]", "", sk) or not sk:
                        err = "invalid accs permissions list; "
                        raise Exception(err)
                    if " " in re.sub(", *", "", sv).strip():
@@ -958,7 +968,7 @@ class AuthSrv(object):
                    self._read_vol_str(sk, sv.replace(" ", ""), daxs[vp], mflags[vp])
                    continue
                except:
-                    err += "accs entries must be 'rwmdgGa: user1, user2, ...'"
+                    err += "accs entries must be 'rwmdgGha: user1, user2, ...'"
                    raise Exception(err + SBADCFG)

            if cat == catf:
@@ -994,7 +1004,7 @@ class AuthSrv(object):
    def _read_vol_str(
        self, lvl: str, uname: str, axs: AXS, flags: dict[str, Any]
    ) -> None:
-        if lvl.strip("crwmdgGa"):
+        if lvl.strip("crwmdgGha"):
            raise Exception("invalid volflag: {},{}".format(lvl, uname))

        if lvl == "c":
@@ -1023,10 +1033,12 @@ class AuthSrv(object):
                ("w", axs.uwrite),
                ("m", axs.umove),
                ("d", axs.udel),
+                ("a", axs.uadmin),
+                ("h", axs.uhtml),
+                ("h", axs.uget),
                ("g", axs.uget),
                ("G", axs.uget),
                ("G", axs.upget),
-                ("a", axs.uadmin),
            ]:  # b bb bbb
                if ch in lvl:
                    if un == "*":
@@ -1099,7 +1111,7 @@ class AuthSrv(object):

        if self.args.v:
            # list of src:dst:permset:permset:...
-            # permset is <rwmdgGa>[,username][,username] or <c>,<flag>[=args]
+            # permset is <rwmdgGha>[,username][,username] or <c>,<flag>[=args]
            for v_str in self.args.v:
                m = re_vol.match(v_str)
                if not m:
@@ -1188,7 +1200,7 @@ class AuthSrv(object):
            vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
            vol.root = vfs

-        for perm in "read write move del get pget admin".split():
+        for perm in "read write move del get pget html admin".split():
            axs_key = "u" + perm
            unames = ["*"] + list(acct.keys())
            umap: dict[str, list[str]] = {x: [] for x in unames}
@@ -1210,6 +1222,7 @@ class AuthSrv(object):
                axs.udel,
                axs.uget,
                axs.upget,
+                axs.uhtml,
                axs.uadmin,
            ]:
                for usr in d:
@@ -1631,6 +1644,7 @@ class AuthSrv(object):
                ["delete", "udel"],
                ["   get", "uget"],
                [" upget", "upget"],
+                ["  html", "uhtml"],
                ["uadmin", "uadmin"],
            ]:
                u = list(sorted(getattr(zv.axs, attr)))
@@ -1669,7 +1683,7 @@ class AuthSrv(object):
                self.log(t.format(zv.realpath), c=1)

        try:
-            zv, _ = vfs.get("/", "*", False, True)
+            zv, _ = vfs.get("", "*", False, True, err=999)
            if self.warn_anonwrite and os.getcwd() == zv.realpath:
                t = "anyone can write to the current directory: {}\n"
                self.log(t.format(zv.realpath), c=1)
@@ -1798,6 +1812,7 @@ class AuthSrv(object):
                vc.udel,
                vc.uget,
                vc.upget,
+                vc.uhtml,
                vc.uadmin,
            ]
            self.log(t.format(*vs))
@@ -1939,6 +1954,7 @@ class AuthSrv(object):
                "d": "udel",
                "g": "uget",
                "G": "upget",
+                "h": "uhtml",
                "a": "uadmin",
            }
            users = {}
@@ -2142,7 +2158,7 @@ def upgrade_cfg_fmt(
            else:
                sn = sn.replace(",", ", ")
            ret.append("    " + sn)
-        elif sn[:1] in "rwmdgGa":
+        elif sn[:1] in "rwmdgGha":
            if cat != catx:
                cat = catx
                ret.append(cat)
--- a/copyparty/cfg.py
+++ b/copyparty/cfg.py
@@ -62,6 +62,8 @@ permdescs = {
    "d": "delete; permanently delete files and folders",
    "g": "get; download files, but cannot see folder contents",
    "G": 'upget; same as "g" but can see filekeys of their own uploads',
+    "h": 'html; same as "g" but folders return their index.html',
+    "a": "admin; can see uploader IPs, config-reload",
 }


--- a/copyparty/ftpd.py
+++ b/copyparty/ftpd.py
@@ -9,12 +9,19 @@ import stat
 import sys
 import time

+from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
+
+try:
+    import asynchat
+except:
+    sys.path.append(os.path.join(E.mod, "vend"))
+
 from pyftpdlib.authorizers import AuthenticationFailed, DummyAuthorizer
 from pyftpdlib.filesystems import AbstractedFS, FilesystemError
 from pyftpdlib.handlers import FTPHandler
+from pyftpdlib.ioloop import IOLoop
 from pyftpdlib.servers import FTPServer

-from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
 from .authsrv import VFS
 from .bos import bos
 from .util import (
@@ -30,15 +37,6 @@ from .util import (
    vjoin,
 )

-try:
-    from pyftpdlib.ioloop import IOLoop
-except ImportError:
-    p = os.path.join(E.mod, "vend")
-    print("loading asynchat from " + p)
-    sys.path.append(p)
-    from pyftpdlib.ioloop import IOLoop
-
-
 if TYPE_CHECKING:
    from .svchub import SvcHub

--- a/copyparty/httpcli.py
+++ b/copyparty/httpcli.py
@@ -333,10 +333,12 @@ class HttpCli(object):
        # split req into vpath + uparam
        uparam = {}
        if "?" not in self.req:
-            self.trailing_slash = self.req.endswith("/")
-            vpath = undot(self.req)
+            vpath = unquotep(self.req)  # not query, so + means +
+            self.trailing_slash = vpath.endswith("/")
+            vpath = undot(vpath)
        else:
            vpath, arglist = self.req.split("?", 1)
+            vpath = unquotep(vpath)
            self.trailing_slash = vpath.endswith("/")
            vpath = undot(vpath)

@@ -351,6 +353,8 @@ class HttpCli(object):
            for k in arglist.split("&"):
                if "=" in k:
                    k, zs = k.split("=", 1)
+                    # x-www-form-urlencoded (url query part) uses
+                    # either + or %20 for 0x20 so handle both
                    uparam[k.lower()] = unquotep(zs.strip().replace("+", " "))
                else:
                    uparam[k.lower()] = ""
@@ -385,7 +389,7 @@ class HttpCli(object):

        self.uparam = uparam
        self.cookies = cookies
-        self.vpath = unquotep(vpath)  # not query, so + means +
+        self.vpath = vpath
        self.vpaths = (
            self.vpath + "/" if self.trailing_slash and self.vpath else self.vpath
        )
@@ -564,8 +568,8 @@ class HttpCli(object):
            self.out_headers.update(NO_CACHE)
            return

-        n = "604869" if cache == "i" else cache or "69"
-        self.out_headers["Cache-Control"] = "max-age=" + n
+        n = 69 if not cache else 604869 if cache == "i" else int(cache)
+        self.out_headers["Cache-Control"] = "max-age=" + str(n)

    def k304(self) -> bool:
        k304 = self.cookies.get("k304")
@@ -625,7 +629,17 @@ class HttpCli(object):
        headers: Optional[dict[str, str]] = None,
        volsan: bool = False,
    ) -> bytes:
-        if status > 400 and status in (403, 404, 422):
+        if (
+            status > 400
+            and status in (403, 404, 422)
+            and (
+                status != 422
+                or (
+                    not body.startswith(b"<pre>partial upload exists")
+                    and not body.startswith(b"<pre>source file busy")
+                )
+            )
+        ):
            if status == 404:
                g = self.conn.hsrv.g404
            elif status == 403:
@@ -668,6 +682,11 @@ class HttpCli(object):
        if volsan:
            vols = list(self.asrv.vfs.all_vols.values())
            body = vol_san(vols, body)
+            try:
+                zs = absreal(__file__).rsplit(os.path.sep, 2)[0]
+                body = body.replace(zs.encode("utf-8"), b"PP")
+            except:
+                pass

        self.send_headers(len(body), status, mime, headers)

@@ -2177,7 +2196,8 @@ class HttpCli(object):
        vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
        self._assert_safe_rem(rem)

-        if not new_file.endswith(".md"):
+        ext = "" if "." not in new_file else new_file.split(".")[-1]
+        if not ext or len(ext) > 5 or not self.can_delete:
            new_file += ".md"

        sanitized = sanitize_fn(new_file, "", [])
@@ -2510,7 +2530,7 @@ class HttpCli(object):
        fp = os.path.join(fp, fn)
        rem = "{}/{}".format(rp, fn).strip("/")

-        if not rem.endswith(".md"):
+        if not rem.endswith(".md") and not self.can_delete:
            raise Pebkac(400, "only markdown pls")

        if nullwrite:
@@ -2561,7 +2581,8 @@ class HttpCli(object):
                return True

            mdir, mfile = os.path.split(fp)
-            mfile2 = "{}.{:.3f}.md".format(mfile[:-3], srv_lastmod)
+            fname, fext = mfile.rsplit(".", 1) if "." in mfile else (mfile, "md")
+            mfile2 = "{}.{:.3f}.{}".format(fname, srv_lastmod, fext)
            try:
                dp = os.path.join(mdir, ".hist")
                bos.mkdir(dp)
@@ -2886,12 +2907,26 @@ class HttpCli(object):
        logmsg = "{:4} {} ".format("", self.req)
        self.keepalive = False

+        cancmp = not self.args.no_tarcmp
+
        if fmt == "tar":
-            mime = "application/x-tar"
            packer: Type[StreamArc] = StreamTar
+            if cancmp and "gz" in uarg:
+                mime = "application/gzip"
+                ext = "tar.gz"
+            elif cancmp and "bz2" in uarg:
+                mime = "application/x-bzip"
+                ext = "tar.bz2"
+            elif cancmp and "xz" in uarg:
+                mime = "application/x-xz"
+                ext = "tar.xz"
+            else:
+                mime = "application/x-tar"
+                ext = "tar"
        else:
            mime = "application/zip"
            packer = StreamZip
+            ext = "zip"

        fn = items[0] if items and items[0] else self.vpath
        if fn:
@@ -2916,7 +2951,7 @@ class HttpCli(object):
        ufn = b"".join(zbl).decode("ascii")

        cdis = "attachment; filename=\"{}.{}\"; filename*=UTF-8''{}.{}"
-        cdis = cdis.format(afn, fmt, ufn, fmt)
+        cdis = cdis.format(afn, ext, ufn, ext)
        self.log(cdis)
        self.send_headers(None, mime=mime, headers={"Content-Disposition": cdis})

@@ -2934,7 +2969,13 @@ class HttpCli(object):
                self.log("transcoding to [{}]".format(cfmt))
                fgen = gfilter(fgen, self.thumbcli, self.uname, vpath, cfmt)

-        bgen = packer(self.log, fgen, utf8="utf" in uarg, pre_crc="crc" in uarg)
+        bgen = packer(
+            self.log,
+            fgen,
+            utf8="utf" in uarg,
+            pre_crc="crc" in uarg,
+            cmp=uarg if cancmp or uarg == "pax" else "",
+        )
        bsent = 0
        for buf in bgen.gen():
            if not buf:
@@ -3350,8 +3391,7 @@ class HttpCli(object):
        if not idx or not hasattr(idx, "p_end"):
            raise Pebkac(500, "sqlite3 is not available on the server; cannot unpost")

-        filt = self.uparam.get("filter")
-        filt = unquotep(filt or "")
+        filt = self.uparam.get("filter") or ""
        lm = "ups [{}]".format(filt)
        self.log(lm)

@@ -3449,9 +3489,6 @@ class HttpCli(object):
        if not dst:
            raise Pebkac(400, "need dst vpath")

-        # x-www-form-urlencoded (url query part) uses
-        # either + or %20 for 0x20 so handle both
-        dst = unquotep(dst.replace("+", " "))
        return self._mv(self.vpath, dst.lstrip("/"))

    def _mv(self, vsrc: str, vdst: str) -> bool:
@@ -3582,6 +3619,7 @@ class HttpCli(object):
            self.out_headers.pop("X-Robots-Tag", None)

        is_dir = stat.S_ISDIR(st.st_mode)
+        fk_pass = False
        icur = None
        if is_dir and (e2t or e2d):
            idx = self.conn.get_u2idx()
@@ -3630,8 +3668,38 @@ class HttpCli(object):

                return self.tx_ico(rem)

+        elif self.can_get and self.avn:
+            axs = self.avn.axs
+            if self.uname not in axs.uhtml and "*" not in axs.uhtml:
+                pass
+            elif is_dir:
+                for fn in ("index.htm", "index.html"):
+                    ap2 = os.path.join(abspath, fn)
+                    try:
+                        st2 = bos.stat(ap2)
+                    except:
+                        continue
+
+                    # might as well be extra careful
+                    if not stat.S_ISREG(st2.st_mode):
+                        continue
+
+                    if not self.trailing_slash:
+                        return self.redirect(
+                            self.vpath + "/", flavor="redirecting to", use302=True
+                        )
+
+                    fk_pass = True
+                    is_dir = False
+                    rem = vjoin(rem, fn)
+                    vrem = vjoin(vrem, fn)
+                    abspath = ap2
+                    break
+            elif self.vpath.rsplit("/", 1)[1] in ("index.htm", "index.html"):
+                fk_pass = True
+
        if not is_dir and (self.can_read or self.can_get):
-            if not self.can_read and "fk" in vn.flags:
+            if not self.can_read and not fk_pass and "fk" in vn.flags:
                correct = self.gen_fk(
                    self.args.fk_salt, abspath, st.st_size, 0 if ANYWIN else st.st_ino
                )[: vn.flags["fk"]]
@@ -3641,7 +3709,7 @@ class HttpCli(object):
                    return self.tx_404()

            if (
-                abspath.endswith(".md")
+                (abspath.endswith(".md") or self.can_delete)
                and "nohtml" not in vn.flags
                and (
                    "v" in self.uparam
@@ -3775,10 +3843,14 @@ class HttpCli(object):
        }

        if self.args.js_browser:
-            j2a["js"] = self.args.js_browser
+            zs = self.args.js_browser
+            zs += "&" if "?" in zs else "?"
+            j2a["js"] = zs

        if self.args.css_browser:
-            j2a["css"] = self.args.css_browser
+            zs = self.args.css_browser
+            zs += "&" if "?" in zs else "?"
+            j2a["css"] = zs

        if not self.conn.hsrv.prism:
            j2a["no_prism"] = True
--- a/copyparty/smbd.py
+++ b/copyparty/smbd.py
@@ -32,6 +32,8 @@ class SMB(object):
        self.asrv = hub.asrv
        self.log = hub.log
        self.files: dict[int, tuple[float, str]] = {}
+        self.noacc = self.args.smba
+        self.accs = not self.args.smba

        lg.setLevel(logging.DEBUG if self.args.smbvvv else logging.INFO)
        for x in ["impacket", "impacket.smbserver"]:
@@ -94,6 +96,14 @@ class SMB(object):

        port = int(self.args.smb_port)
        srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port)
+        try:
+            if self.accs:
+                srv.setAuthCallback(self._auth_cb)
+        except:
+            self.accs = False
+            self.noacc = True
+            t = "impacket too old; access permissions will not work! all accounts are admin!"
+            self.log("smb", t, 1)

        ro = "no" if self.args.smbw else "yes"  # (does nothing)
        srv.addShare("A", "/", readOnly=ro)
@@ -119,24 +129,73 @@ class SMB(object):
    def start(self) -> None:
        Daemon(self.srv.start)

-    def _v2a(self, caller: str, vpath: str, *a: Any) -> tuple[VFS, str]:
+    def _auth_cb(self, *a, **ka):
+        debug("auth-result: %s %s", a, ka)
+        conndata = ka["connData"]
+        auth_ok = conndata["Authenticated"]
+        uname = ka["user_name"] if auth_ok else "*"
+        uname = self.asrv.iacct.get(uname, uname) or "*"
+        oldname = conndata.get("partygoer", "*") or "*"
+        cli_ip = conndata["ClientIP"]
+        cli_hn = ka["host_name"]
+        if uname != "*":
+            conndata["partygoer"] = uname
+            info("client %s [%s] authed as %s", cli_ip, cli_hn, uname)
+        elif oldname != "*":
+            info("client %s [%s] keeping old auth as %s", cli_ip, cli_hn, oldname)
+        elif auth_ok:
+            info("client %s [%s] authed as [*] (anon)", cli_ip, cli_hn)
+        else:
+            info("client %s [%s] rejected", cli_ip, cli_hn)
+
+    def _uname(self) -> str:
+        if self.noacc:
+            return LEELOO_DALLAS
+
+        try:
+            # you found it! my single worst bit of code so far
+            # (if you can think of a better way to track users through impacket i'm all ears)
+            cf0 = inspect.currentframe().f_back.f_back
+            cf = cf0.f_back
+            for n in range(3):
+                cl = cf.f_locals
+                if "connData" in cl:
+                    return cl["connData"]["partygoer"]
+                cf = cf.f_back
+        except:
+            warning(
+                "nyoron... %s <<-- %s <<-- %s <<-- %s",
+                cf0.f_code.co_name,
+                cf0.f_back.f_code.co_name,
+                cf0.f_back.f_back.f_code.co_name,
+                cf0.f_back.f_back.f_back.f_code.co_name,
+            )
+            return "*"
+
+    def _v2a(
+        self, caller: str, vpath: str, *a: Any, uname="", perms=None
+    ) -> tuple[VFS, str]:
        vpath = vpath.replace("\\", "/").lstrip("/")
        # cf = inspect.currentframe().f_back
        # c1 = cf.f_back.f_code.co_name
        # c2 = cf.f_code.co_name
-        debug('%s("%s", %s)\033[K\033[0m', caller, vpath, str(a))
+        if not uname:
+            uname = self._uname()
+        if not perms:
+            perms = [True, True]

-        # TODO find a way to grab `identity` in smbComSessionSetupAndX and smb2SessionSetup
-        vfs, rem = self.asrv.vfs.get(vpath, LEELOO_DALLAS, True, True)
+        debug('%s("%s", %s) %s @%s\033[K\033[0m', caller, vpath, str(a), perms, uname)
+        vfs, rem = self.asrv.vfs.get(vpath, uname, *perms)
        return vfs, vfs.canonical(rem)

    def _listdir(self, vpath: str, *a: Any, **ka: Any) -> list[str]:
        vpath = vpath.replace("\\", "/").lstrip("/")
        # caller = inspect.currentframe().f_back.f_code.co_name
-        debug('listdir("%s", %s)\033[K\033[0m', vpath, str(a))
-        vfs, rem = self.asrv.vfs.get(vpath, LEELOO_DALLAS, False, False)
+        uname = self._uname()
+        # debug('listdir("%s", %s) @%s\033[K\033[0m', vpath, str(a), uname)
+        vfs, rem = self.asrv.vfs.get(vpath, uname, False, False)
        _, vfs_ls, vfs_virt = vfs.ls(
-            rem, LEELOO_DALLAS, not self.args.no_scandir, [[False, False]]
+            rem, uname, not self.args.no_scandir, [[False, False]]
        )
        dirs = [x[0] for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)]
        fils = [x[0] for x in vfs_ls if x[0] not in dirs]
@@ -149,8 +208,8 @@ class SMB(object):
        sz = 112 * 2  # ['.', '..']
        for n, fn in enumerate(ls):
            if sz >= 64000:
-                t = "listing only %d of %d files (%d byte); see impacket#1433"
-                warning(t, n, len(ls), sz)
+                t = "listing only %d of %d files (%d byte) in /%s; see impacket#1433"
+                warning(t, n, len(ls), sz, vpath)
                break

            nsz = len(fn.encode("utf-16", "replace"))
@@ -171,10 +230,12 @@ class SMB(object):
        if wr and not self.args.smbw:
            yeet("blocked write (no --smbw): " + vpath)

-        vfs, ap = self._v2a("open", vpath, *a)
+        uname = self._uname()
+        vfs, ap = self._v2a("open", vpath, *a, uname=uname, perms=[True, wr])
        if wr:
            if not vfs.axs.uwrite:
-                yeet("blocked write (no-write-acc): " + vpath)
+                t = "blocked write (no-write-acc %s): /%s @%s"
+                yeet(t % (vfs.axs.uwrite, vpath, uname))

            xbu = vfs.flags.get("xbu")
            if xbu and not runhook(
@@ -204,7 +265,7 @@ class SMB(object):

        _, vp = self.files.pop(fd)
        vp, fn = os.path.split(vp)
-        vfs, rem = self.hub.asrv.vfs.get(vp, LEELOO_DALLAS, False, True)
+        vfs, rem = self.hub.asrv.vfs.get(vp, self._uname(), False, True)
        vfs, rem = vfs.get_dbv(rem)
        self.hub.up2k.hash_file(
            vfs.realpath,
@@ -224,15 +285,18 @@ class SMB(object):
        vp1 = vp1.lstrip("/")
        vp2 = vp2.lstrip("/")

-        vfs2, ap2 = self._v2a("rename", vp2, vp1)
+        uname = self._uname()
+        vfs2, ap2 = self._v2a("rename", vp2, vp1, uname=uname)
        if not vfs2.axs.uwrite:
-            yeet("blocked rename (no-write-acc): " + vp2)
+            t = "blocked write (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs2.axs.uwrite, vp2, uname))

-        vfs1, _ = self.asrv.vfs.get(vp1, LEELOO_DALLAS, True, True)
+        vfs1, _ = self.asrv.vfs.get(vp1, uname, True, True, True)
        if not vfs1.axs.umove:
-            yeet("blocked rename (no-move-acc): " + vp1)
+            t = "blocked rename (no-move-acc %s): /%s @%s"
+            yeet(t % (vfs1.axs.umove, vp1, uname))

-        self.hub.up2k.handle_mv(LEELOO_DALLAS, vp1, vp2)
+        self.hub.up2k.handle_mv(uname, vp1, vp2)
        try:
            bos.makedirs(ap2)
        except:
@@ -242,52 +306,74 @@ class SMB(object):
        if not self.args.smbw:
            yeet("blocked mkdir (no --smbw): " + vpath)

-        vfs, ap = self._v2a("mkdir", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a("mkdir", vpath, uname=uname)
        if not vfs.axs.uwrite:
-            yeet("blocked mkdir (no-write-acc): " + vpath)
+            t = "blocked mkdir (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs.axs.uwrite, vpath, uname))

        return bos.mkdir(ap)

    def _stat(self, vpath: str, *a: Any, **ka: Any) -> os.stat_result:
-        return bos.stat(self._v2a("stat", vpath, *a)[1], *a, **ka)
+        try:
+            ap = self._v2a("stat", vpath, *a, perms=[True, False])[1]
+            ret = bos.stat(ap, *a, **ka)
+            # debug(" `-stat:ok")
+            return ret
+        except:
+            # white lie: windows freaks out if we raise due to an offline volume
+            # debug(" `-stat:NOPE (faking a directory)")
+            ts = int(time.time())
+            return os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, ts, ts, ts))

    def _unlink(self, vpath: str) -> None:
        if not self.args.smbw:
            yeet("blocked delete (no --smbw): " + vpath)

        # return bos.unlink(self._v2a("stat", vpath, *a)[1])
-        vfs, ap = self._v2a("delete", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a(
+            "delete", vpath, uname=uname, perms=[True, False, False, True]
+        )
        if not vfs.axs.udel:
            yeet("blocked delete (no-del-acc): " + vpath)

        vpath = vpath.replace("\\", "/").lstrip("/")
-        self.hub.up2k.handle_rm(LEELOO_DALLAS, "1.7.6.2", [vpath], [], False)
+        self.hub.up2k.handle_rm(uname, "1.7.6.2", [vpath], [], False)

    def _utime(self, vpath: str, times: tuple[float, float]) -> None:
        if not self.args.smbw:
            yeet("blocked utime (no --smbw): " + vpath)

-        vfs, ap = self._v2a("utime", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a("utime", vpath, uname=uname)
        if not vfs.axs.uwrite:
-            yeet("blocked utime (no-write-acc): " + vpath)
+            t = "blocked utime (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs.axs.uwrite, vpath, uname))

        return bos.utime(ap, times)

    def _p_exists(self, vpath: str) -> bool:
+        # ap = "?"
        try:
-            bos.stat(self._v2a("p.exists", vpath)[1])
+            ap = self._v2a("p.exists", vpath, perms=[True, False])[1]
+            bos.stat(ap)
+            # debug(" `-exists((%s)->(%s)):ok", vpath, ap)
            return True
        except:
+            # debug(" `-exists((%s)->(%s)):NOPE", vpath, ap)
            return False

    def _p_getsize(self, vpath: str) -> int:
-        st = bos.stat(self._v2a("p.getsize", vpath)[1])
+        st = bos.stat(self._v2a("p.getsize", vpath, perms=[True, False])[1])
        return st.st_size

    def _p_isdir(self, vpath: str) -> bool:
        try:
-            st = bos.stat(self._v2a("p.isdir", vpath)[1])
-            return stat.S_ISDIR(st.st_mode)
+            st = bos.stat(self._v2a("p.isdir", vpath, perms=[True, False])[1])
+            ret = stat.S_ISDIR(st.st_mode)
+            # debug(" `-isdir:%s:%s", st.st_mode, ret)
+            return ret
        except:
            return False

--- a/copyparty/ssdp.py
+++ b/copyparty/ssdp.py
@@ -214,7 +214,7 @@ CONFIGID.UPNP.ORG: 1
        srv.sck.sendto(zb, addr[:2])

        if cip not in self.txc.c:
-            self.log("{} [{}] --> {}".format(srv.name, srv.ip, cip), "6")
+            self.log("{} [{}] --> {}".format(srv.name, srv.ip, cip), 6)

        self.txc.add(cip)
        self.txc.cln()
--- a/copyparty/star.py
+++ b/copyparty/star.py
@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals

+import re
 import stat
 import tarfile

@@ -44,6 +45,7 @@ class StreamTar(StreamArc):
        self,
        log: "NamedLogger",
        fgen: Generator[dict[str, Any], None, None],
+        cmp: str = "",
        **kwargs: Any
    ):
        super(StreamTar, self).__init__(log, fgen)
@@ -53,10 +55,36 @@ class StreamTar(StreamArc):
        self.qfile = QFile()
        self.errf: dict[str, Any] = {}

-        # python 3.8 changed to PAX_FORMAT as default,
-        # waste of space and don't care about the new features
+        # python 3.8 changed to PAX_FORMAT as default;
+        # slower, bigger, and no particular advantage
        fmt = tarfile.GNU_FORMAT
-        self.tar = tarfile.open(fileobj=self.qfile, mode="w|", format=fmt)  # type: ignore
+        if "pax" in cmp:
+            # unless a client asks for it (currently
+            # gnu-tar has wider support than pax-tar)
+            fmt = tarfile.PAX_FORMAT
+            cmp = re.sub(r"[^a-z0-9]*pax[^a-z0-9]*", "", cmp)
+
+        try:
+            cmp, lv = cmp.replace(":", ",").split(",")
+            lv = int(lv)
+        except:
+            lv = None
+
+        arg = {"name": None, "fileobj": self.qfile, "mode": "w", "format": fmt}
+        if cmp == "gz":
+            fun = tarfile.TarFile.gzopen
+            arg["compresslevel"] = lv if lv is not None else 3
+        elif cmp == "bz2":
+            fun = tarfile.TarFile.bz2open
+            arg["compresslevel"] = lv if lv is not None else 2
+        elif cmp == "xz":
+            fun = tarfile.TarFile.xzopen
+            arg["preset"] = lv if lv is not None else 1
+        else:
+            fun = tarfile.open
+            arg["mode"] = "w|"
+
+        self.tar = fun(**arg)

        Daemon(self._gen, "star-gen")

--- a/copyparty/szip.py
+++ b/copyparty/szip.py
@@ -221,6 +221,7 @@ class StreamZip(StreamArc):
        fgen: Generator[dict[str, Any], None, None],
        utf8: bool = False,
        pre_crc: bool = False,
+        **kwargs: Any
    ) -> None:
        super(StreamZip, self).__init__(log, fgen)

@@ -275,6 +276,7 @@ class StreamZip(StreamArc):
    def gen(self) -> Generator[bytes, None, None]:
        errf: dict[str, Any] = {}
        errors = []
+        mbuf = b""
        try:
            for f in self.fgen:
                if "err" in f:
@@ -283,13 +285,20 @@ class StreamZip(StreamArc):

                try:
                    for x in self.ser(f):
-                        yield x
+                        mbuf += x
+                        if len(mbuf) >= 16384:
+                            yield mbuf
+                            mbuf = b""
                except GeneratorExit:
                    raise
                except:
                    ex = min_ex(5, True).replace("\n", "\n-- ")
                    errors.append((f["vp"], ex))

+            if mbuf:
+                yield mbuf
+                mbuf = b""
+
            if errors:
                errf, txt = errdesc(errors)
                self.log("\n".join(([repr(errf)] + txt[1:])))
@@ -299,20 +308,23 @@ class StreamZip(StreamArc):
            cdir_pos = self.pos
            for name, sz, ts, crc, h_pos in self.items:
                buf = gen_hdr(h_pos, name, sz, ts, self.utf8, crc, self.pre_crc)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)
+                if len(mbuf) >= 16384:
+                    yield mbuf
+                    mbuf = b""
            cdir_end = self.pos

            _, need_64 = gen_ecdr(self.items, cdir_pos, cdir_end)
            if need_64:
                ecdir64_pos = self.pos
                buf = gen_ecdr64(self.items, cdir_pos, cdir_end)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)

                buf = gen_ecdr64_loc(ecdir64_pos)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)

            ecdr, _ = gen_ecdr(self.items, cdir_pos, cdir_end)
-            yield self._ct(ecdr)
+            yield mbuf + self._ct(ecdr)
        finally:
            if errf:
                bos.unlink(errf["ap"])
--- a/copyparty/up2k.py
+++ b/copyparty/up2k.py
@@ -1050,7 +1050,7 @@ class Up2k(object):
        if WINDOWS:
            rd = rd.replace("\\", "/").strip("/")

-        g = statdir(self.log_func, not self.args.no_scandir, False, cdir)
+        g = statdir(self.log_func, not self.args.no_scandir, True, cdir)
        gl = sorted(g)
        partials = set([x[0] for x in gl if "PARTIAL" in x[0]])
        for iname, inf in gl:
@@ -1065,6 +1065,12 @@ class Up2k(object):
                continue

            lmod = int(inf.st_mtime)
+            if stat.S_ISLNK(inf.st_mode):
+                try:
+                    inf = bos.stat(abspath)
+                except:
+                    continue
+
            sz = inf.st_size
            if fat32 and not ffat and inf.st_mtime % 2:
                fat32 = False
@@ -1445,9 +1451,11 @@ class Up2k(object):
                pf = "v{}, {:.0f}+".format(n_left, b_left / 1024 / 1024)
                self.pp.msg = pf + abspath

-                st = bos.stat(abspath)
+                # throws on broken symlinks (always did)
+                stl = bos.lstat(abspath)
+                st = bos.stat(abspath) if stat.S_ISLNK(stl.st_mode) else stl
+                mt2 = int(stl.st_mtime)
                sz2 = st.st_size
-                mt2 = int(st.st_mtime)

                if nohash or not sz2:
                    w2 = up2k_wark_from_metadata(self.salt, sz2, mt2, rd, fn)
@@ -1469,6 +1477,13 @@ class Up2k(object):
                if w == w2:
                    continue

+                # symlink mtime was inconsistent before v1.9.4; check if that's it
+                if st != stl and (nohash or not sz2):
+                    mt2b = int(st.st_mtime)
+                    w2b = up2k_wark_from_metadata(self.salt, sz2, mt2b, rd, fn)
+                    if w == w2b:
+                        continue
+
                rewark.append((drd, dfn, w2, sz2, mt2))

                t = "hash mismatch: {}\n  db: {} ({} byte, {})\n  fs: {} ({} byte, {})"
--- a/copyparty/util.py
+++ b/copyparty/util.py
@@ -2125,7 +2125,7 @@ def list_ips() -> list[str]:
 def yieldfile(fn: str) -> Generator[bytes, None, None]:
    with open(fsenc(fn), "rb", 512 * 1024) as f:
        while True:
-            buf = f.read(64 * 1024)
+            buf = f.read(128 * 1024)
            if not buf:
                break

--- a/copyparty/web/baguettebox.js
+++ b/copyparty/web/baguettebox.js
@@ -261,7 +261,7 @@ window.baguetteBox = (function () {
            setloop(1);
        else if (k == "BracketRight")
            setloop(2);
-        else if (e.shiftKey)
+        else if (e.shiftKey && k != 'KeyR')
            return;
        else if (k == "ArrowLeft" || k == "KeyJ")
            showPreviousImage();
--- a/copyparty/web/browser.css
+++ b/copyparty/web/browser.css
@@ -493,6 +493,7 @@ html.dz {
 	--err-ts: #500;

 	text-shadow: none;
+	font-family: 'scp', monospace, monospace;
 }
 html.dy {
 	--fg: #000;
@@ -860,7 +861,7 @@ html.y #path a:hover {
 }
 .mdo,
 .mdo * {
-	line-height: 1.4em;
+	line-height: 1.5em;
 }
 #srv_info,
 #srv_info2,
@@ -1397,6 +1398,9 @@ input[type="checkbox"]:checked+label {
 	color: #0e0;
 	color: var(--a);
 }
+html.dz input {
+	font-family: 'scp', monospace, monospace;
+}
 .opwide div>span>input+label {
 	padding: .3em 0 .3em .3em;
 	margin: 0 0 0 -.3em;
@@ -1577,6 +1581,7 @@ html.cz .btn {
 	border-bottom: .2em solid #709;
 }
 html.dz .btn {
+	font-size: 1em;
 	box-shadow: 0 0 0 .1em #080 inset;
 }
 html.dz .tgl.btn.on {
@@ -2766,6 +2771,9 @@ html.c .opbox,
 html.a .opbox {
 	margin: 1.5em 0 0 0;
 }
+html.dz .opview input.i {
+	width: calc(100% - 17em);
+}
 html.c #tree,
 html.c #treeh,
 html.a #tree,
@@ -2818,6 +2826,9 @@ html.a #u2btn {
 html.ay #u2btn {
 	box-shadow: .4em .4em 0 #ccc;
 }
+html.dz #u2btn {
+	letter-spacing: -.033em;
+}
 html.c #u2conf.ww #u2btn,
 html.a #u2conf.ww #u2btn {
 	margin: -2em .5em -3em 0;
--- a/copyparty/web/browser.html
+++ b/copyparty/web/browser.html
@@ -11,7 +11,7 @@
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
 	{%- if css %}
-	<link rel="stylesheet" media="screen" href="{{ css }}?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ css }}_={{ ts }}">
 	{%- endif %}
 </head>

@@ -29,7 +29,7 @@

 	<div id="op_player" class="opview opbox opwide"></div>

-	<div id="op_bup" class="opview opbox act">
+	<div id="op_bup" class="opview opbox {% if not ls0 %}act{% endif %}">
 		<div id="u2err"></div>
 		<form method="post" enctype="multipart/form-data" accept-charset="utf-8" action="{{ url_suf }}">
 			<input type="hidden" name="act" value="bput" />
@@ -39,7 +39,7 @@
 		<a id="bbsw" href="?b=u" rel="nofollow"><br />switch to basic browser</a>
 	</div>

-	<div id="op_mkdir" class="opview opbox act">
+	<div id="op_mkdir" class="opview opbox {% if not ls0 %}act{% endif %}">
 		<form method="post" enctype="multipart/form-data" accept-charset="utf-8" action="{{ url_suf }}">
 			<input type="hidden" name="act" value="mkdir" />
 			📂<input type="text" name="name" class="i" placeholder="awesome mix vol.1">
@@ -55,7 +55,7 @@
 		</form>
 	</div>

-	<div id="op_msg" class="opview opbox act">
+	<div id="op_msg" class="opview opbox {% if not ls0 %}act{% endif %}">
 		<form method="post" enctype="application/x-www-form-urlencoded" accept-charset="utf-8" action="{{ url_suf }}">
 			📟<input type="text" name="msg" class="i" placeholder="lorem ipsum dolor sit amet">
 			<input type="submit" value="send msg to srv log">
@@ -173,7 +173,7 @@
 	<script src="{{ r }}/.cpr/browser.js?_={{ ts }}"></script>
 	<script src="{{ r }}/.cpr/up2k.js?_={{ ts }}"></script>
 	{%- if js %}
-	<script src="{{ js }}?_={{ ts }}"></script>
+	<script src="{{ js }}_={{ ts }}"></script>
 	{%- endif %}
 </body>

--- a/copyparty/web/browser.js
+++ b/copyparty/web/browser.js
@@ -80,6 +80,7 @@ var Ls = {
 				"textfile-viewer",
 				["I/K", "prev/next file"],
 				["M", "close textfile"],
+				["E", "edit textfile"],
 				["S", "select file (for cut/rename)"],
 			]
 		],
@@ -262,7 +263,8 @@ var Ls = {
 		"mm_e403": "Could not play audio; error 403: Access denied.\n\nTry pressing F5 to reload, maybe you got logged out",
 		"mm_e5xx": "Could not play audio; server error ",
 		"mm_nof": "not finding any more audio files nearby",
-		"mm_pwrsv": "<p>it looks like playback is being interrupted by your phone's power-saving settings!</p>" + '<p>please go to <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">the app settings of your browser</a> and then <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">allow unrestricted battery usage</a> to fix it.</p><p>(probably a good idea to use a separate browser dedicated for just music streaming...)</p>',
+		"mm_pwrsv": "<p>it looks like playback is being interrupted by your phone's power-saving settings!</p>" + '<p>please go to <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">the app settings of your browser</a> and then <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">allow unrestricted battery usage</a> to fix it.</p><p><em>however,</em> it could also be due to the browser\'s autoplay settings;</p><p>Firefox: tap the icon on the left side of the address bar, then select "autoplay" and "allow audio"</p><p>Chrome: the problem will gradually dissipate as you play more music on this site</p>',
+		"mm_iosblk": "<p>your web browser thinks the audio playback is unwanted, and it decided to block playback until you start another track manually... unfortunately we are both powerless in telling it otherwise</p><p>supposedly this will get better as you continue playing music on this site, but I'm unfamiliar with apple devices so idk if that's true</p><p>you could try another browser, maybe firefox or chrome?</p>",
 		"mm_hnf": "that song no longer exists",

 		"im_hnf": "that image no longer exists",
@@ -326,6 +328,7 @@ var Ls = {
 		"tvt_prev": "show previous document$NHotkey: i\">⬆ prev",
 		"tvt_next": "show next document$NHotkey: K\">⬇ next",
 		"tvt_sel": "select file &nbsp; ( for cut / delete / ... )$NHotkey: S\">sel",
+		"tvt_edit": "open file in text editor$NHotkey: E\">✏️ edit",

 		"gt_msel": "enable file selection; ctrl-click a file to override$N$N&lt;em&gt;when active: doubleclick a file / folder to open it&lt;/em&gt;$N$NHotkey: S\">multiselect",
 		"gt_zoom": "zoom",
@@ -374,7 +377,10 @@ var Ls = {
 		"fu_xe1": "failed to load unpost list from server:\n\nerror ",
 		"fu_xe2": "404: File not found??",

-		"fz_tar": "plain gnutar file (linux / mac)",
+		"fz_tar": "uncompressed gnu-tar file (linux / mac)",
+		"fz_pax": "uncompressed pax-format tar (slower)",
+		"fz_targz": "gnu-tar with gzip level 3 compression$N$Nthis is usually very slow, so$Nuse uncompressed tar instead",
+		"fz_tarxz": "gnu-tar with xz level 1 compression$N$Nthis is usually very slow, so$Nuse uncompressed tar instead",
 		"fz_zip8": "zip with utf8 filenames (maybe wonky on windows 7 and older)",
 		"fz_zipd": "zip with traditional cp437 filenames, for really old software",
 		"fz_zipc": "cp437 with crc32 computed early,$Nfor MS-DOS PKZIP v2.04g (october 1993)$N(takes longer to process before download can start)",
@@ -543,6 +549,7 @@ var Ls = {
 				"dokumentviser",
 				["I/K", "forr./neste fil"],
 				["M", "lukk tekstdokument"],
+				["E", "rediger tekstdokument"]
 				["S", "velg fil (for F2/ctrl-x/...)"]
 			]
 		],
@@ -725,7 +732,8 @@ var Ls = {
 		"mm_e403": "Avspilling feilet: Tilgang nektet.\n\nKanskje du ble logget ut?\nPrøv å trykk F5 for å laste siden på nytt.",
 		"mm_e5xx": "Avspilling feilet: ",
 		"mm_nof": "finner ikke flere sanger i nærheten",
-		"mm_pwrsv": "<p>det ser ut som musikken ble avbrutt av telefonen sine strømsparings-innstillinger!</p>" + '<p>ta en tur innom <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">app-innstillingene til nettleseren din</a> og så <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">tillat ubegrenset batteriforbruk</a></p><p>(sikkert smart å ha en egen nettleser kun for musikkspilling...)</p>',
+		"mm_pwrsv": "<p>det ser ut som musikken ble avbrutt av telefonen sine strømsparings-innstillinger!</p>" + '<p>ta en tur innom <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">app-innstillingene til nettleseren din</a> og så <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">tillat ubegrenset batteriforbruk</a></p><p>NB: det kan også være pga. autoplay-innstillingene, så prøv dette:</p><p>Firefox: klikk på ikonet i venstre side av addressefeltet, velg "autoplay" og "tillat lyd"</p><p>Chrome: problemet vil minske gradvis jo mer musikk du spiller på denne siden</p>',
+		"mm_iosblk": "<p>nettleseren din tror at musikken er uønsket, og den bestemte seg for å stoppe avspillingen slik at du manuelt må velge en ny sang... dessverre er både du og jeg maktesløse når den har bestemt seg.</p><p>det ryktes at problemet vil minske jo mer musikk du spiller på denne siden, men jeg er ikke godt kjent med apple-dingser så jeg er ikke sikker.</p><p>kanskje firefox eller chrome fungerer bedre?</p>",
 		"mm_hnf": "sangen finnes ikke lenger",

 		"im_hnf": "bildet finnes ikke lenger",
@@ -784,11 +792,12 @@ var Ls = {
 		"tv_xe1": "kunne ikke laste tekstfil:\n\nfeil ",
 		"tv_xe2": "404, Fil ikke funnet",
 		"tv_lst": "tekstfiler i mappen",
-		"tvt_close": "gå tilbake til mappen$NSnarvei: M\">❌ close",
+		"tvt_close": "gå tilbake til mappen$NSnarvei: M\">❌ lukk",
 		"tvt_dl": "last ned denne filen\">💾 last ned",
-		"tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ prev",
-		"tvt_next": "vis neste dokument$NSnarvei: K\">⬇ next",
-		"tvt_sel": "markér filen &nbsp; ( for utklipp / sletting / ... )$NSnarvei: S\">sel",
+		"tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ forr.",
+		"tvt_next": "vis neste dokument$NSnarvei: K\">⬇ neste",
+		"tvt_sel": "markér filen &nbsp; ( for utklipp / sletting / ... )$NSnarvei: S\">merk",
+		"tvt_edit": "redigér filen$NSnarvei: E\">✏️ endre",

 		"gt_msel": "markér filer istedenfor å åpne dem; ctrl-klikk filer for å overstyre$N$N&lt;em&gt;når aktiv: dobbelklikk en fil / mappe for å åpne&lt;/em&gt;$N$NSnarvei: S\">markering",
 		"gt_zoom": "zoom",
@@ -838,6 +847,9 @@ var Ls = {
 		"fu_xe2": "404: Filen finnes ikke??",

 		"fz_tar": "ukomprimert gnu-tar arkiv, for linux og mac",
+		"fz_pax": "ukomprimert pax-tar arkiv, litt tregere",
+		"fz_targz": "gnu-tar pakket med gzip (nivå 3)$N$NNB: denne er veldig treg;$Nukomprimert tar er bedre",
+		"fz_tarxz": "gnu-tar pakket med xz (nivå 1)$N$NNB: denne er veldig treg;$Nukomprimert tar er bedre",
 		"fz_zip8": "zip med filnavn i utf8 (noe problematisk på windows 7 og eldre)",
 		"fz_zipd": "zip med filnavn i cp437, for høggamle maskiner",
 		"fz_zipc": "cp437 med tidlig crc32,$Nfor MS-DOS PKZIP v2.04g (oktober 1993)$N(øker behandlingstid på server)",
@@ -1299,7 +1311,8 @@ function set_files_html(html) {
 }


-var ACtx = window.AudioContext || window.webkitAudioContext,
+// actx breaks background album playback on ios
+var ACtx = !IPHONE && (window.AudioContext || window.webkitAudioContext),
 	noih = /[?&]v\b/.exec('' + location),
 	hash0 = location.hash,
 	mp;
@@ -2184,6 +2197,7 @@ function song_skip(n, dirskip) {

 	if (dirskip && ofs + 1 && ofs > mp.order.length - 2) {
 		toast.inf(10, L.mm_nof);
+		console.log("mm_nof1");
 		mpl.traversals = 0;
 		return;
 	}
@@ -2210,13 +2224,14 @@ function next_song_cmn(e) {
 	}
 	if (mpl.traversals++ < 5) {
 		if (MOBILE && t_fchg && Date.now() - t_fchg > 30 * 1000)
-			modal.alert(L.mm_pwrsv);
+			modal.alert(IPHONE ? L.mm_iosblk : L.mm_pwrsv);

 		t_fchg = document.hasFocus() ? 0 : Date.now();
 		treectl.ls_cb = next_song_cmn;
 		return tree_neigh(1);
 	}
 	toast.inf(10, L.mm_nof);
+	console.log("mm_nof2");
 	mpl.traversals = 0;
 	t_fchg = 0;
 }
@@ -2366,7 +2381,7 @@ var mpui = (function () {
 			// cannot check document.hasFocus to avoid false positives;
 			// it continues on power-on, doesn't need to be in-browser
 			if (MOBILE && Date.now() - t_fchg > 30 * 1000)
-				modal.alert(L.mm_pwrsv);
+				modal.alert(IPHONE ? L.mm_iosblk : L.mm_pwrsv);

 			t_fchg = 0;
 		}
@@ -2932,6 +2947,7 @@ function evau_error(e) {
 		err = e404;

 	toast.warn(15, esc(basenames(err + mfile)));
+	console.log(basenames(err + mfile));

 	if (em.startsWith('MEDIA_ELEMENT_ERROR:')) {
 		// chromish for 40x
@@ -3043,7 +3059,7 @@ function eval_hash() {
 		goto('search');
 		var i = ebi('q_raw');
 		i.value = uricom_dec(v.slice(3));
-		return i.oninput();
+		return i.onkeydown({ 'key': 'Enter' });
 	}

 	if (v.indexOf('#v=') === 0) {
@@ -4009,6 +4025,8 @@ var showfile = (function () {

 		ebi('files').style.display = ebi('gfiles').style.display = ebi('lazy').style.display = ebi('pro').style.display = ebi('epi').style.display = 'none';
 		ebi('dldoc').setAttribute('href', url);
+		ebi('editdoc').setAttribute('href', url + (url.indexOf('?') > 0 ? '&' : '?') + 'edit');
+		ebi('editdoc').style.display = (has(perms, 'write') && (is_md || has(perms, 'delete'))) ? '' : 'none';

 		var wr = ebi('bdoc'),
 			defer = !Prism.highlightElement;
@@ -4020,7 +4038,7 @@ var showfile = (function () {

 				el = el || QS('#doc>code');
 				Prism.highlightElement(el);
-				if (el.className == 'language-ans')
+				if (el.className == 'language-ans' || (!lang && /\x1b\[[0-9;]{0,16}m/.exec(txt.slice(0, 4096))))
 					r.ansify(el);
 			}
 			catch (ex) { }
@@ -4179,6 +4197,7 @@ var showfile = (function () {
 		'<a href="#" class="btn" id="prevdoc" tt="' + L.tvt_prev + '</a>\n' +
 		'<a href="#" class="btn" id="nextdoc" tt="' + L.tvt_next + '</a>\n' +
 		'<a href="#" class="btn" id="seldoc" tt="' + L.tvt_sel + '</a>\n' +
+		'<a href="#" class="btn" id="editdoc" tt="' + L.tvt_edit + '</a>\n' +
 		'</div>'
 	);
 	ebi('xdoc').onclick = function () {
@@ -4881,6 +4900,8 @@ document.onkeydown = function (e) {
 	if (showfile.active()) {
 		if (k == 'KeyS')
 			showfile.tglsel();
+		if (k == 'KeyE' && ebi('editdoc').style.display != 'none')
+			ebi('editdoc').click();
 	}
 };

@@ -4962,7 +4983,7 @@ document.onkeydown = function (e) {
 		search_in_progress = 0;

 	function ev_search_input() {
-		var v = this.value,
+		var v = unsmart(this.value),
 			id = this.getAttribute('id');

 		if (id.slice(-1) == 'v') {
@@ -4999,7 +5020,7 @@ document.onkeydown = function (e) {
 		if (search_in_progress)
 			return;

-		var q = ebi('q_raw').value,
+		var q = unsmart(ebi('q_raw').value),
 			vq = ebi('files').getAttribute('q_raw');

 		srch_msg(false, (q == vq) ? '' : L.sm_prev + (vq ? vq : '(*)'));
@@ -5011,7 +5032,7 @@ document.onkeydown = function (e) {
 			for (var b = 1; b < sconf[a].length; b++) {
 				var k = sconf[a][b][0],
 					chk = 'srch_' + k + 'c',
-					vs = ebi('srch_' + k + 'v').value,
+					vs = unsmart(ebi('srch_' + k + 'v').value),
 					tvs = [];

 				if (a == 1)
@@ -5104,7 +5125,7 @@ document.onkeydown = function (e) {
 		xhr.setRequestHeader('Content-Type', 'text/plain');
 		xhr.onload = xhr.onerror = xhr_search_results;
 		xhr.ts = Date.now();
-		xhr.q_raw = ebi('q_raw').value;
+		xhr.q_raw = unsmart(ebi('q_raw').value);
 		xhr.send(JSON.stringify({ "q": xhr.q_raw, "n": cap }));
 	}

@@ -6666,6 +6687,9 @@ var arcfmt = (function () {
 	var html = [],
 		fmts = [
 			["tar", "tar", L.fz_tar],
+			["pax", "tar=pax", L.fz_pax],
+			["tgz", "tar=gz", L.fz_targz],
+			["txz", "tar=xz", L.fz_tarxz],
 			["zip", "zip=utf8", L.fz_zip8],
 			["zip_dos", "zip", L.fz_zipd],
 			["zip_crc", "zip=crc", L.fz_zipc]
@@ -6695,7 +6719,7 @@ var arcfmt = (function () {

 		for (var a = 0, aa = tds.length; a < aa; a++) {
 			var o = tds[a], txt = o.textContent, href = o.getAttribute('href');
-			if (txt != 'tar' && txt != 'zip')
+			if (!/^(zip|tar|pax|tgz|txz)$/.exec(txt))
 				continue;

 			var ofs = href.lastIndexOf('?');
@@ -7223,7 +7247,7 @@ function sandbox(tgt, rules, cls, html) {
 		'function say(m){window.parent.postMessage(m,"*")};' +
 		'setTimeout(function(){var its=0,pih=-1,f=function(){' +
 		'var ih=2+Math.min(parseInt(getComputedStyle(d).height),d.scrollHeight);' +
-		'if(ih!=pih){pih=ih;say("iheight #' + tid + ' "+ih,"*")}' +
+		'if(ih!=pih&&!isNaN(ih)){pih=ih;say("iheight #' + tid + ' "+ih,"*")}' +
 		'if(++its<20)return setTimeout(f,20);if(its==20)setInterval(f,200)' +
 		'};f();' +
 		'window.onfocus=function(){say("igot #' + tid + '")};' +
--- a/copyparty/web/util.js
+++ b/copyparty/web/util.js
@@ -369,6 +369,15 @@ function import_js(url, cb) {
 }


+function unsmart(txt) {
+    return !IPHONE ? txt : (txt.
+        replace(/[\u2014]/g, "--").
+        replace(/[\u2022]/g, "*").
+        replace(/[\u2018\u2019]/g, "'").
+        replace(/[\u201c\u201d]/g, '"'));
+}
+
+
 var crctab = (function () {
    var c, tab = [];
    for (var n = 0; n < 256; n++) {
@@ -1117,6 +1126,8 @@ var timer = (function () {
    var r = {};
    r.q = [];
    r.last = 0;
+    r.fs = 0;
+    r.fc = 0;

    r.add = function (fun, run) {
        r.rm(fun);
@@ -1142,6 +1153,7 @@ var timer = (function () {
            q[a]();

        r.last = Date.now();
+        //r.fc++; if (r.last - r.fs >= 2000) { console.log(r.last - r.fs, r.fc); r.fs = r.last; r.fc = 0; }
    }
    setInterval(doevents, 100);

@@ -1598,7 +1610,7 @@ function repl_load() {
            ret = [
                'var v=Object.keys(localStorage); v.sort(); JSON.stringify(v)',
                "for (var a of QSA('#files a[id]')) a.setAttribute('download','')",
-                'console.hist.slice(-10).join("\\n")'
+                'console.hist.slice(-50).join("\\n")'
            ];

        ipre.innerHTML = '<option value=""></option>';
@@ -1654,6 +1666,8 @@ function repl(e) {
        if (!cmd)
            return toast.inf(3, 'eval aborted');

+        cmd = unsmart(cmd);
+
        if (cmd.startsWith(',')) {
            evalex_fatal = true;
            return modal.alert(esc(eval(cmd.slice(1)) + ''));
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -1,3 +1,84 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0902-0018  `v1.9.4`  yes symlink times
+
+hello! it's been a while, an entire day even...
+
+## new features
+* download folder as tar.gz, tar.bz2, tar.xz
+  * single-threaded, so extremely slow, but nice for easily compressed data or challenged networks
+  * append `?tar=gz`, `?tar=bz2` or `?tar=xz` to a folder URL to do it
+  * default compression levels are gz:3, bz2:2, xz:1; override with `?tar=gz:9`
+
+# bugfixes
+* c1efd227b7377144a5760bc6cff64f4e87b626d9 symlink-deduplicated files got indexed with the wrong last-modified timestamp
+  * mostly inconsequential; would cause the dupe's uploader-ip to be forgotten on the next server restart since it would reindex to "fix" the timestamp
+* when linking [a search query](https://a.ocv.me/pub/#q=tags%20like%20soundsho*) it loads the results faster
+
+# other changes
+* update readme to mention that iPhones and iPads dislike the preload feature and respond by glitching the audio a bit when a song is exactly 20 seconds away from ending and yet how it's probably a bad idea to disable preloading since i bet it's load-bearing against other iOS bugs
+  * speaking of iPhones and iPads, the [previous version](https://github.com/9001/copyparty/releases/tag/v1.9.3) should have fixed album playback on those
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0831-2211  `v1.9.3`  iOS and http fixes
+
+## new features
+* iPhones and iPads are now able to...
+  * 9986136dfb2364edb35aa9fbb87410641c6d6af3 play entire albums while the screen is off without the music randomly stopping
+    * apple keeps breaking AudioContext in new and interesting ways; time to give up (no more equalizer)
+  * 1c0d978979a703edeb792e552b18d3b7695b2d90 perform search queries and execude js code
+    * by translating [smart-quotes](https://stackoverflow.com/questions/48678359/ios-11-safari-html-disable-smart-punctuation) into regular `'` and `"` characters
+* python 3.12 support
+  * technically a bugfix since it was added [a year ago](https://github.com/9001/copyparty/commit/32e22dfe84d5e0b13914b4d0e15c1b8c9725a76d) way before the first py3.12 alpha was released but turns out i botched it, oh well
+* filter error messages so they never include the filesystem path where copyparty's python files reside
+* print more context in server logs if someone hits an unexpected permission-denied
+
+# bugfixes
+found some iffy stuff combing over the code but, as far as I can tell, luckily none of these were dangerous:
+* URL normalization was a bit funky, but it appears everything access-control-related was unaffected
+* some url parameters were double-decoded, causing the unpost filtering and file renaming to fail if the values contained `%`
+* clients could cause the server to return an invalid cache-control header, but newlines and control-characters got rejected correctly
+* minor cosmetics / qol fixes:
+  * reduced flickering on page load in chrome
+  * fixed some console spam in search results
+  * markdown documents now have the same line-height in directory listings and the editor
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0826-2116  `v1.9.2`  bigger hammer
+
+## new features
+* more ways to automatically ban users! three new sensors, all default-enabled, giving a 1 day ban after 9 hits in 2 minutes:
+  * `--ban-403`: trying to access volumes that dont exist or require authentication
+  * `--ban-422`: invalid POST messages (from brutefocing POST parameters and such)
+  * `--ban-url`: URLs which 404 and also match `--sus-urls` (scanners/crawlers)
+  * if you want to run a vulnerability scan on copyparty, please just [download the server](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and do it locally! takes less than 30 seconds to set up, you get lower latency, and you won't be filling up the logfiles on the demo server with junk, thank you 🙏
+* more ban-related stuff,
+  * new global option `--nonsus-urls` specifies regex of URLs which are OK to 404 and shouldn't ban people
+  * `--turbo` now accepts the value `-1` which makes it impossible for clients to enable it, making `--ban-404` safe to use
+* range-selecting files in the list-view by shift-pgup/pgdn
+* volumes which are currently unavailable (dead nfs share, external HDD which is off, ...) are marked with a ❌ in the directory tree sidebar
+* the toggle-button to see dotfiles is now persisted as a cookie so it also applies on the initial page load
+* more effort is made to prevent `<script>`s inside markdown documents from running in the markdown editor and the fullpage viewer
+  * anyone who wanted to use markdown files for malicious stuff can still just upload an html file instead, so this doesn't make anything more secure, just less confusing
+  * the safest approach is still the `nohtml` volflag which disables markdown rendering outside sandboxes entirely, or only giving out write-access to trustworthy people
+  * enabling markdown plugins with `-emp` now has the side-effect of cancelling this band-aid too
+
+## bugfixes
+* textfile navigation hotkeys broke in the previous version
+
+## other changes
+* example [nginx config](https://github.com/9001/copyparty/blob/hovudstraum/contrib/nginx/copyparty.conf) was not compatible with cloudflare (suggest `$http_cf_connecting_ip` instead of `$proxy_add_x_forwarded_for`)
+* `copyparty.exe` is now built with python 3.11.5 which fixes [CVE-2023-40217](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-40217)
+  * `copyparty32.exe` is not, because python understandably ended win7 support 
+* [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md):
+  * copyparty appears to be 30x faster than nextcloud and seafile at receiving uploads of many small files
+  * seafile has a size limit when zip-downloading folders
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2023-0820-2338  `v1.9.1`  prometheable

--- a/docs/devnotes.md
+++ b/docs/devnotes.md
@@ -125,8 +125,14 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | GET | `?b` | list files/folders at URL as simplified HTML |
 | GET | `?tree=.` | list one level of subdirectories inside URL |
 | GET | `?tree` | list one level of subdirectories for each level until URL |
-| GET | `?tar` | download everything below URL as a tar file |
-| GET | `?zip=utf-8` | download everything below URL as a zip file |
+| GET | `?tar` | download everything below URL as a gnu-tar file |
+| GET | `?tar=gz:9` | ...as a gzip-level-9 gnu-tar file |
+| GET | `?tar=xz:9` | ...as an xz-level-9 gnu-tar file |
+| GET | `?tar=pax` | ...as a pax-tar file |
+| GET | `?tar=pax,xz` | ...as an xz-level-1 pax-tar file |
+| GET | `?zip=utf-8` | ...as a zip file |
+| GET | `?zip` | ...as a WinXP-compatible zip file |
+| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
 | GET | `?ups` | show recent uploads from your IP |
 | GET | `?ups&filter=f` | ...where URL contains `f` |
 | GET | `?mime=foo` | specify return mimetype `foo` |
--- a/docs/versus.md
+++ b/docs/versus.md
@@ -255,7 +255,7 @@ symbol legend,
 | per-file permissions    |   |   |   | █ | █ |   | █ |   | █ |   |   |   |
 | per-file passwords      | █ |   |   | █ | █ |   | █ |   | █ |   |   |   |
 | unmap subfolders        | █ |   |   |   |   |   | █ |   |   | █ | ╱ | • |
-| index.html blocks list  |   |   |   |   |   |   | █ |   |   | • |   |   |
+| index.html blocks list  | ╱ |   |   |   |   |   | █ |   |   | • |   |   |
 | write-only folders      | █ |   |   |   |   |   |   |   |   |   | █ | █ |
 | files stored as-is      | █ | █ | █ | █ |   | █ | █ |   |   | █ | █ | █ |
 | file versioning         |   |   |   | █ | █ |   |   |   |   |   |   |   |
@@ -291,6 +291,7 @@ symbol legend,
  * one-way folder sync from local to server can be done efficiently with [u2c.py](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy), or with webdav and conventional rsync
  * can hot-reload config files (with just a few exceptions)
  * can set per-folder permissions if that folder is made into a separate volume, so there is configuration overhead
+  * `index.html` on its own does not prevent directory listing, but permission `h` (instead of `r`) enforces index.html to be returned instead of folder contents
  * [event hooks](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks) ([discord](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png), [desktop](https://user-images.githubusercontent.com/241032/215335767-9c91ed24-d36e-4b6b-9766-fb95d12d163f.png)) inspired by filebrowser, as well as the more complex [media parser](https://github.com/9001/copyparty/tree/hovudstraum/bin/mtag) alternative
  * upload history can be visualized using [partyjournal](https://github.com/9001/copyparty/blob/hovudstraum/bin/partyjournal.py)
 * `k`/filegator remarks:
--- a/scripts/pyinstaller/deps.sha512
+++ b/scripts/pyinstaller/deps.sha512
@@ -1,9 +1,9 @@
 d5510a24cb5e15d6d30677335bbc7624c319b371c0513981843dc51d9b3a1e027661096dfcfc540634222bb2634be6db55bf95185b30133cb884f1e47652cf53  altgraph-0.17.3-py2.py3-none-any.whl
 eda6c38fc4d813fee897e969ff9ecc5acc613df755ae63df0392217bbd67408b5c1f6c676f2bf5497b772a3eb4e1a360e1245e1c16ee83f0af555f1ab82c3977  Git-2.39.1-32-bit.exe
 17ce52ba50692a9d964f57a23ac163fb74c77fdeb2ca988a6d439ae1fe91955ff43730c073af97a7b3223093ffea3479a996b9b50ee7fba0869247a56f74baa6  pefile-2023.2.7-py3-none-any.whl
-98ba6fa675d83264dbac5496ed8f2bdb0515928714b564e9d837ced95377e394f35d19c4f65a865ddbb053b940bce96c10ce8f6012ebce150a96e3cb7fc0cc60  pyinstaller-5.13.1-py3-none-win32.whl
-b8d4d1a80ef75e60cb2cae67331a6462ba7a05d4c68b526fe3299d99ff3d94c136c2d741f08f9d3cff457ffa188b1ebd91ba5af39abd7af416f833b2ea0e8a35  pyinstaller-5.13.1-py3-none-win_amd64.whl
-e6cd2a8e604f58b04c5694d82ac10460d39c643db89c3fd2643708dc6229f392e39fe423fbdea8921a89e75d7289238380e914dbf82049d87bb980fc6a36779f  pyinstaller_hooks_contrib-2023.7-py2.py3-none-any.whl
+f298e34356b5590dde7477d7b3a88ad39c622a2bcf3fcd7c53870ce8384dd510f690af81b8f42e121a22d3968a767d2e07595036b2ed7049c8ef4d112bcf3a61  pyinstaller-5.13.2-py3-none-win32.whl
+ea73aa54cc6d5db20dfb127e54562dabf890e4cd6171a91b10a51af2bcfc76e1d64cbdce4546df2dcfe42b624724c85b1cd05934be2413425b1f880222727b4f  pyinstaller-5.13.2-py3-none-win_amd64.whl
+2f4e3927a38cf7757bc9a1c06370d79209669a285a80f1b09cf9917137825c7022a50a56b351807e6e687e2c3a7bd7b2c5cc6daeb4d90e11920284c1a04a1cc3  pyinstaller_hooks_contrib-2023.8-py2.py3-none-any.whl
 749a473646c6d4c7939989649733d4c7699fd1c359c27046bf5bc9c070d1a4b8b986bbc65f60d7da725baf16dbfdd75a4c2f5bb8335f2cb5685073f5fee5c2d1  pywin32_ctypes-0.2.2-py3-none-any.whl
 3c5adf0a36516d284a2ede363051edc1bcc9df925c5a8a9fa2e03cab579dd8d847fdad42f7fd5ba35992e08234c97d2dbfec40a9d12eec61c8dc03758f2bd88e  typing_extensions-4.4.0-py3-none-any.whl
 8d16a967a0a7872a7575b1005cf66915deacda6ee8611fbb52f42fc3e3beb2f901a5140c942a5d146bd412b92bfa9cbadd82beeba83df6d70930c6dc26608a5b  upx-4.1.0-win32.zip
@@ -25,6 +25,6 @@ ba91ab0518c61eff13e5612d9e6b532940813f6b56e6ed81ea6c7c4d45acee4d98136a383a250675
 # win10
 00558cca2e0ac813d404252f6e5aeacb50546822ecb5d0570228b8ddd29d94e059fbeb6b90393dee5abcddaca1370aca784dc9b095cbb74e980b3c024767fb24  Jinja2-3.1.2-py3-none-any.whl
 7f8f4daa4f4f2dbf24cdd534b2952ee3fba6334eb42b37465ccda3aa1cccc3d6204aa6bfffb8a83bf42ec59c702b5b5247d4c8ee0d4df906334ae53072ef8c4c  MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl
-4a20aeb52d4fde6aabcba05ee261595eeb5482c72ee27332690f34dd6e7a49c0b3ba3813202ac15c9d21e29f1cd803f2e79ccc1c45ec314fcd0a937016bcbc56  mutagen-1.46.0-py3-none-any.whl
+8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e  mutagen-1.47.0-py3-none-any.whl
 926d408a886059a75cf12706fa061146f9f042b27fb6e65be7d49f398ed23fb0227639d84804586ac014c6bcf7d08cd86a09c1a20793d341aa0802d3d32a546b  Pillow-10.0.0-cp311-cp311-win_amd64.whl
 c86bbeacad3ae3c7bde747f5b4f09c11eced841add14e79ec4a064e5e29ebca35460e543ba735b11bfb882837d5ff4371ce64492d28d096b4686233c9a8cda6d  python-3.11.5-amd64.exe
--- a/scripts/pyinstaller/notes.txt
+++ b/scripts/pyinstaller/notes.txt
@@ -17,13 +17,13 @@ uname -s | grep NT-10 && w10=1 || {
 fns=(
  altgraph-0.17.3-py2.py3-none-any.whl
  pefile-2023.2.7-py3-none-any.whl
-  pyinstaller-5.13.1-py3-none-win_amd64.whl
+  pyinstaller-5.13.2-py3-none-win_amd64.whl
  pyinstaller_hooks_contrib-2023.7-py2.py3-none-any.whl
  pywin32_ctypes-0.2.2-py3-none-any.whl
  upx-4.1.0-win32.zip
 )
 [ $w10 ] && fns+=(
-  mutagen-1.46.0-py3-none-any.whl
+  mutagen-1.47.0-py3-none-any.whl
  Pillow-9.4.0-cp311-cp311-win_amd64.whl
  python-3.11.3-amd64.exe
 }
@@ -47,7 +47,7 @@ fns=(
 )
 [ $w7x32 ] && fns+=(
  windows6.1-kb2533623-x86.msu
-  pyinstaller-5.13.1-py3-none-win32.whl
+  pyinstaller-5.13.2-py3-none-win32.whl
  python-3.7.9.exe
 )
 dl() { curl -fkLOC- "$1" && return 0; echo "$1"; return 1; }
Author	SHA1	Message	Date
ed	a996a09bba	v1.9.5	2023-09-09 13:36:56 +00:00
ed	18c763ac08	smb: upgrade to impacket 0.11, full user account support, permissions are now per-account instead of coalescing also stops windows from freaking out if there's an offline volume	2023-09-09 12:46:37 +00:00
ed	3d9fb753ba	stuff	2023-09-08 21:42:05 +00:00
ed	714fd1811a	add option to generate pax-format tar archives and forgot to commit the nix module	2023-09-08 21:13:23 +00:00
ed	4364581705	fix accidental 422-ban when uploading lots of dupes	2023-09-08 19:49:29 +00:00
ed	ba02c9cc12	readme fix + make hacker theme more hacker	2023-09-08 19:35:12 +00:00
ed	11eefaf968	create / edit non-markdown textfiles (if user has delete-access) also enables the ansi escape code parser if the text looks like ansi	2023-09-08 18:47:31 +00:00
ed	5a968f9e47	add permission 'h': folders redirect to index.html; safest way to make copyparty like a general-purpose webserver where index.html is returned as expected yet directory listing is entirely disabled / unavailable	2023-09-07 23:30:01 +00:00
ed	6420c4bd03	up to 2.6x faster download-as-zip when there's lots of files, and especially small ones and also reduces cpu load by at least 15%	2023-09-05 22:57:03 +00:00
ed	0f9877201b	support cache directives in --css-browser, --js-browser; for example --css-browser=/the.css?cache=600 (seconds) or --js-browser=/.res/the.js?cache=i (7 days)	2023-09-03 19:50:31 +00:00
ed	9ba2dec9b2	lightbox: fix ccw rotation hotkey	2023-09-03 19:23:29 +00:00
ed	ae9cfea939	update pkgs to 1.9.4	2023-09-02 00:45:57 +00:00
ed	cadaeeeace	v1.9.4	2023-09-02 00:18:53 +00:00
ed	767696185b	add ?tar=gz, ?tar=bz2, ?tar=xz with optional level; defaults are ?tar=gz:3, ?tar=bz2:9, ?tar=xz:1	2023-09-01 23:44:10 +00:00
ed	c1efd227b7	fix inconsistent use of symlink mtimes in database; on upload, dupes are by default handled by symlinking to the existing copy on disk, writing the uploader's local mtime into the symlink mtime, which is also what gets indexed in the db this worked as intended, however during an -e2dsa rescan on startup the symlink destination timestamps would be used instead, causing a reindex and the resulting loss of uploader metadata (ip, timestamp) will now always use the symlink's mtime; worst-case 1% slower startup (no dhash) this change will cause a reindex of incorrectly indexed files, however as this has already happened at least once due to the bug being fixed, there will be no additional loss of metadata	2023-09-01 20:29:55 +00:00
ed	a50d0563c3	instantly perform search when URL contains a raw query	2023-09-01 20:16:19 +00:00
ed	e5641ddd16	update pkgs to 1.9.3	2023-08-31 23:08:32 +00:00
ed	700111ffeb	v1.9.3	2023-08-31 22:11:31 +00:00
ed	b8adeb824a	misc http correctness; some of this looks shady af but appears to have been harmless (decent amount of testing came out ok) * some location normalization happened before unquoting; however vfs handled this correctly so the outcome was just confusing messages * some url parameters were double-decoded (unpost filter, move destinations), causing some operations to fail unexpectedly * invalid cache-control headers could be generated, but not in a maliciously-beneficial way (there are safeguards stripping newlines and control-characters) also adds an exception-message cleanup step to strip away the filesystem path that copyparty's python files are located at, in case that could be interesting knowledge	2023-08-31 21:51:58 +00:00
ed	30cc9defcb	cosmetics: * in case someone gets a confusing access-related error message, include more context in serverlogs (exact path) * fix js console spam in search results * same markdown line-height in viewer and browser	2023-08-31 21:27:14 +00:00
ed	61875bd773	slightly reduce flickering during page load on chrome	2023-08-31 20:02:33 +00:00
ed	30905c6f5d	add convenient debugs in case the fight is not over	2023-08-31 20:00:14 +00:00
ed	9986136dfb	apple/ios/iphone: maybe fix background album playback good news: apple finally added support for samplerates other than 44100 for AudioContext, meaning it would now have been possible to set non-100% volume for audio files including opus files bad news: apple broke AudioContext in a way that makes it bug out mediaSessions, causing lockscreen controls to become mostly useless bad news: apple broke AudioContext additionally where it randomly causes playback issues, blocking playback of audio files, even if the AudioContext is sitting idle doing nothing (which is a requirement for reliable upload speeds on other platforms) disable AudioContext on iOS	2023-08-31 19:57:05 +00:00
ed	1c0d978979	ios/iphone: autoreplace smart-quotes with sane quotes, as the iphone keyboard is not able to produce ' or "	2023-08-31 19:29:37 +00:00
ed	0a0364e9f8	FTPd: fix py3.12 support; workaround until next release: run sfx twice with PYTHONPATH=/tmp/pe-copyparty.$(id -u)/copyparty/vend	2023-08-28 00:25:33 +00:00
ed	3376fbde1a	update pkgs to 1.9.2	2023-08-26 22:09:43 +00:00