v1.9.6

if the server is started while an external drive is not mounted,
it would drop the database because all the files are missing

README.md

@@ -287,6 +287,10 @@ server notes:
 * iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
   * `AudioContext` will probably never be a viable workaround as apple introduces new issues faster than they fix current ones
 
+* iPhones: the preload feature (in the media-player-options tab) can cause a tiny audio glitch 20sec before the end of each song, but disabling it may cause worse iOS bugs to appear instead
+  * just a hunch, but disabling preloading may cause playback to stop entirely, or possibly mess with bluetooth speakers
+  * tried to add a tooltip regarding this but looks like apple broke my tooltips
+
 * Windows: folders cannot be accessed if the name ends with `.`
   * python or windows bug
 
@@ -324,6 +328,12 @@ upgrade notes
 * can I make copyparty download a file to my server if I give it a URL?
   * yes, using [hooks](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/wget.py)
 
+* i want to learn python and/or programming and am considering looking at the copyparty source code in that occasion
+    ```bash
+     _|  _      __   _  _|_
+    (_| (_)     | | (_)  |_
+    ```
+
 
 # accounts and volumes
 
@@ -347,6 +357,7 @@ permissions:
 * `d` (delete): delete files/folders
 * `g` (get): only download files, cannot see folder contents or zip/tar
 * `G` (upget): same as `g` except uploaders get to see their own filekeys (see `fk` in examples below)
+* `h` (html): same as `g` except folders return their index.html, and filekeys are not necessary for index.html
 * `a` (admin): can see uploader IPs, config-reload
 
 examples:
@@ -505,10 +516,16 @@ select which type of archive you want in the `[⚙️] config` tab:
 | name | url-suffix | description |
 |--|--|--|
 | `tar` | `?tar` | plain gnutar, works great with `curl \| tar -xv` |
+| `pax` | `?tar=pax` | pax-format tar, futureproof, not as fast |
+| `tgz` | `?tar=gz` | gzip compressed gnu-tar (slow), for `curl \| tar -xvz` |
+| `txz` | `?tar=xz` | gnu-tar with xz / lzma compression (v.slow) |
 | `zip` | `?zip=utf8` | works everywhere, glitchy filenames on win7 and older |
 | `zip_dos` | `?zip` | traditional cp437 (no unicode) to fix glitchy filenames |
 | `zip_crc` | `?zip=crc` | cp437 with crc32 computed early for truly ancient software |
 
+* gzip default level is `3` (0=fast, 9=best), change with `?tar=gz:9`
+* xz default level is `1` (0=fast, 9=best), change with `?tar=xz:9`
+* bz2 default level is `2` (1=fast, 9=best), change with `?tar=bz2:9`
 * hidden files (dotfiles) are excluded unless `-ed`
   * `up2k.db` and `dir.txt` is always excluded
 * `zip_crc` will take longer to download since the server has to read each file twice
@@ -893,15 +910,13 @@ unsafe, slow, not recommended for wan,  enable with `--smb` for read-only or `--
 
 click the [connect](http://127.0.0.1:3923/?hc) button in the control-panel to see connection instructions for windows, linux, macos
 
-dependencies: `python3 -m pip install --user -U impacket==0.10.0`
+dependencies: `python3 -m pip install --user -U impacket==0.11.0`
 * newer versions of impacket will hopefully work just fine but there is monkeypatching so maybe not
 
 some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
 * not entirely confident that read-only is read-only
 * the smb backend is not fully integrated with vfs, meaning there could be security issues (path traversal). Please use `--smb-port` (see below) and [prisonparty](./bin/prisonparty.sh)
-  * account passwords work per-volume as expected, but account permissions are coalesced; all accounts have read-access to all volumes, and if a single account has write-access to some volume then all other accounts also do
-    * if no accounts have write-access to a specific volume, or if `--smbw` is not set, then writing to that volume from smb *should* be impossible
-    * will be fixed once [impacket v0.11.0](https://github.com/SecureAuthCorp/impacket/commit/d923c00f75d54b972bca573a211a82f09b55261a) is released
+  * account passwords work per-volume as expected, and so does account permissions (read/write/move/delete), but `--smbw` must be given to allow write-access from smb
   * [shadowing](#shadowing) probably works as expected but no guarantees
 
 and some minor issues,
@@ -912,7 +927,7 @@ and some minor issues,
   * win10 onwards does not allow connecting anonymously / without accounts
 * on windows, creating a new file through rightclick --> new --> textfile throws an error due to impacket limitations -- hit OK and F5 to get your file
 * python3 only
-* slow
+* slow (the builtin webdav support in windows is 5x faster, and rclone-webdav is 30x faster)
 
 known client bugs:
 * on win7 only, `--smb1` is much faster than smb2 (default) because it keeps rescanning folders on smb2
@@ -1631,6 +1646,8 @@ other misc notes:
   * combine this with volflag `c,fk` to generate filekeys (per-file accesskeys); users which have full read-access will then see URLs with `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
     * the default filekey entropy is fairly small so give `--fk-salt` around 30 characters if you want filekeys longer than 16 chars
   * permissions `wG` lets users upload files and receive their own filekeys, still without being able to see other uploads
+  * permission `h` instead of `r` makes copyparty behave like a traditional webserver with directory listing/index disabled, returning index.html instead
+    * compatibility with filekeys: index.html itself can be retrieved without the correct filekey, but all other files are protected
 
 
 ## gotchas
@@ -1734,7 +1751,7 @@ enable [thumbnails](#thumbnails) of...
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`
 
 enable [smb](#smb-server) support (**not** recommended):
-* `impacket==0.10.0`
+* `impacket==0.11.0`
 
 `pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
 

bin/hooks/msg-log.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python
+# coding: utf-8
+# vim:ts=4:sw=4:softtabstop=4:smarttab:expandtab
+from __future__ import print_function, unicode_literals
+
+import json
+import os
+import sys
+import time
+from datetime import datetime
+
+
+"""
+use copyparty as a dumb messaging server / guestbook thing;
+initially contributed by @clach04 in https://github.com/9001/copyparty/issues/35 (thanks!)
+
+Sample usage:
+
+    python copyparty-sfx.py --xm j,bin/hooks/msg-log.py
+
+Where:
+
+    xm = execute on message-to-server-log
+    j = provide message information as json; not just the text - this script REQUIRES json
+    t10 = timeout and kill download after 10 secs
+"""
+
+
+# output filename
+FILENAME = os.environ.get("COPYPARTY_MESSAGE_FILENAME", "") or "README.md"
+
+# set True to write in descending order (newest message at top of file);
+# note that this becomes very slow/expensive as the file gets bigger
+DESCENDING = True
+
+# the message template; the following parameters are provided by copyparty and can be referenced below:
+# 'ap' = absolute filesystem path where the message was posted
+# 'vp' = virtual path (URL 'path') where the message was posted
+# 'mt' = 'at' = unix-timestamp when the message was posted
+# 'datetime' = ISO-8601 time when the message was posted
+# 'sz' = message size in bytes
+# 'host' = the server hostname which the user was accessing (URL 'host')
+# 'user' = username (if logged in), otherwise '*'
+# 'txt' = the message text itself
+# (uncomment the print(msg_info) to see if additional information has been introduced by copyparty since this was written)
+TEMPLATE = """
+🕒 %(datetime)s, 👤 %(user)s @ %(ip)s
+%(txt)s
+"""
+
+
+def write_ascending(filepath, msg_text):
+    with open(filepath, "a", encoding="utf-8", errors="replace") as outfile:
+        outfile.write(msg_text)
+
+
+def write_descending(filepath, msg_text):
+    lockpath = filepath + ".lock"
+    got_it = False
+    for _ in range(16):
+        try:
+            os.mkdir(lockpath)
+            got_it = True
+            break
+        except:
+            time.sleep(0.1)
+            continue
+
+    if not got_it:
+        return sys.exit(1)
+
+    try:
+        oldpath = filepath + ".old"
+        os.rename(filepath, oldpath)
+        with open(oldpath, "r", encoding="utf-8", errors="replace") as infile, open(
+            filepath, "w", encoding="utf-8", errors="replace"
+        ) as outfile:
+            outfile.write(msg_text)
+            while True:
+                buf = infile.read(4096)
+                if not buf:
+                    break
+                outfile.write(buf)
+    finally:
+        try:
+            os.unlink(oldpath)
+        except:
+            pass
+        os.rmdir(lockpath)
+
+
+def main(argv=None):
+    if argv is None:
+        argv = sys.argv
+
+    msg_info = json.loads(sys.argv[1])
+    # print(msg_info)
+
+    dt = datetime.utcfromtimestamp(msg_info["at"])
+    msg_info["datetime"] = dt.strftime("%Y-%m-%d, %H:%M:%S")
+
+    msg_text = TEMPLATE % msg_info
+
+    filepath = os.path.join(msg_info["ap"], FILENAME)
+
+    if DESCENDING and os.path.exists(filepath):
+        write_descending(filepath, msg_text)
+    else:
+        write_ascending(filepath, msg_text)
+
+    print(msg_text)
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/install-deps.sh

@@ -250,8 +250,9 @@ install_vamp() {
 		rm -- *.tar.gz
 		cd beatroot-vamp-v1.0
 		[ -e ~/pe/vamp-sdk ] &&
-			sed -ri 's`^(CFLAGS :=.*)`\1 -I'$HOME'/pe/vamp-sdk/include`' Makefile.linux
-		make -f Makefile.linux -j4 LDFLAGS=-L$HOME/pe/vamp-sdk/lib
+			sed -ri 's`^(CFLAGS :=.*)`\1 -I'$HOME'/pe/vamp-sdk/include`' Makefile.linux ||
+			sed -ri 's`^(CFLAGS :=.*)`\1 -I/usr/include/vamp-sdk`' Makefile.linux
+		make -f Makefile.linux -j4 LDFLAGS="-L$HOME/pe/vamp-sdk/lib -L/usr/lib64"
 		# /home/ed/vamp /home/ed/.vamp /usr/local/lib/vamp
 		mkdir ~/vamp
 		cp -pv beatroot-vamp.* ~/vamp/

contrib/index.html

@@ -26,8 +26,8 @@ a {
 	<script>
 
 var a = document.getElementById('redir'),
-	proto = window.location.protocol.indexOf('https') === 0 ? 'https' : 'http',
-	loc = window.location.hostname || '127.0.0.1',
+	proto = location.protocol.indexOf('https') === 0 ? 'https' : 'http',
+	loc = location.hostname || '127.0.0.1',
 	port = a.getAttribute('href').split(':').pop().split('/')[0],
 	url = proto + '://' + loc + ':' + port + '/';
 
@@ -35,7 +35,7 @@ a.setAttribute('href', url);
 document.getElementById('desc').innerHTML = 'redirecting to';
 
 setTimeout(function() {
-	window.location.href = url;
+	location.href = url;
 }, 500);
 
 </script>

contrib/nixos/modules/copyparty.nix

@@ -138,7 +138,8 @@ in {
                 "d" (delete): permanently delete files and folders
                 "g" (get):    download files, but cannot see folder contents
                 "G" (upget):  "get", but can see filekeys of their own uploads
-                "a" (upget):  can see uploader IPs, config-reload
+                "h" (html):   "get", but folders return their index.html
+                "a" (admin):  can see uploader IPs, config-reload
 
               For example: "rwmd"
 

contrib/package/arch/PKGBUILD

@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.9.2"
+pkgver="1.9.5"
 pkgrel=1
 pkgdesc="Portable file sharing hub"
 arch=("any")
@@ -20,7 +20,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("d9509c0cf7d325124ce40637258574f62e628f06f74a61b07168fe4afa4ef489")
+sha256sums=("0c68368bab5b17716860994b3e6485e4d396d0ec3eb5f1c8947eca8cb2b18821")
 
 build() {
     cd "${srcdir}/${pkgname}-${pkgver}"

contrib/package/nix/copyparty/pin.json

@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.9.2/copyparty-sfx.py",
-    "version": "1.9.2",
-    "hash": "sha256-Y5sreREKvuHmKZy6kGk/oBHfKTq8b3SZGGWm9u2Iw9E="
+    "url": "https://github.com/9001/copyparty/releases/download/v1.9.5/copyparty-sfx.py",
+    "version": "1.9.5",
+    "hash": "sha256-NT8se/f9vf6iiMVIOxTKdG7jddmVUCv12C0R2yhhy1Q="
 }

copyparty/__main__.py

@@ -492,6 +492,7 @@ def get_sects():
               "d" (delete): permanently delete files and folders
               "g" (get):    download files, but cannot see folder contents
               "G" (upget):  "get", but can see filekeys of their own uploads
+              "h" (html):   "get", but folders return their index.html
               "a" (admin):  can see uploader IPs, config-reload
 
             too many volflags to list here, see --help-flags
@@ -825,7 +826,9 @@ def add_network(ap):
     ap2.add_argument("-i", metavar="IP", type=u, default="::", help="ip to bind (comma-sep.), default: all IPv4 and IPv6")
     ap2.add_argument("-p", metavar="PORT", type=u, default="3923", help="ports to bind (comma/range)")
     ap2.add_argument("--ll", action="store_true", help="include link-local IPv4/IPv6 even if the NIC has routable IPs (breaks some mdns clients)")
-    ap2.add_argument("--rproxy", metavar="DEPTH", type=int, default=1, help="which ip to keep; [\033[32m0\033[0m]=tcp, [\033[32m1\033[0m]=origin (first x-fwd), [\033[32m2\033[0m]=cloudflare, [\033[32m3\033[0m]=nginx, [\033[32m-1\033[0m]=closest proxy")
+    ap2.add_argument("--rproxy", metavar="DEPTH", type=int, default=1, help="which ip to keep; [\033[32m0\033[0m]=tcp, [\033[32m1\033[0m]=origin (first x-fwd, unsafe), [\033[32m2\033[0m]=outermost-proxy, [\033[32m3\033[0m]=second-proxy, [\033[32m-1\033[0m]=closest-proxy")
+    ap2.add_argument("--xff-hdr", metavar="NAME", type=u, default="x-forwarded-for", help="if reverse-proxied, which http header to read the client's real ip from (argument must be lowercase, but not the actual header)")
+    ap2.add_argument("--xff-src", metavar="IP", type=u, default="127., ::1", help="comma-separated list of trusted reverse-proxy IPs; only accept the real-ip header (--xff-hdr) if the incoming connection is from an IP starting with either of these. Can be disabled with [\033[32many\033[0m] if you are behind cloudflare (or similar) and are using --xff-hdr=cf-connecting-ip (or similar)")
     ap2.add_argument("--rp-loc", metavar="PATH", type=u, default="", help="if reverse-proxying on a location instead of a dedicated domain/subdomain, provide the base location here (eg. /foo/bar)")
     if ANYWIN:
         ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
@@ -931,12 +934,13 @@ def add_webdav(ap):
 
 def add_smb(ap):
     ap2 = ap.add_argument_group('SMB/CIFS options')
-    ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless --smb-port is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is dangerous! Never expose to the internet. Account permissions are coalesced; if one account has write-access to a volume, then all accounts do.")
+    ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless --smb-port is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is dangerous! Never expose to the internet!")
     ap2.add_argument("--smbw", action="store_true", help="enable write support (please dont)")
     ap2.add_argument("--smb1", action="store_true", help="disable SMBv2, only enable SMBv1 (CIFS)")
     ap2.add_argument("--smb-port", metavar="PORT", type=int, default=445, help="port to listen on -- if you change this value, you must NAT from TCP:445 to this port using iptables or similar")
     ap2.add_argument("--smb-nwa-1", action="store_true", help="disable impacket#1433 workaround (truncate directory listings to 64kB)")
     ap2.add_argument("--smb-nwa-2", action="store_true", help="disable impacket workaround for filecopy globs")
+    ap2.add_argument("--smba", action="store_true", help="small performance boost: disable per-account permissions, enables account coalescing instead (if one user has write/delete-access, then everyone does)")
     ap2.add_argument("--smbv", action="store_true", help="verbose")
     ap2.add_argument("--smbvv", action="store_true", help="verboser")
     ap2.add_argument("--smbvvv", action="store_true", help="verbosest")
@@ -989,6 +993,7 @@ def add_optouts(ap):
     ap2.add_argument("-nid", action="store_true", help="no info disk-usage -- don't show in UI")
     ap2.add_argument("-nb", action="store_true", help="no powered-by-copyparty branding in UI")
     ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
+    ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
     ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (as specified by the 'lifetime' volflag)")
 
 
@@ -1013,8 +1018,8 @@ def add_safety(ap):
     ap2.add_argument("--ban-403", metavar="N,W,B", type=u, default="9,2,1440", help="hitting more than \033[33mN\033[0m 403's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; [\033[32m1440\033[0m]=day, [\033[32m10080\033[0m]=week, [\033[32m43200\033[0m]=month")
     ap2.add_argument("--ban-422", metavar="N,W,B", type=u, default="9,2,1440", help="hitting more than \033[33mN\033[0m 422's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes (422 is server fuzzing, invalid POSTs and so)")
     ap2.add_argument("--ban-url", metavar="N,W,B", type=u, default="9,2,1440", help="hitting more than \033[33mN\033[0m sus URL's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes (decent replacement for --ban-404 if that can't be used)")
-    ap2.add_argument("--sus-urls", metavar="REGEX", type=u, default=r"\.php$|(^|/)wp-(admin|content|includes)/", help="URLs which are considered sus / eligible for banning; disable with blank or [\033[32mno\033[0m]")
-    ap2.add_argument("--nonsus-urls", metavar="REGEX", type=u, default=r"^(favicon\.ico|robots\.txt)$|^apple-touch-icon|^\.well-known", help="harmless URLs ignored from 404-bans; disable with blank or [\033[32mno\033[0m]")
+    ap2.add_argument("--sus-urls", metavar="R", type=u, default=r"\.php$|(^|/)wp-(admin|content|includes)/", help="URLs which are considered sus / eligible for banning; disable with blank or [\033[32mno\033[0m]")
+    ap2.add_argument("--nonsus-urls", metavar="R", type=u, default=r"^(favicon\.ico|robots\.txt)$|^apple-touch-icon|^\.well-known", help="harmless URLs ignored from 404-bans; disable with blank or [\033[32mno\033[0m]")
     ap2.add_argument("--aclose", metavar="MIN", type=int, default=10, help="if a client maxes out the server connection limit, downgrade it from connection:keep-alive to connection:close for MIN minutes (and also kill its active connections) -- disable with 0")
     ap2.add_argument("--loris", metavar="B", type=int, default=60, help="if a client maxes out the server connection limit without sending headers, ban it for B minutes; disable with [\033[32m0\033[0m]")
     ap2.add_argument("--acao", metavar="V[,V]", type=u, default="*", help="Access-Control-Allow-Origin; list of origins (domains/IPs without port) to accept requests from; [\033[32mhttps://1.2.3.4\033[0m]. Default [\033[32m*\033[0m] allows requests from all sites but removes cookies and http-auth; only ?pw=hunter2 survives")
@@ -1045,7 +1050,7 @@ def add_logging(ap):
     ap2.add_argument("--no-ansi", action="store_true", default=not VT100, help="disable colors; same as environment-variable NO_COLOR")
     ap2.add_argument("--ansi", action="store_true", help="force colors; overrides environment-variable NO_COLOR")
     ap2.add_argument("--no-voldump", action="store_true", help="do not list volumes and permissions on startup")
-    ap2.add_argument("--log-tdec", type=int, default=3, help="timestamp resolution / number of timestamp decimals")
+    ap2.add_argument("--log-tdec", metavar="N", type=int, default=3, help="timestamp resolution / number of timestamp decimals")
     ap2.add_argument("--log-conn", action="store_true", help="debug: print tcp-server msgs")
     ap2.add_argument("--log-htp", action="store_true", help="debug: print http-server threadpool scaling")
     ap2.add_argument("--ihead", metavar="HEADER", type=u, action='append', help="dump incoming header")
@@ -1149,7 +1154,7 @@ def add_ui(ap, retry):
     ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
     ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
-    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI)")
+    ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
     ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
     ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
     ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty @ --name", help="title / service-name to show in html documents")
@@ -1196,7 +1201,10 @@ def run_argparse(
     fk_salt = get_fk_salt(cert_path)
     ah_salt = get_ah_salt()
 
-    hcores = min(CORES, 4)  # optimal on py3.11 @ r5-4500U
+    # alpine peaks at 5 threads for some reason,
+    # all others scale past that (but try to avoid SMT),
+    # 5 should be plenty anyways (3 GiB/s on most machines)
+    hcores = min(CORES, 5 if CORES > 8 else 4)
 
     tty = os.environ.get("TERM", "").lower() == "linux"
 
@@ -1397,7 +1405,7 @@ def main(argv: Optional[list[str]] = None) -> None:
             if re.match("c[^,]", opt):
                 mod = True
                 na.append("c," + opt[1:])
-            elif re.sub("^[rwmdgGa]*", "", opt) and "," not in opt:
+            elif re.sub("^[rwmdgGha]*", "", opt) and "," not in opt:
                 mod = True
                 perm = opt[0]
                 na.append(perm + "," + opt[1:])

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 9, 3)
+VERSION = (1, 9, 6)
 CODENAME = "prometheable"
-BUILD_DT = (2023, 8, 31)
+BUILD_DT = (2023, 9, 23)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -67,6 +67,7 @@ class AXS(object):
         udel: Optional[Union[list[str], set[str]]] = None,
         uget: Optional[Union[list[str], set[str]]] = None,
         upget: Optional[Union[list[str], set[str]]] = None,
+        uhtml: Optional[Union[list[str], set[str]]] = None,
         uadmin: Optional[Union[list[str], set[str]]] = None,
     ) -> None:
         self.uread: set[str] = set(uread or [])
@@ -75,10 +76,11 @@ class AXS(object):
         self.udel: set[str] = set(udel or [])
         self.uget: set[str] = set(uget or [])
         self.upget: set[str] = set(upget or [])
+        self.uhtml: set[str] = set(uhtml or [])
         self.uadmin: set[str] = set(uadmin or [])
 
     def __repr__(self) -> str:
-        ks = "uread uwrite umove udel uget upget uadmin".split()
+        ks = "uread uwrite umove udel uget upget uhtml uadmin".split()
         return "AXS(%s)" % (", ".join("%s=%r" % (k, self.__dict__[k]) for k in ks),)
 
 
@@ -329,6 +331,7 @@ class VFS(object):
         self.adel: dict[str, list[str]] = {}
         self.aget: dict[str, list[str]] = {}
         self.apget: dict[str, list[str]] = {}
+        self.ahtml: dict[str, list[str]] = {}
         self.aadmin: dict[str, list[str]] = {}
 
         if realpath:
@@ -456,6 +459,7 @@ class VFS(object):
             uname in c.upget or "*" in c.upget,
             uname in c.uadmin or "*" in c.uadmin,
         )
+        # skip uhtml because it's rarely needed
 
     def get(
         self,
@@ -488,7 +492,7 @@ class VFS(object):
             (will_get, c.uget, "get"),
         ]:
             if req and (uname not in d and "*" not in d) and uname != LEELOO_DALLAS:
-                if self.log and err != 999:
+                if vpath != cvpath and vpath != "." and self.log:
                     ap = vn.canonical(rem)
                     t = "{} has no {} in [{}] => [{}] => [{}]"
                     self.log("vfs", t.format(uname, msg, vpath, cvpath, ap), 6)
@@ -955,7 +959,7 @@ class AuthSrv(object):
                 try:
                     self._l(ln, 5, "volume access config:")
                     sk, sv = ln.split(":")
-                    if re.sub("[rwmdgGa]", "", sk) or not sk:
+                    if re.sub("[rwmdgGha]", "", sk) or not sk:
                         err = "invalid accs permissions list; "
                         raise Exception(err)
                     if " " in re.sub(", *", "", sv).strip():
@@ -964,7 +968,7 @@ class AuthSrv(object):
                     self._read_vol_str(sk, sv.replace(" ", ""), daxs[vp], mflags[vp])
                     continue
                 except:
-                    err += "accs entries must be 'rwmdgGa: user1, user2, ...'"
+                    err += "accs entries must be 'rwmdgGha: user1, user2, ...'"
                     raise Exception(err + SBADCFG)
 
             if cat == catf:
@@ -1000,7 +1004,7 @@ class AuthSrv(object):
     def _read_vol_str(
         self, lvl: str, uname: str, axs: AXS, flags: dict[str, Any]
     ) -> None:
-        if lvl.strip("crwmdgGa"):
+        if lvl.strip("crwmdgGha"):
             raise Exception("invalid volflag: {},{}".format(lvl, uname))
 
         if lvl == "c":
@@ -1029,10 +1033,12 @@ class AuthSrv(object):
                 ("w", axs.uwrite),
                 ("m", axs.umove),
                 ("d", axs.udel),
+                ("a", axs.uadmin),
+                ("h", axs.uhtml),
+                ("h", axs.uget),
                 ("g", axs.uget),
                 ("G", axs.uget),
                 ("G", axs.upget),
-                ("a", axs.uadmin),
             ]:  # b bb bbb
                 if ch in lvl:
                     if un == "*":
@@ -1105,7 +1111,7 @@ class AuthSrv(object):
 
         if self.args.v:
             # list of src:dst:permset:permset:...
-            # permset is <rwmdgGa>[,username][,username] or <c>,<flag>[=args]
+            # permset is <rwmdgGha>[,username][,username] or <c>,<flag>[=args]
             for v_str in self.args.v:
                 m = re_vol.match(v_str)
                 if not m:
@@ -1194,7 +1200,7 @@ class AuthSrv(object):
             vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
             vol.root = vfs
 
-        for perm in "read write move del get pget admin".split():
+        for perm in "read write move del get pget html admin".split():
             axs_key = "u" + perm
             unames = ["*"] + list(acct.keys())
             umap: dict[str, list[str]] = {x: [] for x in unames}
@@ -1216,6 +1222,7 @@ class AuthSrv(object):
                 axs.udel,
                 axs.uget,
                 axs.upget,
+                axs.uhtml,
                 axs.uadmin,
             ]:
                 for usr in d:
@@ -1637,6 +1644,7 @@ class AuthSrv(object):
                 ["delete", "udel"],
                 ["   get", "uget"],
                 [" upget", "upget"],
+                ["  html", "uhtml"],
                 ["uadmin", "uadmin"],
             ]:
                 u = list(sorted(getattr(zv.axs, attr)))
@@ -1675,7 +1683,7 @@ class AuthSrv(object):
                 self.log(t.format(zv.realpath), c=1)
 
         try:
-            zv, _ = vfs.get("/", "*", False, True, err=999)
+            zv, _ = vfs.get("", "*", False, True, err=999)
             if self.warn_anonwrite and os.getcwd() == zv.realpath:
                 t = "anyone can write to the current directory: {}\n"
                 self.log(t.format(zv.realpath), c=1)
@@ -1804,6 +1812,7 @@ class AuthSrv(object):
                 vc.udel,
                 vc.uget,
                 vc.upget,
+                vc.uhtml,
                 vc.uadmin,
             ]
             self.log(t.format(*vs))
@@ -1945,6 +1954,7 @@ class AuthSrv(object):
                 "d": "udel",
                 "g": "uget",
                 "G": "upget",
+                "h": "uhtml",
                 "a": "uadmin",
             }
             users = {}
@@ -2148,7 +2158,7 @@ def upgrade_cfg_fmt(
             else:
                 sn = sn.replace(",", ", ")
             ret.append("    " + sn)
-        elif sn[:1] in "rwmdgGa":
+        elif sn[:1] in "rwmdgGha":
             if cat != catx:
                 cat = catx
                 ret.append(cat)

copyparty/broker_mp.py

@@ -69,7 +69,7 @@ class BrokerMp(object):
 
         while procs:
             if procs[-1].is_alive():
-                time.sleep(0.1)
+                time.sleep(0.05)
                 continue
 
             procs.pop()

copyparty/cfg.py

@@ -62,6 +62,8 @@ permdescs = {
     "d": "delete; permanently delete files and folders",
     "g": "get; download files, but cannot see folder contents",
     "G": 'upget; same as "g" but can see filekeys of their own uploads',
+    "h": 'html; same as "g" but folders return their index.html',
+    "a": "admin; can see uploader IPs, config-reload",
 }
 
 

copyparty/httpcli.py

@@ -283,22 +283,35 @@ class HttpCli(object):
 
         n = self.args.rproxy
         if n:
-            zso = self.headers.get("x-forwarded-for")
-            if zso and self.conn.addr[0] in ["127.0.0.1", "::1"]:
+            zso = self.headers.get(self.args.xff_hdr)
+            if zso:
                 if n > 0:
                     n -= 1
 
                 zsl = zso.split(",")
                 try:
-                    self.ip = zsl[n].strip()
+                    cli_ip = zsl[n].strip()
                 except:
-                    self.ip = zsl[0].strip()
+                    cli_ip = zsl[0].strip()
                     t = "rproxy={} oob x-fwd {}"
                     self.log(t.format(self.args.rproxy, zso), c=3)
 
-                self.log_src = self.conn.set_rproxy(self.ip)
-                self.is_vproxied = bool(self.args.R)
-                self.host = self.headers.get("x-forwarded-host") or self.host
+                pip = self.conn.addr[0]
+                if self.args.xff_re and not self.args.xff_re.match(pip):
+                    t = 'got header "%s" from untrusted source "%s" claiming the true client ip is "%s" (raw value: "%s");  if you trust this, you must allowlist this proxy with "--xff-src=%s"'
+                    if self.headers.get("cf-connecting-ip"):
+                        t += "  Alternatively, if you are behind cloudflare, it is better to specify these two instead:  --xff-hdr=cf-connecting-ip  --xff-src=any"
+                    zs = (
+                        ".".join(pip.split(".")[:2]) + "."
+                        if "." in pip
+                        else ":".join(pip.split(":")[:4]) + ":"
+                    )
+                    self.log(t % (self.args.xff_hdr, pip, cli_ip, zso, zs), 3)
+                else:
+                    self.ip = cli_ip
+                    self.is_vproxied = bool(self.args.R)
+                    self.log_src = self.conn.set_rproxy(self.ip)
+                    self.host = self.headers.get("x-forwarded-host") or self.host
 
         if self.is_banned():
             return False
@@ -629,7 +642,17 @@ class HttpCli(object):
         headers: Optional[dict[str, str]] = None,
         volsan: bool = False,
     ) -> bytes:
-        if status > 400 and status in (403, 404, 422):
+        if (
+            status > 400
+            and status in (403, 404, 422)
+            and (
+                status != 422
+                or (
+                    not body.startswith(b"<pre>partial upload exists")
+                    and not body.startswith(b"<pre>source file busy")
+                )
+            )
+        ):
             if status == 404:
                 g = self.conn.hsrv.g404
             elif status == 403:
@@ -2186,7 +2209,8 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
-        if not new_file.endswith(".md"):
+        ext = "" if "." not in new_file else new_file.split(".")[-1]
+        if not ext or len(ext) > 5 or not self.can_delete:
             new_file += ".md"
 
         sanitized = sanitize_fn(new_file, "", [])
@@ -2519,7 +2543,7 @@ class HttpCli(object):
         fp = os.path.join(fp, fn)
         rem = "{}/{}".format(rp, fn).strip("/")
 
-        if not rem.endswith(".md"):
+        if not rem.endswith(".md") and not self.can_delete:
             raise Pebkac(400, "only markdown pls")
 
         if nullwrite:
@@ -2570,7 +2594,8 @@ class HttpCli(object):
                 return True
 
             mdir, mfile = os.path.split(fp)
-            mfile2 = "{}.{:.3f}.md".format(mfile[:-3], srv_lastmod)
+            fname, fext = mfile.rsplit(".", 1) if "." in mfile else (mfile, "md")
+            mfile2 = "{}.{:.3f}.{}".format(fname, srv_lastmod, fext)
             try:
                 dp = os.path.join(mdir, ".hist")
                 bos.mkdir(dp)
@@ -2895,12 +2920,26 @@ class HttpCli(object):
         logmsg = "{:4} {} ".format("", self.req)
         self.keepalive = False
 
+        cancmp = not self.args.no_tarcmp
+
         if fmt == "tar":
-            mime = "application/x-tar"
             packer: Type[StreamArc] = StreamTar
+            if cancmp and "gz" in uarg:
+                mime = "application/gzip"
+                ext = "tar.gz"
+            elif cancmp and "bz2" in uarg:
+                mime = "application/x-bzip"
+                ext = "tar.bz2"
+            elif cancmp and "xz" in uarg:
+                mime = "application/x-xz"
+                ext = "tar.xz"
+            else:
+                mime = "application/x-tar"
+                ext = "tar"
         else:
             mime = "application/zip"
             packer = StreamZip
+            ext = "zip"
 
         fn = items[0] if items and items[0] else self.vpath
         if fn:
@@ -2925,7 +2964,7 @@ class HttpCli(object):
         ufn = b"".join(zbl).decode("ascii")
 
         cdis = "attachment; filename=\"{}.{}\"; filename*=UTF-8''{}.{}"
-        cdis = cdis.format(afn, fmt, ufn, fmt)
+        cdis = cdis.format(afn, ext, ufn, ext)
         self.log(cdis)
         self.send_headers(None, mime=mime, headers={"Content-Disposition": cdis})
 
@@ -2943,7 +2982,13 @@ class HttpCli(object):
                 self.log("transcoding to [{}]".format(cfmt))
                 fgen = gfilter(fgen, self.thumbcli, self.uname, vpath, cfmt)
 
-        bgen = packer(self.log, fgen, utf8="utf" in uarg, pre_crc="crc" in uarg)
+        bgen = packer(
+            self.log,
+            fgen,
+            utf8="utf" in uarg,
+            pre_crc="crc" in uarg,
+            cmp=uarg if cancmp or uarg == "pax" else "",
+        )
         bsent = 0
         for buf in bgen.gen():
             if not buf:
@@ -3587,6 +3632,7 @@ class HttpCli(object):
             self.out_headers.pop("X-Robots-Tag", None)
 
         is_dir = stat.S_ISDIR(st.st_mode)
+        fk_pass = False
         icur = None
         if is_dir and (e2t or e2d):
             idx = self.conn.get_u2idx()
@@ -3635,8 +3681,38 @@ class HttpCli(object):
 
                 return self.tx_ico(rem)
 
+        elif self.can_get and self.avn:
+            axs = self.avn.axs
+            if self.uname not in axs.uhtml and "*" not in axs.uhtml:
+                pass
+            elif is_dir:
+                for fn in ("index.htm", "index.html"):
+                    ap2 = os.path.join(abspath, fn)
+                    try:
+                        st2 = bos.stat(ap2)
+                    except:
+                        continue
+
+                    # might as well be extra careful
+                    if not stat.S_ISREG(st2.st_mode):
+                        continue
+
+                    if not self.trailing_slash:
+                        return self.redirect(
+                            self.vpath + "/", flavor="redirecting to", use302=True
+                        )
+
+                    fk_pass = True
+                    is_dir = False
+                    rem = vjoin(rem, fn)
+                    vrem = vjoin(vrem, fn)
+                    abspath = ap2
+                    break
+            elif self.vpath.rsplit("/", 1)[1] in ("index.htm", "index.html"):
+                fk_pass = True
+
         if not is_dir and (self.can_read or self.can_get):
-            if not self.can_read and "fk" in vn.flags:
+            if not self.can_read and not fk_pass and "fk" in vn.flags:
                 correct = self.gen_fk(
                     self.args.fk_salt, abspath, st.st_size, 0 if ANYWIN else st.st_ino
                 )[: vn.flags["fk"]]
@@ -3646,7 +3722,7 @@ class HttpCli(object):
                     return self.tx_404()
 
             if (
-                abspath.endswith(".md")
+                (abspath.endswith(".md") or self.can_delete)
                 and "nohtml" not in vn.flags
                 and (
                     "v" in self.uparam
@@ -3780,10 +3856,14 @@ class HttpCli(object):
         }
 
         if self.args.js_browser:
-            j2a["js"] = self.args.js_browser
+            zs = self.args.js_browser
+            zs += "&" if "?" in zs else "?"
+            j2a["js"] = zs
 
         if self.args.css_browser:
-            j2a["css"] = self.args.css_browser
+            zs = self.args.css_browser
+            zs += "&" if "?" in zs else "?"
+            j2a["css"] = zs
 
         if not self.conn.hsrv.prism:
             j2a["no_prism"] = True

copyparty/mtag.py

@@ -8,7 +8,7 @@ import shutil
 import subprocess as sp
 import sys
 
-from .__init__ import EXE, PY2, WINDOWS, E, unicode
+from .__init__ import ANYWIN, EXE, PY2, WINDOWS, E, unicode
 from .bos import bos
 from .util import (
     FFMPEG_URL,
@@ -29,6 +29,9 @@ if True:  # pylint: disable=using-constant-test
 
 
 def have_ff(scmd: str) -> bool:
+    if ANYWIN:
+        scmd += ".exe"
+
     if PY2:
         print("# checking {}".format(scmd))
         acmd = (scmd + " -version").encode("ascii").split(b" ")

copyparty/smbd.py

@@ -32,6 +32,8 @@ class SMB(object):
         self.asrv = hub.asrv
         self.log = hub.log
         self.files: dict[int, tuple[float, str]] = {}
+        self.noacc = self.args.smba
+        self.accs = not self.args.smba
 
         lg.setLevel(logging.DEBUG if self.args.smbvvv else logging.INFO)
         for x in ["impacket", "impacket.smbserver"]:
@@ -94,6 +96,14 @@ class SMB(object):
 
         port = int(self.args.smb_port)
         srv = smbserver.SimpleSMBServer(listenAddress=ip, listenPort=port)
+        try:
+            if self.accs:
+                srv.setAuthCallback(self._auth_cb)
+        except:
+            self.accs = False
+            self.noacc = True
+            t = "impacket too old; access permissions will not work! all accounts are admin!"
+            self.log("smb", t, 1)
 
         ro = "no" if self.args.smbw else "yes"  # (does nothing)
         srv.addShare("A", "/", readOnly=ro)
@@ -119,24 +129,73 @@ class SMB(object):
     def start(self) -> None:
         Daemon(self.srv.start)
 
-    def _v2a(self, caller: str, vpath: str, *a: Any) -> tuple[VFS, str]:
+    def _auth_cb(self, *a, **ka):
+        debug("auth-result: %s %s", a, ka)
+        conndata = ka["connData"]
+        auth_ok = conndata["Authenticated"]
+        uname = ka["user_name"] if auth_ok else "*"
+        uname = self.asrv.iacct.get(uname, uname) or "*"
+        oldname = conndata.get("partygoer", "*") or "*"
+        cli_ip = conndata["ClientIP"]
+        cli_hn = ka["host_name"]
+        if uname != "*":
+            conndata["partygoer"] = uname
+            info("client %s [%s] authed as %s", cli_ip, cli_hn, uname)
+        elif oldname != "*":
+            info("client %s [%s] keeping old auth as %s", cli_ip, cli_hn, oldname)
+        elif auth_ok:
+            info("client %s [%s] authed as [*] (anon)", cli_ip, cli_hn)
+        else:
+            info("client %s [%s] rejected", cli_ip, cli_hn)
+
+    def _uname(self) -> str:
+        if self.noacc:
+            return LEELOO_DALLAS
+
+        try:
+            # you found it! my single worst bit of code so far
+            # (if you can think of a better way to track users through impacket i'm all ears)
+            cf0 = inspect.currentframe().f_back.f_back
+            cf = cf0.f_back
+            for n in range(3):
+                cl = cf.f_locals
+                if "connData" in cl:
+                    return cl["connData"]["partygoer"]
+                cf = cf.f_back
+        except:
+            warning(
+                "nyoron... %s <<-- %s <<-- %s <<-- %s",
+                cf0.f_code.co_name,
+                cf0.f_back.f_code.co_name,
+                cf0.f_back.f_back.f_code.co_name,
+                cf0.f_back.f_back.f_back.f_code.co_name,
+            )
+            return "*"
+
+    def _v2a(
+        self, caller: str, vpath: str, *a: Any, uname="", perms=None
+    ) -> tuple[VFS, str]:
         vpath = vpath.replace("\\", "/").lstrip("/")
         # cf = inspect.currentframe().f_back
         # c1 = cf.f_back.f_code.co_name
         # c2 = cf.f_code.co_name
-        debug('%s("%s", %s)\033[K\033[0m', caller, vpath, str(a))
+        if not uname:
+            uname = self._uname()
+        if not perms:
+            perms = [True, True]
 
-        # TODO find a way to grab `identity` in smbComSessionSetupAndX and smb2SessionSetup
-        vfs, rem = self.asrv.vfs.get(vpath, LEELOO_DALLAS, True, True)
+        debug('%s("%s", %s) %s @%s\033[K\033[0m', caller, vpath, str(a), perms, uname)
+        vfs, rem = self.asrv.vfs.get(vpath, uname, *perms)
         return vfs, vfs.canonical(rem)
 
     def _listdir(self, vpath: str, *a: Any, **ka: Any) -> list[str]:
         vpath = vpath.replace("\\", "/").lstrip("/")
         # caller = inspect.currentframe().f_back.f_code.co_name
-        debug('listdir("%s", %s)\033[K\033[0m', vpath, str(a))
-        vfs, rem = self.asrv.vfs.get(vpath, LEELOO_DALLAS, False, False)
+        uname = self._uname()
+        # debug('listdir("%s", %s) @%s\033[K\033[0m', vpath, str(a), uname)
+        vfs, rem = self.asrv.vfs.get(vpath, uname, False, False)
         _, vfs_ls, vfs_virt = vfs.ls(
-            rem, LEELOO_DALLAS, not self.args.no_scandir, [[False, False]]
+            rem, uname, not self.args.no_scandir, [[False, False]]
         )
         dirs = [x[0] for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)]
         fils = [x[0] for x in vfs_ls if x[0] not in dirs]
@@ -149,8 +208,8 @@ class SMB(object):
         sz = 112 * 2  # ['.', '..']
         for n, fn in enumerate(ls):
             if sz >= 64000:
-                t = "listing only %d of %d files (%d byte); see impacket#1433"
-                warning(t, n, len(ls), sz)
+                t = "listing only %d of %d files (%d byte) in /%s; see impacket#1433"
+                warning(t, n, len(ls), sz, vpath)
                 break
 
             nsz = len(fn.encode("utf-16", "replace"))
@@ -171,10 +230,12 @@ class SMB(object):
         if wr and not self.args.smbw:
             yeet("blocked write (no --smbw): " + vpath)
 
-        vfs, ap = self._v2a("open", vpath, *a)
+        uname = self._uname()
+        vfs, ap = self._v2a("open", vpath, *a, uname=uname, perms=[True, wr])
         if wr:
             if not vfs.axs.uwrite:
-                yeet("blocked write (no-write-acc): " + vpath)
+                t = "blocked write (no-write-acc %s): /%s @%s"
+                yeet(t % (vfs.axs.uwrite, vpath, uname))
 
             xbu = vfs.flags.get("xbu")
             if xbu and not runhook(
@@ -204,7 +265,7 @@ class SMB(object):
 
         _, vp = self.files.pop(fd)
         vp, fn = os.path.split(vp)
-        vfs, rem = self.hub.asrv.vfs.get(vp, LEELOO_DALLAS, False, True)
+        vfs, rem = self.hub.asrv.vfs.get(vp, self._uname(), False, True)
         vfs, rem = vfs.get_dbv(rem)
         self.hub.up2k.hash_file(
             vfs.realpath,
@@ -224,15 +285,18 @@ class SMB(object):
         vp1 = vp1.lstrip("/")
         vp2 = vp2.lstrip("/")
 
-        vfs2, ap2 = self._v2a("rename", vp2, vp1)
+        uname = self._uname()
+        vfs2, ap2 = self._v2a("rename", vp2, vp1, uname=uname)
         if not vfs2.axs.uwrite:
-            yeet("blocked rename (no-write-acc): " + vp2)
+            t = "blocked write (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs2.axs.uwrite, vp2, uname))
 
-        vfs1, _ = self.asrv.vfs.get(vp1, LEELOO_DALLAS, True, True)
+        vfs1, _ = self.asrv.vfs.get(vp1, uname, True, True, True)
         if not vfs1.axs.umove:
-            yeet("blocked rename (no-move-acc): " + vp1)
+            t = "blocked rename (no-move-acc %s): /%s @%s"
+            yeet(t % (vfs1.axs.umove, vp1, uname))
 
-        self.hub.up2k.handle_mv(LEELOO_DALLAS, vp1, vp2)
+        self.hub.up2k.handle_mv(uname, vp1, vp2)
         try:
             bos.makedirs(ap2)
         except:
@@ -242,52 +306,74 @@ class SMB(object):
         if not self.args.smbw:
             yeet("blocked mkdir (no --smbw): " + vpath)
 
-        vfs, ap = self._v2a("mkdir", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a("mkdir", vpath, uname=uname)
         if not vfs.axs.uwrite:
-            yeet("blocked mkdir (no-write-acc): " + vpath)
+            t = "blocked mkdir (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs.axs.uwrite, vpath, uname))
 
         return bos.mkdir(ap)
 
     def _stat(self, vpath: str, *a: Any, **ka: Any) -> os.stat_result:
-        return bos.stat(self._v2a("stat", vpath, *a)[1], *a, **ka)
+        try:
+            ap = self._v2a("stat", vpath, *a, perms=[True, False])[1]
+            ret = bos.stat(ap, *a, **ka)
+            # debug(" `-stat:ok")
+            return ret
+        except:
+            # white lie: windows freaks out if we raise due to an offline volume
+            # debug(" `-stat:NOPE (faking a directory)")
+            ts = int(time.time())
+            return os.stat_result((16877, -1, -1, 1, 1000, 1000, 8, ts, ts, ts))
 
     def _unlink(self, vpath: str) -> None:
         if not self.args.smbw:
             yeet("blocked delete (no --smbw): " + vpath)
 
         # return bos.unlink(self._v2a("stat", vpath, *a)[1])
-        vfs, ap = self._v2a("delete", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a(
+            "delete", vpath, uname=uname, perms=[True, False, False, True]
+        )
         if not vfs.axs.udel:
             yeet("blocked delete (no-del-acc): " + vpath)
 
         vpath = vpath.replace("\\", "/").lstrip("/")
-        self.hub.up2k.handle_rm(LEELOO_DALLAS, "1.7.6.2", [vpath], [], False)
+        self.hub.up2k.handle_rm(uname, "1.7.6.2", [vpath], [], False)
 
     def _utime(self, vpath: str, times: tuple[float, float]) -> None:
         if not self.args.smbw:
             yeet("blocked utime (no --smbw): " + vpath)
 
-        vfs, ap = self._v2a("utime", vpath)
+        uname = self._uname()
+        vfs, ap = self._v2a("utime", vpath, uname=uname)
         if not vfs.axs.uwrite:
-            yeet("blocked utime (no-write-acc): " + vpath)
+            t = "blocked utime (no-write-acc %s): /%s @%s"
+            yeet(t % (vfs.axs.uwrite, vpath, uname))
 
         return bos.utime(ap, times)
 
     def _p_exists(self, vpath: str) -> bool:
+        # ap = "?"
         try:
-            bos.stat(self._v2a("p.exists", vpath)[1])
+            ap = self._v2a("p.exists", vpath, perms=[True, False])[1]
+            bos.stat(ap)
+            # debug(" `-exists((%s)->(%s)):ok", vpath, ap)
             return True
         except:
+            # debug(" `-exists((%s)->(%s)):NOPE", vpath, ap)
             return False
 
     def _p_getsize(self, vpath: str) -> int:
-        st = bos.stat(self._v2a("p.getsize", vpath)[1])
+        st = bos.stat(self._v2a("p.getsize", vpath, perms=[True, False])[1])
         return st.st_size
 
     def _p_isdir(self, vpath: str) -> bool:
         try:
-            st = bos.stat(self._v2a("p.isdir", vpath)[1])
-            return stat.S_ISDIR(st.st_mode)
+            st = bos.stat(self._v2a("p.isdir", vpath, perms=[True, False])[1])
+            ret = stat.S_ISDIR(st.st_mode)
+            # debug(" `-isdir:%s:%s", st.st_mode, ret)
+            return ret
         except:
             return False
 

copyparty/ssdp.py

@@ -214,7 +214,7 @@ CONFIGID.UPNP.ORG: 1
         srv.sck.sendto(zb, addr[:2])
 
         if cip not in self.txc.c:
-            self.log("{} [{}] --> {}".format(srv.name, srv.ip, cip), "6")
+            self.log("{} [{}] --> {}".format(srv.name, srv.ip, cip), 6)
 
         self.txc.add(cip)
         self.txc.cln()

copyparty/star.py

@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
+import re
 import stat
 import tarfile
 
@@ -44,6 +45,7 @@ class StreamTar(StreamArc):
         self,
         log: "NamedLogger",
         fgen: Generator[dict[str, Any], None, None],
+        cmp: str = "",
         **kwargs: Any
     ):
         super(StreamTar, self).__init__(log, fgen)
@@ -53,10 +55,36 @@ class StreamTar(StreamArc):
         self.qfile = QFile()
         self.errf: dict[str, Any] = {}
 
-        # python 3.8 changed to PAX_FORMAT as default,
-        # waste of space and don't care about the new features
+        # python 3.8 changed to PAX_FORMAT as default;
+        # slower, bigger, and no particular advantage
         fmt = tarfile.GNU_FORMAT
-        self.tar = tarfile.open(fileobj=self.qfile, mode="w|", format=fmt)  # type: ignore
+        if "pax" in cmp:
+            # unless a client asks for it (currently
+            # gnu-tar has wider support than pax-tar)
+            fmt = tarfile.PAX_FORMAT
+            cmp = re.sub(r"[^a-z0-9]*pax[^a-z0-9]*", "", cmp)
+
+        try:
+            cmp, lv = cmp.replace(":", ",").split(",")
+            lv = int(lv)
+        except:
+            lv = None
+
+        arg = {"name": None, "fileobj": self.qfile, "mode": "w", "format": fmt}
+        if cmp == "gz":
+            fun = tarfile.TarFile.gzopen
+            arg["compresslevel"] = lv if lv is not None else 3
+        elif cmp == "bz2":
+            fun = tarfile.TarFile.bz2open
+            arg["compresslevel"] = lv if lv is not None else 2
+        elif cmp == "xz":
+            fun = tarfile.TarFile.xzopen
+            arg["preset"] = lv if lv is not None else 1
+        else:
+            fun = tarfile.open
+            arg["mode"] = "w|"
+
+        self.tar = fun(**arg)
 
         Daemon(self._gen, "star-gen")
 

copyparty/svchub.py

@@ -198,6 +198,10 @@ class SvcHub(object):
         self.log("root", "max clients: {}".format(self.args.nc))
 
         self.tcpsrv = TcpSrv(self)
+
+        if not self.tcpsrv.srv and self.args.ign_ebind_all:
+            self.args.no_fastboot = True
+
         self.up2k = Up2k(self)
 
         decs = {k: 1 for k in self.args.th_dec.split(",")}
@@ -339,12 +343,20 @@ class SvcHub(object):
         if self.httpsrv_up != self.broker.num_workers:
             return
 
-        time.sleep(0.1)  # purely cosmetic dw
+        ar = self.args
+        for _ in range(10 if ar.ftp or ar.ftps else 0):
+            time.sleep(0.03)
+            if self.ftpd:
+                break
+
         if self.tcpsrv.qr:
             self.log("qr-code", self.tcpsrv.qr)
         else:
             self.log("root", "workers OK\n")
 
+        self.after_httpsrv_up()
+
+    def after_httpsrv_up(self) -> None:
         self.up2k.init_vols()
 
         Daemon(self.sd_notify, "sd-notify")
@@ -415,6 +427,12 @@ class SvcHub(object):
         elif al.ban_url == "no":
             al.sus_urls = None
 
+        if al.xff_src in ("any", "0", ""):
+            al.xff_re = None
+        else:
+            zs = al.xff_src.replace(" ", "").replace(".", "\\.").replace(",", "|")
+            al.xff_re = re.compile("^(?:" + zs + ")")
+
         return True
 
     def _setlimits(self) -> None:
@@ -645,19 +663,25 @@ class SvcHub(object):
         ret = 1
         try:
             self.pr("OPYTHAT")
+            tasks = []
             slp = 0.0
 
             if self.mdns:
-                Daemon(self.mdns.stop)
+                tasks.append(Daemon(self.mdns.stop, "mdns"))
                 slp = time.time() + 0.5
 
             if self.ssdp:
-                Daemon(self.ssdp.stop)
+                tasks.append(Daemon(self.ssdp.stop, "ssdp"))
                 slp = time.time() + 0.5
 
             self.broker.shutdown()
             self.tcpsrv.shutdown()
             self.up2k.shutdown()
+
+            if hasattr(self, "smbd"):
+                slp = max(slp, time.time() + 0.5)
+                tasks.append(Daemon(self.smbd.stop, "smbd"))
+
             if self.thumbsrv:
                 self.thumbsrv.shutdown()
 
@@ -667,17 +691,19 @@ class SvcHub(object):
                         break
 
                     if n == 3:
-                        self.pr("waiting for thumbsrv (10sec)...")
+                        self.log("root", "waiting for thumbsrv (10sec)...")
 
             if hasattr(self, "smbd"):
-                slp = max(slp, time.time() + 0.5)
-                Daemon(self.kill9, a=(1,))
-                Daemon(self.smbd.stop)
+                zf = max(time.time() - slp, 0)
+                Daemon(self.kill9, a=(zf + 0.5,))
 
             while time.time() < slp:
-                time.sleep(0.1)
+                if not next((x for x in tasks if x.is_alive), None):
+                    break
 
-            self.pr("nailed it", end="")
+                time.sleep(0.05)
+
+            self.log("root", "nailed it")
             ret = self.retcode
         except:
             self.pr("\033[31m[ error during shutdown ]\n{}\033[0m".format(min_ex()))
@@ -687,7 +713,7 @@ class SvcHub(object):
                 print("\033]0;\033\\", file=sys.stderr, end="")
                 sys.stderr.flush()
 
-            self.pr("\033[0m")
+            self.pr("\033[0m", end="")
             if self.logf:
                 self.logf.close()
 

copyparty/szip.py

@@ -221,6 +221,7 @@ class StreamZip(StreamArc):
         fgen: Generator[dict[str, Any], None, None],
         utf8: bool = False,
         pre_crc: bool = False,
+        **kwargs: Any
     ) -> None:
         super(StreamZip, self).__init__(log, fgen)
 
@@ -275,6 +276,7 @@ class StreamZip(StreamArc):
     def gen(self) -> Generator[bytes, None, None]:
         errf: dict[str, Any] = {}
         errors = []
+        mbuf = b""
         try:
             for f in self.fgen:
                 if "err" in f:
@@ -283,13 +285,20 @@ class StreamZip(StreamArc):
 
                 try:
                     for x in self.ser(f):
-                        yield x
+                        mbuf += x
+                        if len(mbuf) >= 16384:
+                            yield mbuf
+                            mbuf = b""
                 except GeneratorExit:
                     raise
                 except:
                     ex = min_ex(5, True).replace("\n", "\n-- ")
                     errors.append((f["vp"], ex))
 
+            if mbuf:
+                yield mbuf
+                mbuf = b""
+
             if errors:
                 errf, txt = errdesc(errors)
                 self.log("\n".join(([repr(errf)] + txt[1:])))
@@ -299,20 +308,23 @@ class StreamZip(StreamArc):
             cdir_pos = self.pos
             for name, sz, ts, crc, h_pos in self.items:
                 buf = gen_hdr(h_pos, name, sz, ts, self.utf8, crc, self.pre_crc)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)
+                if len(mbuf) >= 16384:
+                    yield mbuf
+                    mbuf = b""
             cdir_end = self.pos
 
             _, need_64 = gen_ecdr(self.items, cdir_pos, cdir_end)
             if need_64:
                 ecdir64_pos = self.pos
                 buf = gen_ecdr64(self.items, cdir_pos, cdir_end)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)
 
                 buf = gen_ecdr64_loc(ecdir64_pos)
-                yield self._ct(buf)
+                mbuf += self._ct(buf)
 
             ecdr, _ = gen_ecdr(self.items, cdir_pos, cdir_end)
-            yield self._ct(ecdr)
+            yield mbuf + self._ct(ecdr)
         finally:
             if errf:
                 bos.unlink(errf["ap"])

copyparty/tcpsrv.py

@@ -15,6 +15,7 @@ from .util import (
     E_ADDR_IN_USE,
     E_ADDR_NOT_AVAIL,
     E_UNREACH,
+    IP6ALL,
     Netdev,
     min_ex,
     sunpack,
@@ -254,6 +255,9 @@ class TcpSrv(object):
         srvs: list[socket.socket] = []
         for srv in self.srv:
             ip, port = srv.getsockname()[:2]
+            if ip == IP6ALL:
+                ip = "::"  # jython
+
             try:
                 srv.listen(self.args.nc)
                 try:
@@ -275,6 +279,8 @@ class TcpSrv(object):
                     srv.close()
                     continue
 
+                t = "\n\nERROR: could not open listening socket, probably because one of the server ports ({}) is busy on one of the requested interfaces ({}); avoid this issue by specifying a different port (-p 3939) and/or a specific interface to listen on (-i 192.168.56.1)\n"
+                self.log("tcpsrv", t.format(port, ip), 1)
                 raise
 
             bound.append((ip, port))

copyparty/up2k.py

@@ -952,6 +952,11 @@ class Up2k(object):
             rtop = absreal(top)
             n_add = n_rm = 0
             try:
+                if not bos.listdir(rtop):
+                    t = "volume /%s at [%s] is empty; will not be indexed as this could be due to an offline filesystem"
+                    self.log(t % (vol.vpath, rtop), 6)
+                    return True, False
+
                 n_add = self._build_dir(
                     db,
                     top,
@@ -1050,7 +1055,7 @@ class Up2k(object):
         if WINDOWS:
             rd = rd.replace("\\", "/").strip("/")
 
-        g = statdir(self.log_func, not self.args.no_scandir, False, cdir)
+        g = statdir(self.log_func, not self.args.no_scandir, True, cdir)
         gl = sorted(g)
         partials = set([x[0] for x in gl if "PARTIAL" in x[0]])
         for iname, inf in gl:
@@ -1065,6 +1070,12 @@ class Up2k(object):
                 continue
 
             lmod = int(inf.st_mtime)
+            if stat.S_ISLNK(inf.st_mode):
+                try:
+                    inf = bos.stat(abspath)
+                except:
+                    continue
+
             sz = inf.st_size
             if fat32 and not ffat and inf.st_mtime % 2:
                 fat32 = False
@@ -1445,9 +1456,11 @@ class Up2k(object):
                 pf = "v{}, {:.0f}+".format(n_left, b_left / 1024 / 1024)
                 self.pp.msg = pf + abspath
 
-                st = bos.stat(abspath)
+                # throws on broken symlinks (always did)
+                stl = bos.lstat(abspath)
+                st = bos.stat(abspath) if stat.S_ISLNK(stl.st_mode) else stl
+                mt2 = int(stl.st_mtime)
                 sz2 = st.st_size
-                mt2 = int(st.st_mtime)
 
                 if nohash or not sz2:
                     w2 = up2k_wark_from_metadata(self.salt, sz2, mt2, rd, fn)
@@ -1469,6 +1482,13 @@ class Up2k(object):
                 if w == w2:
                     continue
 
+                # symlink mtime was inconsistent before v1.9.4; check if that's it
+                if st != stl and (nohash or not sz2):
+                    mt2b = int(st.st_mtime)
+                    w2b = up2k_wark_from_metadata(self.salt, sz2, mt2b, rd, fn)
+                    if w == w2b:
+                        continue
+
                 rewark.append((drd, dfn, w2, sz2, mt2))
 
                 t = "hash mismatch: {}\n  db: {} ({} byte, {})\n  fs: {} ({} byte, {})"

copyparty/util.py

@@ -56,6 +56,8 @@ E_ADDR_IN_USE = _ens("EADDRINUSE WSAEADDRINUSE")
 E_ACCESS = _ens("EACCES WSAEACCES")
 E_UNREACH = _ens("EHOSTUNREACH WSAEHOSTUNREACH ENETUNREACH WSAENETUNREACH")
 
+IP6ALL = "0:0:0:0:0:0:0:0"
+
 
 try:
     import ctypes
@@ -66,7 +68,9 @@ except:
 
 try:
     HAVE_SQLITE3 = True
-    import sqlite3  # pylint: disable=unused-import  # typechk
+    import sqlite3
+
+    assert hasattr(sqlite3, "connect")  # graalpy
 except:
     HAVE_SQLITE3 = False
 
@@ -294,11 +298,19 @@ REKOBO_KEY = {
 REKOBO_LKEY = {k.lower(): v for k, v in REKOBO_KEY.items()}
 
 
+_exestr = "python3 python ffmpeg ffprobe cfssl cfssljson cfssl-certinfo"
+CMD_EXEB = set(_exestr.encode("utf-8").split())
+CMD_EXES = set(_exestr.split())
+
+
 pybin = sys.executable or ""
 if EXE:
     pybin = ""
     for zsg in "python3 python".split():
         try:
+            if ANYWIN:
+                zsg += ".exe"
+
             zsg = shutil.which(zsg)
             if zsg:
                 pybin = zsg
@@ -2125,7 +2137,7 @@ def list_ips() -> list[str]:
 def yieldfile(fn: str) -> Generator[bytes, None, None]:
     with open(fsenc(fn), "rb", 512 * 1024) as f:
         while True:
-            buf = f.read(64 * 1024)
+            buf = f.read(128 * 1024)
             if not buf:
                 break
 
@@ -2446,6 +2458,14 @@ def runcmd(
     bout: bytes
     berr: bytes
 
+    if ANYWIN:
+        if isinstance(argv[0], (bytes, bytearray)):
+            if argv[0] in CMD_EXEB:
+                argv[0] += b".exe"
+        else:
+            if argv[0] in CMD_EXES:
+                argv[0] += ".exe"
+
     p = sp.Popen(argv, stdout=cout, stderr=cerr, **ka)
     if not timeout or PY2:
         bout, berr = p.communicate(sin)

copyparty/web/baguettebox.js

@@ -261,7 +261,7 @@ window.baguetteBox = (function () {
             setloop(1);
         else if (k == "BracketRight")
             setloop(2);
-        else if (e.shiftKey)
+        else if (e.shiftKey && k != 'KeyR')
             return;
         else if (k == "ArrowLeft" || k == "KeyJ")
             showPreviousImage();
@@ -875,7 +875,7 @@ window.baguetteBox = (function () {
 
         if (loopB !== null) {
             timer.add(loopchk);
-            sethash(window.location.hash.slice(1).split('&')[0] + '&t=' + (loopA || 0) + '-' + loopB);
+            sethash(location.hash.slice(1).split('&')[0] + '&t=' + (loopA || 0) + '-' + loopB);
         }
     }
 

copyparty/web/browser.css

@@ -493,6 +493,7 @@ html.dz {
 	--err-ts: #500;
 
 	text-shadow: none;
+	font-family: 'scp', monospace, monospace;
 }
 html.dy {
 	--fg: #000;
@@ -1397,6 +1398,9 @@ input[type="checkbox"]:checked+label {
 	color: #0e0;
 	color: var(--a);
 }
+html.dz input {
+	font-family: 'scp', monospace, monospace;
+}
 .opwide div>span>input+label {
 	padding: .3em 0 .3em .3em;
 	margin: 0 0 0 -.3em;
@@ -1577,6 +1581,7 @@ html.cz .btn {
 	border-bottom: .2em solid #709;
 }
 html.dz .btn {
+	font-size: 1em;
 	box-shadow: 0 0 0 .1em #080 inset;
 }
 html.dz .tgl.btn.on {
@@ -2766,6 +2771,9 @@ html.c .opbox,
 html.a .opbox {
 	margin: 1.5em 0 0 0;
 }
+html.dz .opview input.i {
+	width: calc(100% - 18em);
+}
 html.c #tree,
 html.c #treeh,
 html.a #tree,
@@ -2818,6 +2826,9 @@ html.a #u2btn {
 html.ay #u2btn {
 	box-shadow: .4em .4em 0 #ccc;
 }
+html.dz #u2btn {
+	letter-spacing: -.033em;
+}
 html.c #u2conf.ww #u2btn,
 html.a #u2conf.ww #u2btn {
 	margin: -2em .5em -3em 0;

copyparty/web/browser.html

@@ -11,7 +11,7 @@
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
 	{%- if css %}
-	<link rel="stylesheet" media="screen" href="{{ css }}?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ css }}_={{ ts }}">
 	{%- endif %}
 </head>
 
@@ -173,7 +173,7 @@
 	<script src="{{ r }}/.cpr/browser.js?_={{ ts }}"></script>
 	<script src="{{ r }}/.cpr/up2k.js?_={{ ts }}"></script>
 	{%- if js %}
-	<script src="{{ js }}?_={{ ts }}"></script>
+	<script src="{{ js }}_={{ ts }}"></script>
 	{%- endif %}
 </body>
 

copyparty/web/browser.js

@@ -80,6 +80,7 @@ var Ls = {
 				"textfile-viewer",
 				["I/K", "prev/next file"],
 				["M", "close textfile"],
+				["E", "edit textfile"],
 				["S", "select file (for cut/rename)"],
 			]
 		],
@@ -327,6 +328,7 @@ var Ls = {
 		"tvt_prev": "show previous document$NHotkey: i\">⬆ prev",
 		"tvt_next": "show next document$NHotkey: K\">⬇ next",
 		"tvt_sel": "select file   ( for cut / delete / ... )$NHotkey: S\">sel",
+		"tvt_edit": "open file in text editor$NHotkey: E\">✏️ edit",
 
 		"gt_msel": "enable file selection; ctrl-click a file to override$N$N<em>when active: doubleclick a file / folder to open it</em>$N$NHotkey: S\">multiselect",
 		"gt_zoom": "zoom",
@@ -375,7 +377,10 @@ var Ls = {
 		"fu_xe1": "failed to load unpost list from server:\n\nerror ",
 		"fu_xe2": "404: File not found??",
 
-		"fz_tar": "plain gnutar file (linux / mac)",
+		"fz_tar": "uncompressed gnu-tar file (linux / mac)",
+		"fz_pax": "uncompressed pax-format tar (slower)",
+		"fz_targz": "gnu-tar with gzip level 3 compression$N$Nthis is usually very slow, so$Nuse uncompressed tar instead",
+		"fz_tarxz": "gnu-tar with xz level 1 compression$N$Nthis is usually very slow, so$Nuse uncompressed tar instead",
 		"fz_zip8": "zip with utf8 filenames (maybe wonky on windows 7 and older)",
 		"fz_zipd": "zip with traditional cp437 filenames, for really old software",
 		"fz_zipc": "cp437 with crc32 computed early,$Nfor MS-DOS PKZIP v2.04g (october 1993)$N(takes longer to process before download can start)",
@@ -544,6 +549,7 @@ var Ls = {
 				"dokumentviser",
 				["I/K", "forr./neste fil"],
 				["M", "lukk tekstdokument"],
+				["E", "rediger tekstdokument"]
 				["S", "velg fil (for F2/ctrl-x/...)"]
 			]
 		],
@@ -786,11 +792,12 @@ var Ls = {
 		"tv_xe1": "kunne ikke laste tekstfil:\n\nfeil ",
 		"tv_xe2": "404, Fil ikke funnet",
 		"tv_lst": "tekstfiler i mappen",
-		"tvt_close": "gå tilbake til mappen$NSnarvei: M\">❌ close",
+		"tvt_close": "gå tilbake til mappen$NSnarvei: M\">❌ lukk",
 		"tvt_dl": "last ned denne filen\">💾 last ned",
-		"tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ prev",
-		"tvt_next": "vis neste dokument$NSnarvei: K\">⬇ next",
-		"tvt_sel": "markér filen   ( for utklipp / sletting / ... )$NSnarvei: S\">sel",
+		"tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ forr.",
+		"tvt_next": "vis neste dokument$NSnarvei: K\">⬇ neste",
+		"tvt_sel": "markér filen   ( for utklipp / sletting / ... )$NSnarvei: S\">merk",
+		"tvt_edit": "redigér filen$NSnarvei: E\">✏️ endre",
 
 		"gt_msel": "markér filer istedenfor å åpne dem; ctrl-klikk filer for å overstyre$N$N<em>når aktiv: dobbelklikk en fil / mappe for å åpne</em>$N$NSnarvei: S\">markering",
 		"gt_zoom": "zoom",
@@ -840,6 +847,9 @@ var Ls = {
 		"fu_xe2": "404: Filen finnes ikke??",
 
 		"fz_tar": "ukomprimert gnu-tar arkiv, for linux og mac",
+		"fz_pax": "ukomprimert pax-tar arkiv, litt tregere",
+		"fz_targz": "gnu-tar pakket med gzip (nivå 3)$N$NNB: denne er veldig treg;$Nukomprimert tar er bedre",
+		"fz_tarxz": "gnu-tar pakket med xz (nivå 1)$N$NNB: denne er veldig treg;$Nukomprimert tar er bedre",
 		"fz_zip8": "zip med filnavn i utf8 (noe problematisk på windows 7 og eldre)",
 		"fz_zipd": "zip med filnavn i cp437, for høggamle maskiner",
 		"fz_zipc": "cp437 med tidlig crc32,$Nfor MS-DOS PKZIP v2.04g (oktober 1993)$N(øker behandlingstid på server)",
@@ -3049,7 +3059,7 @@ function eval_hash() {
 		goto('search');
 		var i = ebi('q_raw');
 		i.value = uricom_dec(v.slice(3));
-		return i.oninput();
+		return i.onkeydown({ 'key': 'Enter' });
 	}
 
 	if (v.indexOf('#v=') === 0) {
@@ -3624,7 +3634,7 @@ var fileman = (function () {
 
 			if (!f.length) {
 				toast.ok(2, 'rename OK');
-				treectl.goto(get_evpath());
+				treectl.goto();
 				return rn_cancel();
 			}
 
@@ -3671,7 +3681,7 @@ var fileman = (function () {
 				if (err !== 'xbd')
 					toast.ok(2, L.fd_ok);
 
-				treectl.goto(get_evpath());
+				treectl.goto();
 				return;
 			}
 			toast.show('inf r', 0, esc(L.fd_busy.format(vps.length + 1, vp)), 'r');
@@ -3789,7 +3799,7 @@ var fileman = (function () {
 
 			if (!vp) {
 				toast.ok(2, L.fp_ok);
-				treectl.goto(get_evpath());
+				treectl.goto();
 				r.tx(srcdir);
 				return;
 			}
@@ -3905,9 +3915,9 @@ var showfile = (function () {
 	window.Prism = { 'manual': true };
 	var em = QS('#bdoc>pre');
 	if (em)
-		em = [r.sname(window.location.search), location.hash, em.textContent];
+		em = [r.sname(location.search), location.hash, em.textContent];
 	else {
-		var m = /[?&]doc=([^&]+)/.exec(window.location.search);
+		var m = /[?&]doc=([^&]+)/.exec(location.search);
 		if (m) {
 			setTimeout(function () {
 				r.show(uricom_dec(m[1]), true);
@@ -3927,7 +3937,7 @@ var showfile = (function () {
 	};
 
 	r.active = function () {
-		return document.location.search.indexOf('doc=') + 1;
+		return location.search.indexOf('doc=') + 1;
 	};
 
 	r.getlang = function (fn) {
@@ -4015,6 +4025,8 @@ var showfile = (function () {
 
 		ebi('files').style.display = ebi('gfiles').style.display = ebi('lazy').style.display = ebi('pro').style.display = ebi('epi').style.display = 'none';
 		ebi('dldoc').setAttribute('href', url);
+		ebi('editdoc').setAttribute('href', url + (url.indexOf('?') > 0 ? '&' : '?') + 'edit');
+		ebi('editdoc').style.display = (has(perms, 'write') && (is_md || has(perms, 'delete'))) ? '' : 'none';
 
 		var wr = ebi('bdoc'),
 			defer = !Prism.highlightElement;
@@ -4026,7 +4038,7 @@ var showfile = (function () {
 
 				el = el || QS('#doc>code');
 				Prism.highlightElement(el);
-				if (el.className == 'language-ans')
+				if (el.className == 'language-ans' || (!lang && /\x1b\[[0-9;]{0,16}m/.exec(txt.slice(0, 4096))))
 					r.ansify(el);
 			}
 			catch (ex) { }
@@ -4185,6 +4197,7 @@ var showfile = (function () {
 		'<a href="#" class="btn" id="prevdoc" tt="' + L.tvt_prev + '</a>\n' +
 		'<a href="#" class="btn" id="nextdoc" tt="' + L.tvt_next + '</a>\n' +
 		'<a href="#" class="btn" id="seldoc" tt="' + L.tvt_sel + '</a>\n' +
+		'<a href="#" class="btn" id="editdoc" tt="' + L.tvt_edit + '</a>\n' +
 		'</div>'
 	);
 	ebi('xdoc').onclick = function () {
@@ -4686,6 +4699,39 @@ function hkhelp() {
 
 
 var fselgen, fselctr;
+function fselfunw(e, ae, d, rem) {
+	fselctr = 0;
+	var gen = fselgen = Date.now();
+	if (rem)
+		rem *= window.innerHeight;
+
+	var selfun = function () {
+		var el = ae[d + 'ElementSibling'];
+		if (!el || gen != fselgen)
+			return;
+
+		el.focus();
+		var elh = el.offsetHeight;
+		if (ctrl(e))
+			document.documentElement.scrollTop += (d == 'next' ? 1 : -1) * elh;
+
+		if (e.shiftKey) {
+			clmod(el, 'sel', 't');
+			msel.origin_tr(el);
+			msel.selui();
+		}
+
+		rem -= elh;
+		if (rem > 0) {
+			ae = document.activeElement;
+			if (++fselctr % 5 && rem > elh * (FIREFOX ? 5 : 2))
+				selfun();
+			else
+				setTimeout(selfun, 1);
+		}
+	}
+	selfun();
+}
 document.onkeydown = function (e) {
 	if (e.altKey || e.isComposing)
 		return;
@@ -4736,37 +4782,7 @@ document.onkeydown = function (e) {
 		if (k == 'PageUp') { d = 'previous'; rem = 0.6; }
 		if (k == 'PageDown') { d = 'next'; rem = 0.6; }
 		if (d) {
-			fselctr = 0;
-			var gen = fselgen = Date.now();
-			if (rem)
-				rem *= window.innerHeight;
-
-			function selfun() {
-				var el = ae[d + 'ElementSibling'];
-				if (!el || gen != fselgen)
-					return;
-
-				el.focus();
-				var elh = el.offsetHeight;
-				if (ctrl(e))
-					document.documentElement.scrollTop += (d == 'next' ? 1 : -1) * elh;
-
-				if (e.shiftKey) {
-					clmod(el, 'sel', 't');
-					msel.origin_tr(el);
-					msel.selui();
-				}
-
-				rem -= elh;
-				if (rem > 0) {
-					ae = document.activeElement;
-					if (++fselctr % 5 && rem > elh * (FIREFOX ? 5 : 2))
-						selfun();
-					else
-						setTimeout(selfun, 1);
-				}
-			}
-			selfun();
+			fselfunw(e, ae, d, rem);
 			return ev(e);
 		}
 		if (k == 'Space') {
@@ -4887,6 +4903,8 @@ document.onkeydown = function (e) {
 	if (showfile.active()) {
 		if (k == 'KeyS')
 			showfile.tglsel();
+		if (k == 'KeyE' && ebi('editdoc').style.display != 'none')
+			ebi('editdoc').click();
 	}
 };
 
@@ -5253,7 +5271,10 @@ var filecolwidth = (function () {
 			return;
 
 		lastwidth = w;
-		document.documentElement.style.setProperty('--file-td-w', w + 'em');
+		try {
+			document.documentElement.style.setProperty('--file-td-w', w + 'em');
+		}
+		catch (ex) { }
 	}
 })();
 onresize100.add(filecolwidth, true);
@@ -5262,6 +5283,7 @@ onresize100.add(filecolwidth, true);
 var treectl = (function () {
 	var r = {
 		"hidden": true,
+		"sb_msg": false,
 		"ls_cb": null,
 		"dir_cb": tree_scrollto,
 		"pdir": []
@@ -5278,7 +5300,7 @@ var treectl = (function () {
 	bcfg_bind(r, 'dyn', 'dyntree', true, onresize);
 	bcfg_bind(r, 'csel', 'csel', false);
 	bcfg_bind(r, 'dots', 'dotfiles', false, function (v) {
-		r.goto(get_evpath());
+		r.goto();
 		var xhr = new XHR();
 		xhr.open('GET', SR + '/?setck=dots=' + (v ? 'y' : ''), true);
 		xhr.send();
@@ -5503,6 +5525,9 @@ var treectl = (function () {
 	};
 
 	r.goto = function (url, push, back) {
+		if (!url || !url.startsWith('/'))
+			url = get_evpath() + (url || '');
+
 		get_tree("", url, true);
 		r.reqls(url, push, back);
 	};
@@ -5681,7 +5706,7 @@ var treectl = (function () {
 		}
 		var href = this.getAttribute('href');
 		if (R && !href.startsWith(SR)) {
-			window.location = href;
+			location = href;
 			return;
 		}
 		r.reqls(href, true);
@@ -5700,6 +5725,7 @@ var treectl = (function () {
 		xhr.send();
 
 		r.nvis = r.lim;
+		r.sb_msg = false;
 		r.nextdir = xhr.top;
 		enspin('#tree');
 		enspin(thegrid.en ? '#gfiles' : '#files');
@@ -5726,7 +5752,9 @@ var treectl = (function () {
 			return;
 
 		r.nextdir = null;
-		var cur = ebi('files').getAttribute('ts');
+		var cdir = get_evpath(),
+			cur = ebi('files').getAttribute('ts');
+
 		if (cur && parseInt(cur) > this.ts) {
 			console.log("reject ls");
 			return;
@@ -5737,7 +5765,7 @@ var treectl = (function () {
 			var res = JSON.parse(this.responseText);
 		}
 		catch (ex) {
-			window.location = this.top;
+			location = this.top;
 			return;
 		}
 
@@ -5770,12 +5798,19 @@ var treectl = (function () {
 		despin('#files');
 		despin('#gfiles');
 
-		sandbox(ebi('pro'), sb_lg, '', res.logues ? res.logues[0] || "" : "");
-		sandbox(ebi('epi'), sb_lg, '', res.logues ? res.logues[1] || "" : "");
+		var lg0 = res.logues ? res.logues[0] || "" : "",
+			lg1 = res.logues ? res.logues[1] || "" : "",
+			dirchg = get_evpath() != cdir;
+
+		sandbox(ebi('pro'), sb_lg, '', lg0);
+		if (dirchg)
+			sandbox(ebi('epi'), sb_lg, '', lg1);
 
 		clmod(ebi('epi'), 'mdo');
-		if (res.readme)
+		if (res.readme && treectl.ireadme)
 			show_readme(res.readme);
+		else if (!dirchg)
+			sandbox(ebi('epi'), sb_lg, '', lg1);
 
 		if (this.hpush && !this.back) {
 			var ofs = ebi('wrap').offsetTop;
@@ -5797,7 +5832,7 @@ var treectl = (function () {
 
 		for (var a = 0; a < res.files.length; a++)
 			if (/^index.html?(\?|$)/i.exec(res.files[a].href)) {
-				window.location = vjoin(top, res.files[a].href);
+				location = vjoin(top, res.files[a].href);
 				return true;
 			}
 	};
@@ -6301,10 +6336,11 @@ var filecols = (function () {
 		var ths = QSA('#files>thead th>span');
 		for (var a = 0, aa = ths.length; a < aa; a++) {
 			var th = ths[a].parentElement,
+				toh = ths[a].outerHTML, // !ff10
 				ttv = L.cols[ths[a].textContent];
 
-			if (!MOBILE) {
-				th.innerHTML = '<div class="cfg"><a href="#">-</a></div>' + ths[a].outerHTML;
+			if (!MOBILE && toh) {
+				th.innerHTML = '<div class="cfg"><a href="#">-</a></div>' + toh;
 				th.getElementsByTagName('a')[0].onclick = ev_row_tgl;
 			}
 			if (ttv) {
@@ -6672,6 +6708,9 @@ var arcfmt = (function () {
 	var html = [],
 		fmts = [
 			["tar", "tar", L.fz_tar],
+			["pax", "tar=pax", L.fz_pax],
+			["tgz", "tar=gz", L.fz_targz],
+			["txz", "tar=xz", L.fz_tarxz],
 			["zip", "zip=utf8", L.fz_zip8],
 			["zip_dos", "zip", L.fz_zipd],
 			["zip_crc", "zip=crc", L.fz_zipc]
@@ -6701,7 +6740,7 @@ var arcfmt = (function () {
 
 		for (var a = 0, aa = tds.length; a < aa; a++) {
 			var o = tds[a], txt = o.textContent, href = o.getAttribute('href');
-			if (txt != 'tar' && txt != 'zip')
+			if (!/^(zip|tar|pax|tgz|txz)$/.exec(txt))
 				continue;
 
 			var ofs = href.lastIndexOf('?');
@@ -7032,7 +7071,7 @@ var msel = (function () {
 		clmod(sf, 'vis');
 		sf.textContent = 'sent: "' + this.msg + '"';
 		setTimeout(function () {
-			treectl.goto(get_evpath());
+			treectl.goto();
 		}, 100);
 	}
 })();
@@ -7107,7 +7146,7 @@ function show_md(md, name, div, url, depth) {
 	wfp_debounce.hide();
 	if (!marked) {
 		if (depth)
-			return toast.warn(10, errmsg + 'failed to load marked.js')
+			return toast.warn(10, errmsg + (window.WebAssembly ? 'failed to load marked.js' : 'your browser is too old'));
 
 		wfp_debounce.n--;
 		return import_js(SR + '/.cpr/deps/marked.js', function () {
@@ -7234,6 +7273,7 @@ function sandbox(tgt, rules, cls, html) {
 		'};f();' +
 		'window.onfocus=function(){say("igot #' + tid + '")};' +
 		'window.onblur=function(){say("ilost #' + tid + '")};' +
+		'window.treectl={"goto":function(a){say("goto #' + tid + ' "+(a||""))}};' +
 		'var el="' + want + '"&&ebi("' + want + '");' +
 		'if(el)say("iscroll #' + tid + ' "+el.offsetTop);' +
 		'md_th_set();' +
@@ -7247,16 +7287,24 @@ function sandbox(tgt, rules, cls, html) {
 	fr.setAttribute('title', 'folder ' + tid + 'logue');
 	fr.setAttribute('sandbox', rules ? 'allow-' + rules.replace(/ /g, ' allow-') : '');
 	fr.setAttribute('srcdoc', html);
-	tgt.innerHTML = '';
 	tgt.appendChild(fr);
+	treectl.sb_msg = true;
 	return true;
 }
 window.addEventListener("message", function (e) {
+	if (!treectl.sb_msg)
+		return;
+
 	try {
 		console.log('msg:' + e.data);
 		var t = e.data.split(/ /g);
 		if (t[0] == 'iheight') {
-			var el = QS(t[1] + '>iframe');
+			var el = QSA(t[1] + '>iframe');
+			el = el[el.length - 1];
+			if (wfp_debounce.n)
+				while (el.previousSibling)
+					el.parentNode.removeChild(el.previousSibling);
+
 			el.style.height = (parseInt(t[2]) + SBH) + 'px';
 			el.style.visibility = 'unset';
 			wfp_debounce.show();
@@ -7273,6 +7321,10 @@ window.addEventListener("message", function (e) {
 		else if (t[0] == 'imshow') {
 			thegrid.imshow(e.data.slice(7));
 		}
+		else if (t[0] == 'goto') {
+			var t = e.data.replace(/^[^ ]+ [^ ]+ /, '').split(/[?&]/)[0];
+			treectl.goto(t, !!t);
+		}
 	} catch (ex) {
 		console.log('msg-err: ' + ex);
 	}

copyparty/web/md.html

@@ -31,7 +31,7 @@
 			<span id="lno">L#</span>
 		{%- else %}
 			<a href="{{ arg_base }}edit" tt="good: higher performance$Ngood: same document width as viewer$Nbad: assumes you know markdown">edit (basic)</a>
-			<a href="{{ arg_base }}edit2" tt="not in-house so probably less buggy">edit (fancy)</a>
+			<a href="{{ arg_base }}edit2" id="edit2" tt="not in-house so probably less buggy">edit (fancy)</a>
 			<a href="{{ arg_base }}">view raw</a>
 		{%- endif %}
 	</div>

copyparty/web/md.js

@@ -52,7 +52,7 @@ var img_load = (function () {
     var r = {};
     r.callbacks = [];
 
-    function fire() {
+    var fire = function () {
         for (var a = 0; a < r.callbacks.length; a++)
             r.callbacks[a]();
     }
@@ -472,7 +472,7 @@ img_load.callbacks = [toc.refresh];
 // scroll handler
 var redraw = (function () {
     var sbs = true;
-    function onresize() {
+    var onresize = function () {
         if (window.matchMedia)
             sbs = window.matchMedia('(min-width: 64em)').matches;
 
@@ -485,7 +485,7 @@ var redraw = (function () {
         onscroll();
     }
 
-    function onscroll() {
+    var onscroll = function () {
         toc.refresh();
     }
 
@@ -507,6 +507,13 @@ dom_navtgl.onclick = function () {
     redraw();
 };
 
+if (!HTTPS && location.hostname != '127.0.0.1') try {
+    ebi('edit2').onclick = function (e) {
+        toast.err(0, "the fancy editor is only available over https");
+        return ev(e);
+    }
+} catch (ex) { }
+
 if (sread('hidenav') == 1)
     dom_navtgl.onclick();
 

copyparty/web/md2.js

@@ -92,7 +92,7 @@ var action_stack = null;
 var nlines = 0;
 var draw_md = (function () {
     var delay = 1;
-    function draw_md() {
+    var draw_md = function () {
         var t0 = Date.now();
         var src = dom_src.value;
         convert_markdown(src, dom_pre);
@@ -135,7 +135,7 @@ img_load.callbacks = [function () {
 
 // resize handler
 redraw = (function () {
-    function onresize() {
+    var onresize = function () {
         var y = (dom_hbar.offsetTop + dom_hbar.offsetHeight) + 'px';
         dom_wrap.style.top = y;
         dom_swrap.style.top = y;
@@ -143,12 +143,12 @@ redraw = (function () {
         map_src = genmap(dom_ref, map_src);
         map_pre = genmap(dom_pre, map_pre);
     }
-    function setsbs() {
+    var setsbs = function () {
         dom_wrap.className = '';
         dom_swrap.className = '';
         onresize();
     }
-    function modetoggle() {
+    var modetoggle = function () {
         var mode = dom_nsbs.innerHTML;
         dom_nsbs.innerHTML = mode == 'editor' ? 'preview' : 'editor';
         mode += ' single';
@@ -172,7 +172,7 @@ redraw = (function () {
 (function () {
     var skip_src = false, skip_pre = false;
 
-    function scroll(src, srcmap, dst, dstmap) {
+    var scroll = function (src, srcmap, dst, dstmap) {
         var y = src.scrollTop;
         if (y < 8) {
             dst.scrollTop = 0;
@@ -900,12 +900,12 @@ var set_lno = (function () {
         pv = null,
         lno = ebi('lno');
 
-    function poke() {
+    var poke = function () {
         clearTimeout(t);
         t = setTimeout(fire, 20);
     }
 
-    function fire() {
+    var fire = function () {
         try {
             clearTimeout(t);
 
@@ -930,7 +930,7 @@ var set_lno = (function () {
 
 // hotkeys / toolbar
 (function () {
-    function keydown(ev) {
+    var keydown = function (ev) {
         ev = ev || window.event;
         var kc = ev.code || ev.keyCode || ev.which,
             editing = document.activeElement == dom_src;
@@ -1058,7 +1058,7 @@ action_stack = (function () {
     var ignore = false;
     var ref = dom_src.value;
 
-    function diff(from, to, cpos) {
+    var diff = function (from, to, cpos) {
         if (from === to)
             return null;
 
@@ -1089,14 +1089,14 @@ action_stack = (function () {
         };
     }
 
-    function undiff(from, change) {
+    var undiff = function (from, change) {
         return {
             txt: from.substring(0, change.car) + change.txt + from.substring(change.cdr),
             cpos: change.cpos
         };
     }
 
-    function apply(src, dst) {
+    var apply = function (src, dst) {
         dbg('undos(%d) redos(%d)', hist.un.length, hist.re.length);
 
         if (src.length === 0)
@@ -1120,7 +1120,7 @@ action_stack = (function () {
         return true;
     }
 
-    function schedule_push() {
+    var schedule_push = function () {
         if (ignore) {
             ignore = false;
             return;
@@ -1131,7 +1131,7 @@ action_stack = (function () {
         sched_timer = setTimeout(push, 500);
     }
 
-    function undo() {
+    var undo = function () {
         if (hist.re.length == 0) {
             clearTimeout(sched_timer);
             push();
@@ -1139,11 +1139,11 @@ action_stack = (function () {
         return apply(hist.un, hist.re);
     }
 
-    function redo() {
+    var redo = function () {
         return apply(hist.re, hist.un);
     }
 
-    function push() {
+    var push = function () {
         var newtxt = dom_src.value;
         var change = diff(ref, newtxt, sched_cpos);
         if (change !== null)

copyparty/web/msg.html

@@ -42,7 +42,7 @@
 	{%- if redir %}
 	<script>
 		setTimeout(function() {
-			window.location.replace("{{ redir }}");
+			location.replace("{{ redir }}");
 		}, 1000);
 	</script>
 	{%- endif %}

copyparty/web/up2k.js

@@ -588,7 +588,7 @@ function U2pvis(act, btns, uc, st) {
         btns[a].onclick = function (e) {
             ev(e);
             var newtab = this.getAttribute('act');
-            function go() {
+            var go = function () {
                 for (var b = 0; b < btns.length; b++) {
                     btns[b].className = (
                         btns[b].getAttribute('act') == newtab) ? 'act' : '';
@@ -807,7 +807,7 @@ function up2k_init(subtle) {
     function init_deps() {
         if (!loading_deps && !got_deps()) {
             var fn = 'sha512.' + sha_js + '.js',
-                m = L.u_https1 + ' <a href="' + (window.location + '').replace(':', 's:') + '">' + L.u_https2 + '</a> ' + L.u_https3;
+                m = L.u_https1 + ' <a href="' + (location + '').replace(':', 's:') + '">' + L.u_https2 + '</a> ' + L.u_https3;
 
             showmodal('<h1>loading ' + fn + '</h1>');
             import_js(SR + '/.cpr/deps/' + fn, unmodal);
@@ -1588,7 +1588,7 @@ function up2k_init(subtle) {
             return;
 
         st.oserr = true;
-        var msg = HTTPS ? L.u_emtleak3 : L.u_emtleak2.format((window.location + '').replace(':', 's:'));
+        var msg = HTTPS ? L.u_emtleak3 : L.u_emtleak2.format((location + '').replace(':', 's:'));
         modal.alert(L.u_emtleak1 + msg + (CHROME ? L.u_emtleakc : FIREFOX ? L.u_emtleakf : ''));
     }
 
@@ -1654,11 +1654,11 @@ function up2k_init(subtle) {
         var running = false,
             was_busy = false;
 
-        function defer() {
+        var defer = function () {
             running = false;
         }
 
-        function taskerd() {
+        var taskerd = function () {
             if (running)
                 return;
 
@@ -1688,7 +1688,7 @@ function up2k_init(subtle) {
                     is_busy = st.todo.handshake.length;
                     try {
                         if (!is_busy && !uc.fsearch && !msel.getsel().length && (!mp.au || mp.au.paused))
-                            treectl.goto(get_evpath());
+                            treectl.goto();
                     }
                     catch (ex) { }
                 }
@@ -1956,7 +1956,7 @@ function up2k_init(subtle) {
             st.bytes.hashed += cdr - car;
             st.etac.h++;
 
-            function orz(e) {
+            var orz = function (e) {
                 bpend--;
                 segm_next();
                 hash_calc(nch, e.target.result);
@@ -2160,6 +2160,9 @@ function up2k_init(subtle) {
 
     function exec_head() {
         var t = st.todo.head.shift();
+        if (t.done)
+            return console.log('done; skip head1', t.name, t);
+
         st.busy.head.push(t);
 
         var xhr = new XMLHttpRequest();
@@ -2172,6 +2175,9 @@ function up2k_init(subtle) {
             st.todo.head.unshift(t);
         };
         function orz(e) {
+            if (t.done)
+                return console.log('done; skip head2', t.name, t);
+
             var ok = false;
             if (xhr.status == 200) {
                 var srv_sz = xhr.getResponseHeader('Content-Length'),
@@ -2218,6 +2224,9 @@ function up2k_init(subtle) {
             keepalive = t.keepalive,
             me = Date.now();
 
+        if (t.done)
+            return console.log('done; skip hs', t.name, t);
+
         st.busy.handshake.push(t);
         t.keepalive = undefined;
         t.t_busied = me;
@@ -2227,10 +2236,9 @@ function up2k_init(subtle) {
 
         var xhr = new XMLHttpRequest();
         xhr.onerror = function () {
-            if (t.t_busied != me) {
-                console.log('zombie handshake onerror,', t.name, t);
-                return;
-            }
+            if (t.t_busied != me)  // t.done ok
+                return console.log('zombie handshake onerror', t.name, t);
+
             if (!toast.visible)
                 toast.warn(9.98, L.u_eneths + "\n\nfile: " + t.name, t);
 
@@ -2239,11 +2247,10 @@ function up2k_init(subtle) {
             st.todo.handshake.unshift(t);
             t.keepalive = keepalive;
         };
-        function orz(e) {
-            if (t.t_busied != me) {
-                console.log('zombie handshake onload,', t.name, t);
-                return;
-            }
+        var orz = function (e) {
+            if (t.t_busied != me || t.done)
+                return console.log('zombie handshake onload', t.name, t);
+
             if (xhr.status == 200) {
                 t.t_handshake = Date.now();
                 if (keepalive) {
@@ -2503,14 +2510,17 @@ function up2k_init(subtle) {
     }
 
     function exec_upload() {
-        var upt = st.todo.upload.shift();
+        var upt = st.todo.upload.shift(),
+            t = st.files[upt.nfile],
+            npart = upt.npart,
+            tries = 0;
+
+        if (t.done)
+            return console.log('done; skip chunk', t.name, t);
+
         st.busy.upload.push(upt);
         st.nfile.upload = upt.nfile;
 
-        var npart = upt.npart,
-            t = st.files[upt.nfile],
-            tries = 0;
-
         if (!t.t_uploading)
             t.t_uploading = Date.now();
 
@@ -2523,7 +2533,7 @@ function up2k_init(subtle) {
         if (cdr >= t.size)
             cdr = t.size;
 
-        function orz(xhr) {
+        var orz = function (xhr) {
             var txt = ((xhr.response && xhr.response.err) || xhr.responseText) + '';
             if (txt.indexOf('upload blocked by x') + 1) {
                 apop(st.busy.upload, upt);
@@ -2552,7 +2562,7 @@ function up2k_init(subtle) {
             }
             orz2(xhr);
         }
-        function orz2(xhr) {
+        var orz2 = function (xhr) {
             apop(st.busy.upload, upt);
             apop(t.postlist, npart);
             if (!t.postlist.length) {

copyparty/web/util.js

@@ -13,7 +13,7 @@ var wah = '',
     R = SR.slice(1),
     RS = R ? "/" + R : "",
     HALFMAX = 8192 * 8192 * 8192 * 8192,
-    HTTPS = (window.location + '').indexOf('https:') === 0,
+    HTTPS = ('' + location).indexOf('https:') === 0,
     TOUCH = 'ontouchstart' in window,
     MOBILE = TOUCH,
     CHROME = !!window.chrome,
@@ -140,29 +140,35 @@ catch (ex) {
 }
 var crashed = false, ignexd = {}, evalex_fatal = false;
 function vis_exh(msg, url, lineNo, columnNo, error) {
-    if ((msg + '').indexOf('ResizeObserver') + 1)
+    msg = String(msg);
+    url = String(url);
+
+    if (msg.indexOf('ResizeObserver') + 1)
         return;  // chrome issue 809574 (benign, from <video>)
 
-    if ((msg + '').indexOf('l2d.js') + 1)
+    if (msg.indexOf('l2d.js') + 1)
         return;  // `t` undefined in tapEvent -> hitTestSimpleCustom
 
-    if (!/\.js($|\?)/.exec('' + url))
+    if (!/\.js($|\?)/.exec(url))
         return;  // chrome debugger
 
-    if ((url + '').indexOf(' > eval') + 1 && !evalex_fatal)
+    if (url.indexOf(' > eval') + 1 && !evalex_fatal)
         return;  // md timer
 
     var ekey = url + '\n' + lineNo + '\n' + msg;
     if (ignexd[ekey] || crashed)
         return;
 
+    if (url.indexOf('deps/marked.js') + 1 && !window.WebAssembly)
+        return; // ff<52
+
     crashed = true;
     window.onerror = undefined;
     var html = [
         '<h1>you hit a bug!</h1>',
         '<p style="font-size:1.3em;margin:0;line-height:2em">try to <a href="#" onclick="localStorage.clear();location.reload();">reset copyparty settings</a> if you are stuck here, or <a href="#" onclick="ignex();">ignore this</a> / <a href="#" onclick="ignex(true);">ignore all</a> / <a href="?b=u">basic</a></p>',
         '<p style="color:#fff">please send me a screenshot arigathanks gozaimuch: <a href="<ghi>" target="_blank">new github issue</a></p>',
-        '<p class="b">' + esc(url + ' @' + lineNo + ':' + columnNo), '<br />' + esc(String(msg)).replace(/\n/g, '<br />') + '</p>',
+        '<p class="b">' + esc(url + ' @' + lineNo + ':' + columnNo), '<br />' + esc(msg).replace(/\n/g, '<br />') + '</p>',
         '<p><b>UA:</b> ' + esc(navigator.userAgent + '')
     ];
 
@@ -354,13 +360,13 @@ catch (ex) {
 }
 
 // https://stackoverflow.com/a/950146
-function import_js(url, cb) {
+function import_js(url, cb, ecb) {
     var head = document.head || document.getElementsByTagName('head')[0];
     var script = mknod('script');
     script.type = 'text/javascript';
     script.src = url;
     script.onload = cb;
-    script.onerror = function () {
+    script.onerror = ecb || function () {
         var m = 'Failed to load module:\n' + url;
         console.log(m);
         toast.err(0, m);
@@ -1141,7 +1147,7 @@ var timer = (function () {
         apop(r.q, fun);
     };
 
-    function doevents() {
+    var doevents = function () {
         if (crashed)
             return;
 
@@ -1352,7 +1358,7 @@ var toast = (function () {
     r.p_sec = 0;
     r.p_t = 0;
 
-    function scrollchk() {
+    var scrollchk = function () {
         if (scrolling)
             return;
 
@@ -1367,7 +1373,7 @@ var toast = (function () {
         scrolling = true;
     }
 
-    function unscroll() {
+    var unscroll = function () {
         timer.rm(scrollchk);
         clmod(obj, 'scroll');
         scrolling = false;
@@ -1486,7 +1492,7 @@ var modal = (function () {
         r.busy = false;
         setTimeout(next, 50);
     };
-    function ok(e) {
+    var ok = function (e) {
         ev(e);
         var v = ebi('modali');
         v = v ? v.value : true;
@@ -1494,14 +1500,14 @@ var modal = (function () {
         if (cb_ok)
             cb_ok(v);
     }
-    function ng(e) {
+    var ng = function (e) {
         ev(e);
         r.hide();
         if (cb_ng)
             cb_ng(null);
     }
 
-    function onfocus(e) {
+    var onfocus = function (e) {
         var ctr = ebi('modalc');
         if (!ctr || !ctr.contains || !document.activeElement || ctr.contains(document.activeElement))
             return;
@@ -1513,7 +1519,7 @@ var modal = (function () {
         ev(e);
     }
 
-    function onkey(e) {
+    var onkey = function (e) {
         var k = e.code,
             eok = ebi('modal-ok'),
             eng = ebi('modal-ng'),
@@ -1536,7 +1542,7 @@ var modal = (function () {
             return ng();
     }
 
-    function next() {
+    var next = function () {
         if (!r.busy && q.length)
             q.shift()();
     }
@@ -1547,7 +1553,7 @@ var modal = (function () {
         });
         next();
     };
-    function _alert(html, cb, fun) {
+    var _alert = function (html, cb, fun) {
         cb_ok = cb_ng = cb;
         cb_up = fun;
         html += '<div id="modalb"><a href="#" id="modal-ok">OK</a></div>';
@@ -1560,7 +1566,7 @@ var modal = (function () {
         });
         next();
     }
-    function _confirm(html, cok, cng, fun, btns) {
+    var _confirm = function (html, cok, cng, fun, btns) {
         cb_ok = cok;
         cb_ng = cng === undefined ? cok : cng;
         cb_up = fun;
@@ -1574,7 +1580,7 @@ var modal = (function () {
         });
         next();
     }
-    function _prompt(html, v, cok, cng, fun) {
+    var _prompt = function (html, v, cok, cng, fun) {
         cb_ok = cok;
         cb_ng = cng === undefined ? cok : null;
         cb_up = fun;
@@ -1706,7 +1712,7 @@ function load_md_plug(md_text, plug_type, defer) {
         return md_text;
 
     var find = '\n```copyparty_' + plug_type + '\n',
-        md = md_text.replace(/\r/g, ''),
+        md = '\n' + md_text.replace(/\r/g, '') + '\n',
         ofs = md.indexOf(find),
         ofs2 = md.indexOf('\n```', ofs + 1);
 
@@ -1793,7 +1799,7 @@ var favico = (function () {
     r.en = true;
     r.tag = null;
 
-    function gx(txt) {
+    var gx = function (txt) {
         return (svg_decl +
             '<svg version="1.1" viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg">\n' +
             (r.bg ? '<rect width="100%" height="100%" rx="16" fill="#' + r.bg + '" />\n' : '') +

docs/changelog.md

@@ -1,3 +1,77 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0909-1336  `v1.9.5`  webhotell
+
+[happy 9/9!](https://safebooru.org/index.php?page=post&s=view&id=4027419)
+
+## new features
+* new permission `h` disables directory listing (so works like `g`) except it redirects to the folder's index.html instead of 404
+  * index.html is accessible by anyone with `h` even if filekeys are enabled
+  * well suited for running a shared-webhosting gig (thx kipu) especially now that the...
+* markdown editor can now be used on non-markdown files if account has `w`rite and `d`elete
+  * hotkey `e` to edit a textfile while it's open in the textfile viewer
+* SMB: account permissions now work fully as intended, thanks to impacket 0.11
+  * but enabling `--smb` is still strongly discouraged as it's a massive security hazard
+* download-as-zip can be 2.5x faster on tiny files, at least 15% faster in general
+* download folders as pax-format tarfiles with `?tar=pax` or `?tar=pax,xz:9`
+
+## bugfixes
+* 422-autoban accidentally triggered when uploading lots of duplicate files (thx hiem!)
+* `--css-browser` and `--js-browser` now accepts URLs with cache directives
+  * `--css-browser=/the.css?cache=600` (seconds) or `--js-browser=/.res/the.js?cache=i` (7 days)
+* SMB: avoid windows freaking out and disconnecting if it hits an offline volume
+* hotkey shift-r to rotate pictures counter-clockwise didn't do anything
+* hacker theme wasn't hacker enough (everything is monospace now)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0902-0018  `v1.9.4`  yes symlink times
+
+hello! it's been a while, an entire day even...
+
+## new features
+* download folder as tar.gz, tar.bz2, tar.xz
+  * single-threaded, so extremely slow, but nice for easily compressed data or challenged networks
+  * append `?tar=gz`, `?tar=bz2` or `?tar=xz` to a folder URL to do it
+  * default compression levels are gz:3, bz2:2, xz:1; override with `?tar=gz:9`
+
+# bugfixes
+* c1efd227b7377144a5760bc6cff64f4e87b626d9 symlink-deduplicated files got indexed with the wrong last-modified timestamp
+  * mostly inconsequential; would cause the dupe's uploader-ip to be forgotten on the next server restart since it would reindex to "fix" the timestamp
+* when linking [a search query](https://a.ocv.me/pub/#q=tags%20like%20soundsho*) it loads the results faster
+
+# other changes
+* update readme to mention that iPhones and iPads dislike the preload feature and respond by glitching the audio a bit when a song is exactly 20 seconds away from ending and yet how it's probably a bad idea to disable preloading since i bet it's load-bearing against other iOS bugs
+  * speaking of iPhones and iPads, the [previous version](https://github.com/9001/copyparty/releases/tag/v1.9.3) should have fixed album playback on those
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0831-2211  `v1.9.3`  iOS and http fixes
+
+## new features
+* iPhones and iPads are now able to...
+  * 9986136dfb2364edb35aa9fbb87410641c6d6af3 play entire albums while the screen is off without the music randomly stopping
+    * apple keeps breaking AudioContext in new and interesting ways; time to give up (no more equalizer)
+  * 1c0d978979a703edeb792e552b18d3b7695b2d90 perform search queries and execude js code
+    * by translating [smart-quotes](https://stackoverflow.com/questions/48678359/ios-11-safari-html-disable-smart-punctuation) into regular `'` and `"` characters
+* python 3.12 support
+  * technically a bugfix since it was added [a year ago](https://github.com/9001/copyparty/commit/32e22dfe84d5e0b13914b4d0e15c1b8c9725a76d) way before the first py3.12 alpha was released but turns out i botched it, oh well
+* filter error messages so they never include the filesystem path where copyparty's python files reside
+* print more context in server logs if someone hits an unexpected permission-denied
+
+# bugfixes
+found some iffy stuff combing over the code but, as far as I can tell, luckily none of these were dangerous:
+* URL normalization was a bit funky, but it appears everything access-control-related was unaffected
+* some url parameters were double-decoded, causing the unpost filtering and file renaming to fail if the values contained `%`
+* clients could cause the server to return an invalid cache-control header, but newlines and control-characters got rejected correctly
+* minor cosmetics / qol fixes:
+  * reduced flickering on page load in chrome
+  * fixed some console spam in search results
+  * markdown documents now have the same line-height in directory listings and the editor
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2023-0826-2116  `v1.9.2`  bigger hammer
 

docs/devnotes.md

@@ -125,8 +125,14 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
 | GET | `?b` | list files/folders at URL as simplified HTML |
 | GET | `?tree=.` | list one level of subdirectories inside URL |
 | GET | `?tree` | list one level of subdirectories for each level until URL |
-| GET | `?tar` | download everything below URL as a tar file |
-| GET | `?zip=utf-8` | download everything below URL as a zip file |
+| GET | `?tar` | download everything below URL as a gnu-tar file |
+| GET | `?tar=gz:9` | ...as a gzip-level-9 gnu-tar file |
+| GET | `?tar=xz:9` | ...as an xz-level-9 gnu-tar file |
+| GET | `?tar=pax` | ...as a pax-tar file |
+| GET | `?tar=pax,xz` | ...as an xz-level-1 pax-tar file |
+| GET | `?zip=utf-8` | ...as a zip file |
+| GET | `?zip` | ...as a WinXP-compatible zip file |
+| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
 | GET | `?ups` | show recent uploads from your IP |
 | GET | `?ups&filter=f` | ...where URL contains `f` |
 | GET | `?mime=foo` | specify return mimetype `foo` |

docs/versus.md

@@ -255,7 +255,7 @@ symbol legend,
 | per-file permissions    |   |   |   | █ | █ |   | █ |   | █ |   |   |   |
 | per-file passwords      | █ |   |   | █ | █ |   | █ |   | █ |   |   |   |
 | unmap subfolders        | █ |   |   |   |   |   | █ |   |   | █ | ╱ | • |
-| index.html blocks list  |   |   |   |   |   |   | █ |   |   | • |   |   |
+| index.html blocks list  | ╱ |   |   |   |   |   | █ |   |   | • |   |   |
 | write-only folders      | █ |   |   |   |   |   |   |   |   |   | █ | █ |
 | files stored as-is      | █ | █ | █ | █ |   | █ | █ |   |   | █ | █ | █ |
 | file versioning         |   |   |   | █ | █ |   |   |   |   |   |   |   |
@@ -291,6 +291,7 @@ symbol legend,
   * one-way folder sync from local to server can be done efficiently with [u2c.py](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy), or with webdav and conventional rsync
   * can hot-reload config files (with just a few exceptions)
   * can set per-folder permissions if that folder is made into a separate volume, so there is configuration overhead
+  * `index.html` on its own does not prevent directory listing, but permission `h` (instead of `r`) enforces index.html to be returned instead of folder contents
   * [event hooks](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks) ([discord](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png), [desktop](https://user-images.githubusercontent.com/241032/215335767-9c91ed24-d36e-4b6b-9766-fb95d12d163f.png)) inspired by filebrowser, as well as the more complex [media parser](https://github.com/9001/copyparty/tree/hovudstraum/bin/mtag) alternative
   * upload history can be visualized using [partyjournal](https://github.com/9001/copyparty/blob/hovudstraum/bin/partyjournal.py)
 * `k`/filegator remarks:

scripts/bench/filehash.sh

@@ -0,0 +1,67 @@
+#!/bin/bash
+set -euo pipefail
+
+# check how fast copyparty is able to hash files during indexing
+# assuming an infinitely fast HDD to read from (alternatively,
+# checks whether you will be bottlenecked by CPU or HDD)
+#
+# uses copyparty's default config of using, well, it's complicated:
+# * if you have more than 8 cores, then 5 threads,
+# * if you have between 4 and 8, then 4 threads,
+# * anything less and it takes your number of cores
+#
+# can be adjusted with --hash-mt (but alpine caps out at 5)
+
+[ $# -ge 1 ] || {
+	echo 'need arg 1: path to copyparty-sfx.py'
+	echo ' (remaining args will be passed on to copyparty,'
+	echo '  for example to tweak the hasher settings)'
+	exit 1
+}
+sfx="$1"
+shift
+sfx="$(realpath "$sfx" || readlink -e "$sfx" || echo "$sfx")"
+awk=$(which gawk || which awk)
+
+# try to use /dev/shm to avoid hitting filesystems at all,
+# otherwise fallback to mktemp which probably uses /tmp
+td=/dev/shm/cppbenchtmp
+mkdir $td || td=$(mktemp -d)
+trap "rm -rf $td" INT TERM EXIT
+cd $td
+
+echo creating 256 MiB testfile in $td
+head -c $((1024*1024*256)) /dev/urandom > 1
+
+echo creating 127 symlinks to it
+for n in $(seq 2 128); do ln -s 1 $n; done
+
+echo warming up cache
+cat 1 >/dev/null
+
+echo ok lets go
+python3 "$sfx" -p39204 -e2dsa --dbd=yolo --exit=idx -lo=t "$@"
+
+echo and the results are...
+$awk '/1 volumes in / {printf "%s MiB/s\n", 256*128/$(NF-1)}' <t
+
+echo deleting $td and exiting
+
+##
+## some results:
+
+# MiB/s @ cpu or device  (copyparty, pythonver, distro/os)  // comment
+
+#  3340 @ Ryzen 5 4500U  (cpp 1.9.5, py 3.11.5, fedora 38)  // --hash-mt=6; laptop
+#  2696 @ Ryzen 5 4500U  (cpp 1.9.5, py 3.11.5, fedora 38)  // --hash-mt=4 (old-default)
+#  2202 @ Ryzen 5 4500U  (cpp 1.9.5, py 3.11.5, docker-alpine 3.18.3) ??? alpine slow
+#  2719 @ Ryzen 5 4500U  (cpp 1.9.5, py 3.11.2, docker-debian 12.1)
+
+#  5544 @ Intel i5-12500 (cpp 1.9.5, py 3.11.2, debian 12.0)  // --hash-mt=12; desktop
+#  5197 @ Ryzen 7 3700X  (cpp 1.9.5, py 3.9.18, freebsd 13.2)  // --hash-mt=8; 2u server
+#  2606 @ Ryzen 7 3700X  (cpp 1.9.5, py 3.9.18, freebsd 13.2)  // --hash-mt=4 (old-default)
+#  1436 @ Ryzen 5 5500U  (cpp 1.9.5, py 3.11.4, alpine 3.18.3)  // nuc
+#  1065 @ Pixel 7        (cpp 1.9.5, py 3.11.5, termux 2023-09)
+
+# notes,
+# podman run --rm -it --shm-size 512m --entrypoint /bin/ash localhost/copyparty-min

scripts/deps-docker/Dockerfile

@@ -142,10 +142,11 @@ RUN     ./genprism.sh $ver_prism
 
 
 # compress
-COPY    zopfli.makefile /z/dist/Makefile
+COPY    brotli.makefile zopfli.makefile /z/dist/
 RUN     cd /z/dist \
-        && make -j$(nproc) \
-        && rm Makefile \
+        && make -j$(nproc) -f brotli.makefile \
+        && make -j$(nproc) -f zopfli.makefile \
+        && rm *.makefile \
         && mv no-pk/* . \
         && rmdir no-pk
 

scripts/deps-docker/brotli.makefile

@@ -0,0 +1,4 @@
+all: $(addsuffix .br, $(wildcard easymde*))
+
+%.br: %
+	brotli -jZ $<

scripts/deps-docker/zopfli.makefile

@@ -1,10 +1,6 @@
-all: $(addsuffix .gz, $(wildcard *.*))
+all: $(addsuffix .gz, $(wildcard *.js *.css))
 
 %.gz: %
-	#brotli -q 11 $<
 	pigz -11 -I 573 $<
 
 # pigz -11 -J 34 -I 100 -F < $< > $@.first
-
-# disabling brotli after all since the gain is meh
-#   and it bloats sfx and wheels by like 70%

scripts/make-sfx.sh

@@ -134,6 +134,14 @@ tmv() {
 	touch -r "$1" t
 	mv t "$1"
 }
+iawk() {
+	awk "$1" <"$2" >t
+	tmv "$2"
+}
+ised() {
+	sed -r "$1" <"$2" >t
+	tmv "$2"
+}
 
 stamp=$(
 	for d in copyparty scripts; do
@@ -229,9 +237,7 @@ necho() {
 	mv python-magic-*/magic .
 	rm -rf python-magic-*
 	rm magic/compat.py
-	f=magic/__init__.py
-	awk '/^def _add_compat/{o=1} !o; /^_add_compat/{o=0}' <$f >t
-	tmv "$f"
+	iawk '/^def _add_compat/{o=1} !o; /^_add_compat/{o=0}' magic/__init__.py
 	mv magic ftp/  # doesn't provide a version label anyways
 
 	# enable this to dynamically remove type hints at startup,
@@ -398,7 +404,7 @@ find -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16')
 
 rm -f copyparty/web/deps/*.full.* copyparty/web/dbg-* copyparty/web/Makefile
 
-find copyparty | LC_ALL=C sort | sed 's/\.gz$//;s/$/,/' > have
+find copyparty | LC_ALL=C sort | sed -r 's/\.(gz|br)$//;s/$/,/' > have
 cat have | while IFS= read -r x; do
 	grep -qF -- "$x" ../scripts/sfx.ls || {
 		echo "unexpected file: $x"
@@ -407,6 +413,11 @@ cat have | while IFS= read -r x; do
 done
 rm have
 
+ised /fork_process/d ftp/pyftpdlib/servers.py
+iawk '/^class _Base/{s=1}!s' ftp/pyftpdlib/authorizers.py
+iawk '/^ {0,4}[^ ]/{s=0}/^ {4}def (serve_forever|_loop)/{s=1}!s' ftp/pyftpdlib/servers.py
+rm -f ftp/pyftpdlib/{__main__,prefork}.py
+
 [ $no_ftp ] &&
 	rm -rf copyparty/ftpd.py ftp asyncore.py asynchat.py &&
 	sed -ri '/add_argument\("--ftp/d' copyparty/__main__.py &&
@@ -423,9 +434,7 @@ rm have
 [ $no_cm ] && {
 	rm -rf copyparty/web/mde.* copyparty/web/deps/easymde*
 	echo h > copyparty/web/mde.html
-	f=copyparty/web/md.html
-	sed -r '/edit2">edit \(fancy/d' <$f >t
-	tmv "$f"
+	ised '/edit2">edit \(fancy/d' copyparty/web/md.html
 }
 
 [ $no_hl ] &&
@@ -435,23 +444,20 @@ rm have
 	rm -f copyparty/web/deps/scp.woff2
 	f=copyparty/web/ui.css
 	gzip -d "$f.gz" || true
-	sed -r "s/src:.*scp.*\)/src:local('Consolas')/" <$f >t
-	tmv "$f"
+	ised "s/src:.*scp.*\)/src:local('Consolas')/" $f
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
 	gzip -d "$f.gz" || true
-	sed -r 's/(cursor: ?)url\([^)]+\), ?(pointer)/\1\2/; s/[0-9]+% \{cursor:[^}]+\}//; s/animation: ?cursor[^};]+//' <$f >t
-	tmv "$f"
+	ised 's/(cursor: ?)url\([^)]+\), ?(pointer)/\1\2/; s/[0-9]+% \{cursor:[^}]+\}//; s/animation: ?cursor[^};]+//' $f
 }
 
 [ $langs ] &&
 	for f in copyparty/web/{browser.js,splash.js}; do
 		gzip -d "$f.gz" || true
-		awk '/^\}/{l=0} !l; /^var Ls =/{l=1;next} o; /^\t["}]/{o=0} /^\t"'"$langs"'"/{o=1;print}' <$f >t
-		tmv "$f"
+		iawk '/^\}/{l=0} !l; /^var Ls =/{l=1;next} o; /^\t["}]/{o=0} /^\t"'"$langs"'"/{o=1;print}' $f
 	done
 
 [ ! $repack ] && [ ! $use_ox ] && {
@@ -466,10 +472,20 @@ rm have
 	#	sed -ri '/: TypeAlias = /d' "$x"; done
 }
 
-f=j2/jinja2/constants.py
-awk '/^LOREM_IPSUM_WORDS/{o=1;print "LOREM_IPSUM_WORDS = u\"a\"";next} !o; /"""/{o=0}' <$f >t
-tmv "$f"
-rm -f j2/jinja2/async*
+rm -f j2/jinja2/constants.py
+iawk '/^ {4}def /{s=0}/^ {4}def compile_templates\(/{s=1}!s' j2/jinja2/environment.py
+ised '/generate_lorem_ipsum/d' j2/jinja2/defaults.py
+iawk '/^def /{s=0}/^def generate_lorem_ipsum/{s=1}!s' j2/jinja2/utils.py
+iawk '/^(class|def) /{s=0}/^(class InternationalizationExtension|def _make_new_n?gettext)/{s=1}!s' j2/jinja2/ext.py
+iawk '/^[^ ]/{s=0}/^def babel_extract/{s=1}!s' j2/jinja2/ext.py
+ised '/InternationalizationExtension/d' j2/jinja2/ext.py
+iawk '/^class/{s=0}/^class (Package|Dict|Function|Prefix|Choice|Module)Loader/{s=1}!s' j2/jinja2/loaders.py
+sed -ri '/^from .bccache | (Package|Dict|Function|Prefix|Choice|Module)Loader$/d' j2/jinja2/__init__.py
+rm -f j2/jinja2/async* j2/jinja2/{bccache,sandbox}.py
+cat > j2/jinja2/_identifier.py <<'EOF'
+import re
+pattern = re.compile(r"\w+")
+EOF
 
 grep -rLE '^#[^a-z]*coding: utf-8' j2 |
 while IFS= read -r f; do

scripts/pyinstaller/build.sh

@@ -9,7 +9,7 @@ tee build2.sh | cmp build.sh && rm build2.sh || {
     [[ $r =~ [yY] ]] && mv build{2,}.sh && exec ./build.sh
 }
 
-[ -e up2k.sh ] && ./up2k.sh
+[ -e up2k.sh ] && [ ! "$1" ] && ./up2k.sh
 
 uname -s | grep WOW64 && m=64 || m=32
 uname -s | grep NT-10 && w10=1 || w7=1
@@ -73,6 +73,7 @@ excl=(
     multiprocessing
     pdb
     pickle
+    PIL.EpsImagePlugin
     pyftpdlib.prefork
     urllib.request
     urllib.response

scripts/pyinstaller/deps.sha512

@@ -25,6 +25,6 @@ ba91ab0518c61eff13e5612d9e6b532940813f6b56e6ed81ea6c7c4d45acee4d98136a383a250675
 # win10
 00558cca2e0ac813d404252f6e5aeacb50546822ecb5d0570228b8ddd29d94e059fbeb6b90393dee5abcddaca1370aca784dc9b095cbb74e980b3c024767fb24  Jinja2-3.1.2-py3-none-any.whl
 7f8f4daa4f4f2dbf24cdd534b2952ee3fba6334eb42b37465ccda3aa1cccc3d6204aa6bfffb8a83bf42ec59c702b5b5247d4c8ee0d4df906334ae53072ef8c4c  MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl
-4a20aeb52d4fde6aabcba05ee261595eeb5482c72ee27332690f34dd6e7a49c0b3ba3813202ac15c9d21e29f1cd803f2e79ccc1c45ec314fcd0a937016bcbc56  mutagen-1.46.0-py3-none-any.whl
-926d408a886059a75cf12706fa061146f9f042b27fb6e65be7d49f398ed23fb0227639d84804586ac014c6bcf7d08cd86a09c1a20793d341aa0802d3d32a546b  Pillow-10.0.0-cp311-cp311-win_amd64.whl
+8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e  mutagen-1.47.0-py3-none-any.whl
+08a033202b5c51e50609b2700dd69cbae30edb367f34762fd1633aae08b35949b4f67f12c75f25868a5b62b4956190d0cc8d201b170758d9c04a523bc8442b9b  Pillow-10.0.1-cp311-cp311-win_amd64.whl
 c86bbeacad3ae3c7bde747f5b4f09c11eced841add14e79ec4a064e5e29ebca35460e543ba735b11bfb882837d5ff4371ce64492d28d096b4686233c9a8cda6d  python-3.11.5-amd64.exe

scripts/pyinstaller/notes.txt

@@ -17,14 +17,14 @@ uname -s | grep NT-10 && w10=1 || {
 fns=(
   altgraph-0.17.3-py2.py3-none-any.whl
   pefile-2023.2.7-py3-none-any.whl
-  pyinstaller-5.13.1-py3-none-win_amd64.whl
+  pyinstaller-5.13.2-py3-none-win_amd64.whl
   pyinstaller_hooks_contrib-2023.7-py2.py3-none-any.whl
   pywin32_ctypes-0.2.2-py3-none-any.whl
   upx-4.1.0-win32.zip
 )
 [ $w10 ] && fns+=(
-  mutagen-1.46.0-py3-none-any.whl
-  Pillow-9.4.0-cp311-cp311-win_amd64.whl
+  mutagen-1.47.0-py3-none-any.whl
+  Pillow-10.0.1-cp311-cp311-win_amd64.whl
   python-3.11.3-amd64.exe
 }
 [ $w7 ] && fns+=(
@@ -47,7 +47,7 @@ fns=(
 )
 [ $w7x32 ] && fns+=(
   windows6.1-kb2533623-x86.msu
-  pyinstaller-5.13.1-py3-none-win32.whl
+  pyinstaller-5.13.2-py3-none-win32.whl
   python-3.7.9.exe
 )
 dl() { curl -fkLOC- "$1" && return 0; echo "$1"; return 1; }

scripts/uncomment.py

@@ -55,6 +55,11 @@ def uncomment(fpath):
                 out += '"a"'
         elif token_type != tokenize.COMMENT or is_legalese:
             out += token_string
+        else:
+            if out.rstrip(" ").endswith("\n"):
+                out = out.rstrip() + "\n"
+            else:
+                out = out.rstrip()
 
         prev_toktype = token_type
         last_lineno = end_line

srv/chat.md

@@ -0,0 +1,18 @@
+## chattyparty
+
+this file, combined with the [msg-log](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/msg-log.py) hook, turns copyparty into a makeshift instant-messaging / chat service
+
+name this file `README.md` and run copyparty as such:
+```bash
+python copyparty-sfx.py -emp --xm j,bin/hooks/msg-log.py
+```
+
+only the stuff below is important; delete everything from this line up
+
+```copyparty_post
+render2(dom) {
+ if (/[?&]edit/.test(location)) return;
+ setTimeout(function() { treectl.goto(); }, 1000);
+ // if you wanna go to another folder: treectl.goto('foo <bar> baz/', true);
+}
+```

tests/test_httpcli.py

@@ -119,8 +119,11 @@ class TestHttpCli(unittest.TestCase):
                 # with open(os.path.join(td, "tar"), "wb") as f:
                 #    f.write(b)
                 try:
-                    tar = tarfile.open(fileobj=io.BytesIO(b)).getnames()
+                    tar = tarfile.open(fileobj=io.BytesIO(b), mode="r|").getnames()
                 except:
+                    if "HTTP/1.1 403 Forbidden" not in h and b != b"\nJ2EOT":
+                        eprint("bad tar?", url, h, b)
+                        raise
                     tar = []
                 tar = [x.split("/", 1)[1] for x in tar]
                 tar = ["/".join([y for y in [top, durl, x] if y]) for x in tar]

tests/util.py

@@ -20,7 +20,7 @@ ANYWIN = WINDOWS or sys.platform in ["msys"]
 MACOS = platform.system() == "Darwin"
 
 J2_ENV = jinja2.Environment(loader=jinja2.BaseLoader)
-J2_FILES = J2_ENV.from_string("{{ files|join('\n') }}")
+J2_FILES = J2_ENV.from_string("{{ files|join('\n') }}\nJ2EOT")
 
 
 def nah(*a, **ka):
@@ -109,13 +109,13 @@ class Cfg(Namespace):
     def __init__(self, a=None, v=None, c=None):
         ka = {}
 
-        ex = "daw dav_auth dav_inf dav_mac dav_rt dotsrch e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_thumb no_vthumb no_zip nrand nw rand smb th_no_crop vague_403 vc ver xdev xlink xvol"
+        ex = "daw dav_auth dav_inf dav_mac dav_rt dotsrch e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw rand smb th_no_crop vague_403 vc ver xdev xlink xvol"
         ka.update(**{k: False for k in ex.split()})
 
         ex = "dotpart no_rescan no_sendfile no_voldump plain_ip"
         ka.update(**{k: True for k in ex.split()})
 
-        ex = "css_browser hist js_browser no_forget no_hash no_idx"
+        ex = "css_browser hist js_browser no_forget no_hash no_idx nonsus_urls"
         ka.update(**{k: None for k in ex.split()})
 
         ex = "s_thead s_tbody th_convt"