v1.6.5

* new audio/MPT formats: apac bonk dfpwm ilbc it itgz itr itz mo3 mod mptm mt2 okt s3gz s3m s3r s3z xm xmgz xmr xmz xpk
* new image/PIL formats: blp dcx emf eps fits flc fli fpx im j2k j2p psd spi wmf

.gitignore

@@ -25,6 +25,9 @@ copyparty.egg-info/
 copyparty/res/COPYING.txt
 copyparty/web/deps/
 srv/
+scripts/docker/i/
+contrib/package/arch/pkg/
+contrib/package/arch/src/
 
 # state/logs
 up.*.txt

README.md

@@ -57,7 +57,7 @@ try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running fro
     * [other tricks](#other-tricks)
     * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config) - using arguments or config files, or a mix of both
-    * [zeroconf](#zeroconf) - announce enabled services on the LAN
+    * [zeroconf](#zeroconf) - announce enabled services on the LAN ([pic](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png))
         * [mdns](#mdns) - LAN domain-name and feature announcer
         * [ssdp](#ssdp) - windows-explorer announcer
     * [qr-code](#qr-code) - print a qr-code [(screenshot)](https://user-images.githubusercontent.com/241032/194728533-6f00849b-c6ac-43c6-9359-83e454d11e00.png) for quick access
@@ -108,7 +108,9 @@ try the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running fro
 
 download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
 
-if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
+* or install through pypi (python3 only): `python3 -m pip install --user -U copyparty`
+* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
+* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
 
 running the sfx without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; you may want [accounts and volumes](#accounts-and-volumes)
 
@@ -199,7 +201,7 @@ recommended additional steps on debian  which enable audio metadata and thumbnai
 
 small collection of user feedback
 
-`good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`
+`good enough`, `surprisingly correct`, `certified good software`, `just works`, `why`, `wow this is better than nextcloud`
 
 
 # motivations
@@ -208,7 +210,7 @@ project goals / philosophy
 
 * inverse linux philosophy -- do all the things, and do an *okay* job
   * quick drop-in service to get a lot of features in a pinch
-  * check [the alternatives](./docs/versus.md)
+  * some of [the alternatives](./docs/versus.md) might be a better fit for you
 * run anywhere, support everything
   * as many web-browsers and python versions as possible
     * every browser should at least be able to browse, download, upload files
@@ -305,6 +307,7 @@ upgrade notes
 
 per-folder, per-user permissions  - if your setup is getting complex, consider making a [config file](./docs/example.conf) instead of using arguments
 * much easier to manage, and you can modify the config at runtime with `systemctl reload copyparty` or more conveniently using the `[reload cfg]` button in the control-panel (if logged in as admin)
+  * changes to the `[global]` config section requires a restart to take effect
 
 a quick summary can be seen using `--help-accounts`
 
@@ -689,6 +692,7 @@ using arguments or config files, or a mix of both:
 * config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf) and [./docs/example2.conf](docs/example2.conf)
 * `kill -s USR1` (same as `systemctl reload copyparty`) to reload accounts and volumes from config files without restarting
   * or click the `[reload cfg]` button in the control-panel when logged in as admin 
+  * changes to the `[global]` config section requires a restart to take effect
 
 
 ## zeroconf
@@ -799,7 +803,7 @@ some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
 
 and some minor issues,
 * clients only see the first ~400 files in big folders; [impacket#1433](https://github.com/SecureAuthCorp/impacket/issues/1433)
-* hot-reload of server config (`/?reload=cfg`) only works for volumes, not account passwords
+* hot-reload of server config (`/?reload=cfg`) does not include the `[global]` section (commandline args)
 * listens on the first IPv4 `-i` interface only (default = :: = 0.0.0.0 = all)
 * login doesn't work on winxp, but anonymous access is ok -- remove all accounts from copyparty config for that to work
   * win10 onwards does not allow connecting anonymously / without accounts
@@ -1270,6 +1274,7 @@ other misc notes:
 
 * you can disable directory listings by giving permission `g` instead of `r`, only accepting direct URLs to files
   * combine this with volflag `c,fk` to generate filekeys (per-file accesskeys); users which have full read-access will then see URLs with `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
+    * the default filekey entropy is fairly small so give `--fk-salt` around 30 characters if you want filekeys longer than 16 chars
   * permissions `wG` lets users upload files and receive their own filekeys, still without being able to see other uploads
 
 
@@ -1346,7 +1351,7 @@ enable [thumbnails](#thumbnails) of...
 * **AVIF pictures:** `pyvips` or `ffmpeg` or `pillow-avif-plugin`
 * **JPEG XL pictures:** `pyvips` or `ffmpeg`
 
-enable [smb](#smb-server) support:
+enable [smb](#smb-server) support (**not** recommended):
 * `impacket==0.10.0`
 
 `pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`

bin/hooks/notify.py

@@ -26,9 +26,23 @@ parameters explained,
 """
 
 
+try:
+    from copyparty.util import humansize
+except:
+
+    def humansize(n):
+        return n
+
+
 def main():
-    dp, fn = os.path.split(sys.argv[1])
-    msg = "🏷️ {}\n📁 {}".format(fn, dp)
+    fp = sys.argv[1]
+    dp, fn = os.path.split(fp)
+    try:
+        sz = humansize(os.path.getsize(fp))
+    except:
+        sz = "?"
+
+    msg = "{} ({})\n📁 {}".format(fn, sz, dp)
     title = "File received"
 
     if "com.termux" in sys.executable:

bin/mtag/image-noexif.py

@@ -61,7 +61,7 @@ def main():
 
     os.chdir(cwd)
     f1 = fsenc(fn)
-    f2 = os.path.join(b"noexif", f1)
+    f2 = fsenc(os.path.join(b"noexif", fn))
     cmd = [
         b"exiftool",
         b"-exif:all=",

bin/mtag/install-deps.sh

@@ -57,6 +57,7 @@ hash -r
 	command -v python3 && pybin=python3 || pybin=python
 }
 
+$pybin -c 'import numpy' ||
 $pybin -m pip install --user numpy
 
 

bin/prisonparty.sh

@@ -4,7 +4,7 @@ set -e
 # runs copyparty (or any other program really) in a chroot
 #
 # assumption: these directories, and everything within, are owned by root
-sysdirs=( /bin /lib /lib32 /lib64 /sbin /usr )
+sysdirs=( /bin /lib /lib32 /lib64 /sbin /usr /etc/alternatives )
 
 
 # error-handler
@@ -97,9 +97,11 @@ done
 
 cln() {
 	rv=$?
-	# cleanup if not in use
-	lsof "$jail" | grep -qF "$jail" &&
-		echo "chroot is in use, will not cleanup" ||
+	wait -f -p rv $p || true
+	cd /
+	echo "stopping chroot..."
+	lsof "$jail" | grep -F "$jail" &&
+		echo "chroot is in use; will not unmount" ||
 	{
 		mount | grep -F " on $jail" |
 		awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' |
@@ -124,5 +126,6 @@ export LOGNAME="$USER"
 #echo "cpp [$cpp]"
 chroot --userspec=$uid:$gid "$jail" "$pybin" $pyarg "$cpp" "$@" &
 p=$!
+trap 'kill -USR1 $p' USR1
 trap 'kill $p' INT TERM
 wait

contrib/package/arch/PKGBUILD

@@ -0,0 +1,57 @@
+# Maintainer: icxes <dev.null@need.moe>
+pkgname=copyparty
+pkgver="1.6.4"
+pkgrel=1
+pkgdesc="Portable file sharing hub"
+arch=("any")
+url="https://github.com/9001/${pkgname}"
+license=('MIT')
+depends=("python" "lsof")
+optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tags" 
+            "python-jinja: faster html generator" 
+            "python-mutagen: music tags (alternative)" 
+            "python-pillow: thumbnails for images" 
+            "python-pyvips: thumbnails for images (higher quality, faster, uses more ram)" 
+            "libkeyfinder-git: detection of musical keys" 
+            "qm-vamp-plugins: BPM detection" 
+            "python-pyopenssl: ftps functionality" 
+            "python-impacket-git: smb support (bad idea)"
+)
+source=("${url}/releases/download/v${pkgver}/${pkgname}-sfx.py" 
+        "${pkgname}.conf"
+        "${pkgname}.service"
+        "prisonparty.service"
+        "index.md"
+        "https://raw.githubusercontent.com/9001/${pkgname}/v${pkgver}/bin/prisonparty.sh"
+        "https://raw.githubusercontent.com/9001/${pkgname}/v${pkgver}/LICENSE"
+)
+backup=("etc/${pkgname}.d/init" )
+sha256sums=("d0447c7a8c4738d2f910f0287c66c70f48c6fae4cf941fb7227504e646fe3e78"
+            "b8565eba5e64dedba1cf6c7aac7e31c5a731ed7153d6810288a28f00a36c28b2"
+            "f65c207e0670f9d78ad2e399bda18d5502ff30d2ac79e0e7fc48e7fbdc39afdc"
+            "c4f396b083c9ec02ad50b52412c84d2a82be7f079b2d016e1c9fad22d68285ff"
+            "dba701de9fd584405917e923ea1e59dbb249b96ef23bad479cf4e42740b774c8"
+            "746971e95817c54445ce7f9c8406822dffc814cd5eb8113abd36dd472fd677d7"
+            "cb2ce3d6277bf2f5a82ecf336cc44963bc6490bcf496ffbd75fc9e21abaa75f3"
+)
+
+package() {
+    cd "${srcdir}/"
+
+    install -dm755 "${pkgdir}/etc/${pkgname}.d"
+    install -Dm755 "${pkgname}-sfx.py" "${pkgdir}/usr/bin/${pkgname}"
+    install -Dm755 "prisonparty.sh" "${pkgdir}/usr/bin/prisonparty"
+    install -Dm644 "${pkgname}.conf" "${pkgdir}/etc/${pkgname}.d/init"
+    install -Dm644 "${pkgname}.service" "${pkgdir}/usr/lib/systemd/system/${pkgname}.service"
+    install -Dm644 "prisonparty.service" "${pkgdir}/usr/lib/systemd/system/prisonparty.service"
+    install -Dm644 "index.md" "${pkgdir}/var/lib/${pkgname}-jail/README.md"
+    install -Dm644 "LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
+
+    find /etc/${pkgname}.d -iname '*.conf' 2>/dev/null | grep -qE . && return
+    echo "┏━━━━━━━━━━━━━━━──-"
+    echo "┃ Configure ${pkgname} by adding .conf files into /etc/${pkgname}.d/"
+    echo "┃ and maybe copy+edit one of the following to /etc/systemd/system/:"
+    echo "┣━♦ /usr/lib/systemd/system/${pkgname}.service   (standard)"
+    echo "┣━♦ /usr/lib/systemd/system/prisonparty.service (chroot)"
+    echo "┗━━━━━━━━━━━━━━━──-"
+}

contrib/package/arch/copyparty.conf

@@ -0,0 +1,7 @@
+## import all *.conf files from the current folder (/etc/copyparty.d)
+% ./
+
+# add additional .conf files to this folder;
+# see example config files for reference:
+# https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf
+# https://github.com/9001/copyparty/tree/hovudstraum/docs/copyparty.d

contrib/package/arch/copyparty.service

@@ -0,0 +1,32 @@
+# this will start `/usr/bin/copyparty-sfx.py`
+# and read config from `/etc/copyparty.d/*.conf`
+#
+# you probably want to:
+#   change "User=cpp" and "/home/cpp/" to another user
+#
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
+#   Environment=PYTHONUNBUFFERED=x
+
+[Unit]
+Description=copyparty file server
+
+[Service]
+Type=notify
+SyslogIdentifier=copyparty
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# user to run as + where the TLS certificate is (if any)
+User=cpp
+Environment=XDG_CONFIG_HOME=/home/cpp/.config
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
+ExecStart=/usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init
+
+[Install]
+WantedBy=multi-user.target

contrib/package/arch/index.md

@@ -0,0 +1,3 @@
+this is `/var/lib/copyparty-jail`, the fallback webroot when copyparty has not yet been configured
+
+please add some `*.conf` files to `/etc/copyparty.d/`

contrib/package/arch/prisonparty.service

@@ -0,0 +1,31 @@
+# this will start `/usr/bin/copyparty-sfx.py`
+# in a chroot, preventing accidental access elsewhere
+# and read config from `/etc/copyparty.d/*.conf`
+#
+# expose additional filesystem locations to copyparty
+#   by listing them between the last `1000` and `--`
+#
+# `1000 1000` = what user to run copyparty as
+#
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
+#   Environment=PYTHONUNBUFFERED=x
+
+[Unit]
+Description=copyparty file server
+
+[Service]
+SyslogIdentifier=prisonparty
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
+ExecStart=/bin/bash /usr/bin/prisonparty /var/lib/copyparty-jail 1000 1000 /etc/copyparty.d -- \
+  /usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init
+
+[Install]
+WantedBy=multi-user.target

contrib/plugins/minimal-up2k.html

@@ -1,4 +1,9 @@
 <!--
+  NOTE: DEPRECATED; please use the javascript version instead:
+  https://github.com/9001/copyparty/blob/hovudstraum/contrib/plugins/minimal-up2k.js
+
+  ----
+
   save this as .epilogue.html inside a write-only folder to declutter the UI,  makes it look like
   https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png
 
@@ -11,7 +16,7 @@
 
     /* make the up2k ui REALLY minimal by hiding a bunch of stuff: */
 
-    #ops, #tree, #path, #epi+h2,  /* main tabs and navigators (tree/breadcrumbs) */
+    #ops, #tree, #path, #wfp,  /* main tabs and navigators (tree/breadcrumbs) */
 
     #u2conf tr:first-child>td[rowspan]:not(#u2btn_cw),  /* most of the config options */
 

contrib/plugins/minimal-up2k.js

@@ -17,7 +17,7 @@ almost the same as minimal-up2k.html except this one...:
 var u2min = `
 <style>
 
-#ops, #path, #tree, #files, #epi+div+h2,
+#ops, #path, #tree, #files, #wfp,
 #u2conf td.c+.c, #u2cards, #srch_dz, #srch_zd {
   display: none !important;
 }
@@ -55,5 +55,5 @@ var u2min = `
 if (!has(perms, 'read')) {
   var e2 = mknod('div');
   e2.innerHTML = u2min;
-  ebi('wrap').insertBefore(e2, QS('#epi+h2'));
+  ebi('wrap').insertBefore(e2, QS('#wfp'));
 }

contrib/systemd/prisonparty.service

@@ -6,12 +6,17 @@
 #   1) put copyparty-sfx.py and prisonparty.sh in /usr/local/bin
 #   2) cp -pv prisonparty.service /etc/systemd/system && systemctl enable --now prisonparty
 #
+# expose additional filesystem locations to copyparty
+#   by listing them between the last `1000` and `--`
+#
+# `1000 1000` = what user to run copyparty as
+#
 # you may want to:
 #   change '/mnt::rw' to another location or permission-set
 #    (remember to change the '/mnt' chroot arg too)
 #
-# enable line-buffering for realtime logging (slight performance cost):
-#   inside the [Service] block, add the following line:
+# unless you add -q to disable logging, you may want to remove the
+#   following line to allow buffering (slightly better performance):
 #   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
@@ -19,7 +24,14 @@ Description=copyparty file server
 
 [Service]
 SyslogIdentifier=prisonparty
-WorkingDirectory=/usr/local/bin
+Environment=PYTHONUNBUFFERED=x
+WorkingDirectory=/var/lib/copyparty-jail
+ExecReload=/bin/kill -s USR1 $MAINPID
+
+# stop systemd-tmpfiles-clean.timer from deleting copyparty while it's running
+ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+
+# run copyparty
 ExecStart=/bin/bash /usr/local/bin/prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt -- \
   /usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
 

copyparty/__main__.py

@@ -25,7 +25,8 @@ from textwrap import dedent
 
 from .__init__ import ANYWIN, CORES, PY2, VT100, WINDOWS, E, EnvParams, unicode
 from .__version__ import CODENAME, S_BUILD_DT, S_VERSION
-from .authsrv import expand_config_file, re_vol
+from .authsrv import expand_config_file, re_vol, split_cfg_ln, upgrade_cfg_fmt
+from .cfg import flagcats, onedash
 from .svchub import SvcHub
 from .util import (
     IMPLICATIONS,
@@ -35,8 +36,10 @@ from .util import (
     UNPLICATIONS,
     align_tab,
     ansi_re,
+    is_exe,
     min_ex,
     py_desc,
+    pybin,
     termsize,
     wrap,
 )
@@ -53,8 +56,9 @@ try:
 except:
     HAVE_SSL = False
 
-printed: list[str] = []
 u = unicode
+printed: list[str] = []
+zsid = uuid.uuid4().urn[4:]
 
 
 class RiceFormatter(argparse.HelpFormatter):
@@ -354,27 +358,28 @@ def configure_ssl_ciphers(al: argparse.Namespace) -> None:
 def args_from_cfg(cfg_path: str) -> list[str]:
     lines: list[str] = []
     expand_config_file(lines, cfg_path, "")
+    lines = upgrade_cfg_fmt(None, argparse.Namespace(vc=False), lines, "")
 
     ret: list[str] = []
-    skip = False
+    skip = True
     for ln in lines:
-        if not ln:
+        sn = ln.split("  #")[0].strip()
+        if sn.startswith("["):
+            skip = True
+        if sn.startswith("[global]"):
             skip = False
             continue
-
-        if ln.startswith("#"):
+        if skip or not sn.split("#")[0].strip():
             continue
-
-        if not ln.startswith("-"):
+        for k, v in split_cfg_ln(sn).items():
+            k = k.lstrip("-")
+            if not k:
                 continue
-
-        if skip:
-            continue
-
-        try:
-            ret.extend(ln.split(" ", 1))
-        except:
-            ret.append(ln)
+            prefix = "-" if k in onedash else "--"
+            if v is True:
+                ret.append(prefix + k)
+            else:
+                ret.append(prefix + k + "=" + v)
 
     return ret
 
@@ -468,7 +473,7 @@ def get_sects():
               "g" (get):    download files, but cannot see folder contents
               "G" (upget):  "get", but can see filekeys of their own uploads
 
-            too many volflags to list here, see the other sections
+            too many volflags to list here, see --help-flags
 
             example:\033[35m
               -a ed:hunter2 -v .::r:rw,ed -v ../inc:dump:w:rw,ed:c,nodupe  \033[36m
@@ -495,69 +500,9 @@ def get_sects():
                 """
             volflags are appended to volume definitions, for example,
             to create a write-only volume with the \033[33mnodupe\033[0m and \033[32mnosub\033[0m flags:
-              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub
-
-            \033[0muploads, general:
-              \033[36mnodupe\033[35m rejects existing files (instead of symlinking them)
-              \033[36mhardlink\033[35m does dedup with hardlinks instead of symlinks
-              \033[36mneversymlink\033[35m disables symlink fallback; full copy instead
-              \033[36mcopydupes\033[35m disables dedup, always saves full copies of dupes
-              \033[36mnosub\033[35m forces all uploads into the top folder of the vfs
-              \033[36mmagic$\033[35m enables filetype detection for nameless uploads
-              \033[36mgz\033[35m allows server-side gzip of uploads with ?gz (also c,xz)
-              \033[36mpk\033[35m forces server-side compression, optional arg: xz,9
-
-            \033[0mupload rules:
-              \033[36mmaxn=250,600\033[35m max 250 uploads over 15min
-              \033[36mmaxb=1g,300\033[35m max 1 GiB over 5min (suffixes: b, k, m, g)
-              \033[36msz=1k-3m\033[35m allow filesizes between 1 KiB and 3MiB
-              \033[36mdf=1g\033[35m ensure 1 GiB free disk space
-
-            \033[0mupload rotation:
-            (moves all uploads into the specified folder structure)
-              \033[36mrotn=100,3\033[35m 3 levels of subfolders with 100 entries in each
-              \033[36mrotf=%Y-%m/%d-%H\033[35m date-formatted organizing
-              \033[36mlifetime=3600\033[35m uploads are deleted after 1 hour
-
-            \033[0mdatabase, general:
-              \033[36me2d\033[35m sets -e2d (all -e2* args can be set using ce2* volflags)
-              \033[36md2ts\033[35m disables metadata collection for existing files
-              \033[36md2ds\033[35m disables onboot indexing, overrides -e2ds*
-              \033[36md2t\033[35m disables metadata collection, overrides -e2t*
-              \033[36md2v\033[35m disables file verification, overrides -e2v*
-              \033[36md2d\033[35m disables all database stuff, overrides -e2*
-              \033[36mhist=/tmp/cdb\033[35m puts thumbnails and indexes at that location
-              \033[36mscan=60\033[35m scan for new files every 60sec, same as --re-maxage
-              \033[36mnohash=\\.iso$\033[35m skips hashing file contents if path matches *.iso
-              \033[36mnoidx=\\.iso$\033[35m fully ignores the contents at paths matching *.iso
-              \033[36mnoforget$\033[35m don't forget files when deleted from disk
-              \033[36mdbd=[acid|swal|wal|yolo]\033[35m database speed-durability tradeoff
-              \033[36mxlink$\033[35m cross-volume dupe detection / linking
-              \033[36mxdev\033[35m do not descend into other filesystems
-              \033[36mxvol\033[35m skip symlinks leaving the volume root
-
-            \033[0mdatabase, audio tags:
-            "mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...
-              \033[36mmtp=.bpm=f,audio-bpm.py\033[35m uses the "audio-bpm.py" program to
-                generate ".bpm" tags from uploads (f = overwrite tags)
-              \033[36mmtp=ahash,vhash=media-hash.py\033[35m collects two tags at once
-
-            \033[0mthumbnails:
-              \033[36mdthumb\033[35m disables all thumbnails
-              \033[36mdvthumb\033[35m disables video thumbnails
-              \033[36mdathumb\033[35m disables audio thumbnails (spectrograms)
-              \033[36mdithumb\033[35m disables image thumbnails
-
-            \033[0mclient and ux:
-              \033[36mhtml_head=TXT\033[35m includes TXT in the <head>
-              \033[36mrobots\033[35m allows indexing by search engines (default)
-              \033[36mnorobots\033[35m kindly asks search engines to leave
-
-            \033[0mothers:
-              \033[36mfk=8\033[35m generates per-file accesskeys,
-                which will then be required at the "g" permission
-            \033[0m"""
-            ),
+              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub"""
+            )
+            + build_flags_desc(),
         ],
         [
             "hooks",
@@ -659,6 +604,17 @@ def get_sects():
     ]
 
 
+def build_flags_desc():
+    ret = ""
+    for grp, flags in flagcats.items():
+        ret += "\n\n\033[0m" + grp
+        for k, v in flags.items():
+            v = v.replace("\n", "\n    ")
+            ret += "\n  \033[36m{}\033[35m {}".format(k, v)
+
+    return ret + "\033[0m"
+
+
 # fmt: off
 
 
@@ -704,6 +660,8 @@ def add_upload(ap):
     ap2.add_argument("--no-dedup", action="store_true", help="disable symlink/hardlink creation; copy file contents instead (volflag=copydupes")
     ap2.add_argument("--no-dupe", action="store_true", help="reject duplicate files during upload; only matches within the same volume (volflag=nodupe)")
     ap2.add_argument("--no-snap", action="store_true", help="disable snapshots -- forget unfinished uploads on shutdown; don't create .hist/up2k.snap files -- abandoned/interrupted uploads must be cleaned up manually")
+    ap2.add_argument("--rand", action="store_true", help="force randomized filenames, --nrand chars long (volflag=rand)")
+    ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
     ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
     ap2.add_argument("--df", metavar="GiB", type=float, default=0, help="ensure GiB free disk space by rejecting upload requests")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
@@ -774,7 +732,7 @@ def add_zc_ssdp(ap):
     ap2.add_argument("--zs-off", metavar="NETS", type=u, default="", help="disable zeroconf on the comma-separated list of subnets and/or interface names/indexes")
     ap2.add_argument("--zsv", action="store_true", help="verbose SSDP")
     ap2.add_argument("--zsl", metavar="PATH", type=u, default="/?hc", help="location to include in the url (or a complete external URL), for example [\033[32mpriv/?pw=hunter2\033[0m] (goes directly to /priv/ with password hunter2) or [\033[32m?hc=priv&pw=hunter2\033[0m] (shows mounting options for /priv/ with password)")
-    ap2.add_argument("--zsid", metavar="UUID", type=u, default=uuid.uuid4().urn[4:], help="USN (device identifier) to announce")
+    ap2.add_argument("--zsid", metavar="UUID", type=u, default=zsid, help="USN (device identifier) to announce")
 
 
 def add_ftp(ap):
@@ -808,7 +766,7 @@ def add_smb(ap):
 
 
 def add_hooks(ap):
-    ap2 = ap.add_argument_group('hooks (see --help-hooks)')
+    ap2 = ap.add_argument_group('event hooks (see --help-hooks)')
     ap2.add_argument("--xbu", metavar="CMD", type=u, action="append", help="execute CMD before a file upload starts")
     ap2.add_argument("--xau", metavar="CMD", type=u, action="append", help="execute CMD after  a file upload finishes")
     ap2.add_argument("--xbr", metavar="CMD", type=u, action="append", help="execute CMD before a file move/rename")
@@ -866,7 +824,7 @@ def add_shutdown(ap):
     ap2 = ap.add_argument_group('shutdown options')
     ap2.add_argument("--ign-ebind", action="store_true", help="continue running even if it's impossible to listen on some of the requested endpoints")
     ap2.add_argument("--ign-ebind-all", action="store_true", help="continue running even if it's impossible to receive connections at all")
-    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after WHEN has finished; for example [\033[32midx\033[0m] will do volume indexing + metadata analysis")
+    ap2.add_argument("--exit", metavar="WHEN", type=u, default="", help="shutdown after WHEN has finished; [\033[32mcfg\033[0m] config parsing, [\033[32midx\033[0m] volscan + multimedia indexing")
 
 
 def add_logging(ap):
@@ -908,11 +866,11 @@ def add_thumbnail(ap):
     # https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
     # https://github.com/libvips/libvips
     # ffmpeg -hide_banner -demuxers | awk '/^ D  /{print$2}' | while IFS= read -r x; do ffmpeg -hide_banner -h demuxer=$x; done | grep -E '^Demuxer |extensions:'
-    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="bmp,dib,gif,icns,ico,jpg,jpeg,jp2,jpx,pcx,png,pbm,pgm,ppm,pnm,sgi,tga,tif,tiff,webp,xbm,dds,xpm,heif,heifs,heic,heics,avif,avifs", help="image formats to decode using pillow")
-    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="jpg,jpeg,jp2,jpx,jxl,tif,tiff,png,webp,heic,avif,fit,fits,fts,exr,svg,hdr,ppm,pgm,pfm,gif,nii", help="image formats to decode using pyvips")
-    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
-    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="av1,asf,avi,flv,m4v,mkv,mjpeg,mjpg,mpg,mpeg,mpg2,mpeg2,h264,avc,mts,h265,hevc,mov,3gp,mp4,ts,mpegts,nut,ogv,ogm,rm,vob,webm,wmv", help="video formats to decode using ffmpeg")
-    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,m4a,ogg,opus,flac,alac,mp3,mp2,ac3,dts,wma,ra,wav,aif,aiff,au,alaw,ulaw,mulaw,amr,gsm,ape,tak,tta,wv,mpc", help="audio formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-pil", metavar="T,T", type=u, default="avif,avifs,blp,bmp,dcx,dds,dib,emf,eps,fits,flc,fli,fpx,gif,heic,heics,heif,heifs,icns,ico,im,j2p,j2k,jp2,jpeg,jpg,jpx,pbm,pcx,pgm,png,pnm,ppm,psd,sgi,spi,tga,tif,tiff,webp,wmf,xbm,xpm", help="image formats to decode using pillow")
+    ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
+    ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
+    ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,m4a,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,tak,tta,ulaw,wav,wma,wv,xm,xpk", help="audio formats to decode using ffmpeg")
 
 
 def add_transcoding(ap):
@@ -923,7 +881,7 @@ def add_transcoding(ap):
 
 def add_db_general(ap, hcores):
     ap2 = ap.add_argument_group('general db options')
-    ap2.add_argument("-e2d", action="store_true", help="enable up2k database, making files searchable + enables upload deduplocation")
+    ap2.add_argument("-e2d", action="store_true", help="enable up2k database, making files searchable + enables upload deduplication")
     ap2.add_argument("-e2ds", action="store_true", help="scan writable folders for new files on startup; sets -e2d")
     ap2.add_argument("-e2dsa", action="store_true", help="scans all folders on startup; sets -e2ds")
     ap2.add_argument("-e2v", action="store_true", help="verify file integrity; rehash all files and compare with db")
@@ -944,6 +902,7 @@ def add_db_general(ap, hcores):
     ap2.add_argument("--db-act", metavar="SEC", type=float, default=10, help="defer any scheduled volume reindexing until SEC seconds after last db write (uploads, renames, ...)")
     ap2.add_argument("--srch-time", metavar="SEC", type=int, default=45, help="search deadline -- terminate searches running for more than SEC seconds")
     ap2.add_argument("--srch-hits", metavar="N", type=int, default=7999, help="max search results to allow clients to fetch; 125 results will be shown initially")
+    ap2.add_argument("--dotsrch", action="store_true", help="show dotfiles in search results (volflags: dotsrch | nodotsrch)")
 
 
 def add_db_metadata(ap):
@@ -979,14 +938,17 @@ def add_ui(ap, retry):
     ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
     ap2.add_argument("--doctitle", metavar="TXT", type=u, default="copyparty", help="title / service-name to show in html documents")
     ap2.add_argument("--pb-url", metavar="URL", type=u, default="https://github.com/9001/copyparty", help="powered-by link; disable with -np")
-    ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms modals popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox")
-    ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms modals popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for prologue/epilogue docs  (volflag=lg_sbf)")
+    ap2.add_argument("--ver", action="store_true", help="show version on the control panel (incompatible by -np)")
+    ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox")
+    ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for prologue/epilogue docs  (volflag=lg_sbf)")
     ap2.add_argument("--no-sb-md", action="store_true", help="don't sandbox README.md documents (volflags: no_sb_md | sb_md)")
     ap2.add_argument("--no-sb-lg", action="store_true", help="don't sandbox prologue/epilogue docs (volflags: no_sb_lg | sb_lg); enables non-js support")
 
 
 def add_debug(ap):
     ap2 = ap.add_argument_group('debug options')
+    ap2.add_argument("--vc", action="store_true", help="verbose config file parser (explain config)")
+    ap2.add_argument("--cgen", action="store_true", help="generate config file from current config (best-effort; probably buggy)")
     ap2.add_argument("--no-sendfile", action="store_true", help="disable sendfile; instead using a traditional file read loop")
     ap2.add_argument("--no-scandir", action="store_true", help="disable scandir; instead using listdir + stat on each file")
     ap2.add_argument("--no-fastboot", action="store_true", help="wait for up2k indexing before starting the httpd")
@@ -1105,6 +1067,9 @@ def main(argv: Optional[list[str]] = None) -> None:
         showlic()
         sys.exit(0)
 
+    if is_exe:
+        print("pybin: {}\n".format(pybin), end="")
+
     ensure_locale()
     if HAVE_SSL:
         ensure_cert()
@@ -1137,7 +1102,8 @@ def main(argv: Optional[list[str]] = None) -> None:
         if da:
             argv.extend(["--qr"])
             if ANYWIN or not os.geteuid():
-                argv.extend(["-p80,443,3923", "--ign-ebind"])
+                # win10 allows symlinks if admin; can be unexpected
+                argv.extend(["-p80,443,3923", "--ign-ebind", "--no-dedup"])
     except:
         pass
 
@@ -1159,6 +1125,7 @@ def main(argv: Optional[list[str]] = None) -> None:
     for fmtr in [RiceFormatter, RiceFormatter, Dodge11874, BasicDodge11874]:
         try:
             al = run_argparse(argv, fmtr, retry, nc)
+            dal = run_argparse([], fmtr, retry, nc)
             break
         except SystemExit:
             raise
@@ -1168,6 +1135,7 @@ def main(argv: Optional[list[str]] = None) -> None:
 
     try:
         assert al  # type: ignore
+        assert dal  # type: ignore
         al.E = E  # __init__ is not shared when oxidized
     except:
         sys.exit(1)
@@ -1273,7 +1241,7 @@ def main(argv: Optional[list[str]] = None) -> None:
 
     # signal.signal(signal.SIGINT, sighandler)
 
-    SvcHub(al, argv, "".join(printed)).run()
+    SvcHub(al, dal, argv, "".join(printed)).run()
 
 
 if __name__ == "__main__":

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 6, 2)
+VERSION = (1, 6, 5)
 CODENAME = "cors k"
-BUILD_DT = (2023, 1, 29)
+BUILD_DT = (2023, 2, 12)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -14,6 +14,7 @@ from datetime import datetime
 
 from .__init__ import ANYWIN, TYPE_CHECKING, WINDOWS
 from .bos import bos
+from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
 from .util import (
     IMPLICATIONS,
     META_NOBOTS,
@@ -21,7 +22,7 @@ from .util import (
     UNPLICATIONS,
     Pebkac,
     absreal,
-    fsenc,
+    afsenc,
     get_df,
     humansize,
     relchk,
@@ -36,7 +37,7 @@ if True:  # pylint: disable=using-constant-test
 
     from typing import Any, Generator, Optional, Union
 
-    from .util import RootLogger
+    from .util import NamedLogger, RootLogger
 
 if TYPE_CHECKING:
     pass
@@ -653,11 +654,15 @@ class AuthSrv(object):
         args: argparse.Namespace,
         log_func: Optional["RootLogger"],
         warn_anonwrite: bool = True,
+        dargs: Optional[argparse.Namespace] = None,
     ) -> None:
         self.args = args
+        self.dargs = dargs or args
         self.log_func = log_func
         self.warn_anonwrite = warn_anonwrite
         self.line_ctr = 0
+        self.indent = ""
+        self.desc = []
 
         self.mutex = threading.Lock()
         self.reload()
@@ -690,17 +695,47 @@ class AuthSrv(object):
             raise Exception("invalid config")
 
         if src in mount.values():
-            t = "warning: filesystem-path [{}] mounted in multiple locations:"
+            t = "filesystem-path [{}] mounted in multiple locations:"
             t = t.format(src)
             for v in [k for k, v in mount.items() if v == src] + [dst]:
                 t += "\n  /{}".format(v)
 
             self.log(t, c=3)
+            raise Exception("invalid config")
+
+        if not bos.path.isdir(src):
+            self.log("warning: filesystem-path does not exist: {}".format(src), 3)
 
         mount[dst] = src
         daxs[dst] = AXS()
         mflags[dst] = {}
 
+    def _e(self, desc: Optional[str] = None) -> None:
+        if not self.args.vc or not self.line_ctr:
+            return
+
+        if not desc and not self.indent:
+            self.log("")
+            return
+
+        desc = desc or ""
+        desc = desc.replace("[", "[\033[0m").replace("]", "\033[90m]")
+        self.log(" >>> {}{}".format(self.indent, desc), "90")
+
+    def _l(self, ln: str, c: int, desc: str) -> None:
+        if not self.args.vc or not self.line_ctr:
+            return
+
+        if c < 10:
+            c += 30
+
+        t = "\033[97m{:4} \033[{}m{}{}"
+        if desc:
+            t += "  \033[0;90m# {}\033[0m"
+            desc = desc.replace("[", "[\033[0m").replace("]", "\033[90m]")
+
+        self.log(t.format(self.line_ctr, c, self.indent, ln, desc))
+
     def _parse_config_file(
         self,
         fp: str,
@@ -710,61 +745,140 @@ class AuthSrv(object):
         mflags: dict[str, dict[str, Any]],
         mount: dict[str, str],
     ) -> None:
-        skip = False
-        vol_src = None
-        vol_dst = None
+        self.desc = []
         self.line_ctr = 0
+
         expand_config_file(cfg_lines, fp, "")
+        if self.args.vc:
+            lns = ["{:4}: {}".format(n, s) for n, s in enumerate(cfg_lines, 1)]
+            self.log("expanded config file (unprocessed):\n" + "\n".join(lns))
+
+        cfg_lines = upgrade_cfg_fmt(self.log, self.args, cfg_lines, fp)
+
+        cat = ""
+        catg = "[global]"
+        cata = "[accounts]"
+        catx = "accs:"
+        catf = "flags:"
+        ap: Optional[str] = None
+        vp: Optional[str] = None
         for ln in cfg_lines:
             self.line_ctr += 1
-            if not ln and vol_src is not None:
-                vol_src = None
-                vol_dst = None
-
-            if skip:
-                if not ln:
-                    skip = False
+            ln = ln.split("  #")[0].strip()
+            if not ln.split("#")[0].strip():
                 continue
 
-            if not ln or ln.startswith("#"):
-                continue
+            subsection = ln in (catx, catf)
+            if ln.startswith("[") or subsection:
+                self._e()
+                if ap is None and vp is not None:
+                    t = "the first line after [/{}] must be a filesystem path to share on that volume"
+                    raise Exception(t.format(vp))
 
-            if vol_src is None:
-                if ln.startswith("u "):
-                    u, p = ln[2:].split(":", 1)
-                    acct[u] = p
-                elif ln.startswith("-"):
-                    skip = True  # argv
+                cat = ln
+                if not subsection:
+                    ap = vp = None
+                    self.indent = ""
                 else:
-                    vol_src = ln
+                    self.indent = "  "
+
+                if ln == catg:
+                    t = "begin commandline-arguments (anything from --help; dashes are optional)"
+                    self._l(ln, 6, t)
+                elif ln == cata:
+                    self._l(ln, 5, "begin user-accounts section")
+                elif ln.startswith("[/"):
+                    vp = ln[1:-1].strip("/")
+                    self._l(ln, 2, "define volume at URL [/{}]".format(vp))
+                elif subsection:
+                    if ln == catx:
+                        self._l(ln, 5, "volume access config:")
+                    else:
+                        t = "volume-specific config (anything from --help-flags)"
+                        self._l(ln, 6, t)
+                else:
+                    raise Exception("invalid section header")
+
+                self.indent = "    " if subsection else "  "
                 continue
 
-            if vol_src and vol_dst is None:
-                vol_dst = ln
-                if not vol_dst.startswith("/"):
-                    raise Exception('invalid mountpoint "{}"'.format(vol_dst))
-
-                if vol_src.startswith("~"):
-                    vol_src = os.path.expanduser(vol_src)
-
-                # cfg files override arguments and previous files
-                vol_src = absreal(vol_src)
-                vol_dst = vol_dst.strip("/")
-                self._map_volume(vol_src, vol_dst, mount, daxs, mflags)
+            if cat == catg:
+                self._l(ln, 6, "")
+                zt = split_cfg_ln(ln)
+                for zs, za in zt.items():
+                    zs = zs.lstrip("-")
+                    if za is True:
+                        self._e("└─argument [{}]".format(zs))
+                    else:
+                        self._e("└─argument [{}] with value [{}]".format(zs, za))
                 continue
 
+            if cat == cata:
                 try:
-                lvl, uname = ln.split(" ", 1)
+                    u, p = [zs.strip() for zs in ln.split(":", 1)]
+                    self._l(ln, 5, "account [{}], password [{}]".format(u, p))
+                    acct[u] = p
                 except:
-                lvl = ln
-                uname = "*"
+                    t = 'lines inside the [accounts] section must be "username: password"'
+                    raise Exception(t)
+                continue
 
-            if lvl == "a":
-                t = "WARNING (config-file): permission flag 'a' is deprecated; please use 'rw' instead"
-                self.log(t, 1)
+            if vp is not None and ap is None:
+                ap = ln
+                if ap.startswith("~"):
+                    ap = os.path.expanduser(ap)
 
-            assert vol_dst is not None
-            self._read_vol_str(lvl, uname, daxs[vol_dst], mflags[vol_dst])
+                ap = absreal(ap)
+                self._l(ln, 2, "bound to filesystem-path [{}]".format(ap))
+                self._map_volume(ap, vp, mount, daxs, mflags)
+                continue
+
+            if cat == catx:
+                err = ""
+                try:
+                    self._l(ln, 5, "volume access config:")
+                    sk, sv = ln.split(":")
+                    if re.sub("[rwmdgG]", "", sk) or not sk:
+                        err = "invalid accs permissions list; "
+                        raise Exception(err)
+                    if " " in re.sub(", *", "", sv).strip():
+                        err = "list of users is not comma-separated; "
+                        raise Exception(err)
+                    self._read_vol_str(sk, sv.replace(" ", ""), daxs[vp], mflags[vp])
+                    continue
+                except:
+                    err += "accs entries must be 'rwmdgG: user1, user2, ...'"
+                    raise Exception(err)
+
+            if cat == catf:
+                err = ""
+                try:
+                    self._l(ln, 6, "volume-specific config:")
+                    zd = split_cfg_ln(ln)
+                    fstr = ""
+                    for sk, sv in zd.items():
+                        bad = re.sub(r"[a-z0-9_]", "", sk)
+                        if bad:
+                            err = "bad characters [{}] in volflag name [{}]; "
+                            err = err.format(bad, sk)
+                            raise Exception(err)
+                        if sv is True:
+                            fstr += "," + sk
+                        else:
+                            fstr += ",{}={}".format(sk, sv)
+                            self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
+                            fstr = ""
+                    if fstr:
+                        self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
+                    continue
+                except:
+                    err += "flags entries (volflags) must be one of the following:\n  'flag1, flag2, ...'\n  'key: value'\n  'flag1, flag2, key: value'"
+                    raise Exception(err)
+
+            raise Exception("unprocessable line in config")
+
+        self._e()
+        self.line_ctr = 0
 
     def _read_vol_str(
         self, lvl: str, uname: str, axs: AXS, flags: dict[str, Any]
@@ -803,6 +917,13 @@ class AuthSrv(object):
                 ("G", axs.upget),
             ]:  # b bb bbb
                 if ch in lvl:
+                    if un == "*":
+                        t = "└─add permission [{0}] for [everyone] -- {2}"
+                    else:
+                        t = "└─add permission [{0}] for user [{1}] -- {2}"
+
+                    desc = permdescs.get(ch, "?")
+                    self._e(t.format(ch, un, desc))
                     al.add(un)
 
     def _read_volflag(
@@ -812,7 +933,13 @@ class AuthSrv(object):
         value: Union[str, bool, list[str]],
         is_list: bool,
     ) -> None:
+        desc = flagdescs.get(name, "?").replace("\n", " ")
         if name not in ["mtp", "xbu", "xau", "xbr", "xar", "xbd", "xad", "xm"]:
+            if value is True:
+                t = "└─add volflag [{}] = {}  ({})"
+            else:
+                t = "└─add volflag [{}] = [{}]  ({})"
+            self._e(t.format(name, value, desc))
             flags[name] = value
             return
 
@@ -825,6 +952,7 @@ class AuthSrv(object):
             vals += [value]
 
         flags[name] = vals
+        self._e("volflag [{}] += {}  ({})".format(name, vals, desc))
 
     def reload(self) -> None:
         """
@@ -875,6 +1003,18 @@ class AuthSrv(object):
                 lns: list[str] = []
                 try:
                     self._parse_config_file(cfg_fn, lns, acct, daxs, mflags, mount)
+
+                    zs = "#\033[36m cfg files in "
+                    zst = [x[len(zs) :] for x in lns if x.startswith(zs)]
+                    for zs in list(set(zst)):
+                        self.log("discovered config files in " + zs, 6)
+
+                    zs = "#\033[36m opening cfg file"
+                    zstt = [x.split(" -> ") for x in lns if x.startswith(zs)]
+                    zst = [(max(0, len(x) - 2) * " ") + "└" + x[-1] for x in zstt]
+                    t = "loaded {} config files:\n{}"
+                    self.log(t.format(len(zst), "\n".join(zst)))
+
                 except:
                     lns = lns[: self.line_ctr]
                     slns = ["{:4}: {}".format(n, s) for n, s in enumerate(lns, 1)]
@@ -955,7 +1095,7 @@ class AuthSrv(object):
         promote = []
         demote = []
         for vol in vfs.all_vols.values():
-            zb = hashlib.sha512(fsenc(vol.realpath)).digest()
+            zb = hashlib.sha512(afsenc(vol.realpath)).digest()
             hid = base64.b32encode(zb).decode("ascii").lower()
             vflag = vol.flags.get("hist")
             if vflag == "-":
@@ -974,7 +1114,7 @@ class AuthSrv(object):
                     except:
                         owner = None
 
-                    me = fsenc(vol.realpath).rstrip()
+                    me = afsenc(vol.realpath).rstrip()
                     if owner not in [None, me]:
                         continue
 
@@ -1114,38 +1254,30 @@ class AuthSrv(object):
                 if ptn:
                     vol.flags[vf] = re.compile(ptn)
 
-            for k in ["e2t", "e2ts", "e2tsr", "e2v", "e2vu", "e2vp", "xdev", "xvol"]:
-                if getattr(self.args, k):
-                    vol.flags[k] = True
-
-            for ga, vf in (
-                ("no_sb_md", "no_sb_md"),
-                ("no_sb_lg", "no_sb_lg"),
-                ("no_forget", "noforget"),
-                ("no_dupe", "nodupe"),
-                ("hardlink", "hardlink"),
-                ("never_symlink", "neversymlink"),
-                ("no_dedup", "copydupes"),
-                ("magic", "magic"),
-                ("xlink", "xlink"),
-            ):
+            for ga, vf in vf_bmap().items():
                 if getattr(self.args, ga):
                     vol.flags[vf] = True
 
             for ve, vd in (
-                ("sb_md", "no_sb_md"),
+                ("nodotsrch", "dotsrch"),
                 ("sb_lg", "no_sb_lg"),
+                ("sb_md", "no_sb_md"),
             ):
                 if ve in vol.flags:
                     vol.flags.pop(vd, None)
 
-            for ga, vf in (
-                ("md_sbf", "md_sbf"),
-                ("lg_sbf", "lg_sbf"),
-            ):
+            for ga, vf in vf_vmap().items():
                 if vf not in vol.flags:
                     vol.flags[vf] = getattr(self.args, ga)
 
+            for k in ("nrand",):
+                if k not in vol.flags:
+                    vol.flags[k] = getattr(self.args, k)
+
+            for k in ("nrand",):
+                if k in vol.flags:
+                    vol.flags[k] = int(vol.flags[k])
+
             for k1, k2 in IMPLICATIONS:
                 if k1 in vol.flags:
                     vol.flags[k2] = True
@@ -1483,32 +1615,293 @@ class AuthSrv(object):
         if not flag_r:
             sys.exit(0)
 
+    def cgen(self) -> None:
+        ret = [
+            "## WARNING:",
+            "##  there will probably be mistakes in",
+            "##  commandline-args (and maybe volflags)",
+            "",
+        ]
+
+        csv = set("i p".split())
+        lst = set("c ihead mtm mtp xad xar xau xbd xbr xbu xm".split())
+        askip = set("a v c vc cgen".split())
+
+        # keymap from argv to vflag
+        amap = vf_bmap()
+        amap.update(vf_vmap())
+        amap.update(vf_cmap())
+        vmap = {v: k for k, v in amap.items()}
+
+        args = {k: v for k, v in vars(self.args).items()}
+        pops = []
+        for k1, k2 in IMPLICATIONS:
+            if args.get(k1):
+                pops.append(k2)
+        for pop in pops:
+            args.pop(pop, None)
+
+        if args:
+            ret.append("[global]")
+            for k, v in args.items():
+                if k in askip:
+                    continue
+                if k in csv:
+                    v = ", ".join([str(za) for za in v])
+                try:
+                    v2 = getattr(self.dargs, k)
+                    if v == v2:
+                        continue
+                except:
+                    continue
+
+                dk = "  " + k.replace("_", "-")
+                if k in lst:
+                    for ve in v:
+                        ret.append("{}: {}".format(dk, ve))
+                else:
+                    if v is True:
+                        ret.append(dk)
+                    elif v not in (False, None, ""):
+                        ret.append("{}: {}".format(dk, v))
+            ret.append("")
+
+        if self.acct:
+            ret.append("[accounts]")
+            for u, p in self.acct.items():
+                ret.append("  {}: {}".format(u, p))
+            ret.append("")
+
+        for vol in self.vfs.all_vols.values():
+            ret.append("[/{}]".format(vol.vpath))
+            ret.append("  " + vol.realpath)
+            ret.append("  accs:")
+            perms = {
+                "r": "uread",
+                "w": "uwrite",
+                "m": "umove",
+                "d": "udel",
+                "g": "uget",
+                "G": "upget",
+            }
+            users = {}
+            for pkey in perms.values():
+                for uname in getattr(vol.axs, pkey):
+                    try:
+                        users[uname] += 1
+                    except:
+                        users[uname] = 1
+            lusers = [(v, k) for k, v in users.items()]
+            vperms = {}
+            for _, uname in sorted(lusers):
+                pstr = ""
+                for pchar, pkey in perms.items():
+                    if uname in getattr(vol.axs, pkey):
+                        pstr += pchar
+                if "g" in pstr and "G" in pstr:
+                    pstr = pstr.replace("g", "")
+                try:
+                    vperms[pstr].append(uname)
+                except:
+                    vperms[pstr] = [uname]
+            for pstr, uname in vperms.items():
+                ret.append("    {}: {}".format(pstr, ", ".join(uname)))
+            trues = []
+            vals = []
+            for k, v in sorted(vol.flags.items()):
+                try:
+                    ak = vmap[k]
+                    if getattr(self.args, ak) is v:
+                        continue
+                except:
+                    pass
+
+                if k in lst:
+                    for ve in v:
+                        vals.append("{}: {}".format(k, ve))
+                elif v is True:
+                    trues.append(k)
+                elif v is not False:
+                    vals.append("{}: {}".format(k, v))
+            pops = []
+            for k1, k2 in IMPLICATIONS:
+                if k1 in trues:
+                    pops.append(k2)
+            trues = [x for x in trues if x not in pops]
+            if trues:
+                vals.append(", ".join(trues))
+            if vals:
+                ret.append("  flags:")
+                for zs in vals:
+                    ret.append("    " + zs)
+            ret.append("")
+
+        self.log("generated config:\n\n" + "\n".join(ret))
+
+
+def split_cfg_ln(ln: str) -> dict[str, Any]:
+    # "a, b, c: 3" => {a:true, b:true, c:3}
+    ret = {}
+    while True:
+        ln = ln.strip()
+        if not ln:
+            break
+        ofs_sep = ln.find(",") + 1
+        ofs_var = ln.find(":") + 1
+        if not ofs_sep and not ofs_var:
+            ret[ln] = True
+            break
+        if ofs_sep and (ofs_sep < ofs_var or not ofs_var):
+            k, ln = ln.split(",", 1)
+            ret[k.strip()] = True
+        else:
+            k, ln = ln.split(":", 1)
+            ret[k.strip()] = ln.strip()
+            break
+    return ret
+
 
 def expand_config_file(ret: list[str], fp: str, ipath: str) -> None:
     """expand all % file includes"""
     fp = absreal(fp)
-    ipath += " -> " + fp
-    ret.append("#\033[36m opening cfg file{}\033[0m".format(ipath))
     if len(ipath.split(" -> ")) > 64:
         raise Exception("hit max depth of 64 includes")
 
     if os.path.isdir(fp):
-        for fn in sorted(os.listdir(fp)):
+        names = os.listdir(fp)
+        ret.append("#\033[36m cfg files in {} => {}\033[0m".format(fp, names))
+        for fn in sorted(names):
             fp2 = os.path.join(fp, fn)
-            if not os.path.isfile(fp2):
-                continue  # dont recurse
+            if not fp2.endswith(".conf") or fp2 in ipath:
+                continue
 
             expand_config_file(ret, fp2, ipath)
         return
 
+    ipath += " -> " + fp
+    ret.append("#\033[36m opening cfg file{}\033[0m".format(ipath))
+
     with open(fp, "rb") as f:
-        for ln in [x.decode("utf-8").strip() for x in f]:
+        for oln in [x.decode("utf-8").rstrip() for x in f]:
+            ln = oln.split("  #")[0].strip()
             if ln.startswith("% "):
+                pad = " " * len(oln.split("%")[0])
                 fp2 = ln[1:].strip()
                 fp2 = os.path.join(os.path.dirname(fp), fp2)
+                ofs = len(ret)
                 expand_config_file(ret, fp2, ipath)
+                for n in range(ofs, len(ret)):
+                    ret[n] = pad + ret[n]
                 continue
 
-            ret.append(ln)
+            ret.append(oln)
 
     ret.append("#\033[36m closed{}\033[0m".format(ipath))
+
+
+def upgrade_cfg_fmt(
+    log: Optional["NamedLogger"], args: argparse.Namespace, orig: list[str], cfg_fp: str
+) -> list[str]:
+    """convert from v1 to v2 format"""
+    zst = [x.split("#")[0].strip() for x in orig]
+    zst = [x for x in zst if x]
+    if (
+        "[global]" in zst
+        or "[accounts]" in zst
+        or "accs:" in zst
+        or "flags:" in zst
+        or [x for x in zst if x.startswith("[/")]
+        or len(zst) == len([x for x in zst if x.startswith("%")])
+    ):
+        return orig
+
+    zst = [x for x in orig if "#\033[36m opening cfg file" not in x]
+    incl = len(zst) != len(orig) - 1
+
+    t = "upgrading config file [{}] from v1 to v2"
+    if not args.vc:
+        t += ". Run with argument '--vc' to see the converted config if you want to upgrade"
+    if incl:
+        t += ". Please don't include v1 configs from v2 files or vice versa! Upgrade all of them at the same time."
+    if log:
+        log(t.format(cfg_fp), 3)
+
+    ret = []
+    vp = ""
+    ap = ""
+    cat = ""
+    catg = "[global]"
+    cata = "[accounts]"
+    catx = "  accs:"
+    catf = "  flags:"
+    for ln in orig:
+        sn = ln.strip()
+        if not sn:
+            cat = vp = ap = ""
+        if not sn.split("#")[0]:
+            ret.append(ln)
+        elif sn.startswith("-") and cat in ("", catg):
+            if cat != catg:
+                cat = catg
+                ret.append(cat)
+            sn = sn.lstrip("-")
+            zst = sn.split(" ", 1)
+            if len(zst) > 1:
+                sn = "{}: {}".format(zst[0], zst[1].strip())
+            ret.append("  " + sn)
+        elif sn.startswith("u ") and cat in ("", catg, cata):
+            if cat != cata:
+                cat = cata
+                ret.append(cat)
+            s1, s2 = sn[1:].split(":", 1)
+            ret.append("  {}: {}".format(s1.strip(), s2.strip()))
+        elif not ap:
+            ap = sn
+        elif not vp:
+            vp = "/" + sn.strip("/")
+            cat = "[{}]".format(vp)
+            ret.append(cat)
+            ret.append("  " + ap)
+        elif sn.startswith("c "):
+            if cat != catf:
+                cat = catf
+                ret.append(cat)
+            sn = sn[1:].strip()
+            if "=" in sn:
+                zst = sn.split("=", 1)
+                sn = zst[0].replace(",", ", ")
+                sn += ": " + zst[1]
+            else:
+                sn = sn.replace(",", ", ")
+            ret.append("    " + sn)
+        elif sn[:1] in "rwmdgG":
+            if cat != catx:
+                cat = catx
+                ret.append(cat)
+            zst = sn.split(" ")
+            zst = [x for x in zst if x]
+            if len(zst) == 1:
+                zst.append("*")
+            ret.append("    {}: {}".format(zst[0], ", ".join(zst[1:])))
+        else:
+            t = "did not understand line {} in the config"
+            t1 = t
+            n = 0
+            for ln in orig:
+                n += 1
+                t += "\n{:4} {}".format(n, ln)
+            if log:
+                log(t, 1)
+            else:
+                print("\033[31m" + t)
+            raise Exception(t1)
+
+    if args.vc and log:
+        t = "new config syntax (copy/paste this to upgrade your config):\n"
+        t += "\n# ======================[ begin upgraded config ]======================\n\n"
+        for ln in ret:
+            t += ln + "\n"
+        t += "\n# ======================[ end of upgraded config ]======================\n"
+        log(t)
+
+    return ret

copyparty/cfg.py

@@ -0,0 +1,149 @@
+# coding: utf-8
+from __future__ import print_function, unicode_literals
+
+# awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
+zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nw p q s ss sss v z zv"
+onedash = set(zs.split())
+
+
+def vf_bmap() -> dict[str, str]:
+    """argv-to-volflag: simple bools"""
+    ret = {
+        "never_symlink": "neversymlink",
+        "no_dedup": "copydupes",
+        "no_dupe": "nodupe",
+        "no_forget": "noforget",
+    }
+    for k in (
+        "dotsrch",
+        "e2t",
+        "e2ts",
+        "e2tsr",
+        "e2v",
+        "e2vu",
+        "e2vp",
+        "hardlink",
+        "magic",
+        "no_sb_md",
+        "no_sb_lg",
+        "rand",
+        "xdev",
+        "xlink",
+        "xvol",
+    ):
+        ret[k] = k
+    return ret
+
+
+def vf_vmap() -> dict[str, str]:
+    """argv-to-volflag: simple values"""
+    ret = {}
+    for k in ("lg_sbf", "md_sbf"):
+        ret[k] = k
+    return ret
+
+
+def vf_cmap() -> dict[str, str]:
+    """argv-to-volflag: complex/lists"""
+    ret = {}
+    for k in ("dbd", "html_head", "mte", "mth", "nrand"):
+        ret[k] = k
+    return ret
+
+
+permdescs = {
+    "r": "read; list folder contents, download files",
+    "w": 'write; upload files; need "r" to see the uploads',
+    "m": 'move; move files and folders; need "w" at destination',
+    "d": "delete; permanently delete files and folders",
+    "g": "get; download files, but cannot see folder contents",
+    "G": 'upget; same as "g" but can see filekeys of their own uploads',
+}
+
+
+flagcats = {
+    "uploads, general": {
+        "nodupe": "rejects existing files (instead of symlinking them)",
+        "hardlink": "does dedup with hardlinks instead of symlinks",
+        "neversymlink": "disables symlink fallback; full copy instead",
+        "copydupes": "disables dedup, always saves full copies of dupes",
+        "daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
+        "nosub": "forces all uploads into the top folder of the vfs",
+        "magic": "enables filetype detection for nameless uploads",
+        "gz": "allows server-side gzip of uploads with ?gz (also c,xz)",
+        "pk": "forces server-side compression, optional arg: xz,9",
+    },
+    "upload rules": {
+        "maxn=250,600": "max 250 uploads over 15min",
+        "maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g)",
+        "rand": "force randomized filenames, 9 chars long by default",
+        "nrand=N": "randomized filenames are N chars long",
+        "sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
+        "df=1g": "ensure 1 GiB free disk space",
+    },
+    "upload rotation\n(moves all uploads into the specified folder structure)": {
+        "rotn=100,3": "3 levels of subfolders with 100 entries in each",
+        "rotf=%Y-%m/%d-%H": "date-formatted organizing",
+        "lifetime=3600": "uploads are deleted after 1 hour",
+    },
+    "database, general": {
+        "e2d": "enable database; makes files searchable + enables upload dedup",
+        "e2ds": "scan writable folders for new files on startup; also sets -e2d",
+        "e2dsa": "scans all folders for new files on startup; also sets -e2d",
+        "e2t": "enable multimedia indexing; makes it possible to search for tags",
+        "e2ts": "scan existing files for tags on startup; also sets -e2t",
+        "e2tsa": "delete all metadata from DB (full rescan); also sets -e2ts",
+        "d2ts": "disables metadata collection for existing files",
+        "d2ds": "disables onboot indexing, overrides -e2ds*",
+        "d2t": "disables metadata collection, overrides -e2t*",
+        "d2v": "disables file verification, overrides -e2v*",
+        "d2d": "disables all database stuff, overrides -e2*",
+        "hist=/tmp/cdb": "puts thumbnails and indexes at that location",
+        "scan=60": "scan for new files every 60sec, same as --re-maxage",
+        "nohash=\\.iso$": "skips hashing file contents if path matches *.iso",
+        "noidx=\\.iso$": "fully ignores the contents at paths matching *.iso",
+        "noforget": "don't forget files when deleted from disk",
+        "dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
+        "xlink": "cross-volume dupe detection / linking",
+        "xdev": "do not descend into other filesystems",
+        "xvol": "skip symlinks leaving the volume root",
+        "dotsrch": "show dotfiles in search results",
+        "nodotsrch": "hide dotfiles in search results (default)",
+    },
+    'database, audio tags\n"mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...': {
+        "mtp=.bpm=f,audio-bpm.py": 'uses the "audio-bpm.py" program to\ngenerate ".bpm" tags from uploads (f = overwrite tags)',
+        "mtp=ahash,vhash=media-hash.py": "collects two tags at once",
+    },
+    "thumbnails": {
+        "dthumb": "disables all thumbnails",
+        "dvthumb": "disables video thumbnails",
+        "dathumb": "disables audio thumbnails (spectrograms)",
+        "dithumb": "disables image thumbnails",
+    },
+    "event hooks\n(better explained in --help-hooks)": {
+        "xbu=CMD": "execute CMD before a file upload starts",
+        "xau=CMD": "execute CMD after  a file upload finishes",
+        "xbr=CMD": "execute CMD before a file rename/move",
+        "xar=CMD": "execute CMD after  a file rename/move",
+        "xbd=CMD": "execute CMD before a file delete",
+        "xad=CMD": "execute CMD after  a file delete",
+        "xm=CMD": "execute CMD on message",
+    },
+    "client and ux": {
+        "html_head=TXT": "includes TXT in the <head>",
+        "robots": "allows indexing by search engines (default)",
+        "norobots": "kindly asks search engines to leave",
+        "no_sb_md": "disable js sandbox for markdown files",
+        "no_sb_lg": "disable js sandbox for prologue/epilogue",
+        "sb_md": "enable js sandbox for markdown files (default)",
+        "sb_lg": "enable js sandbox for prologue/epilogue (default)",
+        "md_sbf": "list of markdown-sandbox safeguards to disable",
+        "lg_sbf": "list of *logue-sandbox safeguards to disable",
+    },
+    "others": {
+        "fk=8": 'generates per-file accesskeys,\nwhich will then be required at the "g" permission'
+    },
+}
+
+
+flagdescs = {k.split("=")[0]: v for tab in flagcats.values() for k, v in tab.items()}

copyparty/ftpd.py

@@ -13,9 +13,19 @@ from pyftpdlib.filesystems import AbstractedFS, FilesystemError
 from pyftpdlib.handlers import FTPHandler
 from pyftpdlib.servers import FTPServer
 
-from .__init__ import PY2, TYPE_CHECKING, E
+from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
 from .bos import bos
-from .util import Daemon, Pebkac, exclude_dotfiles, fsenc, ipnorm
+from .util import (
+    Daemon,
+    Pebkac,
+    exclude_dotfiles,
+    fsenc,
+    ipnorm,
+    pybin,
+    relchk,
+    sanitize_fn,
+    vjoin,
+)
 
 try:
     from pyftpdlib.ioloop import IOLoop
@@ -125,6 +135,13 @@ class FtpFs(AbstractedFS):
     ) -> str:
         try:
             vpath = vpath.replace("\\", "/").lstrip("/")
+            rd, fn = os.path.split(vpath)
+            if ANYWIN and relchk(rd):
+                logging.warning("malicious vpath: %s", vpath)
+                raise FilesystemError("unsupported characters in filepath")
+
+            fn = sanitize_fn(fn or "", "", [".prologue.html", ".epilogue.html"])
+            vpath = vjoin(rd, fn)
             vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
             if not vfs.realpath:
                 raise FilesystemError("no filesystem mounted at this path")
@@ -402,7 +419,7 @@ class Ftpd(object):
                 h1 = SftpHandler
             except:
                 t = "\nftps requires pyopenssl;\nplease run the following:\n\n  {} -m pip install --user pyopenssl\n"
-                print(t.format(sys.executable))
+                print(t.format(pybin))
                 sys.exit(1)
 
             h1.certfile = os.path.join(self.args.E.cfg, "cert.pem")
@@ -435,10 +452,18 @@ class Ftpd(object):
         lgr = logging.getLogger("pyftpdlib")
         lgr.setLevel(logging.DEBUG if self.args.ftpv else logging.INFO)
 
+        ips = self.args.i
+        if "::" in ips:
+            ips.append("0.0.0.0")
+
         ioloop = IOLoop()
-        for ip in self.args.i:
+        for ip in ips:
             for h, lp in hs:
+                try:
                     FTPServer((ip, int(lp)), h, ioloop)
+                except:
+                    if ip != "0.0.0.0" or "::" not in ips:
+                        raise
 
         Daemon(ioloop.loop, "ftp")
 

copyparty/httpcli.py

@@ -28,6 +28,7 @@ except:
     pass
 
 from .__init__ import ANYWIN, PY2, TYPE_CHECKING, EnvParams, unicode
+from .__version__ import S_VERSION
 from .authsrv import VFS  # typechk
 from .bos import bos
 from .star import StreamTar
@@ -51,12 +52,14 @@ from .util import (
     guess_mime,
     gzip_orig_sz,
     hashcopy,
+    hidedir,
     html_bescape,
     html_escape,
     humansize,
     ipnorm,
     min_ex,
     quotep,
+    rand_name,
     read_header,
     read_socket,
     read_socket_chunked,
@@ -64,7 +67,6 @@ from .util import (
     relchk,
     ren_open,
     runhook,
-    hidedir,
     s3enc,
     sanitize_fn,
     sendfile_kern,
@@ -1057,11 +1059,14 @@ class HttpCli(object):
         lk = parse_xml(txt)
         assert lk.tag == "{DAV:}lockinfo"
 
-        if not lk.find(r"./{DAV:}depth"):
-            lk.append(mktnod("D:depth", "infinity"))
+        token = str(uuid.uuid4())
 
-        lk.append(mkenod("D:timeout", mktnod("D:href", "Second-3310")))
-        lk.append(mkenod("D:locktoken", mktnod("D:href", uuid.uuid4().urn)))
+        if not lk.find(r"./{DAV:}depth"):
+            depth = self.headers.get("depth", "infinity")
+            lk.append(mktnod("D:depth", depth))
+
+        lk.append(mktnod("D:timeout", "Second-3310"))
+        lk.append(mkenod("D:locktoken", mktnod("D:href", token)))
         lk.append(
             mkenod("D:lockroot", mktnod("D:href", quotep(self.args.SRS + self.vpath)))
         )
@@ -1074,11 +1079,13 @@ class HttpCli(object):
         ret = '<?xml version="1.0" encoding="{}"?>\n'.format(uenc)
         ret += ET.tostring(xroot).decode("utf-8")
 
+        rc = 200
         if self.can_write and not bos.path.isfile(abspath):
             with open(fsenc(abspath), "wb") as _:
-                pass
+                rc = 201
 
-        self.reply(ret.encode(enc, "replace"), 200, "text/xml; charset=" + enc)
+        self.out_headers["Lock-Token"] = "<{}>".format(token)
+        self.reply(ret.encode(enc, "replace"), rc, "text/xml; charset=" + enc)
         return True
 
     def handle_unlock(self) -> bool:
@@ -1388,30 +1395,14 @@ class HttpCli(object):
         params.update(open_ka)
         assert fn
 
-        if rnd and not self.args.nw:
-            fn = self.rand_name(fdir, fn, rnd)
+        if not self.args.nw:
+            if rnd:
+                fn = rand_name(fdir, fn, rnd)
+
+            fn = sanitize_fn(fn or "", "", [".prologue.html", ".epilogue.html"])
 
         path = os.path.join(fdir, fn)
 
-        if is_put and not self.args.no_dav:
-            # allow overwrite if...
-            #  * volflag 'daw' is set
-            #  * and account has delete-access
-            # or...
-            #  * file exists and is empty
-            #  * and there is no .PARTIAL
-
-            tnam = fn + ".PARTIAL"
-            if self.args.dotpart:
-                tnam = "." + tnam
-
-            if (vfs.flags.get("daw") and self.can_delete) or (
-                not bos.path.exists(os.path.join(fdir, tnam))
-                and bos.path.exists(path)
-                and not bos.path.getsize(path)
-            ):
-                params["overwrite"] = "a"
-
         if xbu:
             at = time.time() - lifetime
             if not runhook(
@@ -1430,6 +1421,26 @@ class HttpCli(object):
                 self.log(t, 1)
                 raise Pebkac(403, t)
 
+        if is_put and not self.args.no_dav:
+            # allow overwrite if...
+            #  * volflag 'daw' is set
+            #  * and account has delete-access
+            # or...
+            #  * file exists and is empty
+            #  * and there is no .PARTIAL
+
+            tnam = fn + ".PARTIAL"
+            if self.args.dotpart:
+                tnam = "." + tnam
+
+            if (vfs.flags.get("daw") and self.can_delete) or (
+                not bos.path.exists(os.path.join(fdir, tnam))
+                and bos.path.exists(path)
+                and not bos.path.getsize(path)
+            ):
+                # small toctou, but better than clobbering a hardlink
+                bos.unlink(path)
+
         with ren_open(fn, *open_a, **params) as zfw:
             f, fn = zfw["orz"]
             path = os.path.join(fdir, fn)
@@ -1456,7 +1467,7 @@ class HttpCli(object):
 
             if ext:
                 if rnd:
-                    fn2 = self.rand_name(fdir, "a." + ext, rnd)
+                    fn2 = rand_name(fdir, "a." + ext, rnd)
                 else:
                     fn2 = fn.rsplit(".", 1)[0] + "." + ext
 
@@ -1568,27 +1579,6 @@ class HttpCli(object):
         else:
             self.log("bakflip ok", 2)
 
-    def rand_name(self, fdir: str, fn: str, rnd: int) -> str:
-        ok = False
-        try:
-            ext = "." + fn.rsplit(".", 1)[1]
-        except:
-            ext = ""
-
-        for extra in range(16):
-            for _ in range(16):
-                if ok:
-                    break
-
-                nc = rnd + extra
-                nb = int((6 + 6 * nc) / 8)
-                zb = os.urandom(nb)
-                zb = base64.urlsafe_b64encode(zb)
-                fn = zb[:nc].decode("utf-8") + ext
-                ok = not bos.path.exists(os.path.join(fdir, fn))
-
-        return fn
-
     def _spd(self, nbytes: int, add: bool = True) -> str:
         if add:
             self.conn.nbyte += nbytes
@@ -2017,8 +2007,13 @@ class HttpCli(object):
         return True
 
     def upload_flags(self, vfs: VFS) -> tuple[int, bool, int, list[str], list[str]]:
-        srnd = self.uparam.get("rand", self.headers.get("rand", ""))
-        rnd = int(srnd) if srnd and not self.args.nw else 0
+        if self.args.nw:
+            rnd = 0
+        else:
+            rnd = int(self.uparam.get("rand") or self.headers.get("rand") or 0)
+            if vfs.flags.get("rand"):  # force-enable
+                rnd = max(rnd, vfs.flags["nrand"])
+
         ac = self.uparam.get(
             "want", self.headers.get("accept", "").lower().split(";")[-1]
         )
@@ -2073,7 +2068,7 @@ class HttpCli(object):
                 )
                 if p_file and not nullwrite:
                     if rnd:
-                        fname = self.rand_name(fdir, fname, rnd)
+                        fname = rand_name(fdir, fname, rnd)
 
                     if not bos.path.isdir(fdir):
                         raise Pebkac(404, "that folder does not exist")
@@ -2300,7 +2295,7 @@ class HttpCli(object):
             raise Pebkac(400, "could not read lastmod from request")
 
         nullwrite = self.args.nw
-        vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
+        vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, True, True)
         self._assert_safe_rem(rem)
 
         clen = int(self.headers.get("content-length", -1))
@@ -2382,6 +2377,9 @@ class HttpCli(object):
         if p_field != "body":
             raise Pebkac(400, "expected body, got {}".format(p_field))
 
+        if bos.path.exists(fp):
+            bos.unlink(fp)
+
         with open(fsenc(fp), "wb", 512 * 1024) as f:
             sz, sha512, _ = hashcopy(p_data, f, self.args.s_wr_slp)
 
@@ -2878,6 +2876,7 @@ class HttpCli(object):
             dbwt=vs["dbwt"],
             url_suf=suf,
             k304=self.k304(),
+            ver=S_VERSION if self.args.ver else "",
         )
         self.reply(html.encode("utf-8"))
         return True
@@ -3351,7 +3350,7 @@ class HttpCli(object):
         if not self.args.no_readme and not logues[1]:
             for fn in ["README.md", "readme.md"]:
                 fn = os.path.join(abspath, fn)
-                if bos.path.exists(fn):
+                if bos.path.isfile(fn):
                     with open(fsenc(fn), "rb") as f:
                         readme = f.read().decode("utf-8")
                         break
@@ -3366,6 +3365,7 @@ class HttpCli(object):
             "idx": ("e2d" in vn.flags),
             "itag": ("e2t" in vn.flags),
             "lifetime": vn.flags.get("lifetime") or 0,
+            "frand": bool(vn.flags.get("rand")),
             "perms": perms,
             "logues": logues,
             "readme": readme,
@@ -3378,6 +3378,7 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": json.dumps(perms),
             "lifetime": ls_ret["lifetime"],
+            "frand": bool(vn.flags.get("rand")),
             "taglist": [],
             "def_hcols": [],
             "have_emp": self.args.emp,
@@ -3491,7 +3492,9 @@ class HttpCli(object):
                 if self.args.no_zip:
                     margin = "DIR"
                 else:
-                    margin = '<a href="{}?zip">zip</a>'.format(quotep(href))
+                    margin = '<a href="{}?zip" rel="nofollow">zip</a>'.format(
+                        quotep(href)
+                    )
             elif fn in hist:
                 margin = '<a href="{}.hist/{}">#{}</a>'.format(
                     base, html_escape(hist[fn][2], quot=True, crlf=True), hist[fn][0]

copyparty/mdns.py

@@ -11,6 +11,7 @@ from ipaddress import IPv4Network, IPv6Network
 from .__init__ import TYPE_CHECKING
 from .__init__ import unicode as U
 from .multicast import MC_Sck, MCast
+from .stolen.dnslib import AAAA
 from .stolen.dnslib import CLASS as DC
 from .stolen.dnslib import (
     NSEC,
@@ -20,7 +21,6 @@ from .stolen.dnslib import (
     SRV,
     TXT,
     A,
-    AAAA,
     DNSHeader,
     DNSQuestion,
     DNSRecord,

copyparty/mtag.py

@@ -10,7 +10,18 @@ import sys
 
 from .__init__ import PY2, WINDOWS, E, unicode
 from .bos import bos
-from .util import REKOBO_LKEY, fsenc, min_ex, retchk, runcmd, uncyg
+from .util import (
+    FFMPEG_URL,
+    REKOBO_LKEY,
+    fsenc,
+    is_exe,
+    min_ex,
+    pybin,
+    retchk,
+    runcmd,
+    sfsenc,
+    uncyg,
+)
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any, Union
@@ -285,9 +296,14 @@ class MTag(object):
                 self.log(msg, c=3)
 
         if not self.usable:
+            if is_exe:
+                t = "copyparty.exe cannot use mutagen; need ffprobe.exe to read media tags: "
+                self.log(t + FFMPEG_URL)
+                return
+
             msg = "need Mutagen{} to read media tags so please run this:\n{}{} -m pip install --user mutagen\n"
-            pybin = os.path.basename(sys.executable)
-            self.log(msg.format(or_ffprobe, " " * 37, pybin), c=1)
+            pyname = os.path.basename(pybin)
+            self.log(msg.format(or_ffprobe, " " * 37, pyname), c=1)
             return
 
         # https://picard-docs.musicbrainz.org/downloads/MusicBrainz_Picard_Tag_Map.html
@@ -519,12 +535,15 @@ class MTag(object):
 
         env = os.environ.copy()
         try:
+            if is_exe:
+                raise Exception()
+
             pypath = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
             zsl = [str(pypath)] + [str(x) for x in sys.path if x]
             pypath = str(os.pathsep.join(zsl))
             env["PYTHONPATH"] = pypath
         except:
-            if not E.ox:
+            if not E.ox and not is_exe:
                 raise
 
         ret: dict[str, Any] = {}
@@ -532,7 +551,7 @@ class MTag(object):
             try:
                 cmd = [parser.bin, abspath]
                 if parser.bin.endswith(".py"):
-                    cmd = [sys.executable] + cmd
+                    cmd = [pybin] + cmd
 
                 args = {
                     "env": env,
@@ -551,7 +570,7 @@ class MTag(object):
                 else:
                     cmd = ["nice"] + cmd
 
-                bcmd = [fsenc(x) for x in cmd]
+                bcmd = [sfsenc(x) for x in cmd[:-1]] + [fsenc(cmd[-1])]
                 rc, v, err = runcmd(bcmd, **args)  # type: ignore
                 retchk(rc, bcmd, err, self.log, 5, self.args.mtag_v)
                 v = v.strip()

copyparty/smbd.py

@@ -12,7 +12,7 @@ from types import SimpleNamespace
 from .__init__ import ANYWIN, TYPE_CHECKING
 from .authsrv import LEELOO_DALLAS, VFS
 from .bos import bos
-from .util import Daemon, min_ex
+from .util import Daemon, is_exe, min_ex, pybin
 
 if True:  # pylint: disable=using-constant-test
     from typing import Any
@@ -42,8 +42,12 @@ class SMB(object):
             from impacket import smbserver
             from impacket.ntlm import compute_lmhash, compute_nthash
         except ImportError:
+            if is_exe:
+                print("copyparty.exe cannot do SMB")
+                sys.exit(1)
+
             m = "\033[36m\n{}\033[31m\n\nERROR: need 'impacket'; please run this command:\033[33m\n {} -m pip install --user impacket\n\033[0m"
-            print(m.format(min_ex(), sys.executable))
+            print(m.format(min_ex(), pybin))
             sys.exit(1)
 
         # patch vfs into smbserver.os

copyparty/ssdp.py

@@ -8,7 +8,7 @@ from email.utils import formatdate
 
 from .__init__ import TYPE_CHECKING
 from .multicast import MC_Sck, MCast
-from .util import CachedSet, min_ex, html_escape
+from .util import CachedSet, html_escape, min_ex
 
 if TYPE_CHECKING:
     from .broker_util import BrokerCli

copyparty/svchub.py

@@ -35,6 +35,7 @@ from .tcpsrv import TcpSrv
 from .th_srv import HAVE_PIL, HAVE_VIPS, HAVE_WEBP, ThumbSrv
 from .up2k import Up2k
 from .util import (
+    FFMPEG_URL,
     VERSIONS,
     Daemon,
     Garda,
@@ -42,8 +43,10 @@ from .util import (
     HMaccas,
     alltrace,
     ansi_re,
+    is_exe,
     min_ex,
     mp,
+    pybin,
     start_log_thrs,
     start_stackmon,
 )
@@ -67,8 +70,15 @@ class SvcHub(object):
     put() can return a queue (if want_reply=True) which has a blocking get() with the response.
     """
 
-    def __init__(self, args: argparse.Namespace, argv: list[str], printed: str) -> None:
+    def __init__(
+        self,
+        args: argparse.Namespace,
+        dargs: argparse.Namespace,
+        argv: list[str],
+        printed: str,
+    ) -> None:
         self.args = args
+        self.dargs = dargs
         self.argv = argv
         self.E: EnvParams = args.E
         self.logf: Optional[typing.TextIO] = None
@@ -155,7 +165,14 @@ class SvcHub(object):
             args.log_fk = re.compile(args.log_fk)
 
         # initiate all services to manage
-        self.asrv = AuthSrv(self.args, self.log)
+        self.asrv = AuthSrv(self.args, self.log, dargs=self.dargs)
+
+        if args.cgen:
+            self.asrv.cgen()
+
+        if args.exit == "cfg":
+            sys.exit(0)
+
         if args.ls:
             self.asrv.dbg_ls()
 
@@ -180,6 +197,7 @@ class SvcHub(object):
 
         self.args.th_dec = list(decs.keys())
         self.thumbsrv = None
+        want_ff = False
         if not args.no_thumb:
             t = ", ".join(self.args.th_dec) or "(None available)"
             self.log("thumb", "decoder preference: {}".format(t))
@@ -191,8 +209,12 @@ class SvcHub(object):
             if self.args.th_dec:
                 self.thumbsrv = ThumbSrv(self)
             else:
+                want_ff = True
                 msg = "need either Pillow, pyvips, or FFmpeg to create thumbnails; for example:\n{0}{1} -m pip install --user Pillow\n{0}{1} -m pip install --user pyvips\n{0}apt install ffmpeg"
-                msg = msg.format(" " * 37, os.path.basename(sys.executable))
+                msg = msg.format(" " * 37, os.path.basename(pybin))
+                if is_exe:
+                    msg = "copyparty.exe cannot use Pillow or pyvips; need ffprobe.exe and ffmpeg.exe to create thumbnails"
+
                 self.log("thumb", msg, c=3)
 
         if not args.no_acode and args.no_thumb:
@@ -204,6 +226,10 @@ class SvcHub(object):
             msg = "setting --no-acode because either FFmpeg or FFprobe is not available"
             self.log("thumb", msg, c=6)
             args.no_acode = True
+            want_ff = True
+
+        if want_ff and ANYWIN:
+            self.log("thumb", "download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
 
         args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
 
@@ -399,7 +425,7 @@ class SvcHub(object):
 
             lh = codecs.open(fn, "w", encoding="utf-8", errors="replace")
 
-        argv = [sys.executable] + self.argv
+        argv = [pybin] + self.argv
         if hasattr(shlex, "quote"):
             argv = [shlex.quote(x) for x in argv]
         else:

copyparty/szip.py

@@ -2,8 +2,8 @@
 from __future__ import print_function, unicode_literals
 
 import calendar
-import time
 import stat
+import time
 import zlib
 
 from .bos import bos

copyparty/th_srv.py

@@ -12,14 +12,16 @@ import time
 
 from queue import Queue
 
-from .__init__ import TYPE_CHECKING
+from .__init__ import ANYWIN, TYPE_CHECKING
 from .bos import bos
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
 from .util import (
     BytesIO,
     Cooldown,
     Daemon,
+    FFMPEG_URL,
     Pebkac,
+    afsenc,
     fsenc,
     min_ex,
     runcmd,
@@ -82,14 +84,14 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str) -> str:
     # base64 = 64 = 4096
     rd, fn = vsplit(rem)
     if rd:
-        h = hashlib.sha512(fsenc(rd)).digest()
+        h = hashlib.sha512(afsenc(rd)).digest()
         b64 = base64.urlsafe_b64encode(h).decode("ascii")[:24]
         rd = "{}/{}/".format(b64[:2], b64[2:4]).lower() + b64
     else:
         rd = "top"
 
     # could keep original filenames but this is safer re pathlen
-    h = hashlib.sha512(fsenc(fn)).digest()
+    h = hashlib.sha512(afsenc(fn)).digest()
     fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
 
     if fmt in ("opus", "caf"):
@@ -133,6 +135,8 @@ class ThumbSrv(object):
             msg = "cannot create audio/video thumbnails because some of the required programs are not available: "
             msg += ", ".join(missing)
             self.log(msg, c=3)
+            if ANYWIN and not self.args.no_acode:
+                self.log("download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
 
         if self.args.th_clean:
             Daemon(self.cleaner, "thumb.cln")
@@ -196,12 +200,12 @@ class ThumbSrv(object):
                 self.log("wait {}".format(tpath))
             except:
                 thdir = os.path.dirname(tpath)
-                bos.makedirs(thdir)
+                bos.makedirs(os.path.join(thdir, "w"))
 
                 inf_path = os.path.join(thdir, "dir.txt")
                 if not bos.path.exists(inf_path):
                     with open(inf_path, "wb") as f:
-                        f.write(fsenc(os.path.dirname(abspath)))
+                        f.write(afsenc(os.path.dirname(abspath)))
 
                 self.busy[tpath] = [cond]
                 do_conv = True
@@ -268,9 +272,12 @@ class ThumbSrv(object):
             if not png_ok and tpath.endswith(".png"):
                 raise Pebkac(400, "png only allowed for waveforms")
 
+            tdir, tfn = os.path.split(tpath)
+            ttpath = os.path.join(tdir, "w", tfn)
+
             for fun in funs:
                 try:
-                    fun(abspath, tpath)
+                    fun(abspath, ttpath)
                     break
                 except Exception as ex:
                     msg = "{} could not create thumbnail of {}\n{}"
@@ -279,12 +286,17 @@ class ThumbSrv(object):
                     self.log(msg, c)
                     if getattr(ex, "returncode", 0) != 321:
                         if fun == funs[-1]:
-                            with open(tpath, "wb") as _:
+                            with open(ttpath, "wb") as _:
                                 pass
                     else:
                         # ffmpeg may spawn empty files on windows
                         try:
-                            os.unlink(tpath)
+                            os.unlink(ttpath)
+                        except:
+                            pass
+
+            try:
+                bos.rename(ttpath, tpath)
             except:
                 pass
 

copyparty/u2idx.py

@@ -311,6 +311,7 @@ class U2idx(object):
 
             sret = []
             fk = flags.get("fk")
+            dots = flags.get("dotsrch")
             c = cur.execute(uq, tuple(vuv))
             for hit in c:
                 w, ts, sz, rd, fn, ip, at = hit[:7]
@@ -321,6 +322,10 @@ class U2idx(object):
                 if rd.startswith("//") or fn.startswith("//"):
                     rd, fn = s3dec(rd, fn)
 
+                rp = quotep("/".join([x for x in [vtop, rd, fn] if x]))
+                if not dots and "/." in ("/" + rp):
+                    continue
+
                 if not fk:
                     suf = ""
                 else:
@@ -337,8 +342,7 @@ class U2idx(object):
                         )[:fk]
                     )
 
-                rp = quotep("/".join([x for x in [vtop, rd, fn] if x])) + suf
-                sret.append({"ts": int(ts), "sz": sz, "rp": rp, "w": w[:16]})
+                sret.append({"ts": int(ts), "sz": sz, "rp": rp + suf, "w": w[:16]})
 
             for hit in sret:
                 w = hit["w"]

copyparty/up2k.py

@@ -38,9 +38,12 @@ from .util import (
     db_ex_chk,
     djoin,
     fsenc,
+    gen_filekey,
+    gen_filekey_dbg,
     hidedir,
     min_ex,
     quotep,
+    rand_name,
     ren_open,
     rmdirs,
     rmdirs_up,
@@ -49,6 +52,7 @@ from .util import (
     s3dec,
     s3enc,
     sanitize_fn,
+    sfsenc,
     spack,
     statdir,
     vjoin,
@@ -158,6 +162,7 @@ class Up2k(object):
             Daemon(self._lastmodder, "up2k-lastmod")
 
         self.fstab = Fstab(self.log_func)
+        self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
 
         if self.args.hash_mt < 2:
             self.mth: Optional[MTHash] = None
@@ -213,6 +218,9 @@ class Up2k(object):
 
         self.log_func("up2k", msg, c)
 
+    def _gen_fk(self, salt: str, fspath: str, fsize: int, inode: int) -> str:
+        return gen_filekey_dbg(salt, fspath, fsize, inode, self.log, self.args.log_fk)
+
     def _block(self, why: str) -> None:
         self.blocked = why
         self.log("uploads temporarily blocked due to " + why, 3)
@@ -388,7 +396,7 @@ class Up2k(object):
 
     def _vis_job_progress(self, job: dict[str, Any]) -> str:
         perc = 100 - (len(job["need"]) * 100.0 / len(job["hash"]))
-        path = os.path.join(job["ptop"], job["prel"], job["name"])
+        path = djoin(job["ptop"], job["prel"], job["name"])
         return "{:5.1f}% {}".format(perc, path)
 
     def _vis_reg_progress(self, reg: dict[str, dict[str, Any]]) -> list[str]:
@@ -478,7 +486,7 @@ class Up2k(object):
         if next((zv for zv in vols if "e2ds" in zv.flags), None):
             self._block("indexing")
 
-        if self.args.re_dhash:
+        if self.args.re_dhash or [zv for zv in vols if "e2tsr" in zv.flags]:
             self.args.re_dhash = False
             self._drop_caches()
 
@@ -644,14 +652,8 @@ class Up2k(object):
         ff = "\033[0;35m{}{:.0}"
         fv = "\033[0;36m{}:\033[90m{}"
         fx = set(("html_head",))
-        fd = {
-            "dbd": "dbd",
-            "lg_sbf": "lg_sbf",
-            "md_sbf": "md_sbf",
-            "mte": "mte",
-            "mth": "mth",
-            "mtp": "mtp",
-        }
+        fdl = ("dbd", "lg_sbf", "md_sbf", "mte", "mth", "mtp", "nrand", "rand")
+        fd = {x: x for x in fdl}
         fl = {
             k: v
             for k, v in flags.items()
@@ -690,7 +692,7 @@ class Up2k(object):
                 pass
 
             for k, job in reg2.items():
-                path = os.path.join(job["ptop"], job["prel"], job["name"])
+                path = djoin(job["ptop"], job["prel"], job["name"])
                 if bos.path.exists(path):
                     reg[k] = job
                     job["poke"] = time.time()
@@ -1085,7 +1087,7 @@ class Up2k(object):
             else:
                 rd = drd
 
-            abspath = os.path.join(top, rd)
+            abspath = djoin(top, rd)
             self.pp.msg = "b{} {}".format(ndirs - nchecked, abspath)
             try:
                 if os.path.isdir(abspath):
@@ -1126,7 +1128,7 @@ class Up2k(object):
                 if crd != rd:
                     crd = rd
                     try:
-                        cdc = set(os.listdir(os.path.join(top, rd)))
+                        cdc = set(os.listdir(djoin(top, rd)))
                     except:
                         cdc.clear()
 
@@ -1202,7 +1204,7 @@ class Up2k(object):
                     rd = drd
                     fn = dfn
 
-                abspath = os.path.join(ptop, rd, fn)
+                abspath = djoin(ptop, rd, fn)
                 if rei and rei.search(abspath):
                     continue
 
@@ -1411,7 +1413,7 @@ class Up2k(object):
                     q = "insert into mt values (?,'t:mtp','a')"
                     cur.execute(q, (w[:16],))
 
-            abspath = os.path.join(ptop, rd, fn)
+            abspath = djoin(ptop, rd, fn)
             self.pp.msg = "c{} {}".format(nq, abspath)
             if not mpool:
                 n_tags = self._tagscan_file(cur, entags, w, abspath, ip, at)
@@ -1575,7 +1577,7 @@ class Up2k(object):
                     q = "select rd, fn, ip, at from up where substr(w,1,16)=? limit 1"
                     rd, fn, ip, at = cur.execute(q, (w,)).fetchone()
                     rd, fn = s3dec(rd, fn)
-                    abspath = os.path.join(ptop, rd, fn)
+                    abspath = djoin(ptop, rd, fn)
 
                     q = "select k from mt where w = ?"
                     zq = cur.execute(q, (w,)).fetchall()
@@ -2027,6 +2029,7 @@ class Up2k(object):
             reg = self.registry[ptop]
             vfs = self.asrv.vfs.all_vols[cj["vtop"]]
             n4g = vfs.flags.get("noforget")
+            rand = vfs.flags.get("rand") or cj.get("rand")
             lost: list[tuple["sqlite3.Cursor", str, str]] = []
 
             vols = [(ptop, jcur)] if jcur else []
@@ -2051,7 +2054,7 @@ class Up2k(object):
                     if dp_dir.startswith("//") or dp_fn.startswith("//"):
                         dp_dir, dp_fn = s3dec(dp_dir, dp_fn)
 
-                    dp_abs = "/".join([ptop, dp_dir, dp_fn])
+                    dp_abs = djoin(ptop, dp_dir, dp_fn)
                     try:
                         st = bos.stat(dp_abs)
                         if stat.S_ISLNK(st.st_mode):
@@ -2091,7 +2094,12 @@ class Up2k(object):
                     )
                     alts.append((score, -len(alts), j))
 
-            job = sorted(alts, reverse=True)[0][2] if alts else None
+            if alts:
+                best = sorted(alts, reverse=True)[0]
+                job = best[2]
+            else:
+                job = None
+
             if job and wark in reg:
                 # self.log("pop " + wark + "  " + job["name"] + " handle_json db", 4)
                 del reg[wark]
@@ -2121,7 +2129,7 @@ class Up2k(object):
                     # ensure the files haven't been deleted manually
                     names = [job[x] for x in ["name", "tnam"] if x in job]
                     for fn in names:
-                        path = os.path.join(job["ptop"], job["prel"], fn)
+                        path = djoin(job["ptop"], job["prel"], fn)
                         try:
                             if bos.path.getsize(path) > 0:
                                 # upload completed or both present
@@ -2133,9 +2141,9 @@ class Up2k(object):
                             break
                 else:
                     # file contents match, but not the path
-                    src = os.path.join(job["ptop"], job["prel"], job["name"])
-                    dst = os.path.join(cj["ptop"], cj["prel"], cj["name"])
-                    vsrc = os.path.join(job["vtop"], job["prel"], job["name"])
+                    src = djoin(job["ptop"], job["prel"], job["name"])
+                    dst = djoin(cj["ptop"], cj["prel"], cj["name"])
+                    vsrc = djoin(job["vtop"], job["prel"], job["name"])
                     vsrc = vsrc.replace("\\", "/")  # just for prints anyways
                     if job["need"]:
                         self.log("unfinished:\n  {0}\n  {1}".format(src, dst))
@@ -2162,30 +2170,40 @@ class Up2k(object):
                         # symlink to the client-provided name,
                         # returning the previous upload info
                         job = deepcopy(job)
-                        for k in ["ptop", "vtop", "prel"]:
+                        for k in "ptop vtop prel addr".split():
                             job[k] = cj[k]
 
                         pdir = djoin(cj["ptop"], cj["prel"])
+                        if rand:
+                            job["name"] = rand_name(
+                                pdir, cj["name"], vfs.flags["nrand"]
+                            )
+                        else:
                             job["name"] = self._untaken(pdir, cj, now)
-                        dst = os.path.join(job["ptop"], job["prel"], job["name"])
+
+                        dst = djoin(job["ptop"], job["prel"], job["name"])
                         if not self.args.nw:
-                            bos.unlink(dst)  # TODO ed pls
                             try:
-                                dst_flags = self.flags[job["ptop"]]
-                                self._symlink(src, dst, dst_flags, lmod=cj["lmod"])
+                                dvf = self.flags[job["ptop"]]
+                                self._symlink(src, dst, dvf, lmod=cj["lmod"], rm=True)
                             except:
+                                if bos.path.exists(dst):
+                                    bos.unlink(dst)
                                 if not n4g:
                                     raise
 
                         if cur:
-                            a = [cj[x] for x in "prel name lmod size addr".split()]
-                            a += [cj.get("at") or time.time()]
+                            a = [job[x] for x in "prel name lmod size addr".split()]
+                            a += [job.get("at") or time.time()]
                             self.db_add(cur, wark, *a)
                             cur.connection.commit()
 
             if not job:
-                if vfs.lim:
                 ap1 = djoin(cj["ptop"], cj["prel"])
+                if rand:
+                    cj["name"] = rand_name(ap1, cj["name"], vfs.flags["nrand"])
+
+                if vfs.lim:
                     ap2, cj["prel"] = vfs.lim.all(
                         cj["addr"], cj["prel"], cj["size"], ap1, reg
                     )
@@ -2238,7 +2256,7 @@ class Up2k(object):
             purl = "{}/{}".format(job["vtop"], job["prel"]).strip("/")
             purl = "/{}/".format(purl) if purl else "/"
 
-            return {
+            ret = {
                 "name": job["name"],
                 "purl": purl,
                 "size": job["size"],
@@ -2248,6 +2266,18 @@ class Up2k(object):
                 "wark": wark,
             }
 
+            if (
+                not ret["hash"]
+                and "fk" in vfs.flags
+                and (cj["user"] in vfs.axs.uread or cj["user"] in vfs.axs.upget)
+            ):
+                ap = absreal(djoin(job["ptop"], job["prel"], job["name"]))
+                ino = 0 if ANYWIN else bos.stat(ap).st_ino
+                fk = self.gen_fk(self.args.fk_salt, ap, job["size"], ino)
+                ret["fk"] = fk[: vfs.flags["fk"]]
+
+            return ret
+
     def _untaken(self, fdir: str, job: dict[str, Any], ts: float) -> str:
         fname = job["name"]
         ip = job["addr"]
@@ -2255,7 +2285,7 @@ class Up2k(object):
         if self.args.nw:
             return fname
 
-        fp = os.path.join(fdir, fname)
+        fp = djoin(fdir, fname)
         if job.get("replace") and bos.path.exists(fp):
             self.log("replacing existing file at {}".format(fp))
             bos.unlink(fp)
@@ -2275,6 +2305,7 @@ class Up2k(object):
         dst: str,
         flags: dict[str, Any],
         verbose: bool = True,
+        rm: bool = False,
         lmod: float = 0,
     ) -> None:
         if verbose:
@@ -2314,6 +2345,9 @@ class Up2k(object):
                 lsrc = lsrc.replace("/", "\\")
                 ldst = ldst.replace("/", "\\")
 
+            if rm and bos.path.exists(dst):
+                bos.unlink(dst)
+
             try:
                 if "hardlink" in flags:
                     os.link(fsenc(src), fsenc(dst))
@@ -2364,7 +2398,7 @@ class Up2k(object):
                 t = "that chunk is already being written to:\n  {}\n  {} {}/{}\n  {}"
                 raise Pebkac(400, t.format(wark, chash, idx, nh, job["name"]))
 
-            path = os.path.join(job["ptop"], job["prel"], job["tnam"])
+            path = djoin(job["ptop"], job["prel"], job["tnam"])
 
             chunksize = up2k_chunksize(job["size"])
             ofs = [chunksize * x for x in nchunk]
@@ -2395,9 +2429,9 @@ class Up2k(object):
             self.db_act = time.time()
             try:
                 job = self.registry[ptop][wark]
-                pdir = os.path.join(job["ptop"], job["prel"])
-                src = os.path.join(pdir, job["tnam"])
-                dst = os.path.join(pdir, job["name"])
+                pdir = djoin(job["ptop"], job["prel"])
+                src = djoin(pdir, job["tnam"])
+                dst = djoin(pdir, job["name"])
             except Exception as ex:
                 return "confirm_chunk, wark, " + repr(ex)  # type: ignore
 
@@ -2430,9 +2464,9 @@ class Up2k(object):
         self.db_act = time.time()
         try:
             job = self.registry[ptop][wark]
-            pdir = os.path.join(job["ptop"], job["prel"])
-            src = os.path.join(pdir, job["tnam"])
-            dst = os.path.join(pdir, job["name"])
+            pdir = djoin(job["ptop"], job["prel"])
+            src = djoin(pdir, job["tnam"])
+            dst = djoin(pdir, job["name"])
         except Exception as ex:
             raise Pebkac(500, "finish_upload, wark, " + repr(ex))
 
@@ -2499,7 +2533,7 @@ class Up2k(object):
 
         cur = self.cur.get(ptop)
         for rd, fn, lmod in dupes:
-            d2 = os.path.join(ptop, rd, fn)
+            d2 = djoin(ptop, rd, fn)
             if os.path.exists(d2):
                 continue
 
@@ -2677,7 +2711,7 @@ class Up2k(object):
                         break
 
                 n_files += 1
-                abspath = os.path.join(adir, fn)
+                abspath = djoin(adir, fn)
                 volpath = "{}/{}".format(vrem, fn).strip("/")
                 vpath = "{}/{}".format(dbv.vpath, volpath).strip("/")
                 self.log("rm {}\n  {}".format(vpath, abspath))
@@ -2956,14 +2990,14 @@ class Up2k(object):
         or to first remaining full if no dabs (delete)
         """
         dupes = []
-        sabs = os.path.join(sptop, srem)
+        sabs = djoin(sptop, srem)
         q = "select rd, fn from up where substr(w,1,16)=? and w=?"
         for ptop, cur in self.cur.items():
             for rd, fn in cur.execute(q, (wark[:16], wark)):
                 if rd.startswith("//") or fn.startswith("//"):
                     rd, fn = s3dec(rd, fn)
 
-                dvrem = "/".join([rd, fn]).strip("/")
+                dvrem = vjoin(rd, fn).strip("/")
                 if ptop != sptop or srem != dvrem:
                     dupes.append([ptop, dvrem])
                     self.log("found {} dupe: [{}] {}".format(wark, ptop, dvrem))
@@ -2974,7 +3008,7 @@ class Up2k(object):
         full: dict[str, tuple[str, str]] = {}
         links: dict[str, tuple[str, str]] = {}
         for ptop, vp in dupes:
-            ap = os.path.join(ptop, vp)
+            ap = djoin(ptop, vp)
             try:
                 d = links if bos.path.islink(ap) else full
                 d[ap] = (ptop, vp)
@@ -3078,7 +3112,7 @@ class Up2k(object):
 
     def _new_upload(self, job: dict[str, Any]) -> None:
         pdir = djoin(job["ptop"], job["prel"])
-        if not job["size"] and bos.path.isfile(os.path.join(pdir, job["name"])):
+        if not job["size"] and bos.path.isfile(djoin(pdir, job["name"])):
             return
 
         self.registry[job["ptop"]][job["wark"]] = job
@@ -3123,7 +3157,7 @@ class Up2k(object):
         suffix = "-{:.6f}-{}".format(job["t0"], dip)
         with ren_open(tnam, "wb", fdir=pdir, suffix=suffix) as zfw:
             f, job["tnam"] = zfw["orz"]
-            abspath = os.path.join(pdir, job["tnam"])
+            abspath = djoin(pdir, job["tnam"])
             sprs = job["sprs"]
             sz = job["size"]
             relabel = False
@@ -3222,7 +3256,7 @@ class Up2k(object):
                 x
                 for x in reg.values()
                 if x["need"]
-                and not bos.path.exists(os.path.join(x["ptop"], x["prel"], x["name"]))
+                and not bos.path.exists(djoin(x["ptop"], x["prel"], x["name"]))
             ]
 
         if rm or lost:
@@ -3235,7 +3269,7 @@ class Up2k(object):
                 del reg[job["wark"]]
                 try:
                     # remove the filename reservation
-                    path = os.path.join(job["ptop"], job["prel"], job["name"])
+                    path = djoin(job["ptop"], job["prel"], job["name"])
                     if bos.path.getsize(path) == 0:
                         bos.unlink(path)
                 except:
@@ -3244,7 +3278,7 @@ class Up2k(object):
                 try:
                     if len(job["hash"]) == len(job["need"]):
                         # PARTIAL is empty, delete that too
-                        path = os.path.join(job["ptop"], job["prel"], job["tnam"])
+                        path = djoin(job["ptop"], job["prel"], job["tnam"])
                         bos.unlink(path)
                 except:
                     pass
@@ -3293,7 +3327,7 @@ class Up2k(object):
                 continue
 
             # self.log("\n  " + repr([ptop, rd, fn]))
-            abspath = os.path.join(ptop, rd, fn)
+            abspath = djoin(ptop, rd, fn)
             try:
                 tags = self.mtag.get(abspath)
                 ntags1 = len(tags)
@@ -3343,7 +3377,7 @@ class Up2k(object):
             if "e2d" not in self.flags[ptop]:
                 continue
 
-            abspath = os.path.join(ptop, rd, fn)
+            abspath = djoin(ptop, rd, fn)
             self.log("hashing " + abspath)
             inf = bos.stat(abspath)
             if not inf.st_size:
@@ -3422,6 +3456,6 @@ def up2k_wark_from_hashlist(salt: str, filesize: int, hashes: list[str]) -> str:
 
 
 def up2k_wark_from_metadata(salt: str, sz: int, lastmod: int, rd: str, fn: str) -> str:
-    ret = fsenc("{}\n{}\n{}\n{}\n{}".format(salt, lastmod, sz, rd, fn))
+    ret = sfsenc("{}\n{}\n{}\n{}\n{}".format(salt, lastmod, sz, rd, fn))
     ret = base64.urlsafe_b64encode(hashlib.sha512(ret).digest())
     return "#{}".format(ret.decode("ascii"))[:44]

copyparty/util.py

@@ -14,6 +14,7 @@ import os
 import platform
 import re
 import select
+import shutil
 import signal
 import socket
 import stat
@@ -143,6 +144,8 @@ SYMTIME = sys.version_info > (3, 6) and os.utime in os.supports_follow_symlinks
 
 META_NOBOTS = '<meta name="robots" content="noindex, nofollow">'
 
+FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"
+
 HTTPCODE = {
     200: "OK",
     201: "Created",
@@ -290,6 +293,20 @@ REKOBO_KEY = {
 REKOBO_LKEY = {k.lower(): v for k, v in REKOBO_KEY.items()}
 
 
+pybin = sys.executable or ""
+is_exe = bool(getattr(sys, "frozen", False))
+if is_exe:
+    pybin = ""
+    for p in "python3 python".split():
+        try:
+            p = shutil.which(p)
+            if p:
+                pybin = p
+                break
+        except:
+            pass
+
+
 def py_desc() -> str:
     interp = platform.python_implementation()
     py_ver = ".".join([str(x) for x in sys.version_info])
@@ -1153,20 +1170,12 @@ def ren_open(
     fun = kwargs.pop("fun", open)
     fdir = kwargs.pop("fdir", None)
     suffix = kwargs.pop("suffix", None)
-    overwrite = kwargs.pop("overwrite", None)
 
     if fname == os.devnull:
         with fun(fname, *args, **kwargs) as f:
             yield {"orz": (f, fname)}
             return
 
-    if overwrite:
-        assert fdir
-        fpath = os.path.join(fdir, fname)
-        with fun(fsenc(fpath), *args, **kwargs) as f:
-            yield {"orz": (f, fname)}
-            return
-
     if suffix:
         ext = fname.split(".")[-1]
         if len(ext) < 7:
@@ -1513,6 +1522,28 @@ def read_header(sr: Unrecv) -> list[str]:
         return ret[:ofs].decode("utf-8", "surrogateescape").lstrip("\r\n").split("\r\n")
 
 
+def rand_name(fdir: str, fn: str, rnd: int) -> str:
+    ok = False
+    try:
+        ext = "." + fn.rsplit(".", 1)[1]
+    except:
+        ext = ""
+
+    for extra in range(16):
+        for _ in range(16):
+            if ok:
+                break
+
+            nc = rnd + extra
+            nb = int((6 + 6 * nc) / 8)
+            zb = os.urandom(nb)
+            zb = base64.urlsafe_b64encode(zb)
+            fn = zb[:nc].decode("utf-8") + ext
+            ok = not os.path.exists(fsenc(os.path.join(fdir, fn)))
+
+    return fn
+
+
 def gen_filekey(salt: str, fspath: str, fsize: int, inode: int) -> str:
     return base64.urlsafe_b64encode(
         hashlib.sha512(
@@ -1690,7 +1721,7 @@ def relchk(rp: str) -> str:
 
 def absreal(fpath: str) -> str:
     try:
-        return fsdec(os.path.abspath(os.path.realpath(fsenc(fpath))))
+        return fsdec(os.path.abspath(os.path.realpath(afsenc(fpath))))
     except:
         if not WINDOWS:
             raise
@@ -1810,6 +1841,24 @@ def _w8enc3(txt: str) -> bytes:
     return txt.encode(FS_ENCODING, "surrogateescape")
 
 
+def _msdec(txt: bytes) -> str:
+    ret = txt.decode(FS_ENCODING, "surrogateescape")
+    return ret[4:] if ret.startswith("\\\\?\\") else ret
+
+
+def _msaenc(txt: str) -> bytes:
+    return txt.replace("/", "\\").encode(FS_ENCODING, "surrogateescape")
+
+
+def _msenc(txt: str) -> bytes:
+    txt = txt.replace("/", "\\")
+    if ":" not in txt and not txt.startswith("\\\\"):
+        txt = absreal(txt)
+
+    ret = txt.encode(FS_ENCODING, "surrogateescape")
+    return ret if ret.startswith(b"\\\\") else b"\\\\?\\" + ret
+
+
 w8dec = _w8dec3 if not PY2 else _w8dec2
 w8enc = _w8enc3 if not PY2 else _w8enc2
 
@@ -1824,8 +1873,13 @@ def w8b64enc(txt: str) -> str:
     return base64.urlsafe_b64encode(w8enc(txt)).decode("ascii")
 
 
-if not PY2 or not WINDOWS:
-    fsenc = w8enc
+if not PY2 and WINDOWS:
+    sfsenc = w8enc
+    afsenc = _msaenc
+    fsenc = _msenc
+    fsdec = _msdec
+elif not PY2 or not WINDOWS:
+    fsenc = afsenc = sfsenc = w8enc
     fsdec = w8dec
 else:
     # moonrunes become \x3f with bytestrings,
@@ -1836,7 +1890,7 @@ else:
     def _not_actually_mbcs_dec(txt: bytes) -> str:
         return txt
 
-    fsenc = _not_actually_mbcs_enc
+    fsenc = afsenc = sfsenc = _not_actually_mbcs_enc
     fsdec = _not_actually_mbcs_dec
 
 
@@ -2500,12 +2554,17 @@ def _runhook(
             log(t.format(arg, ocmd))
 
     env = os.environ.copy()
-    # try:
+    try:
+        if is_exe:
+            raise Exception()
+
         pypath = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
         zsl = [str(pypath)] + [str(x) for x in sys.path if x]
         pypath = str(os.pathsep.join(zsl))
         env["PYTHONPATH"] = pypath
-    # except: if not E.ox: raise
+    except:
+        if not is_exe:
+            raise
 
     ka = {
         "env": env,
@@ -2531,9 +2590,9 @@ def _runhook(
 
     acmd = [cmd, arg]
     if cmd.endswith(".py"):
-        acmd = [sys.executable] + acmd
+        acmd = [pybin] + acmd
 
-    bcmd = [fsenc(x) for x in acmd]
+    bcmd = [fsenc(x) if x == ap else sfsenc(x) for x in acmd]
 
     t0 = time.time()
     if fork:

copyparty/web/browser.css

@@ -798,17 +798,24 @@ html.y #path a:hover {
 .logue:empty {
 	display: none;
 }
+#doc>iframe,
 .logue>iframe {
 	background: var(--bgg);
-	border-radius: .3em;
+	border: 1px solid var(--bgg);
+	border-width: 0 .3em 0 .3em;
+	border-radius: .5em;
 	visibility: hidden;
-	border: none;
+	margin: 0 -.3em;
 	width: 100%;
 	height: 0;
 }
+#doc>iframe.focus,
 .logue>iframe.focus {
 	box-shadow: 0 0 .1em .1em var(--a);
 }
+#pro.logue>iframe {
+	height: 100vh;
+}
 #pro.logue {
 	margin-bottom: .8em;
 }
@@ -833,8 +840,9 @@ html.y #path a:hover {
 .mdo {
 	max-width: 52em;
 }
-.mdo.sb {
-	max-width: unset;
+.mdo.sb,
+#epi.logue.mdo>iframe {
+	max-width: 54em;
 }
 .mdo,
 .mdo * {

copyparty/web/browser.html

@@ -36,7 +36,7 @@
 			<input type="file" name="f" multiple /><br />
 			<input type="submit" value="start upload">
 		</form>
-		<a id="bbsw" href="?b=u"><br />switch to basic browser</a>
+		<a id="bbsw" href="?b=u" rel="nofollow"><br />switch to basic browser</a>
 	</div>
 
 	<div id="op_mkdir" class="opview opbox act">
@@ -121,7 +121,7 @@
 	
 	<div id="epi" class="logue">{{ "" if sb_lg else logues[1] }}</div>
 
-	<h2><a href="{{ r }}/?h" id="goh">control-panel</a></h2>
+	<h2 id="wfp"><a href="{{ r }}/?h" id="goh">control-panel</a></h2>
 	
 	<a href="#" id="repl">π</a>
 
@@ -135,6 +135,7 @@
 
 	<script>
 		var SR = {{ r|tojson }},
+			TS = "{{ ts }}",
 			acct = "{{ acct }}",
 			perms = {{ perms }},
 			themes = {{ themes }},
@@ -154,6 +155,7 @@
 			sb_lg = "{{ sb_lg }}",
 			lifetime = {{ lifetime }},
 			turbolvl = {{ turbolvl }},
+			frand = {{ frand|tojson }},
 			u2sort = "{{ u2sort }}",
 			have_emp = {{ have_emp|tojson }},
 			txt_ext = "{{ txt_ext }}",

copyparty/web/browser.js

@@ -108,8 +108,8 @@ var Ls = {
 		"ot_msg": "msg: send a message to the server log",
 		"ot_mp": "media player options",
 		"ot_cfg": "configuration options",
-		"ot_u2i": 'up2k: upload files (if you have write-access) or toggle into the search-mode to see if they exist somewhere on the server$N$Nuploads are resumable, multithreaded, and file timestamps are preserved, but it uses more CPU than the basic uploader<br /><br />during uploads, this icon becomes a progress indicator!',
-		"ot_u2w": 'up2k: upload files with resume support (close your browser and drop the same files in later)$N$Nmultithreaded, and file timestamps are preserved, but it uses more CPU than the basic uploader<br /><br />during uploads, this icon becomes a progress indicator!',
+		"ot_u2i": 'up2k: upload files (if you have write-access) or toggle into the search-mode to see if they exist somewhere on the server$N$Nuploads are resumable, multithreaded, and file timestamps are preserved, but it uses more CPU than [🎈]&nbsp; (the basic uploader)<br /><br />during uploads, this icon becomes a progress indicator!',
+		"ot_u2w": 'up2k: upload files with resume support (close your browser and drop the same files in later)$N$Nmultithreaded, and file timestamps are preserved, but it uses more CPU than [🎈]&nbsp; (the basic uploader)<br /><br />during uploads, this icon becomes a progress indicator!',
 
 		"ab_mkdir": "make directory",
 		"ab_mkdoc": "new markdown doc",
@@ -134,6 +134,7 @@ var Ls = {
 		"wt_next": "next track$NHotkey: L",
 
 		"ul_par": "parallel uploads:",
+		"ut_rand": "randomize filenames",
 		"ut_mt": "continue hashing other files while uploading$N$Nmaybe disable if your CPU or HDD is a bottleneck",
 		"ut_ask": "ask for confirmation before upload starts",
 		"ut_pot": "improve upload speed on slow devices$Nby making the UI less complex",
@@ -158,6 +159,9 @@ var Ls = {
 		"uct_q": "idle, pending",
 
 		"utl_name": "filename",
+		"utl_ulist": "list",
+		"utl_ucopy": "copy",
+		"utl_links": "links",
 		"utl_stat": "status",
 		"utl_prog": "progress",
 
@@ -346,6 +350,7 @@ var Ls = {
 		"s_a1": "specific metadata properties",
 
 		"md_eshow": "cannot show ",
+		"md_off": "[📜<em>readme</em>] disabled in [⚙️] -- document hidden",
 
 		"xhr403": "403: Access denied\n\ntry pressing F5, maybe you got logged out",
 		"cf_ok": "sorry about that -- DD" + wah + "oS protection kicked in\n\nthings should resume in about 30 sec\n\nif nothing happens, hit F5 to reload the page",
@@ -366,7 +371,10 @@ var Ls = {
 		"fz_zipc": "cp437 with crc32 computed early,$Nfor MS-DOS PKZIP v2.04g (october 1993)$N(takes longer to process before download can start)",
 
 		"un_m1": "you can delete your recent uploads below",
-		"un_upd": "refresh list",
+		"un_upd": "refresh",
+		"un_m4": "or share the files visible below:",
+		"un_ulist": "show",
+		"un_ucopy": "copy",
 		"un_flt": "optional filter:&nbsp; URL must contain",
 		"un_fclr": "clear filter",
 		"un_derr": 'unpost-delete failed:\n',
@@ -553,8 +561,8 @@ var Ls = {
 		"ot_msg": "msg: send en beskjed til serverloggen",
 		"ot_mp": "musikkspiller-instillinger",
 		"ot_cfg": "andre innstillinger",
-		"ot_u2i": 'up2k: last opp filer (hvis du har skrivetilgang) eller bytt til søkemodus for å sjekke om filene finnes et-eller-annet sted på serveren$N$Nopplastninger kan gjenopptas etter avbrudd, skjer stykkevis for potensielt høyere ytelse, og ivaretar datostempling -- men bruker litt mer prosessorkraft enn den primitive opplasteren bup<br /><br />mens opplastninger foregår så vises fremdriften her oppe!',
-		"ot_u2w": 'up2k: filopplastning med støtte for å gjenoppta avbrutte opplastninger -- steng ned nettleseren og dra de samme filene inn i nettleseren igjen for å plukke opp igjen der du slapp$N$Nopplastninger skjer stykkevis for potensielt høyere ytelse, og ivaretar datostempling -- men bruker litt mer prosessorkraft enn den primitive opplasteren "bup"<br /><br />mens opplastninger foregår så vises fremdriften her oppe!',
+		"ot_u2i": 'up2k: last opp filer (hvis du har skrivetilgang) eller bytt til søkemodus for å sjekke om filene finnes et-eller-annet sted på serveren$N$Nopplastninger kan gjenopptas etter avbrudd, skjer stykkevis for potensielt høyere ytelse, og ivaretar datostempling -- men bruker litt mer prosessorkraft enn [🎈]&nbsp; (den primitive opplasteren "bup")<br /><br />mens opplastninger foregår så vises fremdriften her oppe!',
+		"ot_u2w": 'up2k: filopplastning med støtte for å gjenoppta avbrutte opplastninger -- steng ned nettleseren og dra de samme filene inn i nettleseren igjen for å plukke opp igjen der du slapp$N$Nopplastninger skjer stykkevis for potensielt høyere ytelse, og ivaretar datostempling -- men bruker litt mer prosessorkraft enn [🎈]&nbsp; (den primitive opplasteren "bup")<br /><br />mens opplastninger foregår så vises fremdriften her oppe!',
 
 		"ab_mkdir": "lag mappe",
 		"ab_mkdoc": "nytt dokument",
@@ -571,14 +579,15 @@ var Ls = {
 		"wt_selinv": "inverter utvalg",
 		"wt_selzip": "last ned de valgte filene som et arkiv",
 		"wt_seldl": "last ned de valgte filene$NSnarvei: Y",
-		"wt_npirc": "kopier sang-info (irc-formattert)",
-		"wt_nptxt": "kopier sang-info",
+		"wt_npirc": "kopiér sang-info (irc-formattert)",
+		"wt_nptxt": "kopiér sang-info",
 		"wt_grid": "bytt mellom ikoner og listevisning$NSnarvei: G",
 		"wt_prev": "forrige sang$NSnarvei: J",
 		"wt_play": "play / pause$NSnarvei: P",
 		"wt_next": "neste sang$NSnarvei: L",
 
 		"ul_par": "samtidige handl.:",
+		"ut_rand": "finn opp nye tilfeldige filnavn",
 		"ut_mt": "fortsett å befare køen mens opplastning foregår$N$Nskru denne av dersom du har en$Ntreg prosessor eller harddisk",
 		"ut_ask": "bekreft filutvalg før opplastning starter",
 		"ut_pot": "forbedre ytelsen på trege enheter ved å$Nforenkle brukergrensesnittet",
@@ -603,6 +612,9 @@ var Ls = {
 		"uct_q": "køen",
 
 		"utl_name": "filnavn",
+		"utl_ulist": "vis",
+		"utl_ucopy": "kopiér",
+		"utl_links": "lenker",
 		"utl_stat": "status",
 		"utl_prog": "fremdrift",
 
@@ -791,6 +803,7 @@ var Ls = {
 		"s_a1": "konkrete egenskaper",
 
 		"md_eshow": "kan ikke vise ",
+		"md_off": "[📜<em>readme</em>] er avskrudd i [⚙️] -- dokument skjult",
 
 		"xhr403": "403: Tilgang nektet\n\nkanskje du ble logget ut? prøv å trykk F5",
 		"cf_ok": "beklager -- liten tilfeldig kontroll, alt OK\n\nting skal fortsette om ca. 30 sekunder\n\nhvis ikkeno skjer, trykk F5 for å laste siden på nytt",
@@ -811,7 +824,10 @@ var Ls = {
 		"fz_zipc": "cp437 med tidlig crc32,$Nfor MS-DOS PKZIP v2.04g (oktober 1993)$N(øker behandlingstid på server)",
 
 		"un_m1": "nedenfor kan du angre / slette filer som du nylig har lastet opp",
-		"un_upd": "oppdater listen",
+		"un_upd": "oppdater",
+		"un_m4": "eller hvis du vil dele nedlastnings-lenkene:",
+		"un_ulist": "vis",
+		"un_ucopy": "kopiér",
 		"un_flt": "valgfritt filter:&nbsp; filnavn / filsti må inneholde",
 		"un_fclr": "nullstill filter",
 		"un_derr": 'unpost-sletting feilet:\n',
@@ -963,8 +979,8 @@ ebi('op_up2k').innerHTML = (
 	'			<label for="potato" tt="' + L.ut_pot + '">🥔</label>\n' +
 	'		</td>\n' +
 	'		<td class="c" rowspan="2">\n' +
-	'			<input type="checkbox" id="ask_up" />\n' +
-	'			<label for="ask_up" tt="' + L.ut_ask + '">💭</label>\n' +
+	'			<input type="checkbox" id="u2rand" />\n' +
+	'			<label for="u2rand" tt="' + L.ut_rand + '">🎲</label>\n' +
 	'		</td>\n' +
 	'		<td class="c" data-perm="read" data-dep="idx" rowspan="2">\n' +
 	'			<input type="checkbox" id="fsearch" />\n' +
@@ -1012,7 +1028,7 @@ ebi('op_up2k').innerHTML = (
 	'<div id="u2tabw" class="na"><table id="u2tab">\n' +
 	'	<thead>\n' +
 	'		<tr>\n' +
-	'			<td>' + L.utl_name + '</td>\n' +
+	'			<td>' + L.utl_name + ' &nbsp;(<a href="#" id="luplinks">' + L.utl_ulist + '</a>/<a href="#" id="cuplinks">' + L.utl_ucopy + '</a>' + L.utl_links + ')</td>\n' +
 	'			<td>' + L.utl_stat + '</td>\n' +
 	'			<td>' + L.utl_prog + '</td>\n' +
 	'		</tr>\n' +
@@ -1073,6 +1089,7 @@ ebi('op_cfg').innerHTML = (
 	'<div>\n' +
 	'	<h3>' + L.cl_uopts + '</h3>\n' +
 	'	<div>\n' +
+	'		<a id="ask_up" class="tgl btn" href="#" tt="' + L.ut_ask + '">💭</a>\n' +
 	'		<a id="hashw" class="tgl btn" href="#" tt="' + L.cut_mt + '">mt</a>\n' +
 	'		<a id="u2turbo" class="tgl btn ttb" href="#" tt="' + L.cut_turbo + '">turbo</a>\n' +
 	'		<a id="u2tdate" class="tgl btn ttb" href="#" tt="' + L.cut_datechk + '">date-chk</a>\n' +
@@ -1200,6 +1217,17 @@ function goto(dest) {
 }
 
 
+var SBW, SBH;  // scrollbar size
+(function () {
+	var el = mknod('div');
+	el.style.cssText = 'overflow:scroll;width:100px;height:100px';
+	document.body.appendChild(el);
+	SBW = el.offsetWidth - el.clientWidth;
+	SBH = el.offsetHeight - el.clientHeight;
+	document.body.removeChild(el);
+})();
+
+
 var have_webp = sread('have_webp');
 (function () {
 	if (have_webp !== null)
@@ -1451,9 +1479,9 @@ try {
 catch (ex) { }
 
 
-var re_au_native = can_ogg ? /\.(opus|ogg|m4a|aac|mp3|wav|flac)$/i :
-	have_acode ? /\.(opus|m4a|aac|mp3|wav|flac)$/i : /\.(m4a|aac|mp3|wav|flac)$/i,
-	re_au_all = /\.(aac|m4a|ogg|opus|flac|alac|mp3|mp2|ac3|dts|wma|ra|wav|aif|aiff|au|alaw|ulaw|mulaw|amr|gsm|ape|tak|tta|wv|mpc)$/i;
+var re_au_native = can_ogg ? /\.(aac|flac|m4a|mp3|ogg|opus|wav)$/i :
+	have_acode ? /\.(aac|flac|m4a|mp3|opus|wav)$/i : /\.(aac|flac|m4a|mp3|wav)$/i,
+	re_au_all = /\.(aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk)$/i;
 
 
 // extract songs + add play column
@@ -1697,29 +1725,9 @@ var widget = (function () {
 
 		m += '[' + cv + s2ms(mp.au.currentTime) + ck + '/' + cv + s2ms(mp.au.duration) + ck + ']';
 
-		var o = mknod('input');
-		o.style.cssText = 'position:fixed;top:45%;left:48%;padding:1em;z-index:9';
-		o.value = m;
-		document.body.appendChild(o);
-
-		var cln = function () {
-			o.value = 'copied to clipboard ';
-			setTimeout(function () {
-				document.body.removeChild(o);
-			}, 500);
-		};
-		var fb = function () {
-			console.log('fb');
-			o.focus();
-			o.select();
-			document.execCommand("copy");
-			cln();
-		};
-		try {
-			// https only
-			navigator.clipboard.writeText(m).then(cln, fb);
-		}
-		catch (ex) { fb(); }
+		cliptxt(m, function () {
+			toast.ok(1, 'copied to clipboard', null, 'top');
+		});
 	};
 	r.set(sread('au_open') == 1);
 	setTimeout(function () {
@@ -1785,6 +1793,9 @@ var pbar = (function () {
 		r.wurl = url;
 		var img = new Image();
 		img.onload = function () {
+			if (r.wurl != url)
+				return;
+
 			r.wimg = img;
 			r.onresize();
 		};
@@ -4528,7 +4539,9 @@ document.onkeydown = function (e) {
 		return seek_au_rel(n) || true;
 
 	if (k == 'KeyY')
-		return msel.getsel().length ? ebi('seldl').click() : dl_song();
+		return msel.getsel().length ? ebi('seldl').click() :
+			showfile.active() ? ebi('dldoc').click() :
+				dl_song();
 
 	n = k == 'KeyI' ? -1 : k == 'KeyK' ? 1 : 0;
 	if (n !== 0)
@@ -5529,7 +5542,7 @@ var treectl = (function () {
 				have_up2k_idx = res.idx;
 				have_tags_idx = res.itag;
 				lifetime = res.lifetime;
-				apply_perms(res.perms);
+				apply_perms(res);
 				fileman.render();
 			}
 			if (sel.length)
@@ -5726,8 +5739,40 @@ function despin(sel) {
 }
 
 
-function apply_perms(newperms) {
-	perms = newperms || [];
+var wfp_debounce = (function () {
+	var r = { 'n': 0, 't': 0 };
+
+	r.hide = function () {
+		if (!sb_lg && !sb_md)
+			return;
+
+		if (++r.n <= 1) {
+			r.n = 1;
+			clearTimeout(r.t);
+			r.t = setTimeout(r.reset, 300);
+			ebi('wfp').style.opacity = 0.1;
+		}
+	};
+	r.show = function () {
+		if (!sb_lg && !sb_md)
+			return;
+
+		if (--r.n <= 0) {
+			r.n = 0;
+			clearTimeout(r.t);
+			ebi('wfp').style.opacity = 'unset';
+		}
+	};
+	r.reset = function () {
+		r.n = 0;
+		r.show();
+	};
+	return r;
+})();
+
+
+function apply_perms(res) {
+	perms = res.perms || [];
 
 	var a = QS('#ops a[data-dest="up2k"]');
 	if (have_up2k_idx) {
@@ -5801,6 +5846,8 @@ function apply_perms(newperms) {
 			(have_write || tds[a].getAttribute('data-perm') == 'read') ?
 				'table-cell' : 'none';
 	}
+	if (res.frand)
+		ebi('u2rand').parentNode.style.display = 'none';
 
 	if (up2k)
 		up2k.set_fsearch();
@@ -6585,6 +6632,28 @@ var globalcss = (function () {
 	};
 })();
 
+var sandboxjs = (function () {
+	var ret = '',
+		busy = false,
+		url = SR + '/.cpr/util.js?_=' + TS,
+		tag = '<script src="' + url + '"></script>';
+
+	return function () {
+		if (ret || busy)
+			return ret || tag;
+
+		var xhr = new XHR();
+		xhr.open('GET', url, true);
+		xhr.onload = function () {
+			if (this.status == 200)
+				ret = '<script>' + this.responseText + '</script>';
+		};
+		xhr.send();
+		busy = true;
+		return tag;
+	};
+})();
+
 
 function show_md(md, name, div, url, depth) {
 	var errmsg = L.md_eshow + name + ':\n\n',
@@ -6594,10 +6663,12 @@ function show_md(md, name, div, url, depth) {
 	if (url != now)
 		return;
 
+	wfp_debounce.hide();
 	if (!marked) {
 		if (depth)
 			return toast.warn(10, errmsg + 'failed to load marked.js')
 
+		wfp_debounce.n--;
 		return import_js(SR + '/.cpr/deps/marked.js', function () {
 			show_md(md, name, div, url, 1);
 		});
@@ -6653,6 +6724,7 @@ function show_md(md, name, div, url, depth) {
 	catch (ex) {
 		toast.warn(10, errmsg + ex);
 	}
+	wfp_debounce.show();
 }
 
 
@@ -6665,7 +6737,7 @@ function set_tabindex() {
 
 function show_readme(md) {
 	if (!treectl.ireadme)
-		return;
+		return sandbox(ebi('epi'), '', '', 'a');
 
 	show_md(md, 'README.md', ebi('epi'));
 }
@@ -6674,16 +6746,24 @@ if (readme)
 
 
 function sandbox(tgt, rules, cls, html) {
+	if (!treectl.ireadme) {
+		tgt.innerHTML = html ? L.md_off : '';
+		return;
+	}
 	if (!rules || (html || '').indexOf('<') == -1) {
 		tgt.innerHTML = html;
 		clmod(tgt, 'sb');
 		return false;
 	}
 	clmod(tgt, 'sb', 1);
+
 	var tid = tgt.getAttribute('id'),
 		hash = location.hash,
 		want = '';
 
+	if (!cls)
+		wfp_debounce.hide();
+
 	if (hash.startsWith('#md-'))
 		want = hash.slice(1);
 
@@ -6696,9 +6776,8 @@ function sandbox(tgt, rules, cls, html) {
 
 	html = '<html class="iframe ' + document.documentElement.className + '"><head><style>' + globalcss() +
 		'</style><base target="_parent"></head><body id="b" class="logue ' + cls + '">' + html +
-		'<script>' + env + '</script>' +
-		'<script src="' + SR + '/.cpr/util.js?_={{ ts }}"></script>' +
-		'<script>var ebi=document.getElementById.bind(document),d=document.documentElement,' +
+		'<script>' + env + '</script>' + sandboxjs() +
+		'<script>var d=document.documentElement,' +
 		'loc=new URL("' + location.href.split('?')[0] + '");' +
 		'function say(m){window.parent.postMessage(m,"*")};' +
 		'setTimeout(function(){var its=0,pih=-1,f=function(){' +
@@ -6729,8 +6808,9 @@ window.addEventListener("message", function (e) {
 		var t = e.data.split(/ /g);
 		if (t[0] == 'iheight') {
 			var el = QS(t[1] + '>iframe');
-			el.style.height = t[2] + 'px';
+			el.style.height = (parseInt(t[2]) + SBH) + 'px';
 			el.style.visibility = 'unset';
+			wfp_debounce.show();
 		}
 		else if (t[0] == 'iscroll') {
 			var y1 = QS(t[1]).offsetTop,
@@ -6779,6 +6859,7 @@ function ev_row_tgl(e) {
 var unpost = (function () {
 	ebi('op_unpost').innerHTML = (
 		L.un_m1 + ' &ndash; <a id="unpost_refresh" href="#">' + L.un_upd + '</a>' +
+		'<p>' + L.un_m4 + ' <a id="unpost_ulist" href="#">' + L.un_ulist + '</a> / <a id="unpost_ucopy" href="#">' + L.un_ucopy + '</a>' +
 		'<p>' + L.un_flt + ' <input type="text" id="unpost_filt" size="20" placeholder="documents/passwords" /><a id="unpost_nofilt" href="#">' + L.un_fclr + '</a></p>' +
 		'<div id="unpost"></div>'
 	);
@@ -6844,6 +6925,16 @@ var unpost = (function () {
 		ct.innerHTML = "<p><em>" + L.un_m3 + "</em></p>";
 	};
 
+	function linklist() {
+		var ret = [],
+			base = document.location.origin.replace(/\/$/, '');
+
+		for (var a = 0; a < r.files.length; a++)
+			ret.push(base + r.files[a].vp);
+
+		return ret.join('\r\n');
+	}
+
 	function unpost_delete_cb() {
 		if (this.status !== 200) {
 			var msg = this.responseText;
@@ -6920,6 +7011,19 @@ var unpost = (function () {
 		goto('unpost');
 	};
 
+	ebi('unpost_ulist').onclick = function (e) {
+		ev(e);
+		modal.alert(linklist());
+	};
+
+	ebi('unpost_ucopy').onclick = function (e) {
+		ev(e);
+		var txt = linklist();
+		cliptxt(txt + '\n', function () {
+			toast.inf(5, txt.split('\n').length + ' links copied to clipboard');
+		});
+	};
+
 	return r;
 })();
 

copyparty/web/splash.html

@@ -99,7 +99,7 @@
 	</div>
 	<a href="#" id="repl">π</a>
 	{%- if not this.args.nb %}
-	<span id="pb"><span>powered by</span> <a href="{{ this.args.pb_url }}">copyparty</a></span>
+	<span id="pb"><span>powered by</span> <a href="{{ this.args.pb_url }}">copyparty {{ver}}</a></span>
 	{%- endif %}
 	<script>
 

copyparty/web/ui.css

@@ -42,6 +42,10 @@ html {
 	text-shadow: 1px 1px 0 #000;
 	color: #fff;
 }
+#toast.top {
+	top: 2em;
+	bottom: unset;
+}
 #toast a {
 	color: inherit;
 	text-shadow: inherit;

copyparty/web/up2k.js

@@ -856,6 +856,7 @@ function up2k_init(subtle) {
         fdom_ctr = 0,
         biggest_file = 0;
 
+    bcfg_bind(uc, 'rand', 'u2rand', false, null, false);
     bcfg_bind(uc, 'multitask', 'multitask', true, null, false);
     bcfg_bind(uc, 'potato', 'potato', false, set_potato, false);
     bcfg_bind(uc, 'ask_up', 'ask_up', true, null, false);
@@ -1363,6 +1364,10 @@ function up2k_init(subtle) {
 
             if (uc.fsearch)
                 entry.srch = 1;
+            else if (uc.rand) {
+                entry.rand = true;
+                entry.name = 'a\n' + entry.name;
+            }
 
             if (biggest_file < entry.size)
                 biggest_file = entry.size;
@@ -1398,7 +1403,7 @@ function up2k_init(subtle) {
         ebi('u2tabw').className = 'ye';
 
         setTimeout(function () {
-            if (!actx || actx.state != 'suspended' || toast.tag == L.u_unpt)
+            if (!actx || actx.state != 'suspended' || toast.visible)
                 return;
 
             toast.warn(30, "<div onclick=\"start_actx();toast.inf(3,'thanks!')\">please click this text to<br />unlock full upload speed</div>");
@@ -1418,6 +1423,35 @@ function up2k_init(subtle) {
     }
     more_one_file();
 
+    function linklist() {
+        var ret = [],
+            base = document.location.origin.replace(/\/$/, '');
+
+        for (var a = 0; a < st.files.length; a++) {
+            var t = st.files[a],
+                url = t.purl + uricom_enc(t.name);
+
+            if (t.fk)
+                url += '?k=' + t.fk;
+
+            ret.push(base + url);
+        }
+        return ret.join('\r\n');
+    }
+
+    ebi('luplinks').onclick = function (e) {
+        ev(e);
+        modal.alert(linklist());
+    };
+
+    ebi('cuplinks').onclick = function (e) {
+        ev(e);
+        var txt = linklist();
+        cliptxt(txt + '\n', function () {
+            toast.inf(5, txt.split('\n').length + ' links copied to clipboard');
+        });
+    };
+
     var etaref = 0, etaskip = 0, utw_minh = 0, utw_read = 0;
     function etafun() {
         var nhash = st.busy.head.length + st.busy.hash.length + st.todo.head.length + st.todo.hash.length,
@@ -2213,13 +2247,24 @@ function up2k_init(subtle) {
 
                 t.sprs = response.sprs;
 
-                var rsp_purl = url_enc(response.purl);
-                if (rsp_purl !== t.purl || response.name !== t.name) {
-                    // server renamed us (file exists / path restrictions)
+                var fk = response.fk,
+                    rsp_purl = url_enc(response.purl),
+                    rename = rsp_purl !== t.purl || response.name !== t.name;
+
+                if (rename || fk) {
+                    if (rename)
                         console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + rsp_purl + "] [" + response.name + "]");
+
                     t.purl = rsp_purl;
                     t.name = response.name;
-                    pvis.seth(t.n, 0, linksplit(t.purl + uricom_enc(t.name)).join(' '));
+
+                    var url = t.purl + uricom_enc(t.name);
+                    if (fk) {
+                        t.fk = fk;
+                        url += '?k=' + fk;
+                    }
+
+                    pvis.seth(t.n, 0, linksplit(url).join(' '));
                 }
 
                 var chunksize = get_chunksize(t.size),
@@ -2368,6 +2413,8 @@ function up2k_init(subtle) {
         };
         if (t.srch)
             req.srch = 1;
+        else if (t.rand)
+            req.rand = true;
 
         xhr.open('POST', t.purl, true);
         xhr.responseType = 'text';
@@ -2871,7 +2918,7 @@ ebi('ico1').onclick = function () {
 if (QS('#op_up2k.act'))
     goto_up2k();
 
-apply_perms(perms);
+apply_perms({ "perms": perms, "frand": frand });
 
 
 (function () {

copyparty/web/util.js

@@ -978,6 +978,7 @@ function sethash(hv) {
     }
 }
 
+
 function dl_file(url) {
     console.log('DL [%s]', url);
     var o = mknod('a');
@@ -987,6 +988,25 @@ function dl_file(url) {
 }
 
 
+function cliptxt(txt, ok) {
+    var fb = function () {
+        console.log('fb');
+        var o = mknod('input');
+        o.value = txt;
+        document.body.appendChild(o);
+        o.focus();
+        o.select();
+        document.execCommand("copy");
+        document.body.removeChild(o);
+        ok();
+    };
+    try {
+        navigator.clipboard.writeText(txt).then(ok, fb);
+    }
+    catch (ex) { fb(); }
+}
+
+
 var timer = (function () {
     var r = {};
     r.q = [];
@@ -1260,17 +1280,17 @@ var toast = (function () {
         r.tag = tag;
     };
 
-    r.ok = function (sec, txt, tag) {
-        r.show('ok', sec, txt, tag);
+    r.ok = function (sec, txt, tag, cls) {
+        r.show('ok ' + (cls || ''), sec, txt, tag);
     };
-    r.inf = function (sec, txt, tag) {
-        r.show('inf', sec, txt, tag);
+    r.inf = function (sec, txt, tag, cls) {
+        r.show('inf ' + (cls || ''), sec, txt, tag);
     };
-    r.warn = function (sec, txt, tag) {
-        r.show('warn', sec, txt, tag);
+    r.warn = function (sec, txt, tag, cls) {
+        r.show('warn ' + (cls || ''), sec, txt, tag);
     };
-    r.err = function (sec, txt, tag) {
-        r.show('err', sec, txt, tag);
+    r.err = function (sec, txt, tag, cls) {
+        r.show('err ' + (cls || ''), sec, txt, tag);
     };
 
     return r;

docs/README.md

@@ -13,15 +13,21 @@
 
 # other stuff
 
+## [`example.conf`](example.conf)
+* example config file for `-c`
+
+## [`versus.md`](versus.md)
+* similar software / alternatives (with pros/cons)
+
 ## [`changelog.md`](changelog.md)
 * occasionally grabbed from github release notes
 
+## [`devnotes.md`](devnotes.md)
+* technical stuff
+
 ## [`rclone.md`](rclone.md)
 * notes on using rclone as a fuse client/server
 
-## [`example.conf`](example.conf)
-* example config file for `-c`
-
 
 
 # junk

docs/changelog.md

@@ -1,3 +1,139 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0211-1802  `v1.6.4`  🔧🎲🔗🐳🇦🎶
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* [1.6 theme song](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c) // [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md)
+
+## new features
+* 🔧 new [config syntax](https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf) (#20)
+  * the new syntax is still kinda esoteric and funky but it's an improvement
+  * old config files are still supported
+    * `--vc` prints the autoconverted config which you can copy back into the config file to upgrade
+  * `--vc` will also [annotate and explain](https://user-images.githubusercontent.com/241032/217356028-eb3e141f-80a6-4bc6-8d04-d8d1d874c3e9.png) the config files
+  * new argument `--cgen` to generate config from commandline arguments
+    * kinda buggy, especially the `[global]` section, so give it a lookover before saving it
+* 🎲 randomize filenames on upload
+  * either optionally, using the 🎲 button in the up2k ui
+  * or force-enabled; globally with `--rand` or per-volume with volflag `rand`
+  * specify filename length with `nrand` (globally or volflag), default 9
+* 🔗 export a list of links to your recent uploads
+  * `copy links` in the up2k tab (🚀) will copy links to all uploads since last page refresh,
+  * `copy` in the unpost tab (🧯) will copy links to all your recent uploads (max 2000 files / 12 hours by default)
+  * filekeys are included if that's enabled and you have access to view those (permissions `G` or `r`)
+* 🇦 [arch package](https://github.com/9001/copyparty/tree/hovudstraum/contrib/package/arch) -- added in #18, thx @icxes 
+  * maybe in aur soon!
+* 🐳 [docker containers](https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker) -- 5 editions,
+  * [min](https://hub.docker.com/r/copyparty/min) (57 MiB), just copyparty without thumbnails or audio transcoding
+  * [im](https://hub.docker.com/r/copyparty/im) (70 MiB), thumbnails of popular image formats + media tags with mutagen
+  * [ac (163 MiB)](https://hub.docker.com/r/copyparty/ac) 🥇 adds audio/video thumbnails + audio transcoding + better tags
+  * [iv](https://hub.docker.com/r/copyparty/iv) (211 MiB), makes heif/avic/jxl faster to thumbnail
+  * [dj](https://hub.docker.com/r/copyparty/dj) (309 MiB), adds optional detection of musical key / bpm
+* 🎶 [chiptune player](https://a.ocv.me/pub/demo/music/chiptunes/#af-f6fb2e5f)
+  * transcodes mod/xm/s3m/it/mo3/mptm/mt2/okt to opus
+  * uses FFmpeg (libopenmpt) so the accuracy is not perfect, but most files play OK enough
+  * not **yet** supported in the docker container since Alpine's FFmpeg was built without libopenmpt
+* windows: support long filepaths (over 260 chars)
+  * uses the `//?/` winapi syntax to also support windows 7
+* `--ver` shows the server version on the control panel
+
+## bugfixes
+* markdown files didn't scale properly in the document browser
+* detect and refuse multiple volume definitions sharing the same filesystem path
+* don't return incomplete transcodes if multiple clients try to play the same flac file
+* [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh): more reliable chroot cleanup, sigusr1 for config reload
+* pypi packaging: compress web resources, include webdav.bat
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0131-2103  `v1.6.3`  sandbox k
+
+* read-only demo server at https://a.ocv.me/pub/demo/
+* and since [1.6.0](https://github.com/9001/copyparty/releases/tag/v1.6.2) only got 2 days of prime time,
+  * [1.6 theme song](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c) (hosted on the demo server)
+  * [similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) / feature comparison
+
+## new features
+* dotfiles are hidden from search results by default
+  * use `--dotsrch` or volflags `dotsrch` / `nodotsrch` to specify otherwise
+  * they were already being excluded from tar/zip-files if `-ed` is not set, so this makes more sense -- dotfiles *should* now be undiscoverable unless `-ed` or `--smb` is set, but please use [volumes](https://github.com/9001/copyparty#accounts-and-volumes) for isolation / access-control instead, much safer
+
+## bugfixes
+* lots of cosmetic fixes for the new readme/prologue/epilogue sandbox
+  * rushed it into the previous release when someone suggested it, bad idea
+  * still flickers a bit (especially prologues), and hotkeys are blocked while the sandboxed document has focus
+  * can be disabled with `--no-sb-md --no-sb-lg` (not recommended)
+* support webdav uploads from davfs2 (fix LOCK response)
+* always unlink files before overwriting them, in case they are hardlinks
+  * was primarily an issue with `--daw` and webdav clients
+* on windows, replace characters in PUT filenames as necessary
+* [prisonparty](https://github.com/9001/copyparty/blob/hovudstraum/bin/prisonparty.sh): support opus transcoding on debian
+  * `rm -rf .hist/ac` to clear the transcode cache if the old version broke some songs
+
+## other changes
+* add `rel="nofollow"` to zip download links, basic-browser link
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2023-0129-1842  `v1.6.2`  cors k
+
+[Ellie Goulding - Stay Awake (kors k Hardcore Bootleg).mp3](https://a.ocv.me/pub/demo/music/.bonus/#af-134e597c)
+* 👆 the read-only demo server at https://a.ocv.me/pub/demo/
+
+## breaking changes
+but nothing is affected (that i know of):
+* all requests must pass [cors validation](https://github.com/9001/copyparty#cors)
+  * but they almost definitely did already
+  * sharex and others are OK since they don't supply an `Origin` header
+* [API calls](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#http-api) `?delete` and `?move` are now POST instead of GET
+  * not aware of any clients using these
+
+## known issues
+* the document sandbox is a bit laggy and sometimes eats hotkeys
+  * disable it with `--no-sb-md --no-sb-lg` if you trust everyone who has write and/or move access
+
+## new features
+* [event hooks](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks) -- run programs on new [uploads](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png), renames, deletes
+* [configurable cors](https://github.com/9001/copyparty#cors) (cross-origin resource sharing) behavior; defaults are mostly same as before
+  * `--allow-csrf` disables all csrf protections and makes it intentionally trivial to send authenticated requests from other domains
+* sandboxed readme.md / prologues / epilogues
+  * documents can still run scripts like before, but can no longer tamper with the web-ui / read the login session, so the old advice of `--no-readme` and `--no-logues` is mostly deprecated
+  * unfortunately disables hotkeys while the text has focus + blocks dragdropping files onto that area, oh well
+* password can be provided through http header `PW:` (instead of cookie `cppwd` or or url-param `?pw`)
+* detect network changes (new NICs, IPs) and reconfigure / reannoucne zeroconf
+  * fixes mdns when running as a systemd service and copyparty is started before networking is up
+* add `--freebind` to start listening on IPs before the NIC is up yet (linux-only)
+* per-volume deduplication-control with volflags `hardlink`, `neversymlink`, `copydupes`
+* detect curl and return a [colorful, sortable plaintext](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png) directory listing instead
+* add optional [powered-by-copyparty](https://user-images.githubusercontent.com/241032/215322626-11d1f02b-25f4-45df-a3d9-f8c51354a8eb.png) footnode on the controlpanel
+  * can be disabled with `-nb` or redirected with `--pb-url`
+
+## bugfixes
+* change some API calls (`?delete`, `?move`) from `GET` to `POST`
+  * don't panic! this was safe against authenticated csrf thanks to [SameSite=Lax](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite#lax)
+  * `--getmod` restores the GETs if you need the convenience and accept the risks
+* [u2cli](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) (command-line uploader):
+  * recover from network hiccups
+  * add `-ns` for slow uefi TTYs
+* separate login cookies for http / https
+  * avoids an https login from getting accidentally sent over plaintext
+  * sadly no longer possible to login with internet explorer 4.0 / windows 3.11
+* tar/zip-download of hidden folders
+* unpost filtering was buggy for non-ascii characters
+* moving a deduplicated file on a volume where deduplication was since disabled
+* improved the [linux 6.0.16](https://utcc.utoronto.ca/~cks/space/blog/linux/KernelBindBugIn6016) kernel bug [workaround](https://github.com/9001/copyparty/commit/9065226c3d634a9fc15b14a768116158bc1761ad) because there is similar funk in 5.x
+* add custom text selection colors because chrome is currently broken on fedora
+* blockdevs (`/dev/nvme0n1`) couldn't be downloaded as files
+* misc fixes for location-based reverse-proxying
+* macos dualstack thing
+
+## other changes
+* added a collection of [cursed usecases](https://github.com/9001/copyparty/tree/hovudstraum/docs/cursed-usecases)
+* and [comparisons to similar software](https://github.com/9001/copyparty/blob/hovudstraum/docs/versus.md) in case you ever wanna jump ship
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2023-0112-0515  `v1.5.6`  many hands
 

docs/copyparty.d/foo/another.conf

@@ -1,5 +1,6 @@
 # this file gets included twice from ../some.conf,
 # setting user permissions for a volume
-rw usr1
-r usr2
-% sibling.conf
+accs:
+  rw: usr1
+  r: usr2
+  % sibling.conf

docs/copyparty.d/foo/sibling.conf

@@ -1,3 +1,3 @@
 # and this config file gets included from ./another.conf,
 # adding a final permission for each of the two volumes in ../some.conf
-m usr1 usr2
+m: usr1, usr2

docs/copyparty.d/some.conf

@@ -1,22 +1,29 @@
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
 # lets make two volumes with the same accounts/permissions for both;
 # first declare the accounts just once:
-u usr1:passw0rd
-u usr2:letmein
+[accounts]
+  usr1: passw0rd
+  usr2: letmein
 
-# and listen on 127.0.0.1 only, port 2434
--i 127.0.0.1
--p 2434
+[global]
+  i: 127.0.0.1  # listen on 127.0.0.1 only,
+  p: 2434       # port 2434
+  e2ds  # enable file indexing+scanning
+  e2ts  # and multimedia indexing+scanning
+  # (inline comments are OK if there is 2 spaces before the #)
 
 # share /usr/share/games from the server filesystem
-/usr/share/games
-/vidya
-# include config file with volume permissions
-% foo/another.conf
+[/vidya]
+  /usr/share/games
+  % foo/another.conf  # include config file with volume permissions
 
 # and share your ~/Music folder too
-~/Music
-/bangers
-% foo/another.conf
+[/bangers]
+  ~/Music
+  % foo/another.conf
 
 # which should result in each of the volumes getting the following permissions:
 # usr1 read/write/move

docs/devnotes.md

@@ -229,7 +229,12 @@ rm -rf copyparty/web/deps
 curl -L https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py >x.py
 python3 x.py --version
 rm x.py
-mv /tmp/pe-copyparty/copyparty/web/deps/ copyparty/web/deps/
+cp -R /tmp/pe-copyparty.$(id -u)/copyparty/web/deps copyparty/web/
+```
+
+or you could build the web-dependencies from source instead (NB: does not include prismjs, need to grab that manually):
+```sh
+make -C scripts/deps-docker
 ```
 
 then build the sfx using any of the following examples:

docs/example.conf

@@ -1,59 +1,69 @@
+# not actually YAML but lets pretend:
+# -*- mode: yaml -*-
+# vim: ft=yaml:
+
 # append some arguments to the commandline;
-# the first space in a line counts as a separator,
-# any additional spaces are part of the value
--e2dsa
--e2ts
--i 127.0.0.1
+# accepts anything listed in --help (leading dashes are optional)
+# and inline comments are OK if there is 2 spaces before the '#'
+[global]
+  p: 8086, 3939  # listen on ports 8086 and 3939
+  e2dsa  # enable file indexing and filesystem scanning
+  e2ts   # and enable multimedia indexing
+  z, qr  # and zeroconf and qrcode (you can comma-separate arguments)
 
 # create users:
-# u username:password
-u ed:123
-u k:k
+[accounts]
+  ed: 123   # username: password
+  k: k
 
-# leave a blank line between volumes
-# (and also between users and volumes)
+# create volumes:
+[/]         # create a volume at "/" (the webroot), which will
+  .         # share the contents of "." (the current directory)
+  accs:
+    r: *    # everyone gets read-access, but
+    rw: ed  # the user "ed" gets read-write
 
-# create a volume:
-# share "." (the current directory)
-# as "/" (the webroot) for the following users:
-# "r" grants read-access for anyone
-# "rw ed" grants read-write to ed
-.
-/
-r
-rw ed
-
-# custom permissions for the "priv" folder:
-# user "k" can only see/read the contents
-# user "ed" gets read-write access
-./priv
-/priv
-r k
-rw ed
-
-# this does the same thing,
-# and will cause an error on startup since /priv is already taken:
-./priv
-/priv
-r ed k
-w ed
+# let's specify different permissions for the "priv" subfolder
+# by creating another volume at that location:
+[/priv]
+  ./priv
+  accs:
+    r: k    # the user "k" can see the contents,
+    rw: ed  # while "ed" gets read-write
 
 # share /home/ed/Music/ as /music and let anyone read it
 # (this will replace any folder called "music" in the webroot)
-/home/ed/Music
-/music
-r
+[/music]
+  /home/ed/Music
+  accs:
+    r: *
+
+# and a folder where anyone can upload, but nobody can see the contents
+[/dump]
+  /home/ed/inc
+  accs:
+    w: *
+  flags:
+    e2d     # the e2d volflag enables the uploads database
+    nodupe  # the nodupe volflag rejects duplicate uploads
+    # (see --help-flags for all available volflags to use)
 
 # and a folder where anyone can upload
-# but nobody can see the contents
-# and set the e2d flag to enable the uploads database
-# and set the nodupe flag to reject duplicate uploads
-/home/ed/inc
-/dump
-w
-c e2d
-c nodupe
+# and anyone can access their own uploads, but nothing else
+[/sharex]
+  /home/ed/inc/sharex
+  accs:
+    wG: *        # wG = write-upget = see your own uploads only
+    rwmd: ed, k  # read-write-modify-delete for users "ed" and "k"
+  flags:
+    e2d, d2t, fk: 4
+    # volflag "e2d" enables the uploads database,
+    # "d2t" disables multimedia parsers (in case the uploads are malicious),
+    # "dthumb" disables thumbnails (same reason),
+    # "fk" enables filekeys (necessary for upget permission) (4 chars long)
+    # -- note that its fine to combine all the volflags on
+    #    one line because only the last volflag has an argument
 
 # this entire config file can be replaced with these arguments:
-# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d,nodupe
+# -u ed:123 -u k:k -v .::r:a,ed -v priv:priv:r,k:rw,ed -v /home/ed/Music:music:r -v /home/ed/inc:dump:w:c,e2d,nodupe -v /home/ed/inc/sharex:sharex:wG:c,e2d,d2t,fk=4
 # but note that the config file always wins in case of conflicts

docs/versus.md

@@ -309,6 +309,7 @@ symbol legend,
 * `audio player » os-integration` = use the [lockscreen](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) or [media hotkeys](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) to play/pause, prev/next song
 * `search by custom tags` = ability to tag files through the UI and search by those
 * `find local file` = drop a file into the browser to see if it exists on the server
+* `undo recent uploads` = accounts without delete permissions have a time window where they can undo their own uploads
 * `a`/copyparty has teeny-tiny skips playing gapless albums depending on audio codec (opus best)
 * `b`/hfs2 has a very basic directory tree view, not showing sibling folders
 * `f`/rclone can do some file management (mkdir, rename, delete) when hosting througn webdav

scripts/docker/Dockerfile.ac

@@ -0,0 +1,21 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk --no-cache add \
+            wget \
+            py3-pillow \
+            ffmpeg \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+COPY    i/dist/copyparty-sfx.py ./
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-ac" \
+        org.opencontainers.image.description="copyparty with Pillow and FFmpeg (image/audio/video thumbnails, audio transcoding, media tags)"

scripts/docker/Dockerfile.dj

@@ -0,0 +1,36 @@
+FROM    alpine:latest
+WORKDIR /z
+
+COPY    i/bin/mtag/install-deps.sh ./
+COPY    i/bin/mtag/audio-bpm.py /mtag/
+COPY    i/bin/mtag/audio-key.py /mtag/
+RUN     apk add -U \
+            wget \
+            py3-pillow py3-pip \
+            ffmpeg \
+            vips-jxl vips-heif vips-poppler vips-magick \
+            py3-numpy fftw libsndfile \
+        && python3 -m pip install pyvips \
+        && apk --no-cache add -t .bd \
+            bash wget gcc g++ make cmake patchelf \
+            python3-dev ffmpeg-dev fftw-dev libsndfile-dev \
+            py3-wheel py3-numpy-dev \
+        && bash install-deps.sh \
+        && apk del py3-pip .bd \
+        && rm -rf /var/cache/apk/* \
+        && chmod 777 /root \
+        && ln -s /root/vamp /root/.local / \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+COPY    i/dist/copyparty-sfx.py ./
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-dj" \
+        org.opencontainers.image.description="copyparty with all optional dependencies, including musical key / bpm detection"

scripts/docker/Dockerfile.im

@@ -0,0 +1,20 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk --no-cache add \
+            wget \
+            py3-pillow py3-mutagen \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+COPY    i/dist/copyparty-sfx.py ./
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-im" \
+        org.opencontainers.image.description="copyparty with Pillow and Mutagen (image thumbnails, media tags)"

scripts/docker/Dockerfile.iv

@@ -0,0 +1,24 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk --no-cache add \
+            wget \
+            py3-pillow py3-pip \
+            ffmpeg \
+            vips-jxl vips-heif vips-poppler vips-magick \
+        && python3 -m pip install pyvips \
+        && apk del py3-pip \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+COPY    i/dist/copyparty-sfx.py ./
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-iv" \
+        org.opencontainers.image.description="copyparty with Pillow, FFmpeg, libvips (image/audio/video thumbnails, audio transcoding, media tags)"

scripts/docker/Dockerfile.min

@@ -0,0 +1,19 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk --no-cache add \
+            python3 \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+COPY    i/dist/copyparty-sfx.py ./
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-min" \
+        org.opencontainers.image.description="just copyparty, no thumbnails / media tags / audio transcoding"

scripts/docker/Dockerfile.min.pip

@@ -0,0 +1,19 @@
+FROM    alpine:latest
+WORKDIR /z
+
+RUN     apk --no-cache add python3 py3-pip \
+        && python3 -m pip install copyparty \
+        && apk del py3-pip \
+        && mkdir /cfg /w \
+        && chmod 777 /cfg /w \
+        && echo % /cfg > initcfg
+
+WORKDIR /w
+EXPOSE  3923
+ENTRYPOINT ["python3", "-m", "copyparty", "-c", "/z/initcfg"]
+
+LABEL   org.opencontainers.image.url="https://github.com/9001/copyparty" \
+        org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
+        org.opencontainers.image.licenses="MIT" \
+        org.opencontainers.image.title="copyparty-min-pip" \
+        org.opencontainers.image.description="just copyparty, no thumbnails, no media tags, no audio transcoding"

scripts/docker/Makefile

@@ -0,0 +1,65 @@
+self := $(dir $(abspath $(lastword $(MAKEFILE_LIST))))
+
+all:
+	-service docker start
+	-systemctl start docker
+	
+	rm -rf i
+	mkdir i
+	tar -cC../.. dist/copyparty-sfx.py bin/mtag | tar -xvCi
+	
+	docker build -t copyparty/min:latest -f Dockerfile.min .
+	echo 'scale=1;'`docker save copyparty/min:latest | pigz -c | wc -c`/1024/1024 | bc
+
+#	docker build -t copyparty/min-pip:latest -f Dockerfile.min.pip .
+#	echo 'scale=1;'`docker save copyparty/min-pip:latest | pigz -c | wc -c`/1024/1024 | bc
+
+	docker build -t copyparty/im:latest -f Dockerfile.im .
+	echo 'scale=1;'`docker save copyparty/im:latest | pigz -c | wc -c`/1024/1024 | bc
+
+	docker build -t copyparty/iv:latest -f Dockerfile.iv .
+	echo 'scale=1;'`docker save copyparty/iv:latest | pigz -c | wc -c`/1024/1024 | bc
+
+	docker build -t copyparty/ac:latest -f Dockerfile.ac .
+	echo 'scale=1;'`docker save copyparty/ac:latest | pigz -c | wc -c`/1024/1024 | bc
+
+	docker build -t copyparty/dj:latest -f Dockerfile.dj .
+	echo 'scale=1;'`docker save copyparty/dj:latest | pigz -c | wc -c`/1024/1024 | bc
+
+	docker image ls
+
+push:
+	docker push copyparty/min
+	docker push copyparty/im
+	docker push copyparty/iv
+	docker push copyparty/ac
+	docker push copyparty/dj
+	docker image tag copyparty/min:latest ghcr.io/9001/copyparty-min:latest
+	docker image tag copyparty/im:latest ghcr.io/9001/copyparty-im:latest
+	docker image tag copyparty/iv:latest ghcr.io/9001/copyparty-iv:latest
+	docker image tag copyparty/ac:latest ghcr.io/9001/copyparty-ac:latest
+	docker image tag copyparty/dj:latest ghcr.io/9001/copyparty-dj:latest
+	docker push ghcr.io/9001/copyparty-min:latest
+	docker push ghcr.io/9001/copyparty-im:latest
+	docker push ghcr.io/9001/copyparty-iv:latest
+	docker push ghcr.io/9001/copyparty-ac:latest
+	docker push ghcr.io/9001/copyparty-dj:latest
+
+clean:
+	-docker kill `docker ps -q`
+	-docker rm   `docker ps -qa`
+	-docker rmi -f `docker images -a | awk '/<none>/{print$$3}'`
+
+hclean:
+	-docker kill `docker ps -q`
+	-docker rm   `docker ps -qa`
+	-docker rmi  `docker images -a | awk '!/^alpine/&&NR>1{print$$3}'`
+
+purge:
+	-docker kill `docker ps -q`
+	-docker rm   `docker ps -qa`
+	-docker rmi  `docker images -qa`
+
+sh:
+	@printf "\n\033[1;31mopening a shell in the most recently created docker image\033[0m\n"
+	docker run --rm -it --entrypoint /bin/ash `docker images -aq | head -n 1`

scripts/docker/README.md

@@ -0,0 +1,74 @@
+copyparty is availabe in these repos:
+* https://hub.docker.com/r/copyparty
+* https://github.com/9001?tab=packages&repo_name=copyparty
+
+
+# getting started
+
+run this command to grab the latest copyparty image and start it:
+```bash
+docker run --rm -it -u 1000 -p 3923:3923 -v /mnt/nas:/w -v $PWD/cfgdir:/cfg copyparty/ac
+```
+
+* `/w` is the path inside the container that gets shared by default, so mount one or more folders to share below there
+* `/cfg` is an optional folder with zero or more config files (*.conf) to load
+* `copyparty/ac` is the recommended [image edition](#editions)
+* you can download the image from github instead by replacing `copyparty/ac` with `ghcr.io/9001/copyparty-ac`
+
+i'm unfamiliar with docker-compose and alternatives so let me know if this section could be better 🙏
+
+
+## configuration
+
+the container has the same default config as the sfx and the pypi module, meaning it will listen on port 3923 and share the "current folder" (`/w` inside the container) as read-write for anyone
+
+the recommended way to configure copyparty inside a container is to mount a folder which has one or more [config files](https://github.com/9001/copyparty/blob/hovudstraum/docs/example.conf) inside; `-v /your/config/folder:/cfg`
+
+* but you can also provide arguments to the docker command if you prefer that
+* config files must be named `something.conf` to get picked up
+
+
+## editions
+
+with image size after installation and when gzipped
+
+* `min` (57 MiB, 20 gz) is just copyparty itself
+* `im` (70 MiB, 25 gz) can thumbnail images with pillow, parse media files with mutagen
+* `ac` (163 MiB, 56 gz) is `im` plus ffmpeg for video/audio thumbs + audio transcoding + better tags
+* `iv` (211 MiB, 73 gz) is `ac` plus vips for faster heif / avic / jxl thumbnails
+* `dj` (309 MiB, 104 gz) is `iv` plus beatroot/keyfinder to detect musical keys and bpm
+
+`ac` is recommended since the additional features available in `iv` and `dj` are rarely useful
+
+
+## detecting bpm and musical key
+
+the `dj` edition comes with `keyfinder` and `beatroot` which can be used to detect music bpm and musical keys
+
+enable them globally in a config file:
+```yaml
+[global]
+e2dsa, e2ts  # enable filesystem indexing and multimedia indexing
+mtp: .bpm=f,t30,/mtag/audio-bpm.py  # should take ~10sec
+mtp: key=f,t190,/mtag/audio-key.py  # should take ~50sec
+```
+
+or enable them for just one volume,
+```yaml
+[/music]  # share name / URL
+  music   # filesystem path inside the docker volume `/w`
+  flags:
+    e2dsa, e2ts
+    mtp: .bpm=f,t30,/mtag/audio-bpm.py
+    mtp: key=f,t190,/mtag/audio-key.py
+```
+
+or using commandline arguments,
+```
+-e2dsa -e2ts -mtp .bpm=f,t30,/mtag/audio-bpm.py -mtp key=f,t190,/mtag/audio-key.py
+```
+
+
+# build the images yourself
+
+put `copyparty-sfx.py` into `../dist/` (or [build that from scratch](../../docs/devnotes.md#just-the-sfx) too) then run `make`

scripts/make-pypi-release.sh

@@ -15,10 +15,12 @@ gtar=$(command -v gtar || command -v gnutar) || true
 }
 
 mode="$1"
+fast="$2"
 
 [ -z "$mode" ] &&
 {
 	echo "need argument 1:  (D)ry, (T)est, (U)pload"
+	echo " optional arg 2:  fast"
 	echo
 	exit 1
 }
@@ -90,10 +92,13 @@ load_env || {
 	load_env
 }
 
+# grab licenses
+scripts/genlic.sh copyparty/res/COPYING.txt
+
 # remove type hints to support python < 3.9
 rm -rf build/pypi
 mkdir -p build/pypi
-cp -pR setup.py README.md LICENSE copyparty tests bin scripts/strip_hints build/pypi/
+cp -pR setup.py README.md LICENSE copyparty contrib bin scripts/strip_hints build/pypi/
 tar -c docs/lics.txt scripts/genlic.sh build/*.txt | tar -xC build/pypi/
 cd build/pypi
 f=../strip-hints-0.1.10.tar.gz
@@ -103,6 +108,34 @@ f=../strip-hints-0.1.10.tar.gz
 tar --strip-components=2 -xf $f strip-hints-0.1.10/src/strip_hints
 python3 -c 'from strip_hints.a import uh; uh("copyparty")'
 
+# resolve symlinks
+find -type l |
+while IFS= read -r f1; do (
+	cd "${f1%/*}"
+	f1="./${f1##*/}"
+	f2="$(readlink "$f1")"
+	[ -e "$f2" ] || f2="../$f2"
+	[ -e "$f2" ] || {
+		echo could not resolve "$f1"
+		exit 1
+	}
+	rm "$f1"
+	cp -p "$f2" "$f1"
+); done
+
+# resolve symlinks on windows
+[ "$OSTYPE" = msys ] &&
+(cd ../..; git ls-files -s | awk '/^120000/{print$4}') |
+while IFS= read -r x; do
+	[ $(wc -l <"$x") -gt 1 ] && continue
+	(cd "${x%/*}"; cp -p "../$(cat "${x##*/}")" ${x##*/})
+done
+
+rm -rf contrib
+[ $fast ] && sed -ri s/5730/10/ copyparty/web/Makefile
+(cd copyparty/web && make -j$(nproc) && rm Makefile)
+
+# build
 ./setup.py clean2
 ./setup.py sdist bdist_wheel --universal
 

scripts/prep.sh

@@ -0,0 +1,43 @@
+#!/bin/bash
+set -e
+
+# general housekeeping before a release
+
+self=$(cd -- "$(dirname "$BASH_SOURCE")"; pwd -P)
+ver=$(awk '/^VERSION/{gsub(/[^0-9]/," ");printf "%d.%d.%d\n",$1,$2,$3}' copyparty/__version__.py)
+
+update_arch_pkgbuild() {
+    cd "$self/../contrib/package/arch"
+    rm -rf x
+    mkdir x
+
+    (echo "$self/../dist/copyparty-sfx.py"
+    awk -v self="$self" '
+        /^\)/{o=0}
+        /^source=/{o=1;next}
+        {
+            sub(/..pkgname./,"copyparty");
+            sub(/.*pkgver./,self "/..");
+            sub(/^ +"/,"");sub(/"/,"")
+        }
+        o&&!/https/' PKGBUILD
+    ) |
+    xargs sha256sum > x/sums
+
+    (awk -v ver=$ver '
+        /^pkgver=/{sub(/[0-9\.]+/,ver)};
+        /^sha256sums=/{exit};
+        1' PKGBUILD
+    echo -n 'sha256sums=('
+    p=; cat x/sums | while read s _; do
+        echo "$p\"$s\""
+        p='            '
+    done
+    awk '/^sha256sums=/{o=1} o&&/^\)/{o=2} o==2' PKGBUILD
+    ) >a
+    mv a PKGBUILD
+
+    rm -rf x
+}
+
+update_arch_pkgbuild

scripts/pyinstaller/build.sh

@@ -44,21 +44,33 @@ read a b c d _ < <(
 )
 sed -r 's/1,2,3,0/'$a,$b,$c,$d'/;s/1\.2\.3/'$a.$b.$c/ <loader.rc >loader.rc2
 
+excl=(
+    copyparty.broker_mp
+    copyparty.broker_mpw
+    ctypes.macholib
+    curses
+    inspect
+    multiprocessing
+    pdb
+    pickle
+    pyftpdlib.prefork
+    urllib.request
+    urllib.response
+    urllib.robotparser
+    zipfile
+)
+false || excl+=(
+    PIL
+    PIL.ExifTags
+    PIL.Image
+    PIL.ImageDraw
+    PIL.ImageOps
+)
+excl=( "${excl[@]/#/--exclude-module }" )
+
 $APPDATA/python/python37/scripts/pyinstaller \
     -y --clean -p mods --upx-dir=. \
-    --exclude-module copyparty.broker_mp \
-    --exclude-module copyparty.broker_mpw \
-    --exclude-module curses \
-    --exclude-module ctypes.macholib \
-    --exclude-module inspect \
-    --exclude-module multiprocessing \
-    --exclude-module pdb \
-    --exclude-module pickle \
-    --exclude-module pyftpdlib.prefork \
-    --exclude-module urllib.request \
-    --exclude-module urllib.response \
-    --exclude-module urllib.robotparser \
-    --exclude-module zipfile \
+    ${excl[*]} \
     --version-file loader.rc2 -i loader.ico -n copyparty -c -F loader.py \
     --add-data 'mods/copyparty/res;copyparty/res' \
     --add-data 'mods/copyparty/web;copyparty/web'

scripts/sfx.ls

@@ -11,6 +11,7 @@ copyparty/broker_mp.py,
 copyparty/broker_mpw.py,
 copyparty/broker_thr.py,
 copyparty/broker_util.py,
+copyparty/cfg.py,
 copyparty/dxml.py,
 copyparty/fsutil.py,
 copyparty/ftpd.py,

setup.py

@@ -29,12 +29,6 @@ with open(here + "/README.md", "rb") as f:
     txt = f.read().decode("utf-8")
     long_description = txt
 
-try:
-    cmd = "bash scripts/genlic.sh copyparty/res/COPYING.txt"
-    sp.Popen(cmd.split()).wait()
-except:
-    pass
-
 about = {}
 if not VERSION:
     with open(os.path.join(here, NAME, "__version__.py"), "rb") as f:
@@ -95,8 +89,6 @@ args = {
         "Development Status :: 5 - Production/Stable",
         "License :: OSI Approved :: MIT License",
         "Programming Language :: Python",
-        "Programming Language :: Python :: 2",
-        "Programming Language :: Python :: 2.7",
         "Programming Language :: Python :: 3",
         "Programming Language :: Python :: 3.3",
         "Programming Language :: Python :: 3.4",
@@ -146,4 +138,7 @@ args = {
     "cmdclass": {"clean2": clean2},
 }
 
+if sys.version_info < (3, 8):
+    args["install_requires"].append("ipaddress")
+
 setup(**args)

tests/util.py

@@ -98,7 +98,7 @@ class Cfg(Namespace):
     def __init__(self, a=None, v=None, c=None):
         ka = {}
 
-        ex = "daw dav_inf dav_mac e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod hardlink ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_thumb no_vthumb no_zip nw xdev xlink xvol"
+        ex = "daw dav_inf dav_mac dotsrch e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod hardlink ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_thumb no_vthumb no_zip nrand nw rand vc xdev xlink xvol"
         ka.update(**{k: False for k in ex.split()})
 
         ex = "dotpart no_rescan no_sendfile no_voldump plain_ip"