v1.13.1

* docker: improve config-not-found warning message
* readme: mention markdown variable expansion
* basic-browser: use zip=crc to support ie4 / win-3.11

.gitignore

@@ -12,6 +12,7 @@ copyparty.egg-info/
 /dist/
 /py2/
 /sfx*
+/pyz/
 /unt/
 /log/
 

README.md

@@ -10,13 +10,16 @@ turn almost any device into a file server with resumable uploads/downloads using
 
 📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer)
 
+🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)
+
 
 ## readme toc
 
 * top
     * [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
+        * [at home](#at-home) - make it accessible over the internet
         * [on servers](#on-servers) - you may also want these, especially on servers
-    * [features](#features)
+    * [features](#features) - also see [comparison to similar software](./docs/versus.md)
     * [testimonials](#testimonials) - small collection of user feedback
 * [motivations](#motivations) - project goals / philosophy
     * [notes](#notes) - general notes
@@ -37,12 +40,14 @@ turn almost any device into a file server with resumable uploads/downloads using
         * [file-search](#file-search) - dropping files into the browser also lets you see if they exist on the server
         * [unpost](#unpost) - undo/delete accidental uploads
         * [self-destruct](#self-destruct) - uploads can be given a lifetime
+        * [race the beam](#race-the-beam) - download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm))
     * [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
     * [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
     * [media player](#media-player) - plays almost every audio format there is
         * [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
         * [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
     * [markdown viewer](#markdown-viewer) - and there are *two* editors
+        * [markdown vars](#markdown-vars) - dynamic docs with serverside variable expansion
     * [other tricks](#other-tricks)
     * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config) - using arguments or config files, or a mix of both
@@ -56,6 +61,7 @@ turn almost any device into a file server with resumable uploads/downloads using
     * [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
     * [smb server](#smb-server) - unsafe, slow, not recommended for wan
     * [browser ux](#browser-ux) - tweaking the ui
+    * [opengraph](#opengraph) - discord and social-media embeds
     * [file indexing](#file-indexing) - enables dedup and music search ++
         * [exclude-patterns](#exclude-patterns) - to save some time
         * [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
@@ -104,8 +110,9 @@ turn almost any device into a file server with resumable uploads/downloads using
 * [dependencies](#dependencies) - mandatory deps
     * [optional dependencies](#optional-dependencies) - install these to enable bonus features
     * [optional gpl stuff](#optional-gpl-stuff)
-* [sfx](#sfx) - the self-contained "binary"
+* [sfx](#sfx) - the self-contained "binary" (recommended!)
     * [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
+    * [zipapp](#zipapp) - another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)
 * [install on android](#install-on-android)
 * [reporting bugs](#reporting-bugs) - ideas for context to include, and where to submit them
 * [devnotes](#devnotes) - for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
@@ -119,6 +126,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
 * or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
 * or install [on arch](#arch-package) ╱ [on NixOS](#nixos-module) ╱ [through nix](#nix-package)
 * or if you are on android, [install copyparty in termux](#install-on-android)
+* or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
 * or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
   * docker has all deps built-in, so skip this step:
 
@@ -126,7 +134,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
 
 * **Alpine:** `apk add py3-pillow ffmpeg`
 * **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
-* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg`
+* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
 * **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
 * **MacOS:** `port install py-Pillow ffmpeg`
 * **MacOS** (alternative): `brew install pillow ffmpeg`
@@ -147,6 +155,17 @@ some recommended options:
   * see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
 
 
+### at home
+
+make it accessible over the internet  by starting a [cloudflare quicktunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/) like so:
+
+first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) and then start the tunnel with `cloudflared tunnel --url http://127.0.0.1:3923`
+
+as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you
+
+since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
+
+
 ### on servers
 
 you may also want these, especially on servers:
@@ -170,6 +189,8 @@ firewall-cmd --reload
 
 ## features
 
+also see [comparison to similar software](./docs/versus.md)
+
 * backend stuff
   * ☑ IPv6
   * ☑ [multiprocessing](#performance) (actual multithreading)
@@ -192,6 +213,7 @@ firewall-cmd --reload
   * ☑ write-only folders
   * ☑ [unpost](#unpost): undo/delete accidental uploads
   * ☑ [self-destruct](#self-destruct) (specified server-side or client-side)
+  * ☑ [race the beam](#race-the-beam) (almost like peer-to-peer)
   * ☑ symlink/discard duplicates (content-matching)
 * download
   * ☑ single files in browser
@@ -216,9 +238,11 @@ firewall-cmd --reload
 * client support
   * ☑ [folder sync](#folder-sync)
   * ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
+  * ☑ [opengraph](#opengraph) (discord embeds)
 * markdown
   * ☑ [viewer](#markdown-viewer)
   * ☑ editor (sure why not)
+  * ☑ [variables](#markdown-vars)
 
 PS: something missing? post any crazy ideas you've got as a [feature request](https://github.com/9001/copyparty/issues/new?assignees=9001&labels=enhancement&template=feature_request.md) or [discussion](https://github.com/9001/copyparty/discussions/new?category=ideas) 🤙
 
@@ -596,9 +620,15 @@ cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3`
 
 ## uploading
 
-drag files/folders into the web-browser to upload  (or use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy))
+drag files/folders into the web-browser to upload
 
-this initiates an upload using `up2k`; there are two uploaders available:
+dragdrop is the recommended way, but you may also:
+
+* select some files (not folders) in your file explorer and press CTRL-V inside the browser window
+* use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy)
+* upload using [curl or sharex](#client-examples)
+
+when uploading files through dragdrop or CTRL-V, this initiates an upload using `up2k`; there are two browser-based uploaders available:
 * `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
 * `[🚀] up2k`, the good / fancy one
 
@@ -617,7 +647,7 @@ up2k has several advantages:
 > it is perfectly safe to restart / upgrade copyparty while someone is uploading to it!  
 > all known up2k clients will resume just fine 💪
 
-see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
+see [up2k](./docs/devnotes.md#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
 
 ![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
 
@@ -683,6 +713,13 @@ clients can specify a shorter expiration time using the [up2k ui](#uploading) --
 specifying a custom expiration time client-side will affect the timespan in which unposts are permitted, so keep an eye on the estimates in the up2k ui
 
 
+### race the beam
+
+download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm))  -- it's almost like peer-to-peer
+
+requires the file to be uploaded using up2k (which is the default drag-and-drop uploader), alternatively the command-line program
+
+
 ## file manager
 
 cut/paste, rename, and delete files/folders (if you have permission)
@@ -818,6 +855,13 @@ other notes,
 * the document preview has a max-width which is the same as an A4 paper when printed
 
 
+### markdown vars
+
+dynamic docs with serverside variable expansion  to replace stuff like `{{self.ip}}` with the client's IP, or `{{srv.htime}}` with the current time on the server
+
+see [./srv/expand/](./srv/expand/) for usage and examples
+
+
 ## other tricks
 
 * you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
@@ -1042,7 +1086,22 @@ tweaking the ui
   * to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Cirle,tags/.tn,tags/Artist,tags/Title,href`
   * to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`
 
-see [./docs/rice](./docs/rice) for more
+see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag
+
+
+## opengraph
+
+discord and social-media embeds
+
+can be enabled globally with `--og` or per-volume with volflag `og`
+
+note that this disables hotlinking because the opengraph spec demands it; to sneak past this intentional limitation, you can enable opengraph selectively by user-agent, for example `--og-ua '(Discord|Twitter|Slack)bot'` (or volflag `og_ua`)
+
+you can also hotlink files regardless by appending `?raw` to the url
+
+NOTE: because discord (and maybe others) strip query args such as `?raw` in opengraph tags, any links which require a filekey or dirkey will not work
+
+if you want to entirely replace the copyparty response with your own jinja2 template, give the template filepath to `--og-tpl` or volflag `og_tpl` (all members of `HttpCli` are available through the `this` object)
 
 
 ## file indexing
@@ -1956,7 +2015,7 @@ these are standalone programs and will never be imported / evaluated by copypart
 
 # sfx
 
-the self-contained "binary"  [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
+the self-contained "binary" (recommended!)  [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
 
 you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
 
@@ -1983,6 +2042,16 @@ meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/d
 then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) every once ina while if you can afford the size
 
 
+## zipapp
+
+another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)  has less features, requires python 3.7 or newer, worse compression, and more importantly is unable to benefit from more recent versions of jinja2 and such (which makes it less secure)... lots of drawbacks with this one really -- but it *may* just work if the regular sfx fails to start because the computer is messed up in certain funky ways, so it's worth a shot if all else fails
+
+run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails
+
+it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
+* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
+
+
 # install on android
 
 install [Termux](https://termux.com/) + its companion app `Termux:API` (see [ocv.me/termux](https://ocv.me/termux/)) and then copy-paste this into Termux (long-tap) all at once:

bin/mtag/install-deps.sh

@@ -231,7 +231,7 @@ install_vamp() {
 	cd "$td"
 	echo '#include <vamp-sdk/Plugin.h>' | g++ -x c++ -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
 		printf '\033[33mcould not find the vamp-sdk, building from source\033[0m\n'
-		(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/2691/vamp-plugin-sdk-2.10.0.tar.gz)
+		(dl_files yolo https://ocv.me/mirror/vamp-plugin-sdk-2.10.0.tar.gz)
 		sha512sum -c <(
 			echo "153b7f2fa01b77c65ad393ca0689742d66421017fd5931d216caa0fcf6909355fff74706fabbc062a3a04588a619c9b515a1dae00f21a57afd97902a355c48ed  -"
 		) <vamp-plugin-sdk-2.10.0.tar.gz
@@ -247,7 +247,7 @@ install_vamp() {
 	cd "$td"
 	have_beatroot || {
 		printf '\033[33mcould not find the vamp beatroot plugin, building from source\033[0m\n'
-		(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/885/beatroot-vamp-v1.0.tar.gz)
+		(dl_files yolo https://ocv.me/mirror/beatroot-vamp-v1.0.tar.gz)
 		sha512sum -c <(
 			echo "1f444d1d58ccf565c0adfe99f1a1aa62789e19f5071e46857e2adfbc9d453037bc1c4dcb039b02c16240e9b97f444aaff3afb625c86aa2470233e711f55b6874  -"
 		) <beatroot-vamp-v1.0.tar.gz

bin/u2c.py

@@ -1,8 +1,8 @@
 #!/usr/bin/env python3
 from __future__ import print_function, unicode_literals
 
-S_VERSION = "1.15"
+S_VERSION = "1.16"
-S_BUILD_DT = "2024-02-18"
+S_BUILD_DT = "2024-04-20"
 
 """
 u2c.py: upload to copyparty
@@ -563,7 +563,7 @@ def handshake(ar, file, search):
     else:
         if ar.touch:
             req["umod"] = True
-        if ar.dr:
+        if ar.ow:
             req["replace"] = True
 
     headers = {"Content-Type": "text/plain"}  # <=1.5.1 compat
@@ -1140,6 +1140,7 @@ source file/folder selection uses rsync syntax, meaning that:
     ap.add_argument("-x", type=unicode, metavar="REGEX", default="", help="skip file if filesystem-abspath matches REGEX, example: '.*/\\.hist/.*'")
     ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
     ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
+    ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
     ap.add_argument("--version", action="store_true", help="show version and exit")
 
     ap = app.add_argument_group("compatibility")
@@ -1148,7 +1149,7 @@ source file/folder selection uses rsync syntax, meaning that:
 
     ap = app.add_argument_group("folder sync")
     ap.add_argument("--dl", action="store_true", help="delete local files after uploading")
-    ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally")
+    ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally (implies --ow)")
     ap.add_argument("--drd", action="store_true", help="delete remote files during upload instead of afterwards; reduces peak disk space usage, but will reupload instead of detecting renames")
 
     ap = app.add_argument_group("performance tweaks")
@@ -1178,6 +1179,9 @@ source file/folder selection uses rsync syntax, meaning that:
     if ar.drd:
         ar.dr = True
 
+    if ar.dr:
+        ar.ow = True
+
     for k in "dl dr drd".split():
         errs = []
         if ar.safe and getattr(ar, k):

contrib/package/arch/PKGBUILD

@@ -1,6 +1,6 @@
 # Maintainer: icxes <dev.null@need.moe>
 pkgname=copyparty
-pkgver="1.12.0"
+pkgver="1.13.0"
 pkgrel=1
 pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
 arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
 )
 source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
 backup=("etc/${pkgname}.d/init" )
-sha256sums=("20d06469924b77f80f104d75eaebbb6ffa8aa4e5701b78f7264c4ce435768e5e")
+sha256sums=("9ad9ea5d4bdf947ed39f4ae571219d3528794c2ec6d4f470a5f3737899787e03")
 
 build() {
     cd "${srcdir}/${pkgname}-${pkgver}"

contrib/package/nix/copyparty/pin.json

@@ -1,5 +1,5 @@
 {
-    "url": "https://github.com/9001/copyparty/releases/download/v1.12.0/copyparty-sfx.py",
+    "url": "https://github.com/9001/copyparty/releases/download/v1.13.0/copyparty-sfx.py",
-    "version": "1.12.0",
+    "version": "1.13.0",
-    "hash": "sha256-Y18pCX/U0POgEH+wcs8Sf7c4lSUYu6bCtKGzIXa/X0M="
+    "hash": "sha256-3/ttK5BXAA5F1MztahiQp32QSSZqhuh3V6xb12T/6nM="
 }

copyparty/__main__.py

@@ -856,9 +856,11 @@ def add_qr(ap, tty):
 
 def add_fs(ap):
     ap2 = ap.add_argument_group("filesystem options")
-    rm_re_def = "5/0.1" if ANYWIN else "0/0"
+    rm_re_def = "15/0.1" if ANYWIN else "0/0"
     ap2.add_argument("--rm-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be deleted because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=rm_retry)")
+    ap2.add_argument("--mv-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be renamed because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=mv_retry)")
     ap2.add_argument("--iobuf", metavar="BYTES", type=int, default=256*1024, help="file I/O buffer-size; if your volumes are on a network drive, try increasing to \033[32m524288\033[0m or even \033[32m4194304\033[0m (and let me know if that improves your performance)")
+    ap2.add_argument("--mtab-age", metavar="SEC", type=int, default=60, help="rebuild mountpoint cache every \033[33mSEC\033[0m to keep track of sparse-files support; keep low on servers with removable media")
 
 
 def add_upload(ap):
@@ -949,6 +951,8 @@ def add_auth(ap):
     ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
     ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
     ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
+    ap2.add_argument("--no-bauth", action="store_true", help="disable basic-authentication support; do not accept passwords from the 'Authenticate' header at all. NOTE: This breaks support for the android app")
+    ap2.add_argument("--bauth-last", action="store_true", help="keeps basic-authentication enabled, but only as a last-resort; if a cookie is also provided then the cookie wins")
 
 
 def add_zeroconf(ap):
@@ -1088,6 +1092,8 @@ def add_optouts(ap):
     ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
     ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
     ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
+    ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
+    ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
 
 
 def add_safety(ap):
@@ -1213,7 +1219,7 @@ def add_db_general(ap, hcores):
     ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
     ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
     ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
-    ap2.add_argument("--re-dhash", action="store_true", help="rebuild the cache if it gets out of sync (for example crash on startup during metadata scanning)")
+    ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
     ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice -- only useful for offloading uploads to a cloud service or something (volflag=noforget)")
     ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see \033[33m--help-dbd\033[0m (volflag=dbd)")
     ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
@@ -1251,19 +1257,38 @@ def add_txt(ap):
     ap2.add_argument("--exp-lg", metavar="V,V,V", type=u, default=DEF_EXP, help="comma/space-separated list of placeholders to expand in prologue/epilogue files (volflag=exp_lg)")
 
 
+def add_og(ap):
+    ap2 = ap.add_argument_group('og / open graph / discord-embed options')
+    ap2.add_argument("--og", action="store_true", help="disable hotlinking and return an html document instead; this is required by open-graph, but can also be useful on its own (volflag=og)")
+    ap2.add_argument("--og-ua", metavar="RE", type=u, default="", help="only disable hotlinking / engage OG behavior if the useragent matches regex \033[33mRE\033[0m (volflag=og_ua)")
+    ap2.add_argument("--og-tpl", metavar="PATH", type=u, default="", help="do not return the regular copyparty html, but instead load the jinja2 template at \033[33mPATH\033[0m (if path contains 'EXT' then EXT will be replaced with the requested file's extension) (volflag=og_tpl)")
+    ap2.add_argument("--og-no-head", action="store_true", help="do not automatically add OG entries into <head> (useful if you're doing this yourself in a template or such) (volflag=og_no_head)")
+    ap2.add_argument("--og-th", metavar="FMT", type=u, default="jf3", help="thumbnail format; j=jpeg, jf=jpeg-uncropped, jf3=jpeg-uncropped-large, w=webm, ... (volflag=og_th)")
+    ap2.add_argument("--og-title", metavar="TXT", type=u, default="", help="fallback title if there is nothing in the \033[33m-e2t\033[0m database (volflag=og_title)")
+    ap2.add_argument("--og-title-a", metavar="T", type=u, default="🎵 {{ artist }} - {{ title }}", help="audio title format; takes any metadata key (volflag=og_title_a)")
+    ap2.add_argument("--og-title-v", metavar="T", type=u, default="{{ title }}", help="video title format; takes any metadata key (volflag=og_title_v)")
+    ap2.add_argument("--og-title-i", metavar="T", type=u, default="{{ title }}", help="image title format; takes any metadata key (volflag=og_title_i)")
+    ap2.add_argument("--og-s-title", action="store_true", help="force default title; do not read from tags (volflag=og_s_title)")
+    ap2.add_argument("--og-desc", metavar="TXT", type=u, default="", help="description text; same for all files, disable with [\033[32m-\033[0m] (volflag=og_desc)")
+    ap2.add_argument("--og-site", metavar="TXT", type=u, default="", help="sitename; defaults to \033[33m--name\033[0m, disable with [\033[32m-\033[0m] (volflag=og_site)")
+    ap2.add_argument("--tcolor", metavar="RGB", type=u, default="333", help="accent color (3 or 6 hex digits); may also affect safari and/or android-chrome (volflag=tcolor)")
+    ap2.add_argument("--uqe", action="store_true", help="query-string parceling; translate a request for \033[33m/foo/.uqe/BASE64\033[0m into \033[33m/foo?TEXT\033[0m, or \033[33m/foo/?TEXT\033[0m if the first character in \033[33mTEXT\033[0m is a slash. Automatically enabled for \033[33m--og\033[0m")
+
+
 def add_ui(ap, retry):
     ap2 = ap.add_argument_group('ui options')
     ap2.add_argument("--grid", action="store_true", help="show grid/thumbnails by default (volflag=grid)")
     ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language; one of the following: \033[32meng nor\033[0m")
     ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use (0..7)")
     ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
+    ap2.add_argument("--au-vol", metavar="0-100", type=int, default=50, choices=range(0, 101), help="default audio/video volume percent")
     ap2.add_argument("--sort", metavar="C,C,C", type=u, default="href", help="default sort order, comma-separated column IDs (see header tooltips), prefix with '-' for descending. Examples: \033[32mhref -href ext sz ts tags/Album tags/.tn\033[0m (volflag=sort)")
     ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
     ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
     ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
     ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
-    ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
+    ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages; can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template (volflag=html_head)")
     ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
     ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
     ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
@@ -1351,6 +1376,7 @@ def run_argparse(
     add_hooks(ap)
     add_stats(ap)
     add_txt(ap)
+    add_og(ap)
     add_ui(ap, retry)
     add_admin(ap)
     add_logging(ap)
@@ -1385,12 +1411,16 @@ def run_argparse(
     return ret
 
 
-def main(argv: Optional[list[str]] = None) -> None:
+def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
     time.strptime("19970815", "%Y%m%d")  # python#7980
     if WINDOWS:
         os.system("rem")  # enables colors
 
     init_E(E)
+
+    if rsrc:  # pyz
+        E.mod = rsrc
+
     if argv is None:
         argv = sys.argv
 

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 12, 1)
+VERSION = (1, 13, 1)
-CODENAME = "locksmith"
+CODENAME = "race the beam"
-BUILD_DT = (2024, 4, 9)
+BUILD_DT = (2024, 5, 6)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -1442,6 +1442,7 @@ class AuthSrv(object):
         elif "" not in mount:
             # there's volumes but no root; make root inaccessible
             vfs = VFS(self.log_func, "", "", AXS(), {})
+            vfs.flags["tcolor"] = self.args.tcolor
             vfs.flags["d2d"] = True
 
         maxdepth = 0
@@ -1727,7 +1728,11 @@ class AuthSrv(object):
             if self.args.e2d or "e2ds" in vol.flags:
                 vol.flags["e2d"] = True
 
-            for ga, vf in [["no_hash", "nohash"], ["no_idx", "noidx"]]:
+            for ga, vf in [
+                ["no_hash", "nohash"],
+                ["no_idx", "noidx"],
+                ["og_ua", "og_ua"],
+            ]:
                 if vf in vol.flags:
                     ptn = re.compile(vol.flags.pop(vf))
                 else:
@@ -1764,13 +1769,21 @@ class AuthSrv(object):
                 if k in vol.flags:
                     vol.flags[k] = float(vol.flags[k])
 
-            try:
+            for k in ("mv_re", "rm_re"):
-                zs1, zs2 = vol.flags["rm_retry"].split("/")
+                try:
-                vol.flags["rm_re_t"] = float(zs1)
+                    zs1, zs2 = vol.flags[k + "try"].split("/")
-                vol.flags["rm_re_r"] = float(zs2)
+                    vol.flags[k + "_t"] = float(zs1)
-            except:
+                    vol.flags[k + "_r"] = float(zs2)
-                t = 'volume "/%s" has invalid rm_retry [%s]'
+                except:
-                raise Exception(t % (vol.vpath, vol.flags.get("rm_retry")))
+                    t = 'volume "/%s" has invalid %stry [%s]'
+                    raise Exception(t % (vol.vpath, k, vol.flags.get(k + "try")))
+
+            if vol.flags.get("og"):
+                self.args.uqe = True
+
+            zs = str(vol.flags.get("tcolor", "")).lstrip("#")
+            if len(zs) == 3:  # fc5 => ffcc55
+                vol.flags["tcolor"] = "".join([x * 2 for x in zs])
 
             for k1, k2 in IMPLICATIONS:
                 if k1 in vol.flags:
@@ -2409,7 +2422,7 @@ def expand_config_file(
         if not cnames:
             t = "warning: tried to read config-files from folder '%s' but it does not contain any "
             if names:
-                t += ".conf files; the following files were ignored: %s"
+                t += ".conf files; the following files/subfolders were ignored: %s"
                 t = t % (fp, ", ".join(names[:8]))
             else:
                 t += "files at all"

copyparty/cert.py

@@ -6,7 +6,8 @@ import os
 import shutil
 import time
 
-from .util import Netdev, runcmd
+from .__init__ import ANYWIN
+from .util import Netdev, runcmd, wrename, wunlink
 
 HAVE_CFSSL = True
 
@@ -14,6 +15,12 @@ if True:  # pylint: disable=using-constant-test
     from .util import RootLogger
 
 
+if ANYWIN:
+    VF = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
+else:
+    VF = {"mv_re_t": 0, "rm_re_t": 0}
+
+
 def ensure_cert(log: "RootLogger", args) -> None:
     """
     the default cert (and the entire TLS support) is only here to enable the
@@ -105,8 +112,12 @@ def _gen_ca(log: "RootLogger", args):
         raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
 
     bname = os.path.join(args.crt_dir, "ca")
-    os.rename(bname + "-key.pem", bname + ".key")
+    try:
-    os.unlink(bname + ".csr")
+        wunlink(log, bname + ".key", VF)
+    except:
+        pass
+    wrename(log, bname + "-key.pem", bname + ".key", VF)
+    wunlink(log, bname + ".csr", VF)
 
     log("cert", "new ca OK", 2)
 
@@ -185,11 +196,11 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
 
     bname = os.path.join(args.crt_dir, "srv")
     try:
-        os.unlink(bname + ".key")
+        wunlink(log, bname + ".key", VF)
     except:
         pass
-    os.rename(bname + "-key.pem", bname + ".key")
+    wrename(log, bname + "-key.pem", bname + ".key", VF)
-    os.unlink(bname + ".csr")
+    wunlink(log, bname + ".csr", VF)
 
     with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:
         ca = f.read()

copyparty/cfg.py

@@ -16,6 +16,7 @@ def vf_bmap() -> dict[str, str]:
         "no_dedup": "copydupes",
         "no_dupe": "nodupe",
         "no_forget": "noforget",
+        "no_pipe": "nopipe",
         "no_robots": "norobots",
         "no_thumb": "dthumb",
         "no_vthumb": "dvthumb",
@@ -38,6 +39,9 @@ def vf_bmap() -> dict[str, str]:
         "magic",
         "no_sb_md",
         "no_sb_lg",
+        "og",
+        "og_no_head",
+        "og_s_title",
         "rand",
         "xdev",
         "xlink",
@@ -60,11 +64,23 @@ def vf_vmap() -> dict[str, str]:
     }
     for k in (
         "dbd",
+        "html_head",
         "lg_sbf",
         "md_sbf",
         "nrand",
+        "og_desc",
+        "og_site",
+        "og_th",
+        "og_title",
+        "og_title_a",
+        "og_title_v",
+        "og_title_i",
+        "og_tpl",
+        "og_ua",
+        "mv_retry",
         "rm_retry",
         "sort",
+        "tcolor",
         "unlist",
         "u2abort",
         "u2ts",
@@ -79,7 +95,6 @@ def vf_cmap() -> dict[str, str]:
     for k in (
         "exp_lg",
         "exp_md",
-        "html_head",
         "mte",
         "mth",
         "mtp",
@@ -199,7 +214,7 @@ flagcats = {
         "grid": "show grid/thumbnails by default",
         "sort": "default sort order",
         "unlist": "dont list files matching REGEX",
-        "html_head=TXT": "includes TXT in the <head>",
+        "html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
         "robots": "allows indexing by search engines (default)",
         "norobots": "kindly asks search engines to leave",
         "no_sb_md": "disable js sandbox for markdown files",
@@ -214,6 +229,7 @@ flagcats = {
         "dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
         "fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
         "fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
+        "mv_retry": "ms-windows: timeout for renaming busy files",
         "rm_retry": "ms-windows: timeout for deleting busy files",
         "davauth": "ask webdav clients to login for all folders",
         "davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",

copyparty/fsutil.py

@@ -1,6 +1,7 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
 
+import argparse
 import os
 import re
 import time
@@ -17,20 +18,26 @@ if True:  # pylint: disable=using-constant-test
 
 
 class Fstab(object):
-    def __init__(self, log: "RootLogger"):
+    def __init__(self, log: "RootLogger", args: argparse.Namespace):
         self.log_func = log
 
+        self.warned = False
         self.trusted = False
         self.tab: Optional[VFS] = None
+        self.oldtab: Optional[VFS] = None
+        self.srctab = "a"
         self.cache: dict[str, str] = {}
         self.age = 0.0
+        self.maxage = args.mtab_age
 
     def log(self, msg: str, c: Union[int, str] = 0) -> None:
         self.log_func("fstab", msg, c)
 
     def get(self, path: str) -> str:
-        if len(self.cache) > 9000:
+        now = time.time()
-            self.age = time.time()
+        if now - self.age > self.maxage or len(self.cache) > 9000:
+            self.age = now
+            self.oldtab = self.tab or self.oldtab
             self.tab = None
             self.cache = {}
 
@@ -75,7 +82,7 @@ class Fstab(object):
         self.trusted = False
 
     def build_tab(self) -> None:
-        self.log("building tab")
+        self.log("inspecting mtab for changes")
 
         sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
         if MACOS:
@@ -84,6 +91,7 @@ class Fstab(object):
         ptn = re.compile(sptn)
         so, _ = chkcmd(["mount"])
         tab1: list[tuple[str, str]] = []
+        atab = []
         for ln in so.split("\n"):
             m = ptn.match(ln)
             if not m:
@@ -91,6 +99,15 @@ class Fstab(object):
 
             zs1, zs2 = m.groups()
             tab1.append((str(zs1), str(zs2)))
+            atab.append(ln)
+
+        # keep empirically-correct values if mounttab unchanged
+        srctab = "\n".join(sorted(atab))
+        if srctab == self.srctab:
+            self.tab = self.oldtab
+            return
+
+        self.log("mtab has changed; reevaluating support for sparse files")
 
         tab1.sort(key=lambda x: (len(x[0]), x[0]))
         path1, fs1 = tab1[0]
@@ -99,6 +116,7 @@ class Fstab(object):
             tab.add(fs, path.lstrip("/"))
 
         self.tab = tab
+        self.srctab = srctab
 
     def relabel(self, path: str, nval: str) -> None:
         assert self.tab
@@ -133,7 +151,9 @@ class Fstab(object):
                 self.trusted = True
             except:
                 # prisonparty or other restrictive environment
-                self.log("failed to build tab:\n{}".format(min_ex()), 3)
+                if not self.warned:
+                    self.warned = True
+                    self.log("failed to build tab:\n{}".format(min_ex()), 3)
                 self.build_fallback()
 
         assert self.tab

copyparty/httpcli.py

@@ -36,6 +36,7 @@ from .bos import bos
 from .star import StreamTar
 from .sutil import StreamArc, gfilter
 from .szip import StreamZip
+from .up2k import up2k_chunksize
 from .util import unquote  # type: ignore
 from .util import (
     APPLESAN_RE,
@@ -83,12 +84,16 @@ from .util import (
     sanitize_vpath,
     sendfile_kern,
     sendfile_py,
+    ub64dec,
+    ub64enc,
+    ujoin,
     undot,
     unescape_cookie,
     unquotep,
     vjoin,
     vol_san,
     vsplit,
+    wrename,
     wunlink,
     yieldfile,
 )
@@ -126,6 +131,7 @@ class HttpCli(object):
         self.ico = conn.ico  # mypy404
         self.thumbcli = conn.thumbcli  # mypy404
         self.u2fh = conn.u2fh  # mypy404
+        self.pipes = conn.pipes  # mypy404
         self.log_func = conn.log_func  # mypy404
         self.log_src = conn.log_src  # mypy404
         self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
@@ -214,6 +220,13 @@ class HttpCli(object):
         ka["favico"] = self.args.favico
         ka["s_name"] = self.args.bname
         ka["s_doctitle"] = self.args.doctitle
+        ka["tcolor"] = self.vn.flags["tcolor"]
+
+        zso = self.vn.flags.get("html_head")
+        if zso:
+            ka["this"] = self
+            self._build_html_head(zso, ka)
+
         ka["html_head"] = self.html_head
         return tpl.render(**ka)  # type: ignore
 
@@ -361,6 +374,21 @@ class HttpCli(object):
         if "&" in self.req and "?" not in self.req:
             self.hint = "did you mean '?' instead of '&'"
 
+        if self.args.uqe and "/.uqe/" in self.req:
+            try:
+                vpath, query = self.req.split("?")[0].split("/.uqe/")
+                query = query.split("/")[0]  # discard trailing junk
+                # (usually a "filename" to trick discord into behaving)
+                query = ub64dec(query.encode("utf-8")).decode("utf-8", "replace")
+                if query.startswith("/"):
+                    self.req = "%s/?%s" % (vpath, query[1:])
+                else:
+                    self.req = "%s?%s" % (vpath, query)
+            except Exception as ex:
+                t = "bad uqe in request [%s]: %r" % (self.req, ex)
+                self.loud_reply(t, status=400)
+                return False
+
         # split req into vpath + uparam
         uparam = {}
         if "?" not in self.req:
@@ -427,7 +455,8 @@ class HttpCli(object):
             cookie_pw = ""
 
         if len(uparam) > 10 or len(cookies) > 50:
-            raise Pebkac(400, "u wot m8")
+            self.loud_reply("u wot m8", status=400)
+            return False
 
         self.uparam = uparam
         self.cookies = cookies
@@ -443,7 +472,11 @@ class HttpCli(object):
 
         zso = self.headers.get("authorization")
         bauth = ""
-        if zso:
+        if (
+            zso
+            and not self.args.no_bauth
+            and (not cookie_pw or not self.args.bauth_last)
+        ):
             try:
                 zb = zso.split(" ")[1].encode("ascii")
                 zs = base64.b64decode(zb).decode("utf-8")
@@ -711,6 +744,31 @@ class HttpCli(object):
             or ("; Trident/" in self.ua and not k304)
         )
 
+    def _build_html_head(self, maybe_html: Any, kv: dict[str, Any]) -> bool:
+        html = str(maybe_html)
+        is_jinja = html[:2] in "%@%"
+        if is_jinja:
+            html = html.replace("%", "", 1)
+
+        if html.startswith("@"):
+            with open(html[1:], "rb") as f:
+                html = f.read().decode("utf-8")
+
+        if html.startswith("%"):
+            html = html[1:]
+            is_jinja = True
+
+        if is_jinja:
+            print("applying jinja")
+            with self.conn.hsrv.mutex:
+                if html not in self.conn.hsrv.j2:
+                    j2env = jinja2.Environment()
+                    tpl = j2env.from_string(html)
+                    self.conn.hsrv.j2[html] = tpl
+                html = self.conn.hsrv.j2[html].render(**kv)
+
+        self.html_head += html + "\n"
+
     def send_headers(
         self,
         length: Optional[int],
@@ -1800,7 +1858,7 @@ class HttpCli(object):
                     f, fn = zfw["orz"]
 
                 path2 = os.path.join(fdir, fn2)
-                atomic_move(path, path2)
+                atomic_move(self.log, path, path2, vfs.flags)
                 fn = fn2
                 path = path2
 
@@ -1881,7 +1939,9 @@ class HttpCli(object):
         self.reply(t.encode("utf-8"), 201, headers=h)
         return True
 
-    def bakflip(self, f: typing.BinaryIO, ofs: int, sz: int, sha: str) -> None:
+    def bakflip(
+        self, f: typing.BinaryIO, ofs: int, sz: int, sha: str, flags: dict[str, Any]
+    ) -> None:
         if not self.args.bak_flips or self.args.nw:
             return
 
@@ -1909,7 +1969,7 @@ class HttpCli(object):
 
         if nrem:
             self.log("bakflip truncated; {} remains".format(nrem), 1)
-            atomic_move(fp, fp + ".trunc")
+            atomic_move(self.log, fp, fp + ".trunc", flags)
         else:
             self.log("bakflip ok", 2)
 
@@ -2175,7 +2235,7 @@ class HttpCli(object):
 
                 if sha_b64 != chash:
                     try:
-                        self.bakflip(f, cstart[0], post_sz, sha_b64)
+                        self.bakflip(f, cstart[0], post_sz, sha_b64, vfs.flags)
                     except:
                         self.log("bakflip failed: " + min_ex())
 
@@ -2242,6 +2302,10 @@ class HttpCli(object):
     def handle_login(self) -> bool:
         assert self.parser
         pwd = self.parser.require("cppwd", 64)
+        try:
+            uhash = self.parser.require("uhash", 256)
+        except:
+            uhash = ""
         self.parser.drop()
 
         self.out_headerlist = [
@@ -2254,6 +2318,11 @@ class HttpCli(object):
 
         dst += self.ourlq()
 
+        uhash = uhash.lstrip("#")
+        if uhash not in ("", "-"):
+            dst += "&" if "?" in dst else "?"
+            dst += "_=1#" + html_escape(uhash, True, True)
+
         msg = self.get_pwd_cookie(pwd)
         html = self.j2s("msg", h1=msg, h2='<a href="' + dst + '">ack</a>', redir=dst)
         self.reply(html.encode("utf-8"))
@@ -2527,7 +2596,7 @@ class HttpCli(object):
                             raise
 
                     if not nullwrite:
-                        atomic_move(tabspath, abspath)
+                        atomic_move(self.log, tabspath, abspath, vfs.flags)
 
                     tabspath = ""
 
@@ -2767,7 +2836,7 @@ class HttpCli(object):
                 hidedir(dp)
             except:
                 pass
-            bos.rename(fp, os.path.join(mdir, ".hist", mfile2))
+            wrename(self.log, fp, os.path.join(mdir, ".hist", mfile2), vfs.flags)
 
         assert self.parser.gen
         p_field, _, p_data = next(self.parser.gen)
@@ -2922,17 +2991,42 @@ class HttpCli(object):
 
         return txt
 
-    def tx_file(self, req_path: str) -> bool:
+    def tx_file(self, req_path: str, ptop: Optional[str] = None) -> bool:
         status = 200
         logmsg = "{:4} {} ".format("", self.req)
         logtail = ""
 
+        if ptop is not None:
+            try:
+                dp, fn = os.path.split(req_path)
+                tnam = fn + ".PARTIAL"
+                if self.args.dotpart:
+                    tnam = "." + tnam
+                ap_data = os.path.join(dp, tnam)
+                st_data = bos.stat(ap_data)
+                if not st_data.st_size:
+                    raise Exception("partial is empty")
+                x = self.conn.hsrv.broker.ask("up2k.find_job_by_ap", ptop, req_path)
+                job = json.loads(x.get())
+                if not job:
+                    raise Exception("not found in registry")
+                self.pipes.set(req_path, job)
+            except Exception as ex:
+                self.log("will not pipe [%s]; %s" % (ap_data, ex), 6)
+                ptop = None
+
         #
         # if request is for foo.js, check if we have foo.js.gz
 
         file_ts = 0.0
         editions: dict[str, tuple[str, int]] = {}
         for ext in ("", ".gz"):
+            if ptop is not None:
+                sz = job["size"]
+                file_ts = job["lmod"]
+                editions["plain"] = (ap_data, sz)
+                break
+
             try:
                 fs_path = req_path + ext
                 st = bos.stat(fs_path)
@@ -3089,6 +3183,11 @@ class HttpCli(object):
             self.send_headers(length=upper - lower, status=status, mime=mime)
             return True
 
+        if ptop is not None:
+            return self.tx_pipe(
+                ptop, req_path, ap_data, job, lower, upper, status, mime, logmsg
+            )
+
         ret = True
         with open_func(*open_args) as f:
             self.send_headers(length=upper - lower, status=status, mime=mime)
@@ -3108,6 +3207,143 @@ class HttpCli(object):
 
         return ret
 
+    def tx_pipe(
+        self,
+        ptop: str,
+        req_path: str,
+        ap_data: str,
+        job: dict[str, Any],
+        lower: int,
+        upper: int,
+        status: int,
+        mime: str,
+        logmsg: str,
+    ) -> bool:
+        M = 1048576
+        self.send_headers(length=upper - lower, status=status, mime=mime)
+        wr_slp = self.args.s_wr_slp
+        wr_sz = self.args.s_wr_sz
+        file_size = job["size"]
+        chunk_size = up2k_chunksize(file_size)
+        num_need = -1
+        data_end = 0
+        remains = upper - lower
+        broken = False
+        spins = 0
+        tier = 0
+        tiers = ["uncapped", "reduced speed", "one byte per sec"]
+
+        while lower < upper and not broken:
+            with self.u2mutex:
+                job = self.pipes.get(req_path)
+                if not job:
+                    x = self.conn.hsrv.broker.ask("up2k.find_job_by_ap", ptop, req_path)
+                    job = json.loads(x.get())
+                    if job:
+                        self.pipes.set(req_path, job)
+
+            if not job:
+                t = "pipe: OK, upload has finished; yeeting remainder"
+                self.log(t, 2)
+                data_end = file_size
+                break
+
+            if num_need != len(job["need"]) and data_end - lower < 8 * M:
+                num_need = len(job["need"])
+                data_end = 0
+                for cid in job["hash"]:
+                    if cid in job["need"]:
+                        break
+                    data_end += chunk_size
+                t = "pipe: can stream %.2f MiB; requested range is %.2f to %.2f"
+                self.log(t % (data_end / M, lower / M, upper / M), 6)
+                with self.u2mutex:
+                    if data_end > self.u2fh.aps.get(ap_data, data_end):
+                        try:
+                            fhs = self.u2fh.cache[ap_data].all_fhs
+                            for fh in fhs:
+                                fh.flush()
+                            self.u2fh.aps[ap_data] = data_end
+                            self.log("pipe: flushed %d up2k-FDs" % (len(fhs),))
+                        except Exception as ex:
+                            self.log("pipe: u2fh flush failed: %r" % (ex,))
+
+            if lower >= data_end:
+                if data_end:
+                    t = "pipe: uploader is too slow; aborting download at %.2f MiB"
+                    self.log(t % (data_end / M))
+                    raise Pebkac(416, "uploader is too slow")
+
+                raise Pebkac(416, "no data available yet; please retry in a bit")
+
+            slack = data_end - lower
+            if slack >= 8 * M:
+                ntier = 0
+                winsz = M
+                bufsz = wr_sz
+                slp = wr_slp
+            else:
+                winsz = max(40, int(M * (slack / (12 * M))))
+                base_rate = M if not wr_slp else wr_sz / wr_slp
+                if winsz > base_rate:
+                    ntier = 0
+                    bufsz = wr_sz
+                    slp = wr_slp
+                elif winsz > 300:
+                    ntier = 1
+                    bufsz = winsz // 5
+                    slp = 0.2
+                else:
+                    ntier = 2
+                    bufsz = winsz = slp = 1
+
+            if tier != ntier:
+                tier = ntier
+                self.log("moved to tier %d (%s)" % (tier, tiers[tier]))
+
+            try:
+                with open(ap_data, "rb", self.args.iobuf) as f:
+                    f.seek(lower)
+                    page = f.read(min(winsz, data_end - lower, upper - lower))
+                if not page:
+                    raise Exception("got 0 bytes (EOF?)")
+            except Exception as ex:
+                self.log("pipe: read failed at %.2f MiB: %s" % (lower / M, ex), 3)
+                with self.u2mutex:
+                    self.pipes.c.pop(req_path, None)
+                spins += 1
+                if spins > 3:
+                    raise Pebkac(500, "file became unreadable")
+                time.sleep(2)
+                continue
+
+            spins = 0
+            pofs = 0
+            while pofs < len(page):
+                if slp:
+                    time.sleep(slp)
+
+                try:
+                    buf = page[pofs : pofs + bufsz]
+                    self.s.sendall(buf)
+                    zi = len(buf)
+                    remains -= zi
+                    lower += zi
+                    pofs += zi
+                except:
+                    broken = True
+                    break
+
+        if lower < upper and not broken:
+            with open(req_path, "rb") as f:
+                remains = sendfile_py(self.log, lower, upper, f, self.s, wr_sz, wr_slp)
+
+        spd = self._spd((upper - lower) - remains)
+        if self.do_log:
+            self.log("{},  {}".format(logmsg, spd))
+
+        return not broken
+
     def tx_zip(
         self,
         fmt: str,
@@ -3308,7 +3544,6 @@ class HttpCli(object):
         targs = {
             "r": self.args.SR if self.is_vproxied else "",
             "ts": self.conn.hsrv.cachebuster(),
-            "html_head": self.html_head,
             "edit": "edit" in self.uparam,
             "title": html_escape(self.vpath, crlf=True),
             "lastmod": int(ts_md * 1000),
@@ -3319,6 +3554,13 @@ class HttpCli(object):
             "md": boundary,
             "arg_base": arg_base,
         }
+
+        zfv = self.vn.flags.get("html_head")
+        if zfv:
+            targs["this"] = self
+            self._build_html_head(zfv, targs)
+
+        targs["html_head"] = self.html_head
         zs = template.render(**targs).encode("utf-8", "replace")
         html = zs.split(boundary.encode("utf-8"))
         if len(html) != 2:
@@ -3434,8 +3676,6 @@ class HttpCli(object):
             self.reply(zb, mime="text/plain; charset=utf-8")
             return True
 
-        self.html_head += self.vn.flags.get("html_head", "")
-
         html = self.j2s(
             "splash",
             this=self,
@@ -3480,7 +3720,7 @@ class HttpCli(object):
         return True
 
     def set_cfg_reset(self) -> bool:
-        for k in ("k304", "js", "idxh", "cppwd", "cppws"):
+        for k in ("k304", "js", "idxh", "dots", "cppwd", "cppws"):
             cookie = gencookie(k, "x", self.args.R, False)
             self.out_headerlist.append(("Set-Cookie", cookie))
 
@@ -3691,7 +3931,7 @@ class HttpCli(object):
         allvols = [x for x in allvols if "e2d" in x.flags]
 
         for vol in allvols:
-            cur = idx.get_cur(vol.realpath)
+            cur = idx.get_cur(vol)
             if not cur:
                 continue
 
@@ -3745,7 +3985,7 @@ class HttpCli(object):
         if not allvols:
             ret = [{"kinshi": 1}]
 
-        jtxt = '{"u":%s,"c":%s}' % (uret, json.dumps(ret, indent=0))
+        jtxt = '{"u":%s,"c":%s}' % (uret, json.dumps(ret, separators=(",\n", ": ")))
         zi = len(uret.split('\n"pd":')) - 1
         self.log("%s #%d+%d %.2fsec" % (lm, zi, len(ret), time.time() - t0))
         self.reply(jtxt.encode("utf-8", "replace"), mime="application/json")
@@ -3906,7 +4146,17 @@ class HttpCli(object):
         e2d = "e2d" in vn.flags
         e2t = "e2t" in vn.flags
 
-        self.html_head += vn.flags.get("html_head", "")
+        add_og = "og" in vn.flags
+        if add_og:
+            if "th" in self.uparam or "raw" in self.uparam:
+                og_ua = add_og = False
+            elif self.args.og_ua:
+                og_ua = add_og = self.args.og_ua.search(self.ua)
+            else:
+                og_ua = False
+                add_og = True
+            og_fn = ""
+
         if "b" in self.uparam:
             self.out_headers["X-Robots-Tag"] = "noindex, nofollow"
 
@@ -3914,13 +4164,15 @@ class HttpCli(object):
         is_dk = False
         fk_pass = False
         icur = None
-        if is_dir and (e2t or e2d):
+        if (e2t or e2d) and (is_dir or add_og):
             idx = self.conn.get_u2idx()
             if idx and hasattr(idx, "p_end"):
-                icur = idx.get_cur(dbv.realpath)
+                icur = idx.get_cur(dbv)
 
         th_fmt = self.uparam.get("th")
-        if self.can_read or (self.can_get and vn.flags.get("dk")):
+        if self.can_read or (
+            self.can_get and (vn.flags.get("dk") or "fk" not in vn.flags)
+        ):
             if th_fmt is not None:
                 nothumb = "dthumb" in dbv.flags
                 if is_dir:
@@ -3967,7 +4219,7 @@ class HttpCli(object):
         elif self.can_write and th_fmt is not None:
             return self.tx_svg("upload\nonly")
 
-        elif self.can_get and self.avn:
+        if not self.can_read and self.can_get and self.avn:
             axs = self.avn.axs
             if self.uname not in axs.uhtml:
                 pass
@@ -4013,6 +4265,17 @@ class HttpCli(object):
                     self.log(t % (correct, got, self.req, abspath), 6)
                     return self.tx_404()
 
+            if add_og:
+                if og_ua or self.host not in self.headers.get("referer", ""):
+                    self.vpath, og_fn = vsplit(self.vpath)
+                    vpath = self.vpath
+                    vn, rem = self.asrv.vfs.get(self.vpath, self.uname, False, False)
+                    abspath = vn.dcanonical(rem)
+                    dbv, vrem = vn.get_dbv(rem)
+                    is_dir = stat.S_ISDIR(st.st_mode)
+                    is_dk = True
+                    vpnodes.pop()
+
             if (
                 (abspath.endswith(".md") or self.can_delete)
                 and "nohtml" not in vn.flags
@@ -4024,7 +4287,10 @@ class HttpCli(object):
             ):
                 return self.tx_md(vn, abspath)
 
-            return self.tx_file(abspath)
+            if not add_og or not og_fn:
+                return self.tx_file(
+                    abspath, None if st.st_size or "nopipe" in vn.flags else vn.realpath
+                )
 
         elif is_dir and not self.can_read:
             if self._use_dirkey(abspath):
@@ -4071,7 +4337,11 @@ class HttpCli(object):
         is_ls = "ls" in self.uparam
         is_js = self.args.force_js or self.cookies.get("js") == "y"
 
-        if not is_ls and (self.ua.startswith("curl/") or self.ua.startswith("fetch")):
+        if (
+            not is_ls
+            and not add_og
+            and (self.ua.startswith("curl/") or self.ua.startswith("fetch"))
+        ):
             self.uparam["ls"] = "v"
             is_ls = True
 
@@ -4145,6 +4415,7 @@ class HttpCli(object):
             "dsort": vf["sort"],
             "dcrop": vf["crop"],
             "dth3x": vf["th3x"],
+            "dvol": self.args.au_vol,
             "themes": self.args.themes,
             "turbolvl": self.args.turbo,
             "u2j": self.args.u2j,
@@ -4196,7 +4467,7 @@ class HttpCli(object):
 
         for k in ["zip", "tar"]:
             v = self.uparam.get(k)
-            if v is not None:
+            if v is not None and (not add_og or not og_fn):
                 return self.tx_zip(k, v, self.vpath, vn, rem, [])
 
         fsroot, vfs_ls, vfs_virt = vn.ls(
@@ -4210,6 +4481,10 @@ class HttpCli(object):
         ls_names = [x[0] for x in vfs_ls]
         ls_names.extend(list(vfs_virt.keys()))
 
+        if add_og and og_fn and not self.can_read:
+            ls_names = [og_fn]
+            is_js = True
+
         # check for old versions of files,
         # [num-backups, most-recent, hist-path]
         hist: dict[str, tuple[int, float, str]] = {}
@@ -4271,12 +4546,14 @@ class HttpCli(object):
                     margin = "DIR"
                 elif add_dk:
                     zs = absreal(fspath)
-                    margin = '<a href="%s?k=%s&zip" rel="nofollow">zip</a>' % (
+                    margin = '<a href="%s?k=%s&zip=crc" rel="nofollow">zip</a>' % (
                         quotep(href),
                         self.gen_fk(2, self.args.dk_salt, zs, 0, 0)[:add_dk],
                     )
                 else:
-                    margin = '<a href="%s?zip" rel="nofollow">zip</a>' % (quotep(href),)
+                    margin = '<a href="%s?zip=crc" rel="nofollow">zip</a>' % (
+                        quotep(href),
+                    )
             elif fn in hist:
                 margin = '<a href="%s.hist/%s">#%s</a>' % (
                     base,
@@ -4420,6 +4697,9 @@ class HttpCli(object):
         else:
             taglist = list(tagset)
 
+        if not files and not dirs and not readme and not logues[0] and not logues[1]:
+            logues[1] = "this folder is empty"
+
         if is_ls:
             ls_ret["dirs"] = dirs
             ls_ret["files"] = files
@@ -4471,6 +4751,148 @@ class HttpCli(object):
         if "mth" in vn.flags:
             j2a["def_hcols"] = list(vn.flags["mth"])
 
+        if add_og and "raw" not in self.uparam:
+            j2a["this"] = self
+            cgv["og_fn"] = og_fn
+            if og_fn and vn.flags.get("og_tpl"):
+                tpl = vn.flags["og_tpl"]
+                if "EXT" in tpl:
+                    zs = og_fn.split(".")[-1].lower()
+                    tpl2 = tpl.replace("EXT", zs)
+                    if os.path.exists(tpl2):
+                        tpl = tpl2
+                with self.conn.hsrv.mutex:
+                    if tpl not in self.conn.hsrv.j2:
+                        tdir, tname = os.path.split(tpl)
+                        j2env = jinja2.Environment()
+                        j2env.loader = jinja2.FileSystemLoader(tdir)
+                        self.conn.hsrv.j2[tpl] = j2env.get_template(tname)
+            thumb = ""
+            is_pic = is_vid = is_au = False
+            covernames = self.args.th_coversd
+            for fn in ls_names:
+                if fn.lower() in covernames:
+                    thumb = fn
+                    break
+            if og_fn:
+                ext = og_fn.split(".")[-1].lower()
+                if ext in self.thumbcli.thumbable:
+                    is_pic = (
+                        ext in self.thumbcli.fmt_pil
+                        or ext in self.thumbcli.fmt_vips
+                        or ext in self.thumbcli.fmt_ffi
+                    )
+                    is_vid = ext in self.thumbcli.fmt_ffv
+                    is_au = ext in self.thumbcli.fmt_ffa
+                    if not thumb or not is_au:
+                        thumb = og_fn
+                file = next((x for x in files if x["name"] == og_fn), None)
+            else:
+                file = None
+
+            url_base = "%s://%s/%s" % (
+                "https" if self.is_https else "http",
+                self.host,
+                self.args.RS + quotep(vpath),
+            )
+            j2a["og_is_pic"] = is_pic
+            j2a["og_is_vid"] = is_vid
+            j2a["og_is_au"] = is_au
+            if thumb:
+                fmt = vn.flags.get("og_th", "j")
+                th_base = ujoin(url_base, quotep(thumb))
+                query = "th=%s&cache" % (fmt,)
+                query = ub64enc(query.encode("utf-8")).decode("utf-8")
+                # discord looks at file extension, not content-type...
+                query += "/a.jpg" if "j" in fmt else "/a.webp"
+                j2a["og_thumb"] = "%s/.uqe/%s" % (th_base, query)
+
+            j2a["og_fn"] = og_fn
+            j2a["og_file"] = file
+            if og_fn:
+                og_fn_q = quotep(og_fn)
+                query = ub64enc(b"raw").decode("utf-8")
+                if "." in og_fn:
+                    query += "/a.%s" % (og_fn.split(".")[-1])
+
+                j2a["og_url"] = ujoin(url_base, og_fn_q)
+                j2a["og_raw"] = j2a["og_url"] + "/.uqe/" + query
+            else:
+                j2a["og_url"] = j2a["og_raw"] = url_base
+
+            if not vn.flags.get("og_no_head"):
+                ogh = {"twitter:card": "summary"}
+
+                title = str(vn.flags.get("og_title") or "")
+
+                if thumb:
+                    ogh["og:image"] = j2a["og_thumb"]
+
+                zso = vn.flags.get("og_desc") or ""
+                if zso != "-":
+                    ogh["og:description"] = str(zso)
+
+                zs = vn.flags.get("og_site") or self.args.name
+                if zs not in ("", "-"):
+                    ogh["og:site_name"] = zs
+
+                tagmap = {}
+                if is_au:
+                    title = str(vn.flags.get("og_title_a") or "")
+                    ogh["og:type"] = "music.song"
+                    ogh["og:audio"] = j2a["og_raw"]
+                    tagmap = {
+                        "artist": "og:music:musician",
+                        "album": "og:music:album",
+                        ".dur": "og:music:duration",
+                    }
+                elif is_vid:
+                    title = str(vn.flags.get("og_title_v") or "")
+                    ogh["og:type"] = "video.other"
+                    ogh["og:video"] = j2a["og_raw"]
+                    tagmap = {
+                        "title": "og:title",
+                        ".dur": "og:video:duration",
+                    }
+                elif is_pic:
+                    title = str(vn.flags.get("og_title_i") or "")
+                    ogh["twitter:card"] = "summary_large_image"
+                    ogh["twitter:image"] = ogh["og:image"] = j2a["og_raw"]
+
+                try:
+                    for k, v in file["tags"].items():
+                        zs = "{{ %s }}" % (k,)
+                        title = title.replace(zs, str(v))
+                except:
+                    pass
+                title = re.sub(r"\{\{ [^}]+ \}\}", "", title)
+                while title.startswith(" - "):
+                    title = title[3:]
+                while title.endswith(" - "):
+                    title = title[:3]
+
+                if vn.flags.get("og_s_title") or not title:
+                    title = str(vn.flags.get("og_title") or "")
+
+                for tag, hname in tagmap.items():
+                    try:
+                        v = file["tags"][tag]
+                        if not v:
+                            continue
+                        ogh[hname] = int(v) if tag == ".dur" else v
+                    except:
+                        pass
+
+                ogh["og:title"] = title
+
+                oghs = [
+                    '\t<meta property="%s" content="%s">'
+                    % (k, html_escape(str(v), True, True))
+                    for k, v in ogh.items()
+                ]
+                zs = self.html_head + "\n%s\n" % ("\n".join(oghs),)
+                self.html_head = zs.replace("\n\n", "\n")
+
         html = self.j2s(tpl, **j2a)
         self.reply(html.encode("utf-8", "replace"))
         return True

copyparty/httpconn.py

@@ -55,6 +55,7 @@ class HttpConn(object):
         self.E: EnvParams = self.args.E
         self.asrv: AuthSrv = hsrv.asrv  # mypy404
         self.u2fh: Util.FHC = hsrv.u2fh  # mypy404
+        self.pipes: Util.CachedDict = hsrv.pipes  # mypy404
         self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
         self.xff_nm: Optional[NetMap] = hsrv.xff_nm
         self.xff_lan: NetMap = hsrv.xff_lan  # type: ignore

copyparty/httpsrv.py

@@ -61,6 +61,7 @@ from .u2idx import U2idx
 from .util import (
     E_SCK,
     FHC,
+    CachedDict,
     Daemon,
     Garda,
     Magician,
@@ -130,6 +131,7 @@ class HttpSrv(object):
         self.t_periodic: Optional[threading.Thread] = None
 
         self.u2fh = FHC()
+        self.pipes = CachedDict(0.2)
         self.metrics = Metrics(self)
         self.nreq = 0
         self.nsus = 0

copyparty/metrics.py

@@ -179,7 +179,7 @@ class Metrics(object):
             tnbytes = 0
             tnfiles = 0
             for vpath, vol in allvols:
-                cur = idx.get_cur(vol.realpath)
+                cur = idx.get_cur(vol)
                 if not cur:
                     continue
 

copyparty/svchub.py

@@ -424,6 +424,12 @@ class SvcHub(object):
             t = "WARNING: found config files in [%s]: %s\n  config files are not expected here, and will NOT be loaded (unless your setup is intentionally hella funky)"
             self.log("root", t % (E.cfg, ", ".join(hits)), 3)
 
+        if self.args.no_bauth:
+            t = "WARNING: --no-bauth disables support for the Android app; you may want to use --bauth-last instead"
+            self.log("root", t, 3)
+            if self.args.bauth_last:
+                self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
+
     def _process_config(self) -> bool:
         al = self.args
 
@@ -520,7 +526,7 @@ class SvcHub(object):
         al.exp_md = odfusion(exp, al.exp_md.replace(" ", ","))
         al.exp_lg = odfusion(exp, al.exp_lg.replace(" ", ","))
 
-        for k in ["no_hash", "no_idx"]:
+        for k in ["no_hash", "no_idx", "og_ua"]:
             ptn = getattr(self.args, k)
             if ptn:
                 setattr(self.args, k, re.compile(ptn))
@@ -544,6 +550,17 @@ class SvcHub(object):
         except:
             raise Exception("invalid --rm-retry [%s]" % (self.args.rm_retry,))
 
+        try:
+            zf1, zf2 = self.args.mv_retry.split("/")
+            self.args.mv_re_t = float(zf1)
+            self.args.mv_re_r = float(zf2)
+        except:
+            raise Exception("invalid --mv-retry [%s]" % (self.args.mv_retry,))
+
+        al.tcolor = al.tcolor.lstrip("#")
+        if len(al.tcolor) == 3:  # fc5 => ffcc55
+            al.tcolor = "".join([x * 2 for x in al.tcolor])
+
         return True
 
     def _ipa2re(self, txt) -> Optional[re.Pattern]:

copyparty/tcpsrv.py

@@ -463,6 +463,12 @@ class TcpSrv(object):
         sys.stderr.flush()
 
     def _qr(self, t1: dict[str, list[int]], t2: dict[str, list[int]]) -> str:
+        t2c = {zs: zli for zs, zli in t2.items() if zs in ("127.0.0.1", "::1")}
+        t2b = {zs: zli for zs, zli in t2.items() if ":" in zs and zs not in t2c}
+        t2 = {zs: zli for zs, zli in t2.items() if zs not in t2b and zs not in t2c}
+        t2.update(t2b)  # first ipv4, then ipv6...
+        t2.update(t2c)  # ...and finally localhost
+
         ip = None
         ips = list(t1) + list(t2)
         qri = self.args.qri

copyparty/th_srv.py

@@ -28,6 +28,7 @@ from .util import (
     runcmd,
     statdir,
     vsplit,
+    wrename,
     wunlink,
 )
 
@@ -346,7 +347,7 @@ class ThumbSrv(object):
                             pass
 
             try:
-                bos.rename(ttpath, tpath)
+                wrename(self.log, ttpath, tpath, vn.flags)
             except:
                 pass
 

copyparty/u2idx.py

@@ -62,6 +62,17 @@ class U2idx(object):
     def log(self, msg: str, c: Union[int, str] = 0) -> None:
         self.log_func("u2idx", msg, c)
 
+    def shutdown(self) -> None:
+        for cur in self.cur.values():
+            db = cur.connection
+            try:
+                db.interrupt()
+            except:
+                pass
+
+            cur.close()
+            db.close()
+
     def fsearch(
         self, uname: str, vols: list[VFS], body: dict[str, Any]
     ) -> list[dict[str, Any]]:
@@ -81,14 +92,18 @@ class U2idx(object):
         except:
             raise Pebkac(500, min_ex())
 
-    def get_cur(self, ptop: str) -> Optional["sqlite3.Cursor"]:
+    def get_cur(self, vn: VFS) -> Optional["sqlite3.Cursor"]:
         if not HAVE_SQLITE3:
             return None
 
-        cur = self.cur.get(ptop)
+        cur = self.cur.get(vn.realpath)
         if cur:
             return cur
 
+        if "e2d" not in vn.flags:
+            return None
+
+        ptop = vn.realpath
         histpath = self.asrv.vfs.histtab.get(ptop)
         if not histpath:
             self.log("no histpath for [{}]".format(ptop))
@@ -317,7 +332,7 @@ class U2idx(object):
             ptop = vol.realpath
             flags = vol.flags
 
-            cur = self.get_cur(ptop)
+            cur = self.get_cur(vol)
             if not cur:
                 continue
 

copyparty/up2k.py

@@ -91,6 +91,9 @@ CV_EXTS = set(zsg.split(","))
 HINT_HISTPATH = "you could try moving the database to another location (preferably an SSD or NVME drive) using either the --hist argument (global option for all volumes), or the hist volflag (just for this volume)"
 
 
+VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
+
+
 class Dbw(object):
     def __init__(self, c: "sqlite3.Cursor", n: int, t: float) -> None:
         self.c = c
@@ -136,6 +139,7 @@ class Up2k(object):
         self.need_rescan: set[str] = set()
         self.db_act = 0.0
 
+        self.reg_mutex = threading.Lock()
         self.registry: dict[str, dict[str, dict[str, Any]]] = {}
         self.flags: dict[str, dict[str, Any]] = {}
         self.droppable: dict[str, list[str]] = {}
@@ -143,7 +147,7 @@ class Up2k(object):
         self.volsize: dict["sqlite3.Cursor", int] = {}
         self.volstate: dict[str, str] = {}
         self.vol_act: dict[str, float] = {}
-        self.busy_aps: set[str] = set()
+        self.busy_aps: dict[str, int] = {}
         self.dupesched: dict[str, list[tuple[str, str, float]]] = {}
         self.snap_prev: dict[str, Optional[tuple[int, float]]] = {}
 
@@ -182,7 +186,7 @@ class Up2k(object):
             t = "could not initialize sqlite3, will use in-memory registry only"
             self.log(t, 3)
 
-        self.fstab = Fstab(self.log_func)
+        self.fstab = Fstab(self.log_func, self.args)
         self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
 
         if self.args.hash_mt < 2:
@@ -200,11 +204,15 @@ class Up2k(object):
         Daemon(self.deferred_init, "up2k-deferred-init")
 
     def reload(self, rescan_all_vols: bool) -> None:
-        """mutex me"""
+        """mutex(main) me"""
         self.log("reload #{} scheduled".format(self.gid + 1))
         all_vols = self.asrv.vfs.all_vols
 
-        scan_vols = [k for k, v in all_vols.items() if v.realpath not in self.registry]
+        with self.reg_mutex:
+            scan_vols = [
+                k for k, v in all_vols.items() if v.realpath not in self.registry
+            ]
+
         if rescan_all_vols:
             scan_vols = list(all_vols.keys())
 
@@ -217,7 +225,7 @@ class Up2k(object):
         if self.stop:
             # up-mt consistency not guaranteed if init is interrupted;
             # drop caches for a full scan on next boot
-            with self.mutex:
+            with self.mutex, self.reg_mutex:
                 self._drop_caches()
 
             if self.pp:
@@ -283,10 +291,27 @@ class Up2k(object):
                 min(1000 * 24 * 60 * 60 - 1, time.time() - self.db_act)
             ),
         }
-        return json.dumps(ret, indent=4)
+        return json.dumps(ret, separators=(",\n", ": "))
+
+    def find_job_by_ap(self, ptop: str, ap: str) -> str:
+        try:
+            if ANYWIN:
+                ap = ap.replace("\\", "/")
+
+            vp = ap[len(ptop) :].strip("/")
+            dn, fn = vsplit(vp)
+            with self.reg_mutex:
+                tab2 = self.registry[ptop]
+                for job in tab2.values():
+                    if job["prel"] == dn and job["name"] == fn:
+                        return json.dumps(job, separators=(",\n", ": "))
+        except:
+            pass
+
+        return "{}"
 
     def get_unfinished_by_user(self, uname, ip) -> str:
-        if PY2 or not self.mutex.acquire(timeout=2):
+        if PY2 or not self.reg_mutex.acquire(timeout=2):
             return '[{"timeout":1}]'
 
         ret: list[tuple[int, str, int, int, int]] = []
@@ -315,17 +340,25 @@ class Up2k(object):
                     )
                     ret.append(zt5)
         finally:
-            self.mutex.release()
+            self.reg_mutex.release()
+
+        if ANYWIN:
+            ret = [(x[0], x[1].replace("\\", "/"), x[2], x[3], x[4]) for x in ret]
 
         ret.sort(reverse=True)
         ret2 = [
-            {"at": at, "vp": "/" + vp, "pd": 100 - ((nn * 100) // (nh or 1)), "sz": sz}
+            {
+                "at": at,
+                "vp": "/" + quotep(vp),
+                "pd": 100 - ((nn * 100) // (nh or 1)),
+                "sz": sz,
+            }
             for (at, vp, sz, nn, nh) in ret
         ]
-        return json.dumps(ret2, indent=0)
+        return json.dumps(ret2, separators=(",\n", ": "))
 
     def get_unfinished(self) -> str:
-        if PY2 or not self.mutex.acquire(timeout=0.5):
+        if PY2 or not self.reg_mutex.acquire(timeout=0.5):
             return ""
 
         ret: dict[str, tuple[int, int]] = {}
@@ -347,17 +380,17 @@ class Up2k(object):
 
                 ret[ptop] = (nbytes, nfiles)
         finally:
-            self.mutex.release()
+            self.reg_mutex.release()
 
-        return json.dumps(ret, indent=4)
+        return json.dumps(ret, separators=(",\n", ": "))
 
     def get_volsize(self, ptop: str) -> tuple[int, int]:
-        with self.mutex:
+        with self.reg_mutex:
             return self._get_volsize(ptop)
 
     def get_volsizes(self, ptops: list[str]) -> list[tuple[int, int]]:
         ret = []
-        with self.mutex:
+        with self.reg_mutex:
             for ptop in ptops:
                 ret.append(self._get_volsize(ptop))
 
@@ -385,7 +418,7 @@ class Up2k(object):
     def _rescan(
         self, all_vols: dict[str, VFS], scan_vols: list[str], wait: bool, fscan: bool
     ) -> str:
-        """mutex me"""
+        """mutex(main) me"""
         if not wait and self.pp:
             return "cannot initiate; scan is already in progress"
 
@@ -667,7 +700,7 @@ class Up2k(object):
                 self.log(msg, c=3)
 
         live_vols = []
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             # only need to protect register_vpath but all in one go feels right
             for vol in vols:
                 try:
@@ -709,7 +742,7 @@ class Up2k(object):
 
         if self.args.re_dhash or [zv for zv in vols if "e2tsr" in zv.flags]:
             self.args.re_dhash = False
-            with self.mutex:
+            with self.mutex, self.reg_mutex:
                 self._drop_caches()
 
         for vol in vols:
@@ -786,7 +819,9 @@ class Up2k(object):
             self.volstate[vol.vpath] = "online (mtp soon)"
 
         for vol in need_vac:
-            reg = self.register_vpath(vol.realpath, vol.flags)
+            with self.mutex, self.reg_mutex:
+                reg = self.register_vpath(vol.realpath, vol.flags)
+
             assert reg
             cur, _ = reg
             with self.mutex:
@@ -800,7 +835,9 @@ class Up2k(object):
             if vol.flags["dbd"] == "acid":
                 continue
 
-            reg = self.register_vpath(vol.realpath, vol.flags)
+            with self.mutex, self.reg_mutex:
+                reg = self.register_vpath(vol.realpath, vol.flags)
+
             try:
                 assert reg
                 cur, db_path = reg
@@ -847,6 +884,7 @@ class Up2k(object):
     def register_vpath(
         self, ptop: str, flags: dict[str, Any]
     ) -> Optional[tuple["sqlite3.Cursor", str]]:
+        """mutex(main,reg) me"""
         histpath = self.asrv.vfs.histtab.get(ptop)
         if not histpath:
             self.log("no histpath for [{}]".format(ptop))
@@ -869,7 +907,7 @@ class Up2k(object):
         ft = "\033[0;32m{}{:.0}"
         ff = "\033[0;35m{}{:.0}"
         fv = "\033[0;36m{}:\033[90m{}"
-        fx = set(("html_head", "rm_re_t", "rm_re_r"))
+        fx = set(("html_head", "rm_re_t", "rm_re_r", "mv_re_t", "mv_re_r"))
         fd = vf_bmap()
         fd.update(vf_cmap())
         fd.update(vf_vmap())
@@ -1030,7 +1068,9 @@ class Up2k(object):
         dev = cst.st_dev if vol.flags.get("xdev") else 0
 
         with self.mutex:
-            reg = self.register_vpath(top, vol.flags)
+            with self.reg_mutex:
+                reg = self.register_vpath(top, vol.flags)
+
             assert reg and self.pp
             cur, db_path = reg
 
@@ -1627,7 +1667,7 @@ class Up2k(object):
 
     def _build_tags_index(self, vol: VFS) -> tuple[int, int, bool]:
         ptop = vol.realpath
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             reg = self.register_vpath(ptop, vol.flags)
 
         assert reg and self.pp
@@ -1648,6 +1688,7 @@ class Up2k(object):
         return ret
 
     def _drop_caches(self) -> None:
+        """mutex(main,reg) me"""
         self.log("dropping caches for a full filesystem scan")
         for vol in self.asrv.vfs.all_vols.values():
             reg = self.register_vpath(vol.realpath, vol.flags)
@@ -1823,7 +1864,7 @@ class Up2k(object):
         params: tuple[Any, ...],
         flt: int,
     ) -> tuple[tempfile.SpooledTemporaryFile[bytes], int]:
-        """mutex me"""
+        """mutex(main) me"""
         n = 0
         c2 = cur.connection.cursor()
         tf = tempfile.SpooledTemporaryFile(1024 * 1024 * 8, "w+b", prefix="cpp-tq-")
@@ -2157,7 +2198,7 @@ class Up2k(object):
         ip: str,
         at: float,
     ) -> int:
-        """will mutex"""
+        """will mutex(main)"""
         assert self.mtag
 
         try:
@@ -2189,7 +2230,7 @@ class Up2k(object):
         abspath: str,
         tags: dict[str, Union[str, float]],
     ) -> int:
-        """mutex me"""
+        """mutex(main) me"""
         assert self.mtag
 
         if not bos.path.isfile(abspath):
@@ -2474,28 +2515,36 @@ class Up2k(object):
 
         cur.connection.commit()
 
-    def _job_volchk(self, cj: dict[str, Any]) -> None:
+    def handle_json(
-        if not self.register_vpath(cj["ptop"], cj["vcfg"]):
+        self, cj: dict[str, Any], busy_aps: dict[str, int]
-            if cj["ptop"] not in self.registry:
+    ) -> dict[str, Any]:
-                raise Pebkac(410, "location unavailable")
+        # busy_aps is u2fh (always undefined if -j0) so this is safe
-
-    def handle_json(self, cj: dict[str, Any], busy_aps: set[str]) -> dict[str, Any]:
         self.busy_aps = busy_aps
+        got_lock = False
         try:
             # bit expensive; 3.9=10x 3.11=2x
             if self.mutex.acquire(timeout=10):
-                self._job_volchk(cj)
+                got_lock = True
-                self.mutex.release()
+                with self.reg_mutex:
+                    return self._handle_json(cj)
             else:
                 t = "cannot receive uploads right now;\nserver busy with {}.\nPlease wait; the client will retry..."
                 raise Pebkac(503, t.format(self.blocked or "[unknown]"))
         except TypeError:
             if not PY2:
                 raise
-            with self.mutex:
+            with self.mutex, self.reg_mutex:
-                self._job_volchk(cj)
+                return self._handle_json(cj)
+        finally:
+            if got_lock:
+                self.mutex.release()
 
+    def _handle_json(self, cj: dict[str, Any]) -> dict[str, Any]:
         ptop = cj["ptop"]
+        if not self.register_vpath(ptop, cj["vcfg"]):
+            if ptop not in self.registry:
+                raise Pebkac(410, "location unavailable")
+
         cj["name"] = sanitize_fn(cj["name"], "", [".prologue.html", ".epilogue.html"])
         cj["poke"] = now = self.db_act = self.vol_act[ptop] = time.time()
         wark = self._get_wark(cj)
@@ -2510,7 +2559,7 @@ class Up2k(object):
         # refuse out-of-order / multithreaded uploading if sprs False
         sprs = self.fstab.get(pdir) != "ng"
 
-        with self.mutex:
+        if True:
             jcur = self.cur.get(ptop)
             reg = self.registry[ptop]
             vfs = self.asrv.vfs.all_vols[cj["vtop"]]
@@ -2948,7 +2997,7 @@ class Up2k(object):
     def handle_chunk(
         self, ptop: str, wark: str, chash: str
     ) -> tuple[int, list[int], str, float, bool]:
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             self.db_act = self.vol_act[ptop] = time.time()
             job = self.registry[ptop].get(wark)
             if not job:
@@ -2991,7 +3040,7 @@ class Up2k(object):
         return chunksize, ofs, path, job["lmod"], job["sprs"]
 
     def release_chunk(self, ptop: str, wark: str, chash: str) -> bool:
-        with self.mutex:
+        with self.reg_mutex:
             job = self.registry[ptop].get(wark)
             if job:
                 job["busy"].pop(chash, None)
@@ -2999,7 +3048,7 @@ class Up2k(object):
         return True
 
     def confirm_chunk(self, ptop: str, wark: str, chash: str) -> tuple[int, str]:
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             self.db_act = self.vol_act[ptop] = time.time()
             try:
                 job = self.registry[ptop][wark]
@@ -3022,16 +3071,16 @@ class Up2k(object):
 
             if self.args.nw:
                 self.regdrop(ptop, wark)
-                return ret, dst
 
         return ret, dst
 
     def finish_upload(self, ptop: str, wark: str, busy_aps: set[str]) -> None:
         self.busy_aps = busy_aps
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             self._finish_upload(ptop, wark)
 
     def _finish_upload(self, ptop: str, wark: str) -> None:
+        """mutex(main,reg) me"""
         try:
             job = self.registry[ptop][wark]
             pdir = djoin(job["ptop"], job["prel"])
@@ -3044,12 +3093,11 @@ class Up2k(object):
             t = "finish_upload {} with remaining chunks {}"
             raise Pebkac(500, t.format(wark, job["need"]))
 
-        # self.log("--- " + wark + "  " + dst + " finish_upload atomic " + dst, 4)
-        atomic_move(src, dst)
-
         upt = job.get("at") or time.time()
         vflags = self.flags[ptop]
 
+        atomic_move(self.log, src, dst, vflags)
+
         times = (int(time.time()), int(job["lmod"]))
         self.log(
             "no more chunks, setting times {} ({}) on {}".format(
@@ -3105,6 +3153,7 @@ class Up2k(object):
             cur.connection.commit()
 
     def regdrop(self, ptop: str, wark: str) -> None:
+        """mutex(main,reg) me"""
         olds = self.droppable[ptop]
         if wark:
             olds.append(wark)
@@ -3199,16 +3248,23 @@ class Up2k(object):
         at: float,
         skip_xau: bool = False,
     ) -> None:
+        """mutex(main) me"""
         self.db_rm(db, rd, fn, sz)
 
+        if not ip:
+            db_ip = ""
+        else:
+            # plugins may expect this to look like an actual IP
+            db_ip = "1.1.1.1" if self.args.no_db_ip else ip
+
         sql = "insert into up values (?,?,?,?,?,?,?)"
-        v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
+        v = (wark, int(ts), sz, rd, fn, db_ip, int(at or 0))
         try:
             db.execute(sql, v)
         except:
             assert self.mem_cur
             rd, fn = s3enc(self.mem_cur, rd, fn)
-            v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
+            v = (wark, int(ts), sz, rd, fn, db_ip, int(at or 0))
             db.execute(sql, v)
 
         self.volsize[db] += sz
@@ -3312,7 +3368,7 @@ class Up2k(object):
             vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
             vn, rem = vn.get_dbv(rem)
             ptop = vn.realpath
-            with self.mutex:
+            with self.mutex, self.reg_mutex:
                 abrt_cfg = self.flags.get(ptop, {}).get("u2abort", 1)
                 addr = (ip or "\n") if abrt_cfg in (1, 2) else ""
                 user = (uname or "\n") if abrt_cfg in (1, 3) else ""
@@ -3320,7 +3376,10 @@ class Up2k(object):
                 for wark, job in reg.items():
                     if (user and user != job["user"]) or (addr and addr != job["addr"]):
                         continue
-                    if djoin(job["prel"], job["name"]) == rem:
+                    jrem = djoin(job["prel"], job["name"])
+                    if ANYWIN:
+                        jrem = jrem.replace("\\", "/")
+                    if jrem == rem:
                         if job["ptop"] != ptop:
                             t = "job.ptop [%s] != vol.ptop [%s] ??"
                             raise Exception(t % (job["ptop"] != ptop))
@@ -3416,7 +3475,7 @@ class Up2k(object):
                         continue
 
                 n_files += 1
-                with self.mutex:
+                with self.mutex, self.reg_mutex:
                     cur = None
                     try:
                         ptop = dbv.realpath
@@ -3534,6 +3593,7 @@ class Up2k(object):
     def _mv_file(
         self, uname: str, svp: str, dvp: str, curs: set["sqlite3.Cursor"]
     ) -> str:
+        """mutex(main) me;  will mutex(reg)"""
         svn, srem = self.asrv.vfs.get(svp, uname, True, False, True)
         svn, srem = svn.get_dbv(srem)
 
@@ -3614,7 +3674,9 @@ class Up2k(object):
             if c2 and c2 != c1:
                 self._copy_tags(c1, c2, w)
 
-            has_dupes = self._forget_file(svn.realpath, srem, c1, w, is_xvol, fsize)
+            with self.reg_mutex:
+                has_dupes = self._forget_file(svn.realpath, srem, c1, w, is_xvol, fsize)
+
             if not is_xvol:
                 has_dupes = self._relink(w, svn.realpath, srem, dabs)
 
@@ -3653,7 +3715,7 @@ class Up2k(object):
                 self._symlink(dlink, dabs, dvn.flags, lmod=ftime)
                 wunlink(self.log, sabs, svn.flags)
             else:
-                atomic_move(sabs, dabs)
+                atomic_move(self.log, sabs, dabs, svn.flags)
 
         except OSError as ex:
             if ex.errno != errno.EXDEV:
@@ -3744,7 +3806,10 @@ class Up2k(object):
         drop_tags: bool,
         sz: int,
     ) -> bool:
-        """forgets file in db, fixes symlinks, does not delete"""
+        """
+        mutex(main,reg) me
+        forgets file in db, fixes symlinks, does not delete
+        """
         srd, sfn = vsplit(vrem)
         has_dupes = False
         self.log("forgetting {}".format(vrem))
@@ -3830,8 +3895,7 @@ class Up2k(object):
             self.log("linkswap [{}] and [{}]".format(sabs, slabs))
             mt = bos.path.getmtime(slabs, False)
             flags = self.flags.get(ptop) or {}
-            wunlink(self.log, slabs, flags)
+            atomic_move(self.log, sabs, slabs, flags)
-            bos.rename(sabs, slabs)
             bos.utime(slabs, (int(time.time()), int(mt)), False)
             self._symlink(slabs, sabs, flags, False)
             full[slabs] = (ptop, rem)
@@ -4070,7 +4134,7 @@ class Up2k(object):
                 self.do_snapshot()
 
     def do_snapshot(self) -> None:
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             for k, reg in self.registry.items():
                 self._snap_reg(k, reg)
 
@@ -4138,11 +4202,11 @@ class Up2k(object):
 
         path2 = "{}.{}".format(path, os.getpid())
         body = {"droppable": self.droppable[ptop], "registry": reg}
-        j = json.dumps(body, indent=2, sort_keys=True).encode("utf-8")
+        j = json.dumps(body, sort_keys=True, separators=(",\n", ": ")).encode("utf-8")
         with gzip.GzipFile(path2, "wb") as f:
             f.write(j)
 
-        atomic_move(path2, path)
+        atomic_move(self.log, path2, path, VF_CAREFUL)
 
         self.log("snap: {} |{}|".format(path, len(reg.keys())))
         self.snap_prev[ptop] = etag
@@ -4211,7 +4275,7 @@ class Up2k(object):
                 raise Exception("invalid hash task")
 
             try:
-                if not self._hash_t(task):
+                if not self._hash_t(task) and self.stop:
                     return
             except Exception as ex:
                 self.log("failed to hash %s: %s" % (task, ex), 1)
@@ -4221,7 +4285,7 @@ class Up2k(object):
     ) -> bool:
         ptop, vtop, flags, rd, fn, ip, at, usr, skip_xau = task
         # self.log("hashq {} pop {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             if not self.register_vpath(ptop, flags):
                 return True
 
@@ -4239,7 +4303,7 @@ class Up2k(object):
 
             wark = up2k_wark_from_hashlist(self.salt, inf.st_size, hashes)
 
-        with self.mutex:
+        with self.mutex, self.reg_mutex:
             self.idx_wark(
                 self.flags[ptop],
                 rd,
@@ -4315,6 +4379,18 @@ class Up2k(object):
         for x in list(self.spools):
             self._unspool(x)
 
+        for cur in self.cur.values():
+            db = cur.connection
+            try:
+                db.interrupt()
+            except:
+                pass
+
+            cur.close()
+            db.close()
+
+        self.registry = {}
+
 
 def up2k_chunksize(filesize: int) -> int:
     chunksize = 1024 * 1024

copyparty/util.py

@@ -35,6 +35,9 @@ from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, VT100, WINDOWS
 from .__version__ import S_BUILD_DT, S_VERSION
 from .stolen import surrogateescape
 
+ub64dec = base64.urlsafe_b64decode
+ub64enc = base64.urlsafe_b64encode
+
 try:
     from datetime import datetime, timezone
 
@@ -759,15 +762,46 @@ class CachedSet(object):
             self.oldest = now
 
 
+class CachedDict(object):
+    def __init__(self, maxage: float) -> None:
+        self.c: dict[str, tuple[float, Any]] = {}
+        self.maxage = maxage
+        self.oldest = 0.0
+
+    def set(self, k: str, v: Any) -> None:
+        now = time.time()
+        self.c[k] = (now, v)
+        if now - self.oldest < self.maxage:
+            return
+
+        c = self.c = {k: v for k, v in self.c.items() if now - v[0] < self.maxage}
+        try:
+            self.oldest = min([x[0] for x in c.values()])
+        except:
+            self.oldest = now
+
+    def get(self, k: str) -> Optional[tuple[str, Any]]:
+        try:
+            ts, ret = self.c[k]
+            now = time.time()
+            if now - ts > self.maxage:
+                del self.c[k]
+                return None
+            return ret
+        except:
+            return None
+
+
 class FHC(object):
     class CE(object):
         def __init__(self, fh: typing.BinaryIO) -> None:
             self.ts: float = 0
             self.fhs = [fh]
+            self.all_fhs = set([fh])
 
     def __init__(self) -> None:
         self.cache: dict[str, FHC.CE] = {}
-        self.aps: set[str] = set()
+        self.aps: dict[str, int] = {}
 
     def close(self, path: str) -> None:
         try:
@@ -779,7 +813,7 @@ class FHC(object):
             fh.close()
 
         del self.cache[path]
-        self.aps.remove(path)
+        del self.aps[path]
 
     def clean(self) -> None:
         if not self.cache:
@@ -800,9 +834,12 @@ class FHC(object):
         return self.cache[path].fhs.pop()
 
     def put(self, path: str, fh: typing.BinaryIO) -> None:
-        self.aps.add(path)
+        if path not in self.aps:
+            self.aps[path] = 0
+
         try:
             ce = self.cache[path]
+            ce.all_fhs.add(fh)
             ce.fhs.append(fh)
         except:
             ce = self.CE(fh)
@@ -1997,6 +2034,7 @@ def vsplit(vpath: str) -> tuple[str, str]:
     return vpath.rsplit("/", 1)  # type: ignore
 
 
+# vpath-join
 def vjoin(rd: str, fn: str) -> str:
     if rd and fn:
         return rd + "/" + fn
@@ -2004,6 +2042,14 @@ def vjoin(rd: str, fn: str) -> str:
         return rd or fn
 
 
+# url-join
+def ujoin(rd: str, fn: str) -> str:
+    if rd and fn:
+        return rd.rstrip("/") + "/" + fn.lstrip("/")
+    else:
+        return rd or fn
+
+
 def _w8dec2(txt: bytes) -> str:
     """decodes filesystem-bytes to wtf8"""
     return surrogateescape.decodefilename(txt)
@@ -2125,26 +2171,29 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
         log("lsof failed; " + min_ex(), 3)
 
 
-def atomic_move(usrc: str, udst: str) -> None:
+def _fs_mvrm(
-    src = fsenc(usrc)
+    log: "NamedLogger", src: str, dst: str, atomic: bool, flags: dict[str, Any]
-    dst = fsenc(udst)
+) -> bool:
-    if not PY2:
+    bsrc = fsenc(src)
-        os.replace(src, dst)
+    bdst = fsenc(dst)
+    if atomic:
+        k = "mv_re_"
+        act = "atomic-rename"
+        osfun = os.replace
+        args = [bsrc, bdst]
+    elif dst:
+        k = "mv_re_"
+        act = "rename"
+        osfun = os.rename
+        args = [bsrc, bdst]
     else:
-        if os.path.exists(dst):
+        k = "rm_re_"
-            os.unlink(dst)
+        act = "delete"
+        osfun = os.unlink
+        args = [bsrc]
 
-        os.rename(src, dst)
+    maxtime = flags.get(k + "t", 0.0)
-
+    chill = flags.get(k + "r", 0.0)
-
-def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
-    maxtime = flags.get("rm_re_t", 0.0)
-    bpath = fsenc(abspath)
-    if not maxtime:
-        os.unlink(bpath)
-        return True
-
-    chill = flags.get("rm_re_r", 0.0)
     if chill < 0.001:
         chill = 0.1
 
@@ -2152,14 +2201,19 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
     t0 = now = time.time()
     for attempt in range(90210):
         try:
-            if ino and os.stat(bpath).st_ino != ino:
+            if ino and os.stat(bsrc).st_ino != ino:
-                log("inode changed; aborting delete")
+                t = "src inode changed; aborting %s %s"
+                log(t % (act, src), 1)
                 return False
-            os.unlink(bpath)
+            if (dst and not atomic) and os.path.exists(bdst):
+                t = "something appeared at dst; aborting rename [%s] ==> [%s]"
+                log(t % (src, dst), 1)
+                return False
+            osfun(*args)
             if attempt:
                 now = time.time()
-                t = "deleted in %.2f sec, attempt %d"
+                t = "%sd in %.2f sec, attempt %d: %s"
-                log(t % (now - t0, attempt + 1))
+                log(t % (act, now - t0, attempt + 1, src))
             return True
         except OSError as ex:
             now = time.time()
@@ -2169,15 +2223,45 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
                 raise
             if not attempt:
                 if not PY2:
-                    ino = os.stat(bpath).st_ino
+                    ino = os.stat(bsrc).st_ino
-                t = "delete failed (err.%d); retrying for %d sec: %s"
+                t = "%s failed (err.%d); retrying for %d sec: [%s]"
-                log(t % (ex.errno, maxtime + 0.99, abspath))
+                log(t % (act, ex.errno, maxtime + 0.99, src))
 
         time.sleep(chill)
 
     return False  # makes pylance happy
 
 
+def atomic_move(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> None:
+    bsrc = fsenc(src)
+    bdst = fsenc(dst)
+    if PY2:
+        if os.path.exists(bdst):
+            _fs_mvrm(log, dst, "", False, flags)  # unlink
+
+        _fs_mvrm(log, src, dst, False, flags)  # rename
+    elif flags.get("mv_re_t"):
+        _fs_mvrm(log, src, dst, True, flags)
+    else:
+        os.replace(bsrc, bdst)
+
+
+def wrename(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> bool:
+    if not flags.get("mv_re_t"):
+        os.rename(fsenc(src), fsenc(dst))
+        return True
+
+    return _fs_mvrm(log, src, dst, False, flags)
+
+
+def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
+    if not flags.get("rm_re_t"):
+        os.unlink(fsenc(abspath))
+        return True
+
+    return _fs_mvrm(log, abspath, "", False, flags)
+
+
 def get_df(abspath: str) -> tuple[Optional[int], Optional[int]]:
     try:
         # some fuses misbehave

copyparty/web/baguettebox.js

@@ -740,6 +740,7 @@ window.baguetteBox = (function () {
         });
         image.setAttribute('src', imageSrc);
         if (is_vid) {
+            image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
             image.setAttribute('controls', 'controls');
             image.onended = vidEnd;
         }

copyparty/web/browser.css

@@ -28,6 +28,8 @@
 	--row-alt: #282828;
 
 	--scroll: #eb0;
+	--sel-fg: var(--bg-d1);
+	--sel-bg: var(--fg);
 
 	--a: #fc5;
 	--a-b: #c90;
@@ -330,6 +332,8 @@ html.c {
 }
 html.cz {
 	--bgg: var(--bg-u2);
+	--sel-bg: var(--bg-u5);
+	--sel-fg: var(--fg);
 	--srv-3: #fff;
 	--u2-tab-b1: var(--bg-d3);
 }
@@ -343,6 +347,8 @@ html.cy {
 	--bg-d3: #f77;
 	--bg-d2: #ff0;
 
+	--sel-bg: #f77;
+
 	--a: #fff;
 	--a-hil: #fff;
 	--a-h-bg: #000;
@@ -588,8 +594,8 @@ html.dy {
 	line-height: 1.2em;
 }
 ::selection {
-	color: var(--bg-d1);
+	color: var(--sel-fg);
-	background: var(--fg);
+	background: var(--sel-bg);
 	text-shadow: none;
 }
 html,body,tr,th,td,#files,a {
@@ -699,12 +705,12 @@ a:hover {
 .s0:after,
 .s1:after {
 	content: '⌄';
-	margin-left: -.1em;
+	margin-left: -.15em;
 }
 .s0r:after,
 .s1r:after {
 	content: '⌃';
-	margin-left: -.1em;
+	margin-left: -.15em;
 }
 .s0:after,
 .s0r:after {
@@ -715,7 +721,7 @@ a:hover {
 	color: var(--sort-2);
 }
 #files thead th:after {
-	margin-right: -.7em;
+	margin-right: -.5em;
 }
 #files tbody tr:hover td,
 #files tbody tr:hover td+td {
@@ -744,6 +750,15 @@ html #files.hhpick thead th {
 	word-wrap: break-word;
 	overflow: hidden;
 }
+#files tr.fade a {
+	color: #999;
+	color: rgba(255, 255, 255, 0.4);
+	font-style: italic;
+}
+html.y #files tr.fade a {
+	color: #999;
+	color: rgba(0, 0, 0, 0.4);
+}
 #files tr:nth-child(2n) td {
 	background: var(--row-alt);
 }
@@ -1600,6 +1615,7 @@ html {
 	padding: .2em .4em;
 	font-size: 1.2em;
 	margin: .2em;
+	display: inline-block;
 	white-space: pre;
 	position: relative;
 	top: -.12em;
@@ -1736,6 +1752,7 @@ html.y #tree.nowrap .ntree a+a:hover {
 }
 #files th span {
 	position: relative;
+	white-space: nowrap;
 }
 #files>thead>tr>th.min,
 #files td.min {
@@ -1773,9 +1790,6 @@ html.y #tree.nowrap .ntree a+a:hover {
 	margin: .7em 0 .7em .5em;
 	padding-left: .5em;
 }
-.opwide>div>div>a {
-	line-height: 2em;
-}
 .opwide>div>h3 {
 	color: var(--fg-weak);
 	margin: 0 .4em;

copyparty/web/browser.html

@@ -6,7 +6,7 @@
 	<title>{{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
 {{ html_head }}

copyparty/web/browser.js

@@ -1,6 +1,8 @@
 "use strict";
 
-var XHR = XMLHttpRequest;
+var XHR = XMLHttpRequest,
+	img_re = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i;
+
 var Ls = {
 	"eng": {
 		"tt": "English",
@@ -283,6 +285,7 @@ var Ls = {
 
 		"im_hnf": "that image no longer exists",
 
+		"f_empty": 'this folder is empty',
 		"f_chide": 'this will hide the column «{0}»\n\nyou can unhide columns in the settings tab',
 		"f_bigtxt": "this file is {0} MiB large -- really view as text?",
 		"fbd_more": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_more">show {2}</a> or <a href="#" id="bd_all">show all</a></div>',
@@ -290,6 +293,8 @@ var Ls = {
 
 		"f_dls": 'the file links in the current folder have\nbeen changed into download links',
 
+		"f_partial": "To safely download a file which is currently being uploaded, please click the file which has the same filename, but without the <code>.PARTIAL</code> file extension. Please press CANCEL or Escape to do this.\n\nPressing OK / Enter will ignore this warning and continue downloading the <code>.PARTIAL</code> scratchfile instead, which will almost definitely give you corrupted data.",
+
 		"ft_paste": "paste {0} items$NHotkey: ctrl-V",
 		"fr_eperm": 'cannot rename:\nyou do not have “move” permission in this folder',
 		"fd_eperm": 'cannot delete:\nyou do not have “delete” permission in this folder',
@@ -335,6 +340,9 @@ var Ls = {
 		"fp_err": "move failed:\n",
 		"fp_confirm": "move these {0} items here?",
 		"fp_etab": 'failed to read clipboard from other browser tab',
+		"fp_name": "uploading a file from your device. Give it a name:",
+		"fp_both_m": '<h6>choose what to paste</h6><code>Enter</code> = Move {0} files from «{1}»\n<code>ESC</code> = Upload {2} files from your device',
+		"fp_both_b": '<a href="#" id="modal-ok">Move</a><a href="#" id="modal-ng">Upload</a>',
 
 		"mk_noname": "type a name into the text field on the left before you do that :p",
 
@@ -420,6 +428,7 @@ var Ls = {
 		"un_fclr": "clear filter",
 		"un_derr": 'unpost-delete failed:\n',
 		"un_f5": 'something broke, please try a refresh or hit F5',
+		"un_uf5": "sorry but you have to refresh the page (for example by pressing F5 or CTRL-R) before this upload can be aborted",
 		"un_nou": '<b>warning:</b> server too busy to show unfinished uploads; click the "refresh" link in a bit',
 		"un_noc": '<b>warning:</b> unpost of fully uploaded files is not enabled/permitted in server config',
 		"un_max": "showing first 2000 files (use the filter)",
@@ -785,6 +794,7 @@ var Ls = {
 
 		"im_hnf": "bildet finnes ikke lenger",
 
+		"f_empty": 'denne mappen er tom',
 		"f_chide": 'dette vil skjule kolonnen «{0}»\n\nfanen for "andre innstillinger" lar deg vise kolonnen igjen',
 		"f_bigtxt": "denne filen er hele {0} MiB -- vis som tekst?",
 		"fbd_more": '<div id="blazy">viser <code>{0}</code> av <code>{1}</code> filer; <a href="#" id="bd_more">vis {2}</a> eller <a href="#" id="bd_all">vis alle</a></div>',
@@ -792,6 +802,8 @@ var Ls = {
 
 		"f_dls": 'linkene i denne mappen er nå\nomgjort til nedlastningsknapper',
 
+		"f_partial": "For å laste ned en fil som enda ikke er ferdig opplastet, klikk på filen som har samme filnavn som denne, men uten <code>.PARTIAL</code> på slutten. Da vil serveren passe på at nedlastning går bra. Derfor anbefales det sterkt å trykke ABRYT eller Escape-tasten.\n\nHvis du virkelig ønsker å laste ned denne <code>.PARTIAL</code>-filen på en ukontrollert måte, trykk OK / Enter for å ignorere denne advarselen. Slik vil du høyst sannsynlig motta korrupt data.",
+
 		"ft_paste": "Lim inn {0} filer$NSnarvei: ctrl-V",
 		"fr_eperm": 'kan ikke endre navn:\ndu har ikke “move”-rettigheten i denne mappen',
 		"fd_eperm": 'kan ikke slette:\ndu har ikke “delete”-rettigheten i denne mappen',
@@ -837,6 +849,9 @@ var Ls = {
 		"fp_err": "flytting feilet:\n",
 		"fp_confirm": "flytt disse {0} filene hit?",
 		"fp_etab": 'kunne ikke lese listen med filer ifra den andre nettleserfanen',
+		"fp_name": "Laster opp én fil fra enheten din. Velg filnavn:",
+		"fp_both_m": '<h6>hva skal limes inn her?</h6><code>Enter</code> = Flytt {0} filer fra «{1}»\n<code>ESC</code> = Last opp {2} filer fra enheten din',
+		"fp_both_b": '<a href="#" id="modal-ok">Flytt</a><a href="#" id="modal-ng">Last opp</a>',
 
 		"mk_noname": "skriv inn et navn i tekstboksen til venstre først :p",
 
@@ -922,6 +937,7 @@ var Ls = {
 		"un_fclr": "nullstill filter",
 		"un_derr": 'unpost-sletting feilet:\n',
 		"un_f5": 'noe gikk galt, prøv å oppdatere listen eller trykk F5',
+		"un_uf5": "beklager, men du må laste siden på nytt (f.eks. ved å trykke F5 eller CTRL-R) før denne opplastningen kan avbrytes",
 		"un_nou": '<b>advarsel:</b> kan ikke vise ufullstendige opplastninger akkurat nå; klikk på oppdater-linken om litt',
 		"un_noc": '<b>advarsel:</b> angring av fullførte opplastninger er deaktivert i serverkonfigurasjonen',
 		"un_max": "viser de første 2000 filene (bruk filteret for å innsnevre)",
@@ -1407,6 +1423,12 @@ var ACtx = !IPHONE && (window.AudioContext || window.webkitAudioContext),
 	dk, mp;
 
 
+if (window.og_fn) {
+	hash0 = 1;
+	hist_replace(vsplit(get_evpath())[0]);
+}
+
+
 var mpl = (function () {
 	var have_mctl = 'mediaSession' in navigator && window.MediaMetadata;
 
@@ -1539,16 +1561,18 @@ var mpl = (function () {
 	set_tint();
 
 	r.acode = function (url) {
-		var c = true;
+		var c = true,
+			cs = url.split('?')[0];
+
 		if (!have_acode)
 			c = false;
-		else if (/\.(wav|flac)$/i.exec(url))
+		else if (/\.(wav|flac)$/i.exec(cs))
 			c = r.ac_flac;
-		else if (/\.(aac|m4a)$/i.exec(url))
+		else if (/\.(aac|m4a)$/i.exec(cs))
 			c = r.ac_aac;
-		else if (/\.(ogg|opus)$/i.exec(url) && !can_ogg)
+		else if (/\.(ogg|opus)$/i.exec(cs) && !can_ogg)
 			c = true;
-		else if (re_au_native.exec(url))
+		else if (re_au_native.exec(cs))
 			c = false;
 
 		if (!c)
@@ -1706,7 +1730,7 @@ function MPlayer() {
 		}
 	}
 
-	r.vol = clamp(fcfg_get('vol', IPHONE ? 1 : 0.5), 0, 1);
+	r.vol = clamp(fcfg_get('vol', IPHONE ? 1 : dvol / 100), 0, 1);
 
 	r.expvol = function (v) {
 		return 0.5 * v + 0.5 * v * v;
@@ -1804,7 +1828,7 @@ function MPlayer() {
 
 	r.preload = function (url, full) {
 		var t0 = Date.now(),
-			fname = uricom_dec(url.split('/').pop());
+			fname = uricom_dec(url.split('/').pop().split('?')[0]);
 
 		url = addq(mpl.acode(url), 'cache=987&_=' + ACB);
 		mpl.preload_url = full ? url : null;
@@ -3275,6 +3299,21 @@ function scan_hash(v) {
 function eval_hash() {
 	window.onpopstate = treectl.onpopfun;
 
+	if (hash0 && window.og_fn) {
+		var all = msel.getall(), mi;
+		for (var a = 0; a < all.length; a++)
+			if (og_fn == uricom_dec(vsplit(all[a].vp)[1].split('?')[0])) {
+				mi = all[a];
+				break;
+			}
+
+		if (mi && img_re.exec(og_fn))
+			hash0 = '#g' + mi.id;
+
+		if (ebi('a' + mi.id))
+			hash0 = '#a' + mi.id;
+	}
+
 	var v = hash0;
 	hash0 = null;
 	if (!v)
@@ -3588,6 +3627,7 @@ var fileman = (function () {
 		bdel = ebi('fdel'),
 		bcut = ebi('fcut'),
 		bpst = ebi('fpst'),
+		t_paste,
 		r = {};
 
 	r.clip = null;
@@ -4065,8 +4105,73 @@ var fileman = (function () {
 		}
 	};
 
-	r.paste = function (e) {
+	document.onpaste = function (e) {
-		ev(e);
+		var xfer = e.clipboardData || window.clipboardData;
+		if (!xfer || !xfer.files || !xfer.files.length)
+			return;
+
+		var files = [];
+		for (var a = 0, aa = xfer.files.length; a < aa; a++)
+			files.push(xfer.files[a]);
+
+		clearTimeout(t_paste);
+
+		if (!r.clip.length)
+			return r.clip_up(files);
+
+		var src = r.clip.length == 1 ? r.clip[0] : vsplit(r.clip[0])[0],
+			msg = L.fp_both_m.format(r.clip.length, src, files.length);
+
+		modal.confirm(msg, r.paste, function () { r.clip_up(files); }, null, L.fp_both_b);
+	};
+
+	r.clip_up = function (files) {
+		goto_up2k();
+		var good = [], nil = [], bad = [];
+		for (var a = 0, aa = files.length; a < aa; a++) {
+			var fobj = files[a], dst = good;
+			try {
+				if (fobj.size < 1)
+					dst = nil;
+			}
+			catch (ex) {
+				dst = bad;
+			}
+			dst.push([fobj, fobj.name]);
+		}
+		var doit = function (is_img) {
+			jwrite('fman_clip', [Date.now()]);
+			r.clip = [];
+
+			var x = up2k.uc.ask_up;
+			if (is_img)
+				up2k.uc.ask_up = false;
+
+			up2k.gotallfiles[0](good, nil, bad, up2k.gotallfiles.slice(1));
+			up2k.uc.ask_up = x;
+		};
+		if (good.length != 1)
+			return doit();
+
+		var fn = good[0][1],
+			ofs = fn.lastIndexOf('.');
+
+		// stop linux-chrome from adding the fs-path into the <input>
+		setTimeout(function () {
+			modal.prompt(L.fp_name, fn, function (v) {
+				good[0][1] = v;
+				doit(true);
+			}, null, null, 0, ofs > 0 ? ofs : undefined);
+		}, 1);
+	};
+
+	r.d_paste = function () {
+		// gets called before onpaste; defer
+		clearTimeout(t_paste);
+		t_paste = setTimeout(r.paste, 50);
+	};
+
+	r.paste = function () {
 		if (clgot(bpst, 'hide'))
 			return toast.err(3, L.fp_eperm);
 
@@ -4676,7 +4781,7 @@ var thegrid = (function () {
 			aplay = ebi('a' + fid),
 			atext = ebi('t' + fid),
 			is_txt = atext && showfile.getlang(href),
-			is_img = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i.test(href),
+			is_img = img_re.test(href),
 			is_dir = href.endsWith('/'),
 			is_srch = !!ebi('unsearch'),
 			in_tree = is_dir && treectl.find(oth.textContent.slice(0, -1)),
@@ -5170,6 +5275,9 @@ document.onkeydown = function (e) {
 		if (ebi('hkhelp'))
 			return qsr('#hkhelp');
 
+		if (toast.visible)
+			return toast.hide();
+
 		if (ebi('rn_cancel'))
 			return ebi('rn_cancel').click();
 
@@ -5242,7 +5350,7 @@ document.onkeydown = function (e) {
 			return fileman.cut();
 
 		if (k == 'KeyV' || k == 'v')
-			return fileman.paste();
+			return fileman.d_paste();
 
 		if (k == 'KeyK' || k == 'k')
 			return fileman.delete();
@@ -6278,6 +6386,9 @@ var treectl = (function () {
 			lg1 = res.logues ? res.logues[1] || "" : "",
 			dirchg = get_evpath() != cdir;
 
+		if (lg1 === Ls.eng.f_empty)
+			lg1 = L.f_empty;
+
 		sandbox(ebi('pro'), sb_lg, '', lg0);
 		if (dirchg)
 			sandbox(ebi('epi'), sb_lg, '', lg1);
@@ -6379,8 +6490,9 @@ var treectl = (function () {
 					'" class="doc' + (lang ? ' bri' : '') +
 					'" hl="' + id + '" name="' + hname + '">-txt-</a>';
 
-			var ln = ['<tr><td>' + tn.lead + '</td><td><a href="' +
+			var cl = /\.PARTIAL$/.exec(fname) ? ' class="fade"' : '',
-				top + tn.href + '" id="' + id + '">' + hname + '</a>', tn.sz];
+				ln = ['<tr' + cl + '><td>' + tn.lead + '</td><td><a href="' +
+					top + tn.href + '" id="' + id + '">' + hname + '</a>', tn.sz];
 
 			for (var b = 0; b < res.taglist.length; b++) {
 				var k = res.taglist[b],
@@ -7914,6 +8026,9 @@ window.addEventListener("message", function (e) {
 
 
 if (sb_lg && logues.length) {
+	if (logues[1] === Ls.eng.f_empty)
+		logues[1] = L.f_empty;
+
 	sandbox(ebi('pro'), sb_lg, '', logues[0]);
 	sandbox(ebi('epi'), sb_lg, '', logues[1]);
 }
@@ -8090,7 +8205,17 @@ var unpost = (function () {
 			if (!links.length)
 				continue;
 
-			req.push(uricom_dec(r.files[a].vp.split('?')[0]));
+			var f = r.files[a];
+			if (f.k == 'u') {
+				var vp = vsplit(f.vp.split('?')[0]),
+					dfn = uricom_dec(vp[1]);
+				for (var iu = 0; iu < up2k.st.files.length; iu++) {
+					var uf = up2k.st.files[iu];
+					if (uf.name == dfn && uf.purl == vp[0])
+						return modal.alert(L.un_uf5);
+				}
+			}
+			req.push(uricom_dec(f.vp.split('?')[0]));
 			for (var b = 0; b < links.length; b++) {
 				links[b].removeAttribute('href');
 				links[b].innerHTML = '[busy]';
@@ -8213,6 +8338,13 @@ ebi('files').onclick = ebi('docul').onclick = function (e) {
 		treectl.reqls(tgt.getAttribute('href'), true);
 		return ev(e);
 	}
+	if (tgt && /\.PARTIAL(\?|$)/.exec('' + tgt.getAttribute('href')) && !window.partdlok) {
+		ev(e);
+		modal.confirm(L.f_partial, function () {
+			window.partdlok = 1;
+			tgt.click();
+		}, null);
+	}
 
 	tgt = e.target.closest('a[hl]');
 	if (tgt) {

copyparty/web/md.html

@@ -3,7 +3,7 @@
 	<title>📝 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
 	{%- if edit %}

copyparty/web/mde.html

@@ -3,7 +3,7 @@
 	<title>📝 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<link rel="stylesheet" href="{{ r }}/.cpr/mde.css?_={{ ts }}">
 	<link rel="stylesheet" href="{{ r }}/.cpr/deps/mini-fa.css?_={{ ts }}">

copyparty/web/msg.html

@@ -6,7 +6,7 @@
 	<title>{{ s_doctitle }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
 {{ html_head }}
 </head>

copyparty/web/splash.css

@@ -190,11 +190,21 @@ input {
 	padding: .5em .7em;
 	margin: 0 .5em 0 0;
 }
+input::placeholder {
+	font-size: 1.2em;
+	font-style: italic;
+	letter-spacing: .04em;
+	opacity: 0.64;
+	color: #930;
+}
 html.z input {
 	color: #fff;
 	background: #626;
 	border-color: #c2c;
 }
+html.z input::placeholder {
+	color: #fff;
+}
 html.z .num {
 	border-color: #777;
 }

copyparty/web/splash.html

@@ -6,7 +6,7 @@
 	<title>{{ s_doctitle }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 {{ html_head }}
@@ -94,7 +94,8 @@
 		<div>
 			<form method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
 				<input type="hidden" name="act" value="login" />
-				<input type="password" name="cppwd" />
+				<input type="password" name="cppwd" placeholder=" password" />
+				<input type="hidden" name="uhash" id="uhash" value="x" />
 				<input type="submit" value="Login" />
 				{% if ahttps %}
 				<a id="w" href="{{ ahttps }}">switch to https</a>

copyparty/web/splash.js

@@ -66,3 +66,5 @@ if (!ebi('c') && o.offsetTop + o.offsetHeight < window.innerHeight)
 o = ebi('u');
 if (o && /[0-9]+$/.exec(o.innerHTML))
 	o.innerHTML = shumantime(o.innerHTML);
+
+ebi('uhash').value = '' + location.hash;

copyparty/web/svcs.html

@@ -6,7 +6,7 @@
 	<title>{{ s_doctitle }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<meta name="theme-color" content="#333">
+	<meta name="theme-color" content="#{{ tcolor }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
 	<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
 	<style>ul{padding-left:1.3em}li{margin:.4em 0}</style>

copyparty/web/util.js

@@ -148,6 +148,8 @@ try {
     hook('error');
 }
 catch (ex) {
+    if (console.stdlog)
+        console.log = console.stdlog;
     console.log('console capture failed', ex);
 }
 var crashed = false, ignexd = {}, evalex_fatal = false;
@@ -1537,6 +1539,8 @@ var modal = (function () {
         cb_up = null,
         cb_ok = null,
         cb_ng = null,
+        sel_0 = 0,
+        sel_1 = 0,
         tok, tng, prim, sec, ok_cancel;
 
     r.load = function () {
@@ -1570,7 +1574,7 @@ var modal = (function () {
         (inp || a).focus();
         if (inp)
             setTimeout(function () {
-                inp.setSelectionRange(0, inp.value.length, "forward");
+                inp.setSelectionRange(sel_0, sel_1, "forward");
             }, 0);
 
         document.addEventListener('focus', onfocus);
@@ -1693,16 +1697,18 @@ var modal = (function () {
         r.show(html);
     }
 
-    r.prompt = function (html, v, cok, cng, fun) {
+    r.prompt = function (html, v, cok, cng, fun, so0, so1) {
         q.push(function () {
-            _prompt(lf2br(html), v, cok, cng, fun);
+            _prompt(lf2br(html), v, cok, cng, fun, so0, so1);
         });
         next();
     }
-    var _prompt = function (html, v, cok, cng, fun) {
+    var _prompt = function (html, v, cok, cng, fun, so0, so1) {
         cb_ok = cok;
         cb_ng = cng === undefined ? cok : null;
         cb_up = fun;
+        sel_0 = so0 || 0;
+        sel_1 = so1 === undefined ? v.length : so1;
         html += '<input id="modali" type="text" ' + NOAC + ' /><div id="modalb">' + ok_cancel + '</div>';
         r.show(html);
 

docs/changelog.md

@@ -1,3 +1,46 @@
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0412-2110  `v1.12.2`  ie11 fix
+
+## new features
+
+* new option `--bauth-last` for when you're hosting other [basic-auth](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication) services on the same domain 7b94e4ed
+  * makes it possible to log into copyparty as intended, but it still sees the passwords from the other service until you do
+  * alternatively, the other new option `--no-bauth` entirely disables basic-auth support, but that also kills [the android app](https://github.com/9001/party-up)
+
+## bugfixes
+
+* internet explorer isn't working?! FIX IT!!! 9e5253ef
+* audio transcoding was buggy with filekeys enabled b8733653
+* on windows, theoretical chance that antivirus could interrupt renaming files, so preemptively guard against that c8e3ed3a
+
+## other changes
+
+* add a "password" placeholder on the login page since you might think it's asking for a username da26ec36
+* config buttons were jank on iOS b772a4f8
+* readme: [making your homeserver accessible from the internet](https://github.com/9001/copyparty#at-home)
+
+
+
+▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
+# 2024-0409-2334  `v1.12.1`  scrolling stuff
+
+## new features
+
+* while viewing pictures/videos, the scrollwheel can be used to view the prev/next file 844d16b9
+
+## bugfixes
+
+* #81 (scrolling suddenly getting disabled) properly fixed after @icxes found another way to reproduce it (thx) 4f0cad54
+* and fixed at least one javascript glitch introduced in v1.12.0 while adding dirkeys 989cc613
+  * directory tree sidebar could fail to render when popping browser history into the lightbox
+
+## other changes
+
+* music preloader is slightly less hyper f89de6b3
+* u2c.exe: updated TLS-certs and deps ab18893c
+
+
+
 ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀  
 # 2024-0406-2011  `v1.12.0`  locksmith
 

docs/devnotes.md

@@ -22,7 +22,6 @@
     * [complete release](#complete-release)
 * [debugging](#debugging)
     * [music playback halting on phones](#music-playback-halting-on-phones) - mostly fine on android
-* [todo](#todo) - roughly sorted by priority
     * [discarded ideas](#discarded-ideas)
 
 
@@ -312,17 +311,11 @@ mostly fine on android,  but still haven't find a way to massage iphones into be
 * conditionally starting/stopping mp.fau according to mp.au.readyState <3 or <4 doesn't help
 * loop=true doesn't work, and manually looping mp.fau from an onended also doesn't work (it does nothing)
 * assigning fau.currentTime in a timer doesn't work, as safari merely pretends to assign it
+* on ios 16.7.7, mp.fau can sometimes make everything visibly work correctly, but no audio is actually hitting the speakers
 
 can be reproduced with `--no-sendfile --s-wr-sz 8192 --s-wr-slp 0.3 --rsp-slp 6` and then play a collection of small audio files with the screen off, `ffmpeg -i track01.cdda.flac -c:a libopus -b:a 128k -segment_time 12 -f segment smol-%02d.opus`
 
 
-# todo
-
-roughly sorted by priority
-
-* nothing! currently
-
-
 ## discarded ideas
 
 * optimization attempts which didn't improve performance

docs/nuitka.txt

@@ -1,82 +1,71 @@
 # recipe for building an exe with nuitka (extreme jank edition)
-#
+
-# NOTE: win7 and win10 builds both work on win10 but
+NOTE: copyparty runs SLOWER when compiled with nuitka;
-#   on win7 they immediately c0000005 in kernelbase.dll
+  just use copyparty-sfx.py and/or pyinstaller instead
-#
+
-# first install python-3.6.8-amd64.exe
+  ( the sfx and the pyinstaller EXEs are equally fast if you
-#   [x] add to path
+  have the latest jinja2 installed, but the older jinja that
-#
+  comes bundled with the sfx is slightly faster yet )
+
+  roughly, copyparty-sfx.py is 6% faster than copyparty.exe
+  (win10-pyinstaller), and copyparty.exe is 10% faster than
+  nuitka, making copyparty-sfx.py 17% faster than nuitka
+
+NOTE: every time a nuitka-compiled copyparty.exe is launched,
+  it will show the windows firewall prompt since nuitka will
+  pick a new unique location in %TEMP% to unpack an exe into,
+  unlike pyinstaller which doesn't fork itself on startup...
+  might be fixable by configuring nuitka differently, idk
+
+NOTE: nuitka EXEs are larger than pyinstaller ones;
+  a minimal nuitka build of just the sfx (with its bundled
+  dependencies) was already the same size as the pyinstaller
+  copyparty.exe which also includes Mutagen and Pillow
+
+NOTE: nuitka takes a lot longer to build than pyinstaller
+  (due to actual compilation of course, but still)
+
+NOTE: binaries built with nuitka cannot run on windows7,
+  even when compiled with python 3.6 on windows 7 itself
+
+NOTE: `--python-flags=-m` is the magic sauce to
+  correctly compile `from .util import Daemon`
+  (which otherwise only explodes at runtime)
+
+NOTE: `--deployment` doesn't seem to affect performance
+
+########################################################################
 # copypaste the rest of this file into cmd
 
-rem from pypi
-cd \users\ed\downloads
-python -m pip install --user Nuitka-0.6.14.7.tar.gz
 
-rem https://github.com/brechtsanders/winlibs_mingw/releases/download/10.2.0-11.0.0-8.0.0-r5/winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
-mkdir C:\Users\ed\AppData\Local\Nuitka\
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\
-copy c:\users\ed\downloads\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
 
-rem https://github.com/ccache/ccache/releases/download/v3.7.12/ccache-3.7.12-windows-32.zip
+python -m pip install --user -U nuitka
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\
-copy c:\users\ed\downloads\ccache-3.7.12-windows-32.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\ccache-3.7.12-windows-32.zip
 
-rem https://dependencywalker.com/depends22_x64.zip
+cd %homedrive%
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\
+cd %homepath%\downloads
-mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\
-copy c:\users\ed\downloads\depends22_x64.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\depends22_x64.zip
 
-cd \
+rd /s /q copypuitka
-rd /s /q %appdata%\..\local\temp\pe-copyparty
+mkdir copypuitka
-cd \users\ed\downloads
+cd copypuitka
-python copyparty-sfx.py -h
-cd %appdata%\..\local\temp\pe-copyparty\copyparty
 
-python
+rd /s /q %temp%\pe-copyparty
-import os, re
+python ..\copyparty-sfx.py --version
-os.rename('../dep-j2/jinja2', '../jinja2')
-os.rename('../dep-j2/markupsafe', '../markupsafe')
 
-print("# nuitka dies if .__init__.stuff is imported")
+move %temp%\pe-copyparty\copyparty .\
-with open('__init__.py','r',encoding='utf-8') as f:
+move %temp%\pe-copyparty\partftpy .\
- t1 = f.read()
+move %temp%\pe-copyparty\ftp\pyftpdlib .\
+move %temp%\pe-copyparty\j2\jinja2 .\
+move %temp%\pe-copyparty\j2\markupsafe .\
 
-with open('util.py','r',encoding='utf-8') as f:
+rd /s /q %temp%\pe-copyparty
- t2 = f.read().split('\n')[3:]
 
-t2 = [x for x in t2 if 'from .__init__' not in x]
+python -m nuitka ^
-t = t1 + '\n'.join(t2)
+  --onefile --deployment --python-flag=-m ^
-with open('__init__.py','w',encoding='utf-8') as f:
+  --include-package=markupsafe ^
- f.write('\n')
+  --include-package=jinja2 ^
+  --include-package=partftpy ^
+  --include-package=pyftpdlib ^
+  --include-data-dir=copyparty\web=copyparty\web ^
+  --include-data-dir=copyparty\res=copyparty\res ^
+  --run copyparty
 
-with open('util.py','w',encoding='utf-8') as f:
- f.write(t)
-
-print("# local-imports fail, prefix module names")
-ptn = re.compile(r'^( *from )(\.[^ ]+ import .*)')
-for d, _, fs in os.walk('.'):
- for f in fs:
-  fp = os.path.join(d, f)
-  if not fp.endswith('.py'):
-   continue
-  t = ''
-  with open(fp,'r',encoding='utf-8') as f:
-   for ln in [x.rstrip('\r\n') for x in f]:
-    m = ptn.match(ln)
-    if not m:
-     t += ln + '\n'
-     continue
-    p1, p2 = m.groups()
-    t += "{}copyparty{}\n".format(p1, p2).replace("__init__", "util")
-  with open(fp,'w',encoding='utf-8') as f:
-   f.write(t)
-
-exit()
-
-cd ..
-
-rd /s /q bout & python -m nuitka --standalone --onefile --windows-onefile-tempdir --python-flag=no_site --assume-yes-for-downloads --include-data-dir=copyparty\web=copyparty\web --include-data-dir=copyparty\res=copyparty\res --run --output-dir=bout --mingw64 --include-package=markupsafe --include-package=jinja2 copyparty

docs/rice/README.md

@@ -47,3 +47,11 @@ and if you want to have a monospace font in the fancy markdown editor, do this:
 
 NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative
 
+
+# `<head>`
+
+to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
+
+if you give it the value `@ASDF` it will try to open a file named ASDF and send the text within
+
+if the value starts with `%` it will assume a jinja2 template and expand it; the template has access to the `HttpCli` object through a property named `this` as well as everything in `j2a` and the stuff added by `self.j2s`; see [browser.html](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.html) for inspiration or look under the hood in [httpcli.py](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/httpcli.py)

docs/versus.md

@@ -48,6 +48,7 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
     * [filebrowser](#filebrowser)
     * [filegator](#filegator)
     * [sftpgo](#sftpgo)
+    * [arozos](#arozos)
     * [updog](#updog)
     * [goshs](#goshs)
     * [gimme-that](#gimme-that)
@@ -93,6 +94,7 @@ the softwares,
 * `j` = [filebrowser](https://github.com/filebrowser/filebrowser)
 * `k` = [filegator](https://github.com/filegator/filegator)
 * `l` = [sftpgo](https://github.com/drakkan/sftpgo)
+* `m` = [arozos](https://github.com/tobychui/arozos)
 
 some softwares not in the matrixes,
 * [updog](#updog)
@@ -113,22 +115,22 @@ symbol legend,
 
 ## general
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| intuitive UX            |   | ╱ | █ | █ | █ |   | █ | █ | █ | █ | █ | █ |
+| intuitive UX            |   | ╱ | █ | █ | █ |   | █ | █ | █ | █ | █ | █ | █ |
-| config GUI              |   | █ | █ | █ | █ |   |   | █ | █ | █ |   | █ |
+| config GUI              |   | █ | █ | █ | █ |   |   | █ | █ | █ |   | █ | █ |
-| good documentation      |   |   |   | █ | █ | █ | █ |   |   | █ | █ | ╱ |
+| good documentation      |   |   |   | █ | █ | █ | █ |   |   | █ | █ | ╱ | ╱ |
-| runs on iOS             | ╱ |   |   |   |   | ╱ |   |   |   |   |   |   |
+| runs on iOS             | ╱ |   |   |   |   | ╱ |   |   |   |   |   |   |   |
-| runs on Android         | █ |   |   |   |   | █ |   |   |   |   |   |   |
+| runs on Android         | █ |   |   |   |   | █ |   |   |   |   |   |   |   |
-| runs on WinXP           | █ | █ |   |   |   | █ |   |   |   |   |   |   |
+| runs on WinXP           | █ | █ |   |   |   | █ |   |   |   |   |   |   |   |
-| runs on Windows         | █ | █ | █ | █ | █ | █ | █ | ╱ | █ | █ | █ | █ |
+| runs on Windows         | █ | █ | █ | █ | █ | █ | █ | ╱ | █ | █ | █ | █ | ╱ |
-| runs on Linux           | █ | ╱ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
+| runs on Linux           | █ | ╱ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
-| runs on Macos           | █ |   | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
+| runs on Macos           | █ |   | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |   |
-| runs on FreeBSD         | █ |   |   | • | █ | █ | █ | • | █ | █ |   | █ |
+| runs on FreeBSD         | █ |   |   | • | █ | █ | █ | • | █ | █ |   | █ |   |
-| portable binary         | █ | █ | █ |   |   | █ | █ |   |   | █ |   | █ |
+| portable binary         | █ | █ | █ |   |   | █ | █ |   |   | █ |   | █ | █ |
-| zero setup, just go     | █ | █ | █ |   |   | ╱ | █ |   |   | █ |   | ╱ |
+| zero setup, just go     | █ | █ | █ |   |   | ╱ | █ |   |   | █ |   | ╱ | █ |
-| android app             | ╱ |   |   | █ | █ |   |   |   |   |   |   |   |
+| android app             | ╱ |   |   | █ | █ |   |   |   |   |   |   |   |   |
-| iOS app                 | ╱ |   |   | █ | █ |   |   |   |   |   |   |   |
+| iOS app                 | ╱ |   |   | █ | █ |   |   |   |   |   |   |   |   |
 
 * `zero setup` = you can get a mostly working setup by just launching the app, without having to install any software or configure whatever
 * `a`/copyparty remarks:
@@ -140,37 +142,39 @@ symbol legend,
 * `f`/rclone must be started with the command `rclone serve webdav .` or similar
 * `h`/chibisafe has undocumented windows support
 * `i`/sftpgo must be launched with a command
+* `m`/arozos has partial windows support
 
 
 ## file transfer
 
 *the thing that copyparty is actually kinda good at*
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| download folder as zip  | █ | █ | █ | █ | ╱ |   | █ |   | █ | █ | ╱ | █ |
+| download folder as zip  | █ | █ | █ | █ | ╱ |   | █ |   | █ | █ | ╱ | █ | ╱ |
-| download folder as tar  | █ |   |   |   |   |   |   |   |   | █ |   |   |
+| download folder as tar  | █ |   |   |   |   |   |   |   |   | █ |   |   |   |
-| upload                  | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
+| upload                  | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
-| parallel uploads        | █ |   |   | █ | █ |   | • |   | █ |   | █ |   |
+| parallel uploads        | █ |   |   | █ | █ |   | • |   | █ |   | █ |   | █ |
-| resumable uploads       | █ |   |   |   |   |   |   |   | █ |   | █ | ╱ |
+| resumable uploads       | █ |   |   |   |   |   |   |   | █ |   | █ | ╱ |   |
-| upload segmenting       | █ |   |   |   |   |   |   | █ | █ |   | █ | ╱ |
+| upload segmenting       | █ |   |   |   |   |   |   | █ | █ |   | █ | ╱ | █ |
-| upload acceleration     | █ |   |   |   |   |   |   |   | █ |   | █ |   |
+| upload acceleration     | █ |   |   |   |   |   |   |   | █ |   | █ |   |   |
-| upload verification     | █ |   |   | █ | █ |   |   |   | █ |   |   |   |
+| upload verification     | █ |   |   | █ | █ |   |   |   | █ |   |   |   |   |
-| upload deduplication    | █ |   |   |   | █ |   |   |   | █ |   |   |   |
+| upload deduplication    | █ |   |   |   | █ |   |   |   | █ |   |   |   |   |
-| upload a 999 TiB file   | █ |   |   |   | █ | █ | • |   | █ |   | █ | ╱ |
+| upload a 999 TiB file   | █ |   |   |   | █ | █ | • |   | █ |   | █ | ╱ | ╱ |
-| keep last-modified time | █ |   |   | █ | █ | █ |   |   |   |   |   | █ |
+| race the beam ("p2p")   | █ |   |   |   |   |   |   |   |   | • |   |   |   |
-| upload rules            | ╱ | ╱ | ╱ | ╱ | ╱ |   |   | ╱ | ╱ |   | ╱ | ╱ |
+| keep last-modified time | █ |   |   | █ | █ | █ |   |   |   |   |   | █ |   |
-| ┗ max disk usage        | █ | █ |   |   | █ |   |   |   | █ |   |   | █ |
+| upload rules            | ╱ | ╱ | ╱ | ╱ | ╱ |   |   | ╱ | ╱ |   | ╱ | ╱ | ╱ |
-| ┗ max filesize          | █ |   |   |   |   |   |   | █ |   |   | █ | █ |
+| ┗ max disk usage        | █ | █ |   |   | █ |   |   |   | █ |   |   | █ | █ |
-| ┗ max items in folder   | █ |   |   |   |   |   |   |   |   |   |   | ╱ |
+| ┗ max filesize          | █ |   |   |   |   |   |   | █ |   |   | █ | █ | █ |
-| ┗ max file age          | █ |   |   |   |   |   |   |   | █ |   |   |   |
+| ┗ max items in folder   | █ |   |   |   |   |   |   |   |   |   |   | ╱ |   |
-| ┗ max uploads over time | █ |   |   |   |   |   |   |   |   |   |   | ╱ |
+| ┗ max file age          | █ |   |   |   |   |   |   |   | █ |   |   |   |   |
-| ┗ compress before write | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ max uploads over time | █ |   |   |   |   |   |   |   |   |   |   | ╱ |   |
-| ┗ randomize filename    | █ |   |   |   |   |   |   | █ | █ |   |   |   |
+| ┗ compress before write | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| ┗ mimetype reject-list  | ╱ |   |   |   |   |   |   |   | • | ╱ |   | ╱ |
+| ┗ randomize filename    | █ |   |   |   |   |   |   | █ | █ |   |   |   |   |
-| ┗ extension reject-list | ╱ |   |   |   |   |   |   | █ | • | ╱ |   | ╱ |
+| ┗ mimetype reject-list  | ╱ |   |   |   |   |   |   |   | • | ╱ |   | ╱ | • |
-| checksums provided      |   |   |   | █ | █ |   |   |   | █ | ╱ |   |   |
+| ┗ extension reject-list | ╱ |   |   |   |   |   |   | █ | • | ╱ |   | ╱ | • |
-| cloud storage backend   | ╱ | ╱ | ╱ | █ | █ | █ | ╱ |   |   | ╱ | █ | █ |
+| checksums provided      |   |   |   | █ | █ |   |   |   | █ | ╱ |   |   |   |
+| cloud storage backend   | ╱ | ╱ | ╱ | █ | █ | █ | ╱ |   |   | ╱ | █ | █ | ╱ |
 
 * `upload segmenting` = files are sliced into chunks, making it possible to upload files larger than 100 MiB on cloudflare for example
 
@@ -178,6 +182,8 @@ symbol legend,
 
 * `upload verification` = uploads are checksummed or otherwise confirmed to have been transferred correctly
 
+* `race the beam` = files can be downloaded while they're still uploading; downloaders are slowed down such that the uploader is always ahead
+
 * `checksums provided` = when downloading a file from the server, the file's checksum is provided for verification client-side
 
 * `cloud storage backend` = able to serve files from (and write to) s3 or similar cloud services; `╱` means the software can do this with some help from `rclone mount` as a bridge
@@ -192,26 +198,27 @@ symbol legend,
   * resumable/segmented uploads only over SFTP, not over HTTP
   * upload rules are totals only, not over time
   * can probably do extension/mimetype rejection similar to copyparty
+* `m`/arozos download-as-zip is not streaming; it creates the full zipfile before download can start, and fails on big folders
 
 
 ## protocols and client support
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| serve https             | █ |   | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
+| serve https             | █ |   | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
-| serve webdav            | █ |   |   | █ | █ | █ | █ |   | █ |   |   | █ |
+| serve webdav            | █ |   |   | █ | █ | █ | █ |   | █ |   |   | █ | █ |
-| serve ftp  (tcp)        | █ |   |   |   |   | █ |   |   |   |   |   | █ |
+| serve ftp  (tcp)        | █ |   |   |   |   | █ |   |   |   |   |   | █ | █ |
-| serve ftps (tls)        | █ |   |   |   |   | █ |   |   |   |   |   | █ |
+| serve ftps (tls)        | █ |   |   |   |   | █ |   |   |   |   |   | █ |   |
-| serve tftp (udp)        | █ |   |   |   |   |   |   |   |   |   |   |   |
+| serve tftp (udp)        | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| serve sftp (ssh)        |   |   |   |   |   | █ |   |   |   |   |   | █ |
+| serve sftp (ssh)        |   |   |   |   |   | █ |   |   |   |   |   | █ | █ |
-| serve smb/cifs          | ╱ |   |   |   |   | █ |   |   |   |   |   |   |
+| serve smb/cifs          | ╱ |   |   |   |   | █ |   |   |   |   |   |   |   |
-| serve dlna              |   |   |   |   |   | █ |   |   |   |   |   |   |
+| serve dlna              |   |   |   |   |   | █ |   |   |   |   |   |   |   |
-| listen on unix-socket   |   |   |   | █ | █ |   | █ | █ | █ |   | █ | █ |
+| listen on unix-socket   |   |   |   | █ | █ |   | █ | █ | █ |   | █ | █ |   |
-| zeroconf                | █ |   |   |   |   |   |   |   |   |   |   |   |
+| zeroconf                | █ |   |   |   |   |   |   |   |   |   |   |   | █ |
-| supports netscape 4     | ╱ |   |   |   |   | █ |   |   |   |   | • |   |
+| supports netscape 4     | ╱ |   |   |   |   | █ |   |   |   |   | • |   | ╱ |
-| ...internet explorer 6  | ╱ | █ |   | █ |   | █ |   |   |   |   | • |   |
+| ...internet explorer 6  | ╱ | █ |   | █ |   | █ |   |   |   |   | • |   | ╱ |
-| mojibake filenames      | █ |   |   | • | • | █ | █ | • | • | • |   | ╱ |
+| mojibake filenames      | █ |   |   | • | • | █ | █ | • | █ | • |   | ╱ |   |
-| undecodable filenames   | █ |   |   | • | • | █ |   | • | • |   |   | ╱ |
+| undecodable filenames   | █ |   |   | • | • | █ |   | • |   |   |   | ╱ |   |
 
 * `webdav` = protocol convenient for mounting a remote server as a local filesystem; see zeroconf:
 * `zeroconf` = the server announces itself on the LAN, [automatically appearing](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png) on other zeroconf-capable devices
@@ -222,61 +229,66 @@ symbol legend,
   * extremely minimal samba/cifs server
   * netscape 4 / ie6 support is mostly listed as a joke altho some people have actually found it useful ([ie4 tho](https://user-images.githubusercontent.com/241032/118192791-fb31fe00-b446-11eb-9647-898ea8efc1f7.png))
 * `l`/sftpgo translates mojibake filenames into valid utf-8 (information loss)
+* `m`/arozos has readonly-support for older browsers; no uploading
 
 
 ## server configuration
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| config from cmd args    | █ |   |   |   |   | █ | █ |   |   | █ |   | ╱ |
+| config from cmd args    | █ |   |   |   |   | █ | █ |   |   | █ |   | ╱ | ╱ |
-| config files            | █ | █ | █ | ╱ | ╱ | █ |   | █ |   | █ | • | ╱ |
+| config files            | █ | █ | █ | ╱ | ╱ | █ |   | █ |   | █ | • | ╱ | ╱ |
-| runtime config reload   | █ | █ | █ |   |   |   |   | █ | █ | █ | █ |   |
+| runtime config reload   | █ | █ | █ |   |   |   |   | █ | █ | █ | █ |   | █ |
-| same-port http / https  | █ |   |   |   |   |   |   |   |   |   |   |   |
+| same-port http / https  | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| listen multiple ports   | █ |   |   |   |   |   |   |   |   |   |   | █ |
+| listen multiple ports   | █ |   |   |   |   |   |   |   |   |   |   | █ |   |
-| virtual file system     | █ | █ | █ |   |   |   | █ |   |   |   |   | █ |
+| virtual file system     | █ | █ | █ |   |   |   | █ |   |   |   |   | █ |   |
-| reverse-proxy ok        | █ |   | █ | █ | █ | █ | █ | █ | • | • | • | █ |
+| reverse-proxy ok        | █ |   | █ | █ | █ | █ | █ | █ | • | • | • | █ | ╱ |
-| folder-rproxy ok        | █ |   |   |   | █ | █ |   | • | • | • | • |   |
+| folder-rproxy ok        | █ |   |   |   | █ | █ |   | • | • | • | • |   | • |
 
 * `folder-rproxy` = reverse-proxying without dedicating an entire (sub)domain, using a subfolder instead
 * `l`/sftpgo:
   * config: users must be added through gui / api calls
+* `m`/arozos:
+  * configuration is primarily through GUI
+  * reverse-proxy is not guaranteed to see the correct client IP
 
 
 ## server capabilities
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| accounts                | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
+| accounts                | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
-| per-account chroot      |   |   |   |   |   |   |   |   |   |   |   | █ |
+| per-account chroot      |   |   |   |   |   |   |   |   |   |   |   | █ |   |
-| single-sign-on          | ╱ |   |   | █ | █ |   |   |   | • |   |   |   |
+| single-sign-on          | ╱ |   |   | █ | █ |   |   |   | • |   |   |   |   |
-| token auth              | ╱ |   |   | █ | █ |   |   | █ |   |   |   |   |
+| token auth              | ╱ |   |   | █ | █ |   |   | █ |   |   |   |   | █ |
-| 2fa                     | ╱ |   |   | █ | █ |   |   |   |   |   |   | █ |
+| 2fa                     | ╱ |   |   | █ | █ |   |   |   |   |   |   | █ | ╱ |
-| per-volume permissions  | █ | █ | █ | █ | █ | █ | █ |   | █ | █ | ╱ | █ |
+| per-volume permissions  | █ | █ | █ | █ | █ | █ | █ |   | █ | █ | ╱ | █ | █ |
-| per-folder permissions  | ╱ |   |   | █ | █ |   | █ |   | █ | █ | ╱ | █ |
+| per-folder permissions  | ╱ |   |   | █ | █ |   | █ |   | █ | █ | ╱ | █ | █ |
-| per-file permissions    |   |   |   | █ | █ |   | █ |   | █ |   |   |   |
+| per-file permissions    |   |   |   | █ | █ |   | █ |   | █ |   |   |   | █ |
-| per-file passwords      | █ |   |   | █ | █ |   | █ |   | █ |   |   |   |
+| per-file passwords      | █ |   |   | █ | █ |   | █ |   | █ |   |   |   | █ |
-| unmap subfolders        | █ |   |   |   |   |   | █ |   |   | █ | ╱ | • |
+| unmap subfolders        | █ |   |   |   |   |   | █ |   |   | █ | ╱ | • |   |
-| index.html blocks list  | ╱ |   |   |   |   |   | █ |   |   | • |   |   |
+| index.html blocks list  | ╱ |   |   |   |   |   | █ |   |   | • |   |   |   |
-| write-only folders      | █ |   |   |   |   |   |   |   |   |   | █ | █ |
+| write-only folders      | █ |   |   |   |   |   |   |   |   |   | █ | █ |   |
-| files stored as-is      | █ | █ | █ | █ |   | █ | █ |   |   | █ | █ | █ |
+| files stored as-is      | █ | █ | █ | █ |   | █ | █ |   |   | █ | █ | █ | █ |
-| file versioning         |   |   |   | █ | █ |   |   |   |   |   |   |   |
+| file versioning         |   |   |   | █ | █ |   |   |   |   |   |   |   |   |
-| file encryption         |   |   |   | █ | █ | █ |   |   |   |   |   | █ |
+| file encryption         |   |   |   | █ | █ | █ |   |   |   |   |   | █ |   |
-| file indexing           | █ |   | █ | █ | █ |   |   | █ | █ | █ |   |   |
+| file indexing           | █ |   | █ | █ | █ |   |   | █ | █ | █ |   |   |   |
-| ┗ per-volume db         | █ |   | • | • | • |   |   | • | • |   |   |   |
+| ┗ per-volume db         | █ |   | • | • | • |   |   | • | • |   |   |   |   |
-| ┗ db stored in folder   | █ |   |   |   |   |   |   | • | • | █ |   |   |
+| ┗ db stored in folder   | █ |   |   |   |   |   |   | • | • | █ |   |   |   |
-| ┗ db stored out-of-tree | █ |   | █ | █ | █ |   |   | • | • | █ |   |   |
+| ┗ db stored out-of-tree | █ |   | █ | █ | █ |   |   | • | • | █ |   |   |   |
-| ┗ existing file tree    | █ |   | █ |   |   |   |   |   |   | █ |   |   |
+| ┗ existing file tree    | █ |   | █ |   |   |   |   |   |   | █ |   |   |   |
-| file action event hooks | █ |   |   |   |   |   |   |   |   | █ |   | █ |
+| file action event hooks | █ |   |   |   |   |   |   |   |   | █ |   | █ | • |
-| one-way folder sync     | █ |   |   | █ | █ | █ |   |   |   |   |   |   |
+| one-way folder sync     | █ |   |   | █ | █ | █ |   |   |   |   |   |   |   |
-| full sync               |   |   |   | █ | █ |   |   |   |   |   |   |   |
+| full sync               |   |   |   | █ | █ |   |   |   |   |   |   |   |   |
-| speed throttle          |   | █ | █ |   |   | █ |   |   | █ |   |   | █ |
+| speed throttle          |   | █ | █ |   |   | █ |   |   | █ |   |   | █ |   |
-| anti-bruteforce         | █ | █ | █ | █ | █ |   |   |   | • |   |   | █ |
+| anti-bruteforce         | █ | █ | █ | █ | █ |   |   |   | • |   |   | █ | • |
-| dyndns updater          |   | █ |   |   |   |   |   |   |   |   |   |   |
+| dyndns updater          |   | █ |   |   |   |   |   |   |   |   |   |   |   |
-| self-updater            |   |   | █ |   |   |   |   |   |   |   |   |   |
+| self-updater            |   |   | █ |   |   |   |   |   |   |   |   |   | █ |
-| log rotation            | █ |   | █ | █ | █ |   |   | • | █ |   |   | █ |
+| log rotation            | █ |   | █ | █ | █ |   |   | • | █ |   |   | █ | • |
-| upload tracking / log   | █ | █ | • | █ | █ |   |   | █ | █ |   |   | ╱ |
+| upload tracking / log   | █ | █ | • | █ | █ |   |   | █ | █ |   |   | ╱ | █ |
-| curl-friendly ls        | █ |   |   |   |   |   |   |   |   |   |   |   |
+| prometheus metrics      | █ |   |   | █ |   |   |   |   |   |   |   | █ |   |
-| curl-friendly upload    | █ |   |   |   |   | █ | █ | • |   |   |   |   |
+| curl-friendly ls        | █ |   |   |   |   |   |   |   |   |   |   |   |   |
+| curl-friendly upload    | █ |   |   |   |   | █ | █ | • |   |   |   |   |   |
 
 * `unmap subfolders` = "shadowing"; mounting a local folder in the middle of an existing filesystem tree in order to disable access below that path
 * `files stored as-is` = uploaded files are trivially readable from the server HDD, not sliced into chunks or in weird folder structures or anything like that
@@ -302,49 +314,51 @@ symbol legend,
 * `l`/sftpgo:
   * `file action event hooks` also include on-download triggers
   * `upload tracking / log` in main logfile
+* `m`/arozos:
+  * `2fa` maybe possible through LDAP/Oauth
 
 
 ## client features
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------  | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------  | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| single-page app         | █ |   | █ | █ | █ |   |   | █ | █ | █ | █ |   |
+| single-page app         | █ |   | █ | █ | █ |   |   | █ | █ | █ | █ |   | █ |
-| themes                  | █ | █ |   | █ |   |   |   |   | █ |   |   |   |
+| themes                  | █ | █ |   | █ |   |   |   |   | █ |   |   |   |   |
-| directory tree nav      | █ | ╱ |   |   | █ |   |   |   | █ |   | ╱ |   |
+| directory tree nav      | █ | ╱ |   |   | █ |   |   |   | █ |   | ╱ |   |   |
-| multi-column sorting    | █ |   |   |   |   |   |   |   |   |   |   |   |
+| multi-column sorting    | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| thumbnails              | █ |   |   | ╱ | ╱ |   |   | █ | █ | ╱ |   |   |
+| thumbnails              | █ |   |   | ╱ | ╱ |   |   | █ | █ | ╱ |   |   | █ |
-| ┗ image thumbnails      | █ |   |   | █ | █ |   |   | █ | █ | █ |   |   |
+| ┗ image thumbnails      | █ |   |   | █ | █ |   |   | █ | █ | █ |   |   | █ |
-| ┗ video thumbnails      | █ |   |   | █ | █ |   |   |   | █ |   |   |   |
+| ┗ video thumbnails      | █ |   |   | █ | █ |   |   |   | █ |   |   |   | █ |
-| ┗ audio spectrograms    | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ audio spectrograms    | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| audio player            | █ |   |   | █ | █ |   |   |   | █ | ╱ |   |   |
+| audio player            | █ |   |   | █ | █ |   |   |   | █ | ╱ |   |   | █ |
-| ┗ gapless playback      | █ |   |   |   |   |   |   |   | • |   |   |   |
+| ┗ gapless playback      | █ |   |   |   |   |   |   |   | • |   |   |   |   |
-| ┗ audio equalizer       | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ audio equalizer       | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| ┗ waveform seekbar      | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ waveform seekbar      | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| ┗ OS integration        | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ OS integration        | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| ┗ transcode to lossy    | █ |   |   |   |   |   |   |   |   |   |   |   |
+| ┗ transcode to lossy    | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| video player            | █ |   |   | █ | █ |   |   |   | █ | █ |   |   |
+| video player            | █ |   |   | █ | █ |   |   |   | █ | █ |   |   | █ |
-| ┗ video transcoding     |   |   |   |   |   |   |   |   | █ |   |   |   |
+| ┗ video transcoding     |   |   |   |   |   |   |   |   | █ |   |   |   |   |
-| audio BPM detector      | █ |   |   |   |   |   |   |   |   |   |   |   |
+| audio BPM detector      | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| audio key detector      | █ |   |   |   |   |   |   |   |   |   |   |   |
+| audio key detector      | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| search by path / name   | █ | █ | █ | █ | █ |   | █ |   | █ | █ | ╱ |   |
+| search by path / name   | █ | █ | █ | █ | █ |   | █ |   | █ | █ | ╱ |   |   |
-| search by date / size   | █ |   |   |   | █ |   |   | █ | █ |   |   |   |
+| search by date / size   | █ |   |   |   | █ |   |   | █ | █ |   |   |   |   |
-| search by bpm / key     | █ |   |   |   |   |   |   |   |   |   |   |   |
+| search by bpm / key     | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| search by custom tags   |   |   |   |   |   |   |   | █ | █ |   |   |   |
+| search by custom tags   |   |   |   |   |   |   |   | █ | █ |   |   |   |   |
-| search in file contents |   |   |   | █ | █ |   |   |   | █ |   |   |   |
+| search in file contents |   |   |   | █ | █ |   |   |   | █ |   |   |   |   |
-| search by custom parser | █ |   |   |   |   |   |   |   |   |   |   |   |
+| search by custom parser | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| find local file         | █ |   |   |   |   |   |   |   |   |   |   |   |
+| find local file         | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| undo recent uploads     | █ |   |   |   |   |   |   |   |   |   |   |   |
+| undo recent uploads     | █ |   |   |   |   |   |   |   |   |   |   |   |   |
-| create directories      | █ |   |   | █ | █ | ╱ | █ | █ | █ | █ | █ | █ |
+| create directories      | █ |   |   | █ | █ | ╱ | █ | █ | █ | █ | █ | █ | █ |
-| image viewer            | █ |   |   | █ | █ |   |   |   | █ | █ | █ |   |
+| image viewer            | █ |   |   | █ | █ |   |   |   | █ | █ | █ |   | █ |
-| markdown viewer         | █ |   |   |   | █ |   |   |   | █ | ╱ | ╱ |   |
+| markdown viewer         | █ |   |   |   | █ |   |   |   | █ | ╱ | ╱ |   | █ |
-| markdown editor         | █ |   |   |   | █ |   |   |   | █ | ╱ | ╱ |   |
+| markdown editor         | █ |   |   |   | █ |   |   |   | █ | ╱ | ╱ |   | █ |
-| readme.md in listing    | █ |   |   | █ |   |   |   |   |   |   |   |   |
+| readme.md in listing    | █ |   |   | █ |   |   |   |   |   |   |   |   |   |
-| rename files            | █ | █ | █ | █ | █ | ╱ | █ |   | █ | █ | █ | █ |
+| rename files            | █ | █ | █ | █ | █ | ╱ | █ |   | █ | █ | █ | █ | █ |
-| batch rename            | █ |   |   |   |   |   |   |   | █ |   |   |   |
+| batch rename            | █ |   |   |   |   |   |   |   | █ |   |   |   |   |
-| cut / paste files       | █ | █ |   | █ | █ |   |   |   | █ |   |   |   |
+| cut / paste files       | █ | █ |   | █ | █ |   |   |   | █ |   |   |   | █ |
-| move files              | █ | █ |   | █ | █ |   | █ |   | █ | █ | █ |   |
+| move files              | █ | █ |   | █ | █ |   | █ |   | █ | █ | █ |   | █ |
-| delete files            | █ | █ |   | █ | █ | ╱ | █ | █ | █ | █ | █ | █ |
+| delete files            | █ | █ |   | █ | █ | ╱ | █ | █ | █ | █ | █ | █ | █ |
-| copy files              |   |   |   |   | █ |   |   |   | █ | █ | █ |   |
+| copy files              |   |   |   |   | █ |   |   |   | █ | █ | █ |   | █ |
 
 * `single-page app` = multitasking; possible to continue navigating while uploading
 * `audio player » os-integration` = use the [lockscreen](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) or [media hotkeys](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) to play/pause, prev/next song
@@ -360,14 +374,14 @@ symbol legend,
 
 ## integration
 
-| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l |
+| feature / software      | a | b | c | d | e | f | g | h | i | j | k | l | m |
-| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
+| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
-| OS alert on upload      | █ |   |   |   |   |   |   |   |   | ╱ |   | ╱ |
+| OS alert on upload      | █ |   |   |   |   |   |   |   |   | ╱ |   | ╱ |   |
-| discord                 | █ |   |   |   |   |   |   |   |   | ╱ |   | ╱ |
+| discord                 | █ |   |   |   |   |   |   |   |   | ╱ |   | ╱ |   |
-| ┗ announce uploads      | █ |   |   |   |   |   |   |   |   |   |   | ╱ |
+| ┗ announce uploads      | █ |   |   |   |   |   |   |   |   |   |   | ╱ |   |
-| ┗ custom embeds         |   |   |   |   |   |   |   |   |   |   |   | ╱ |
+| ┗ custom embeds         |   |   |   |   |   |   |   |   |   |   |   | ╱ |   |
-| sharex                  | █ |   |   | █ |   | █ | ╱ | █ |   |   |   |   |
+| sharex                  | █ |   |   | █ |   | █ | ╱ | █ |   |   |   |   |   |
-| flameshot               |   |   |   |   |   | █ |   |   |   |   |   |   |
+| flameshot               |   |   |   |   |   | █ |   |   |   |   |   |   |   |
 
 * sharex `╱` = yes, but does not provide example sharex config
 * `a`/copyparty remarks:
@@ -393,6 +407,7 @@ symbol legend,
 | filebrowser        | go     | █ apl2 |  20 MB |
 | filegator          | php    | █ mit  |    •   |
 | sftpgo             | go     | ‼ agpl |  44 MB |
+| arozos             | go     | ░ gpl3 | 531 MB |
 | updog              | python | █ mit  |  17 MB |
 | goshs              | go     | █ mit  |  11 MB |
 | gimme-that         | python | █ mit  | 4.8 MB |
@@ -504,12 +519,14 @@ symbol legend,
 * ✅ token auth (api keys)
 
 ## [kodbox](https://github.com/kalcaddle/kodbox)
-* this thing is insane
+* this thing is insane (but is getting competition from [arozos](#arozos))
 * php; [docker](https://hub.docker.com/r/kodcloud/kodbox)
 * 🔵 *upload segmenting, acceleration, and integrity checking!*
   * ⚠️ but uploads are not resumable(?)
 * ⚠️ not portable
 * ⚠️ isolated on-disk file hierarchy, incompatible with other software
+* ⚠️ uploading small files to copyparty is 16x faster
+* ⚠️ uploading large files to copyparty is 3x faster
 * ⚠️ http/webdav only; no ftp or zeroconf
 * ⚠️ some parts of the GUI are in chinese
 * ✅ fantastic ui/ux
@@ -569,6 +586,24 @@ symbol legend,
 * ✅ on-download event hook (otherwise same as copyparty)
 * ✅ more extensive permissions control
 
+## [arozos](https://github.com/tobychui/arozos)
+* big suite of applications similar to [kodbox](#kodbox), copyparty is better at downloading/uploading/music/indexing but arozos has other advantages
+* go; primarily linux (limited support for windows)
+* ⚠️ uploads not resumable / integrity-checked
+* ⚠️ uploading small files to copyparty is 2.7x faster
+* ⚠️ uploading large files to copyparty is at least 10% faster
+  * arozos is websocket-based, 512 KiB chunks; writes each chunk to separate files and then merges
+  * copyparty splices directly into the final file; faster and better for the HDD and filesystem
+* ⚠️ no directory tree navpane; not as easy to navigate
+* ⚠️ download-as-zip is not streaming; creates a temp.file on the server
+* ⚠️ not self-contained (pulls from jsdelivr)
+* ⚠️ has an audio player, but supports less filetypes
+* ⚠️ limited support for configuring real-ip detection
+* ✅ sftp server
+* ✅ settings gui
+* ✅ good-looking gui
+* ✅ an IDE, msoffice viewer, rich host integration, much more
+
 ## [updog](https://github.com/sc0tfree/updog)
 * python; cross-platform
 * basic directory listing with upload feature

scripts/make-pyz.sh

@@ -0,0 +1,61 @@
+#!/bin/bash
+set -e
+echo
+
+# port install gnutar gsed coreutils
+gtar=$(command -v gtar || command -v gnutar) || true
+[ ! -z "$gtar" ] && command -v gsed >/dev/null && {
+	tar()  { $gtar "$@"; }
+	sed()  { gsed  "$@"; }
+	command -v grealpath >/dev/null &&
+		realpath() { grealpath "$@"; }
+}
+
+targs=(--owner=1000 --group=1000)
+[ "$OSTYPE" = msys ] &&
+	targs=()
+
+[ -e copyparty/__main__.py ] || cd ..
+[ -e copyparty/__main__.py ] || {
+	echo "run me from within the project root folder"
+	echo
+	exit 1
+}
+
+[ -e sfx/copyparty/__main__.py ] || {
+    echo "run ./scripts/make-sfx.py first"
+    echo
+    exit 1
+}
+
+rm -rf pyz
+mkdir -p pyz
+cd pyz
+
+cp -pR ../sfx/{copyparty,partftpy} .
+cp -pR ../sfx/{ftp,j2}/* .
+
+ts=$(date -u +%s)
+hts=$(date -u +%Y-%m%d-%H%M%S)
+ver="$(cat ../sfx/ver)"
+
+mkdir -p ../dist
+pyz_out=../dist/copyparty.pyz
+
+echo creating z.tar
+( cd copyparty
+  tar -cf z.tar "${targs[@]}" --numeric-owner web res
+  rm -rf web res
+)
+
+echo creating loader
+sed -r 's/^(VER = ).*/\1"'"$ver"'"/; s/^(STAMP = ).*/\1'$(date +%s)/ \
+    <../scripts/ziploader.py \
+    >__main__.py
+
+echo creating pyz
+rm -f $pyz_out
+zip -9 -q -r $pyz_out *
+
+echo done:
+echo "  $(realpath $pyz_out)"

scripts/make-sfx.sh

@@ -99,9 +99,6 @@ pybin=$(command -v python3 || command -v python) || {
 	exit 1
 }
 
-[ $CSN ] ||
-	CSN=sfx
-
 langs=
 use_gz=
 zopf=2560
@@ -148,9 +145,9 @@ stamp=$(
 	done | sort | tail -n 1 | sha1sum | cut -c-16
 )
 
-rm -rf $CSN/*
+rm -rf sfx$CSN/*
-mkdir -p $CSN build
+mkdir -p sfx$CSN build
-cd $CSN
+cd sfx$CSN
 
 tmpdir="$(
 	printf '%s\n' "$TMPDIR" /tmp |
@@ -386,11 +383,13 @@ git describe --tags >/dev/null 2>/dev/null && {
 	ver="$(awk '/^VERSION *= \(/ {
 		gsub(/[^0-9,a-g-]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
 
+echo "$ver" >ver  # pyz
+
 ts=$(date -u +%s)
 hts=$(date -u +%Y-%m%d-%H%M%S) # --date=@$ts (thx osx)
 
 mkdir -p ../dist
-sfx_out=../dist/copyparty-$CSN
+sfx_out=../dist/copyparty-sfx$CSN
 
 echo cleanup
 find -name '*.pyc' -delete
@@ -542,7 +541,7 @@ gzres() {
 }
 
 
-zdir="$tmpdir/cpp-mk$CSN"
+zdir="$tmpdir/cpp-mksfx$CSN"
 [ -e "$zdir/$stamp" ] || rm -rf "$zdir"
 mkdir -p "$zdir"
 echo a > "$zdir/$stamp"

scripts/rls.sh

@@ -42,7 +42,7 @@ $f$s.py --version >/dev/null
     min=99999999
     for ((a=0; a<$parallel; a++)); do
         while [ -e .sfx-run ]; do
-            CSN=sfx$a ./make-sfx.sh re "$@"
+            CSN=$a ./make-sfx.sh re "$@"
             sz=$(wc -c <$f$a$s.py | awk '{print$1}')
             [ $sz -ge $min ] && continue
             mv $f$a$s.py $f$s.py.$sz

scripts/run-tests.sh

@@ -1,6 +1,21 @@
 #!/bin/bash
 set -ex
 
+if uname | grep -iE '^(msys|mingw)'; then
+    pids=()
+
+    python -m unittest discover -s tests >/dev/null &
+    pids+=($!)
+
+    python scripts/test/smoketest.py &
+    pids+=($!)
+
+    for pid in ${pids[@]}; do
+        wait $pid
+    done
+    exit $?
+fi
+
 # osx support
 gtar=$(command -v gtar || command -v gnutar) || true
 [ ! -z "$gtar" ] && command -v gfind >/dev/null && {

scripts/ziploader.py

@@ -0,0 +1,112 @@
+#!/usr/bin/env python3
+
+import atexit
+import os
+import platform
+import sys
+import tarfile
+import tempfile
+import threading
+import time
+import traceback
+
+
+VER = None
+STAMP = None
+WINDOWS = sys.platform in ["win32", "msys"]
+
+
+def msg(*a, **ka):
+    if a:
+        a = ["[ZIP]", a[0]] + list(a[1:])
+
+    ka["file"] = sys.stderr
+    print(*a, **ka)
+
+
+def utime(top):
+    # avoid cleaners
+    files = [os.path.join(dp, p) for dp, dd, df in os.walk(top) for p in dd + df]
+    try:
+        while True:
+            t = int(time.time())
+            for f in [top] + files:
+                os.utime(f, (t, t))
+
+            time.sleep(78123)
+    except Exception as ex:
+        print("utime:", ex, f)
+
+
+def confirm(rv):
+    msg()
+    msg("retcode", rv if rv else traceback.format_exc())
+    if WINDOWS:
+        msg("*** hit enter to exit ***")
+        try:
+            input()
+        except:
+            pass
+
+    sys.exit(rv or 1)
+
+
+def run():
+    import copyparty
+    from copyparty.__main__ import main as cm
+
+    td = tempfile.TemporaryDirectory(prefix="")
+    atexit.register(td.cleanup)
+    rsrc = td.name
+
+    try:
+        from importlib.resources import files
+
+        f = files(copyparty).joinpath("z.tar").open("rb")
+    except:
+        from importlib.resources import open_binary
+
+        f = open_binary("copyparty", "z.tar")
+
+    with tarfile.open(fileobj=f) as tf:
+        try:
+            tf.extractall(rsrc, filter="tar")
+        except TypeError:
+            tf.extractall(rsrc)  # nosec (archive is safe)
+
+    f.close()
+    f = None
+
+    msg("  rsrc dir:", rsrc)
+    msg()
+
+    t = threading.Thread(target=utime, args=(rsrc,))
+    t.daemon = True
+    t.start()
+
+    cm(rsrc=rsrc)
+
+
+def main():
+    sysver = str(sys.version).replace("\n", "\n" + " " * 18)
+    pktime = time.strftime("%Y-%m-%d, %H:%M:%S", time.gmtime(STAMP))
+    msg()
+    msg("   this is: copyparty", VER)
+    msg(" packed at:", pktime, "UTC,", STAMP)
+    msg("python bin:", sys.executable)
+    msg("python ver:", platform.python_implementation(), sysver)
+
+    try:
+        run()
+    except SystemExit as ex:
+        c = ex.code
+        if c not in [0, -15]:
+            confirm(ex.code)
+    except KeyboardInterrupt:
+        pass
+    except:
+        confirm(0)
+
+
+if __name__ == "__main__":
+    main()

tests/test_dots.py

@@ -3,10 +3,8 @@
 from __future__ import print_function, unicode_literals
 
 import io
-import os
-import time
 import json
-import pprint
+import os
 import shutil
 import tarfile
 import tempfile
@@ -82,7 +80,8 @@ class TestHttpCli(unittest.TestCase):
         x = " ".join(sorted([x["rp"] for x in x[0]]))
         self.assertEqual(x, ".f0 .t/.f2 .t/f2 a/da/f4 a/f3 f0 t/.f1 t/f1")
 
-        self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"], dotsrch=False)
+        u2idx.shutdown()
+        self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"], dotsrch=False, e2d=True)
         self.asrv = AuthSrv(self.args, self.log)
         u2idx = U2idx(self)
 
@@ -91,6 +90,8 @@ class TestHttpCli(unittest.TestCase):
         # u1 can see dotfiles in volB so they should be included
         xe = "a/da/f4 a/f3 f0 t/f1"
         self.assertEqual(x, xe)
+        u2idx.shutdown()
+        up2k.shutdown()
 
         ##
         ## dirkeys
@@ -149,4 +150,4 @@ class TestHttpCli(unittest.TestCase):
         return conn.s._reply.decode("utf-8").split("\r\n\r\n", 1)
 
     def log(self, src, msg, c=0):
-        print(msg)
+        print(msg, "\033[0m")

tests/test_idp.py

@@ -6,6 +6,7 @@ import json
 import os
 import unittest
 
+from copyparty.__init__ import ANYWIN
 from copyparty.authsrv import AuthSrv
 from tests.util import Cfg
 
@@ -51,6 +52,12 @@ class TestVFS(unittest.TestCase):
         vn = self.nav(au, vp)
         self.assertNodes(vn, expected)
 
+    def assertApEq(self, ap, rhs):
+        if ANYWIN and len(ap) > 2 and ap[1] == ":":
+            ap = ap[2:].replace("\\", "/")
+
+        return self.assertEqual(ap, rhs)
+
     def prep(self):
         here = os.path.abspath(os.path.dirname(__file__))
         cfgdir = os.path.join(here, "res", "idp")
@@ -70,7 +77,7 @@ class TestVFS(unittest.TestCase):
         au = AuthSrv(Cfg(c=[cfgdir + "/1.conf"], **xcfg), self.log)
 
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "/")
+        self.assertApEq(au.vfs.realpath, "/")
         self.assertNodes(au.vfs, ["vb"])
         self.assertNodes(au.vfs.nodes["vb"], [])
 
@@ -85,7 +92,7 @@ class TestVFS(unittest.TestCase):
         au = AuthSrv(Cfg(c=[cfgdir + "/2.conf"], **xcfg), self.log)
 
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "/")
+        self.assertApEq(au.vfs.realpath, "/")
         self.assertNodes(au.vfs, ["vb", "vc"])
         self.assertNodes(au.vfs.nodes["vb"], [])
         self.assertNodes(au.vfs.nodes["vc"], [])
@@ -103,7 +110,7 @@ class TestVFS(unittest.TestCase):
         au = AuthSrv(Cfg(c=[cfgdir + "/3.conf"], **xcfg), self.log)
 
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "")
+        self.assertApEq(au.vfs.realpath, "")
         self.assertNodes(au.vfs, [])
         self.assertAxs(au.vfs.axs, [])
 
@@ -112,8 +119,8 @@ class TestVFS(unittest.TestCase):
         self.assertNodesAt(au, "vu", ["iua"])  # same as:
         self.assertNodes(au.vfs.nodes["vu"], ["iua"])
         self.assertNodes(au.vfs.nodes["vg"], ["iga"])
-        self.assertEqual(au.vfs.nodes["vu"].realpath, "")
+        self.assertApEq(au.vfs.nodes["vu"].realpath, "")
-        self.assertEqual(au.vfs.nodes["vg"].realpath, "")
+        self.assertApEq(au.vfs.nodes["vg"].realpath, "")
         self.assertAxs(au.vfs.axs, [])
         self.assertAxsAt(au, "vu/iua", [["iua"]])  # same as:
         self.assertAxs(self.nav(au, "vu/iua").axs, [["iua"]])
@@ -127,7 +134,7 @@ class TestVFS(unittest.TestCase):
         au = AuthSrv(Cfg(c=[cfgdir + "/4.conf"], **xcfg), self.log)
 
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "")
+        self.assertApEq(au.vfs.realpath, "")
         self.assertNodes(au.vfs, ["vu"])
         self.assertNodesAt(au, "vu", ["ua", "ub"])
         self.assertAxs(au.vfs.axs, [])
@@ -147,10 +154,10 @@ class TestVFS(unittest.TestCase):
         self.assertAxsAt(au, "vg", [])
         self.assertAxsAt(au, "vg/iga1", [["iua"]])
         self.assertAxsAt(au, "vg/iga2", [["iua", "ua"]])
-        self.assertEqual(self.nav(au, "vu/ua").realpath, "/u-ua")
+        self.assertApEq(self.nav(au, "vu/ua").realpath, "/u-ua")
-        self.assertEqual(self.nav(au, "vu/iua").realpath, "/u-iua")
+        self.assertApEq(self.nav(au, "vu/iua").realpath, "/u-iua")
-        self.assertEqual(self.nav(au, "vg/iga1").realpath, "/g1-iga")
+        self.assertApEq(self.nav(au, "vg/iga1").realpath, "/g1-iga")
-        self.assertEqual(self.nav(au, "vg/iga2").realpath, "/g2-iga")
+        self.assertApEq(self.nav(au, "vg/iga2").realpath, "/g2-iga")
 
         au.idp_checkin(None, "iub", "iga")
         self.assertAxsAt(au, "vu/iua", [["iua"]])
@@ -165,7 +172,7 @@ class TestVFS(unittest.TestCase):
         au = AuthSrv(Cfg(c=[cfgdir + "/5.conf"], **xcfg), self.log)
 
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "")
+        self.assertApEq(au.vfs.realpath, "")
         self.assertNodes(au.vfs, ["g", "ga", "gb"])
         self.assertAxs(au.vfs.axs, [])
 
@@ -196,7 +203,7 @@ class TestVFS(unittest.TestCase):
 
         self.assertAxs(au.vfs.axs, [])
         self.assertEqual(au.vfs.vpath, "")
-        self.assertEqual(au.vfs.realpath, "")
+        self.assertApEq(au.vfs.realpath, "")
         self.assertNodes(au.vfs, [])
 
         au.idp_checkin(None, "iua", "")

tests/util.py

@@ -44,7 +44,7 @@ if MACOS:
 from copyparty.__init__ import E
 from copyparty.__main__ import init_E
 from copyparty.u2idx import U2idx
-from copyparty.util import FHC, Garda, Unrecv
+from copyparty.util import FHC, CachedDict, Garda, Unrecv
 
 init_E(E)
 
@@ -110,25 +110,25 @@ class Cfg(Namespace):
     def __init__(self, a=None, v=None, c=None, **ka0):
         ka = {}
 
-        ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw q rand smb srch_dbg stats vague_403 vc ver xdev xlink xvol"
+        ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_pipe no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw og og_no_head og_s_title q rand smb srch_dbg stats uqe vague_403 vc ver xdev xlink xvol"
         ka.update(**{k: False for k in ex.split()})
 
-        ex = "dotpart dotsrch no_dhash no_fastboot no_rescan no_sendfile no_voldump re_dhash plain_ip"
+        ex = "dotpart dotsrch no_dhash no_fastboot no_rescan no_sendfile no_snap no_voldump re_dhash plain_ip"
         ka.update(**{k: True for k in ex.split()})
 
-        ex = "ah_cli ah_gen css_browser hist js_browser no_forget no_hash no_idx nonsus_urls"
+        ex = "ah_cli ah_gen css_browser hist js_browser no_forget no_hash no_idx nonsus_urls og_tpl og_ua"
         ka.update(**{k: None for k in ex.split()})
 
         ex = "hash_mt srch_time u2abort u2j"
         ka.update(**{k: 1 for k in ex.split()})
 
-        ex = "reg_cap s_thead s_tbody th_convt"
+        ex = "au_vol mtab_age reg_cap s_thead s_tbody th_convt"
         ka.update(**{k: 9 for k in ex.split()})
 
         ex = "db_act df k304 loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
         ka.update(**{k: 0 for k in ex.split()})
 
-        ex = "ah_alg bname doctitle exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name textfiles unlist vname R RS SR"
+        ex = "ah_alg bname doctitle exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i tcolor textfiles unlist vname R RS SR"
         ka.update(**{k: "" for k in ex.split()})
 
         ex = "grp on403 on404 xad xar xau xban xbd xbr xbu xiu xm"
@@ -155,6 +155,7 @@ class Cfg(Namespace):
             mte={"a": True},
             mth={},
             mtp=[],
+            mv_retry="0/0",
             rm_retry="0/0",
             s_rd_sz=256 * 1024,
             s_wr_sz=256 * 1024,
@@ -250,6 +251,7 @@ class VHttpConn(object):
         self.log_func = log
         self.log_src = "a"
         self.mutex = threading.Lock()
+        self.pipes = CachedDict(1)
         self.u2mutex = threading.Lock()
         self.nbyte = 0
         self.nid = None
@@ -258,3 +260,7 @@ class VHttpConn(object):
         self.u2fh = FHC()
 
         self.get_u2idx = self.hsrv.get_u2idx
+
+
+if WINDOWS:
+    os.system("rem")