v0.13.2

README.md

@@ -16,11 +16,6 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
 
 
-## breaking changes \o/
-
-this is the readme for v0.12 which has a different expression for volume permissions (`-v`); see [the v0.11.x readme](https://github.com/9001/copyparty/tree/15b59822112dda56cee576df30f331252fc62628#readme) for stuff regarding the [current stable release](https://github.com/9001/copyparty/releases/tag/v0.11.47)
-
-
 ## readme toc
 
 * top
@@ -41,11 +36,13 @@ this is the readme for v0.12 which has a different expression for volume permiss
     * [uploading](#uploading)
         * [file-search](#file-search)
     * [file manager](#file-manager)
+    * [batch rename](#batch-rename)
     * [markdown viewer](#markdown-viewer)
     * [other tricks](#other-tricks)
-* [searching](#searching)
+    * [searching](#searching)
-    * [search configuration](#search-configuration)
+* [server config](#server-config)
-    * [database location](#database-location)
+    * [upload rules](#upload-rules)
+    * [database location](#database-location)upload rules
     * [metadata from audio files](#metadata-from-audio-files)
     * [file parser plugins](#file-parser-plugins)
     * [complete examples](#complete-examples)
@@ -125,29 +122,31 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ [accounts](#accounts-and-volumes)
 * upload
   * ☑ basic: plain multipart, ie6 support
-  * ☑ up2k: js, resumable, multithreaded
+  * ☑ [up2k](#uploading): js, resumable, multithreaded
   * ☑ stash: simple PUT filedropper
+  * ☑ unpost: undo/delete accidental uploads
   * ☑ symlink/discard existing files (content-matching)
 * download
   * ☑ single files in browser
-  * ☑ folders as zip / tar files
+  * ☑ [folders as zip / tar files](#zip-downloads)
   * ☑ FUSE client (read-only)
 * browser
   * ☑ navpane (directory tree sidebar)
+  * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
   * ☑ audio player (with OS media controls)
-  * ☑ thumbnails
+  * ☑ image gallery with webm player
+  * ☑ [thumbnails](#thumbnails)
     * ☑ ...of images using Pillow
     * ☑ ...of videos using FFmpeg
     * ☑ cache eviction (max-age; maybe max-size eventually)
-  * ☑ image gallery with webm player
   * ☑ SPA (browse while uploading)
     * if you use the navpane to navigate, not folders in the file list
 * server indexing
-  * ☑ locate files by contents
+  * ☑ [locate files by contents](#file-search)
   * ☑ search by name/path/date/size
-  * ☑ search by ID3-tags etc.
+  * ☑ [search by ID3-tags etc.](#searching)
 * markdown
-  * ☑ viewer
+  * ☑ [viewer](#markdown-viewer)
   * ☑ editor (sure why not)
 
 
@@ -180,7 +179,7 @@ small collection of user feedback
   * this is an msys2 bug, the regular windows edition of python is fine
 
 * VirtualBox: sqlite throws `Disk I/O Error` when running in a VM and the up2k database is in a vboxsf
-  * use `--hist` or the `hist` volflag (`-v [...]:chist=/tmp/foo`) to place the db inside the vm instead
+  * use `--hist` or the `hist` volflag (`-v [...]:c,hist=/tmp/foo`) to place the db inside the vm instead
 
 
 # accounts and volumes
@@ -215,10 +214,11 @@ example:
 ## tabs
 
 * `[🔎]` search by size, date, path/name, mp3-tags ... see [searching](#searching)
+* `[🧯]` unpost: undo/delete accidental uploads
 * `[🚀]` and `[🎈]` are the uploaders, see [uploading](#uploading)
-* `[📂]` mkdir, create directories
+* `[📂]` mkdir: create directories
-* `[📝]` new-md, create a new markdown document
+* `[📝]` new-md: create a new markdown document
-* `[📟]` send-msg, either to server-log or into textfiles if `--urlform save`
+* `[📟]` send-msg: either to server-log or into textfiles if `--urlform save`
 * `[🎺]` audio-player config options
 * `[⚙️]` general client config options
 
@@ -248,6 +248,7 @@ the browser has the following hotkeys (assumes qwerty, ignores actual layout)
 * when viewing images / playing videos:
   * `J/L, Left/Right` prev/next file
   * `Home/End` first/last file
+  * `S` toggle selection
   * `Esc` close viewer
   * videos:
     * `U/O` skip 10sec back/forward
@@ -312,8 +313,10 @@ you can also zip a selection of files or folders by clicking them in the browser
 ## uploading
 
 two upload methods are available in the html client:
-* `🎈 bup`, the basic uploader, supports almost every browser since netscape 4.0
+* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
-* `🚀 up2k`, the fancy one
+* `[🚀] up2k`, the fancy one
+
+you can undo/delete uploads using `[🧯] unpost` if the server is running with `-e2d`
 
 up2k has several advantages:
 * you can drop folders into the browser (files are added recursively)
@@ -369,6 +372,53 @@ if you have the required permissions, you can cut/paste, rename, and delete file
 you can move files across browser tabs (cut in one tab, paste in another)
 
 
+## batch rename
+
+![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
+
+select some files and press F2 to bring up the rename UI
+
+quick explanation of the buttons,  
+* `[✅ apply rename]` confirms and begins renaming
+* `[❌ cancel]` aborts and closes the rename window
+* `[↺ reset]` reverts any filename changes back to the original name
+* `[decode]` does a URL-decode on the filename, fixing stuff like `&` and `%20`
+* `[advanced]` toggles advanced mode
+
+advanced mode: rename files based on rules to decide the new names, based on the original name (regex), or based on the tags collected from the file (artist/title/...), or a mix of both
+
+in advanced mode,  
+* `[case]` toggles case-sensitive regex
+* `regex` is the regex pattern to apply to the original filename; any files which don't match will be skipped
+* `format` is the new filename, taking values from regex capturing groups and/or from file tags
+  * very loosely based on foobar2000 syntax
+* `presets` lets you save rename rules for later
+
+available functions:
+* `$lpad(text, length, pad_char)`
+* `$rpad(text, length, pad_char)`
+
+so,
+
+say you have a file named [`meganeko - Eclipse - 07 Sirius A.mp3`](https://www.youtube.com/watch?v=-dtb0vDPruI) (absolutely fantastic album btw) and the tags are: `Album:Eclipse`, `Artist:meganeko`, `Title:Sirius A`, `tn:7`
+
+you could use just regex to rename it:
+* `regex` = `(.*) - (.*) - ([0-9]{2}) (.*)`
+* `format` = `(3). (1) - (4)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+or you could use just tags:
+* `format` = `$lpad((tn),2,0). (artist) - (title).(ext)`
+* `output` = `7. meganeko - Sirius A.mp3`
+
+or a mix of both:
+* `regex` = ` - ([0-9]{2}) `
+* `format` = `(1). (artist) - (title).(ext)`
+* `output` = `07. meganeko - Sirius A.mp3`
+
+the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
+
+
 ## markdown viewer
 
 ![copyparty-md-read-fs8](https://user-images.githubusercontent.com/241032/115978057-66419080-a57d-11eb-8539-d2be843991aa.png)
@@ -383,7 +433,7 @@ you can move files across browser tabs (cut in one tab, paste in another)
 * if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
 
 
-# searching
+## searching
 
 ![copyparty-search-fs8](https://user-images.githubusercontent.com/241032/115978060-6772bd80-a57d-11eb-81d3-174e869b72c3.png)
 
@@ -395,12 +445,12 @@ path/name queries are space-separated, AND'ed together, and words are negated wi
 * path: `shibayan -bossa` finds all files where one of the folders contain `shibayan` but filters out any results where `bossa` exists somewhere in the path
 * name: `demetori styx` gives you [good stuff](https://www.youtube.com/watch?v=zGh0g14ZJ8I&list=PL3A147BD151EE5218&index=9)
 
-add `-e2ts` to also scan/index tags from music files:
+add the argument `-e2ts` to also scan/index tags from music files, which brings us over to:
 
 
-## search configuration
+# server config
 
-searching relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
+file indexing relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
 
 through arguments:
 * `-e2d` enables file indexing on upload
@@ -411,20 +461,60 @@ through arguments:
 * `-e2tsr` deletes all existing tags, does a full reindex
 
 the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
-* `-v ~/music::r:ce2dsa:ce2tsr` does a full reindex of everything on startup
+* `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
-* `-v ~/music::r:cd2d` disables **all** indexing, even if any `-e2*` are on
+* `-v ~/music::r:c,d2d` disables **all** indexing, even if any `-e2*` are on
-* `-v ~/music::r:cd2t` disables all `-e2t*` (tags), does not affect `-e2d*`
+* `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
 
 note:
 * `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
 * the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
 
-you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `cdhash`, this has the following consequences:
+you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
 * initial indexing is way faster, especially when the volume is on a network disk
 * makes it impossible to [file-search](#file-search)
 * if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
 
-if you set `--no-hash`, you can enable hashing for specific volumes using flag `cehash`
+if you set `--no-hash`, you can enable hashing for specific volumes using flag `:c,ehash`
+
+
+## upload rules
+
+you can set upload rules using volume flags, some examples:
+
+* `:c,sz=1k-3m` sets allowed filesize between 1 KiB and 3 MiB inclusive (suffixes: b, k, m, g)
+* `:c,nosub` disallow uploading into subdirectories; goes well with `rotn` and `rotf`:
+* `:c,rotn=1000,2` moves uploads into subfolders, up to 1000 files in each folder before making a new one, two levels deep (must be at least 1)
+* `:c,rotf=%Y/%m/%d/%H` enforces files to be uploaded into a structure of subfolders according to that date format
+  * if someone uploads to `/foo/bar` the path would be rewritten to `/foo/bar/2021/08/06/23` for example
+  * but the actual value is not verified, just the structure, so the uploader can choose any values which conform to the format string
+    * just to avoid additional complexity in up2k which is enough of a mess already
+* `:c,lifetime=300` delete uploaded files when they become 5 minutes old
+
+you can also set transaction limits which apply per-IP and per-volume, but these assume `-j 1` (default) otherwise the limits will be off, for example `-j 4` would allow anywhere between 1x and 4x the limits you set depending on which processing node the client gets routed to
+
+* `:c,maxn=250,3600` allows 250 files over 1 hour from each IP (tracked per-volume)
+* `:c,maxb=1g,300` allows 1 GiB total over 5 minutes from each IP (tracked per-volume)
+
+
+## compress uploads
+
+files can be autocompressed on upload, either on user-request (if config allows) or forced by server-config
+
+* volume flag `gz` allows gz compression
+* volume flag `xz` allows lzma compression
+* volume flag `pk` **forces** compression on all files
+* url parameter `pk` requests compression with server-default algorithm
+* url parameter `gz` or `xz` requests compression with a specific algorithm
+* url parameter `xz` requests xz compression
+
+things to note,
+* the `gz` and `xz` arguments take a single optional argument, the compression level (range 0 to 9)
+* the `pk` volume flag takes the optional argument `ALGORITHM,LEVEL` which will then be forced for all uploads, for example `gz,9` or `xz,0`
+* default compression is gzip level 9
+* all upload methods except up2k are supported
+* the files will be indexed after compression, so dupe-detection and file-search will not work as expected
+
+some examples,
 
 
 ## database location
@@ -432,7 +522,7 @@ if you set `--no-hash`, you can enable hashing for specific volumes using flag `
 copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
 
 this can instead be kept in a single place using the `--hist` argument, or the `hist=` volume flag, or a mix of both:
-* `--hist ~/.cache/copyparty -v ~/music::r:chist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
+* `--hist ~/.cache/copyparty -v ~/music::r:c,hist=-` sets `~/.cache/copyparty` as the default place to put volume info, but `~/music` gets the regular `.hist` subfolder (`-` restores default behavior)
 
 note:
 * markdown edits are always stored in a local `.hist` subdirectory
@@ -443,10 +533,12 @@ note:
 ## metadata from audio files
 
 `-mte` decides which tags to index and display in the browser (and also the display order), this can be changed per-volume:
-* `-v ~/music::r:cmte=title,artist` indexes and displays *title* followed by *artist*
+* `-v ~/music::r:c,mte=title,artist` indexes and displays *title* followed by *artist*
 
 if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
 
+but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
+
 `-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
 
 tags that start with a `.` such as `.bpm` and `.dur`(ation) indicate numeric value
@@ -467,7 +559,7 @@ copyparty can invoke external programs to collect additional metadata for files
 
 * `-mtp .bpm=~/bin/audio-bpm.py` will execute `~/bin/audio-bpm.py` with the audio file as argument 1 to provide the `.bpm` tag, if that does not exist in the audio metadata
 * `-mtp key=f,t5,~/bin/audio-key.py` uses `~/bin/audio-key.py` to get the `key` tag, replacing any existing metadata tag (`f,`), aborting if it takes longer than 5sec (`t5,`)
-* `-v ~/music::r:cmtp=.bpm=~/bin/audio-bpm.py:cmtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
+* `-v ~/music::r:c,mtp=.bpm=~/bin/audio-bpm.py:c,mtp=key=f,t5,~/bin/audio-key.py` both as a per-volume config wow this is getting ugly
 
 *but wait, there's more!* `-mtp` can be used for non-audio files as well using the `a` flag: `ay` only do audio files, `an` only do non-audio files, or `ad` do all files (d as in dontcare) 
 
@@ -531,11 +623,11 @@ quick summary of more eccentric web-browsers trying to view a directory index:
   * `var xhr = new XMLHttpRequest(); xhr.open('POST', 'https://127.0.0.1:3923/msgs?raw'); xhr.send('foo');`
 
 * curl/wget: upload some files (post=file, chunk=stdin)
-  * `post(){ curl -b cppwd=wark http://127.0.0.1:3923/ -F act=bput -F f=@"$1";}`  
+  * `post(){ curl -b cppwd=wark -F act=bput -F f=@"$1" http://127.0.0.1:3923/;}`  
     `post movie.mkv`
-  * `post(){ wget --header='Cookie: cppwd=wark' http://127.0.0.1:3923/?raw --post-file="$1" -O-;}`  
+  * `post(){ wget --header='Cookie: cppwd=wark' --post-file="$1" -O- http://127.0.0.1:3923/?raw;}`  
     `post movie.mkv`
-  * `chunk(){ curl -b cppwd=wark http://127.0.0.1:3923/ -T-;}`  
+  * `chunk(){ curl -b cppwd=wark -T- http://127.0.0.1:3923/;}`  
     `chunk <movie.mkv`
 
 * FUSE: mount a copyparty server as a local filesystem

bin/README.md

@@ -61,3 +61,8 @@ cd /mnt/nas/music/.hist
 ~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy key
 ~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy .bpm -vac
 ```
+
+
+# [`prisonparty.sh`](prisonparty.sh)
+* run copyparty in a chroot, preventing any accidental file access
+* creates bindmounts for /bin, /lib, and so on, see `sysdirs=`

bin/mtag/README.md

@@ -4,6 +4,7 @@ some of these rely on libraries which are not MIT-compatible
 
 * [audio-bpm.py](./audio-bpm.py) detects the BPM of music using the BeatRoot Vamp Plugin; imports GPL2
 * [audio-key.py](./audio-key.py) detects the melodic key of music using the Mixxx fork of keyfinder; imports GPL3
+* [media-hash.py](./media-hash.py) generates checksums for audio and video streams; uses FFmpeg (LGPL or GPL)
 
 
 # dependencies
@@ -18,7 +19,10 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 
 # usage from copyparty
 
-`copyparty -e2dsa -e2ts -mtp key=f,audio-key.py -mtp .bpm=f,audio-bpm.py`
+`copyparty -e2dsa -e2ts` followed by any combination of these:
+* `-mtp key=f,audio-key.py`
+* `-mtp .bpm=f,audio-bpm.py`
+* `-mtp ahash,vhash=f,media-hash.py`
 
 * `f,` makes the detected value replace any existing values
 * the `.` in `.bpm` indicates numeric value
@@ -29,6 +33,9 @@ run [`install-deps.sh`](install-deps.sh) to build/install most dependencies requ
 ## usage with volume-flags
 
 instead of affecting all volumes, you can set the options for just one volume like so:
-```
+
-copyparty -v /mnt/nas/music:/music:r:cmtp=key=f,audio-key.py:cmtp=.bpm=f,audio-bpm.py:ce2dsa:ce2ts
+`copyparty -v /mnt/nas/music:/music:r:c,e2dsa:c,e2ts` immediately followed by any combination of these:
-```
+
+* `:c,mtp=key=f,audio-key.py`
+* `:c,mtp=.bpm=f,audio-bpm.py`
+* `:c,mtp=ahash,vhash=f,media-hash.py`

bin/mtag/media-hash.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python
+
+import re
+import sys
+import json
+import time
+import base64
+import hashlib
+import subprocess as sp
+
+try:
+    from copyparty.util import fsenc
+except:
+
+    def fsenc(p):
+        return p
+
+
+"""
+dep: ffmpeg
+"""
+
+
+def det():
+    # fmt: off
+    cmd = [
+        "ffmpeg",
+        "-nostdin",
+        "-hide_banner",
+        "-v", "fatal",
+        "-i", fsenc(sys.argv[1]),
+        "-f", "framemd5",
+        "-"
+    ]
+    # fmt: on
+
+    p = sp.Popen(cmd, stdout=sp.PIPE)
+    # ps = io.TextIOWrapper(p.stdout, encoding="utf-8")
+    ps = p.stdout
+
+    chans = {}
+    for ln in ps:
+        if ln.startswith(b"#stream#"):
+            break
+
+        m = re.match(r"^#media_type ([0-9]): ([a-zA-Z])", ln.decode("utf-8"))
+        if m:
+            chans[m.group(1)] = m.group(2)
+
+    hashers = [hashlib.sha512(), hashlib.sha512()]
+    for ln in ps:
+        n = int(ln[:1])
+        v = ln.rsplit(b",", 1)[-1].strip()
+        hashers[n].update(v)
+
+    r = {}
+    for k, v in chans.items():
+        dg = hashers[int(k)].digest()[:12]
+        dg = base64.urlsafe_b64encode(dg).decode("ascii")
+        r[v[0].lower() + "hash"] = dg
+
+    print(json.dumps(r, indent=4))
+
+
+def main():
+    try:
+        det()
+    except:
+        pass  # mute
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/res/yt-ipr.conf

@@ -0,0 +1,39 @@
+# example config file to use copyparty as a youtube manifest collector,
+# use with copyparty like:  python copyparty.py -c yt-ipr.conf
+#
+# see docs/example.conf for a better explanation of the syntax, but
+# newlines are block separators, so adding blank lines inside a volume definition is bad
+# (use comments as separators instead)
+
+
+# create user ed, password wark
+u ed:wark
+
+
+# create a volume at /ytm which stores files at ./srv/ytm
+./srv/ytm
+/ytm
+# write-only, but read-write for user ed
+w
+rw ed
+# rescan the volume on startup
+c e2dsa
+# collect tags from all new files since last scan
+c e2ts
+# optionally enable compression to make the files 50% smaller
+c pk
+# only allow uploads which are between 16k and 1m large
+c sz=16k-1m
+# allow up to 10 uploads over 5 minutes from each ip
+c maxn=10,300
+# move uploads into subfolders: YEAR-MONTH / DAY-HOUR / <upload>
+c rotf=%Y-%m/%d-%H
+# delete uploads when they are 24 hours old
+c lifetime=86400
+# add the parser and tell copyparty what tags it can expect from it
+c mtp=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires=bin/mtag/yt-ipr.py
+# decide which tags we want to index and in what order
+c mte=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires
+
+
+# create any other volumes you'd like down here, or merge this with an existing config file

bin/mtag/res/yt-ipr.user.js

@@ -0,0 +1,47 @@
+// ==UserScript==
+// @name    youtube-playerdata-hub
+// @match   https://youtube.com/*
+// @match   https://*.youtube.com/*
+// @version 1.0
+// @grant   GM_addStyle
+// ==/UserScript==
+
+function main() {
+    var server = 'https://127.0.0.1:3923/ytm?pw=wark',
+        interval = 60; // sec
+
+    var sent = {};
+    function send(txt, mf_url, desc) {
+        if (sent[mf_url])
+            return;
+
+        fetch(server + '&_=' + Date.now(), { method: "PUT", body: txt });
+        console.log('[yt-pdh] yeet %d bytes, %s', txt.length, desc);
+        sent[mf_url] = 1;
+    }
+
+    function collect() {
+        try {
+            var pd = document.querySelector('ytd-watch-flexy');
+            if (!pd)
+                return console.log('[yt-pdh] no video found');
+
+            pd = pd.playerData;
+            var mu = pd.streamingData.dashManifestUrl || pd.streamingData.hlsManifestUrl;
+            if (!mu || !mu.length)
+                return console.log('[yt-pdh] no manifest found');
+
+            var desc = pd.videoDetails.videoId + ', ' + pd.videoDetails.title;
+            send(JSON.stringify(pd), mu, desc);
+        }
+        catch (ex) {
+            console.log("[yt-pdh]", ex);
+        }
+    }
+    setInterval(collect, interval * 1000);
+}
+
+var scr = document.createElement('script');
+scr.textContent = '(' + main.toString() + ')();';
+(document.head || document.getElementsByTagName('head')[0]).appendChild(scr);
+console.log('[yt-pdh] a');

bin/mtag/yt-ipr.py

@@ -0,0 +1,198 @@
+#!/usr/bin/env python
+
+import re
+import os
+import sys
+import gzip
+import json
+import base64
+import string
+import urllib.request
+from datetime import datetime
+
+"""
+youtube initial player response
+
+it's probably best to use this through a config file; see res/yt-ipr.conf
+
+but if you want to use plain arguments instead then:
+  -v srv/ytm:ytm:w:rw,ed
+       :c,e2ts:c,e2dsa
+       :c,sz=16k-1m:c,maxn=10,300:c,rotf=%Y-%m/%d-%H
+       :c,mtp=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires=bin/mtag/yt-ipr.py
+       :c,mte=yt-id,yt-title,yt-author,yt-channel,yt-views,yt-private,yt-manifest,yt-expires
+
+see res/yt-ipr.user.js for the example userscript to go with this
+"""
+
+
+def main():
+    try:
+        with gzip.open(sys.argv[1], "rt", encoding="utf-8", errors="replace") as f:
+            txt = f.read()
+    except:
+        with open(sys.argv[1], "r", encoding="utf-8", errors="replace") as f:
+            txt = f.read()
+
+    txt = "{" + txt.split("{", 1)[1]
+
+    try:
+        pd = json.loads(txt)
+    except json.decoder.JSONDecodeError as ex:
+        pd = json.loads(txt[: ex.pos])
+
+    # print(json.dumps(pd, indent=2))
+
+    if "videoDetails" in pd:
+        parse_youtube(pd)
+    else:
+        parse_freg(pd)
+
+
+def get_expiration(url):
+    et = re.search(r"[?&]expire=([0-9]+)", url).group(1)
+    et = datetime.utcfromtimestamp(int(et))
+    return et.strftime("%Y-%m-%d, %H:%M")
+
+
+def parse_youtube(pd):
+    vd = pd["videoDetails"]
+    sd = pd["streamingData"]
+
+    et = sd["adaptiveFormats"][0]["url"]
+    et = get_expiration(et)
+
+    mf = []
+    if "dashManifestUrl" in sd:
+        mf.append("dash")
+    if "hlsManifestUrl" in sd:
+        mf.append("hls")
+
+    r = {
+        "yt-id": vd["videoId"],
+        "yt-title": vd["title"],
+        "yt-author": vd["author"],
+        "yt-channel": vd["channelId"],
+        "yt-views": vd["viewCount"],
+        "yt-private": vd["isPrivate"],
+        # "yt-expires": sd["expiresInSeconds"],
+        "yt-manifest": ",".join(mf),
+        "yt-expires": et,
+    }
+    print(json.dumps(r))
+
+    freg_conv(pd)
+
+
+def parse_freg(pd):
+    md = pd["metadata"]
+    r = {
+        "yt-id": md["id"],
+        "yt-title": md["title"],
+        "yt-author": md["channelName"],
+        "yt-channel": md["channelURL"].strip("/").split("/")[-1],
+        "yt-expires": get_expiration(list(pd["video"].values())[0]),
+    }
+    print(json.dumps(r))
+
+
+def freg_conv(pd):
+    # based on getURLs.js v1.5 (2021-08-07)
+    # fmt: off
+    priority = {
+        "video": [
+            337, 315, 266, 138,  # 2160p60
+            313, 336,  # 2160p
+            308,  # 1440p60
+            271, 264,  # 1440p
+            335, 303, 299,  # 1080p60
+            248, 169, 137,  # 1080p
+            334, 302, 298,  # 720p60
+            247, 136  # 720p
+        ],
+        "audio": [
+            251, 141, 171, 140, 250, 249, 139
+        ]
+    }
+
+    vid_id = pd["videoDetails"]["videoId"]
+    chan_id = pd["videoDetails"]["channelId"]
+
+    try:
+        thumb_url = pd["microformat"]["playerMicroformatRenderer"]["thumbnail"]["thumbnails"][0]["url"]
+        start_ts = pd["microformat"]["playerMicroformatRenderer"]["liveBroadcastDetails"]["startTimestamp"]
+    except:
+        thumb_url = f"https://img.youtube.com/vi/{vid_id}/maxresdefault.jpg"
+        start_ts = ""
+
+    # fmt: on
+
+    metadata = {
+        "title": pd["videoDetails"]["title"],
+        "id": vid_id,
+        "channelName": pd["videoDetails"]["author"],
+        "channelURL": "https://www.youtube.com/channel/" + chan_id,
+        "description": pd["videoDetails"]["shortDescription"],
+        "thumbnailUrl": thumb_url,
+        "startTimestamp": start_ts,
+    }
+
+    if [x for x in vid_id if x not in string.ascii_letters + string.digits + "_-"]:
+        print(f"malicious json", file=sys.stderr)
+        return
+
+    basepath = os.path.dirname(sys.argv[1])
+
+    thumb_fn = f"{basepath}/{vid_id}.jpg"
+    tmp_fn = f"{thumb_fn}.{os.getpid()}"
+    if not os.path.exists(thumb_fn) and (
+        thumb_url.startswith("https://img.youtube.com/vi/")
+        or thumb_url.startswith("https://i.ytimg.com/vi/")
+    ):
+        try:
+            with urllib.request.urlopen(thumb_url) as fi:
+                with open(tmp_fn, "wb") as fo:
+                    fo.write(fi.read())
+
+            os.rename(tmp_fn, thumb_fn)
+        except:
+            if os.path.exists(tmp_fn):
+                os.unlink(tmp_fn)
+
+    try:
+        with open(thumb_fn, "rb") as f:
+            thumb = base64.b64encode(f.read()).decode("ascii")
+    except:
+        thumb = "/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAMCAgICAgMCAgIDAwMDBAYEBAQEBAgGBgUGCQgKCgkICQkKDA8MCgsOCwkJDRENDg8QEBEQCgwSExIQEw8QEBD/yQALCAABAAEBAREA/8wABgAQEAX/2gAIAQEAAD8A0s8g/9k="
+
+    metadata["thumbnail"] = "data:image/jpeg;base64," + thumb
+
+    ret = {
+        "metadata": metadata,
+        "version": "1.5",
+        "createTime": datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ"),
+    }
+
+    for stream, itags in priority.items():
+        for itag in itags:
+            url = None
+            for afmt in pd["streamingData"]["adaptiveFormats"]:
+                if itag == afmt["itag"]:
+                    url = afmt["url"]
+                    break
+
+            if url:
+                ret[stream] = {itag: url}
+                break
+
+    fn = f"{basepath}/{vid_id}.urls.json"
+    with open(fn, "w", encoding="utf-8", errors="replace") as f:
+        f.write(json.dumps(ret, indent=4))
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except:
+        # raise
+        pass

bin/prisonparty.sh

@@ -0,0 +1,99 @@
+#!/bin/bash
+set -e
+
+# runs copyparty (or any other program really) in a chroot
+#
+# assumption: these directories, and everything within, are owned by root
+sysdirs=( /bin /lib /lib32 /lib64 /sbin /usr )
+
+
+# error-handler
+help() { cat <<'EOF'
+
+usage:
+  ./prisonparty.sh <ROOTDIR> <UID> <GID> [VOLDIR [VOLDIR...]] -- copyparty-sfx.py [...]"
+
+example:
+  ./prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt/nas/music -- copyparty-sfx.py -v /mnt/nas/music::rwmd"
+
+EOF
+exit 1
+}
+
+
+# read arguments
+trap help EXIT
+jail="$(realpath "$1")"; shift
+uid="$1"; shift
+gid="$1"; shift
+
+vols=()
+while true; do
+	v="$1"; shift
+	[ "$v" = -- ] && break  # end of volumes
+	[ "$#" -eq 0 ] && break  # invalid usage
+	vols+=( "$(realpath "$v")" )
+done
+pybin="$1"; shift
+pybin="$(realpath "$pybin")"
+cpp="$1"; shift
+cpp="$(realpath "$cpp")"
+cppdir="$(dirname "$cpp")"
+trap - EXIT
+
+
+# debug/vis
+echo
+echo "chroot-dir = $jail"
+echo "user:group = $uid:$gid"
+echo " copyparty = $cpp"
+echo
+printf '\033[33m%s\033[0m\n' "copyparty can access these folders and all their subdirectories:"
+for v in "${vols[@]}"; do
+	printf '\033[36m ├─\033[0m %s \033[36m ── added by (You)\033[0m\n' "$v"
+done
+printf '\033[36m ├─\033[0m %s \033[36m ── where the copyparty binary is\033[0m\n' "$cppdir"
+printf '\033[36m ╰─\033[0m %s \033[36m ── the folder you are currently in\033[0m\n' "$PWD"
+vols+=("$cppdir" "$PWD")
+echo
+
+
+# remove any trailing slashes
+jail="${jail%/}"
+cppdir="${cppdir%/}"
+
+
+# bind-mount system directories and volumes
+printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | LC_ALL=C sort |
+while IFS= read -r v; do
+	[ -e "$v" ] || {
+		# printf '\033[1;31mfolder does not exist:\033[0m %s\n' "/$v"
+		continue
+	}
+	i1=$(stat -c%D.%i "$v"      2>/dev/null || echo a)
+	i2=$(stat -c%D.%i "$jail$v" 2>/dev/null || echo b)
+	[ $i1 = $i2 ] && continue
+	
+	mkdir -p "$jail$v"
+	mount --bind "$v" "$jail$v"
+done
+
+
+# create a tmp
+mkdir -p "$jail/tmp"
+chmod 777 "$jail/tmp"
+
+
+# run copyparty
+/sbin/chroot --userspec=$uid:$gid "$jail" "$pybin" "$cpp" "$@" && rv=0 || rv=$?
+
+
+# cleanup if not in use
+lsof "$jail" | grep -qF "$jail" &&
+	echo "chroot is in use, will not cleanup" ||
+{
+	mount | grep -qF " on $jail" |
+	awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' |
+	LC_ALL=C sort -r  | tee /dev/stderr | tr '\n' '\0' | xargs -r0 umount
+}
+exit $rv

contrib/openrc/copyparty

@@ -8,11 +8,11 @@
 #
 # you may want to:
 #   change '/usr/bin/python' to another interpreter
-#   change '/mnt::a' to another location or permission-set
+#   change '/mnt::rw' to another location or permission-set
 
 name="$SVCNAME"
 command_background=true
 pidfile="/var/run/$SVCNAME.pid"
 
 command="/usr/bin/python /usr/local/bin/copyparty-sfx.py"
-command_args="-q -v /mnt::a"
+command_args="-q -v /mnt::rw"

contrib/systemd/copyparty.service

@@ -6,13 +6,20 @@
 #
 # you may want to:
 #   change '/usr/bin/python' to another interpreter
-#   change '/mnt::a' to another location or permission-set
+#   change '/mnt::rw' to another location or permission-set
 #
 # with `Type=notify`, copyparty will signal systemd when it is ready to
 #   accept connections; correctly delaying units depending on copyparty.
 #   But note that journalctl will get the timestamps wrong due to
 #   python disabling line-buffering, so messages are out-of-order:
 #   https://user-images.githubusercontent.com/241032/126040249-cb535cc7-c599-4931-a796-a5d9af691bad.png
+#
+# enable line-buffering for realtime logging (slight performance cost):
+#   modify ExecStart and prefix it with `/usr/bin/stdbuf -oL` like so:
+#   ExecStart=/usr/bin/stdbuf -oL /usr/bin/python3 [...]
+# but some systemd versions require this instead (higher performance cost):
+#   inside the [Service] block, add the following line:
+#   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
 Description=copyparty file server
@@ -20,7 +27,7 @@ Description=copyparty file server
 [Service]
 Type=notify
 SyslogIdentifier=copyparty
-ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::a
+ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
 ExecStartPre=/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
 
 [Install]

contrib/systemd/prisonparty.service

@@ -0,0 +1,27 @@
+# this will start `/usr/local/bin/copyparty-sfx.py`
+# in a chroot, preventing accidental access elsewhere
+# and share '/mnt' with anonymous read+write
+#
+# installation:
+#   1) put copyparty-sfx.py and prisonparty.sh in /usr/local/bin
+#   2) cp -pv prisonparty.service /etc/systemd/system && systemctl enable --now prisonparty
+#
+# you may want to:
+#   change '/mnt::rw' to another location or permission-set
+#    (remember to change the '/mnt' chroot arg too)
+#
+# enable line-buffering for realtime logging (slight performance cost):
+#   inside the [Service] block, add the following line:
+#   Environment=PYTHONUNBUFFERED=x
+
+[Unit]
+Description=copyparty file server
+
+[Service]
+SyslogIdentifier=prisonparty
+WorkingDirectory=/usr/local/bin
+ExecStart=/bin/bash /usr/local/bin/prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt -- \
+  /usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
+
+[Install]
+WantedBy=multi-user.target

copyparty/__init__.py

@@ -25,6 +25,28 @@ ANYWIN = WINDOWS or sys.platform in ["msys"]
 MACOS = platform.system() == "Darwin"
 
 
+def get_unix_home():
+    try:
+        v = os.environ["XDG_CONFIG_HOME"]
+        if not v:
+            raise Exception()
+        ret = os.path.normpath(v)
+        os.listdir(ret)
+        return ret
+    except:
+        pass
+
+    try:
+        v = os.path.expanduser("~/.config")
+        if v.startswith("~"):
+            raise Exception()
+        ret = os.path.normpath(v)
+        os.listdir(ret)
+        return ret
+    except:
+        return "/tmp"
+
+
 class EnvParams(object):
     def __init__(self):
         self.t0 = time.time()
@@ -37,10 +59,7 @@ class EnvParams(object):
         elif sys.platform == "darwin":
             self.cfg = os.path.expanduser("~/Library/Preferences/copyparty")
         else:
-            self.cfg = os.path.normpath(
+            self.cfg = get_unix_home() + "/copyparty"
-                os.getenv("XDG_CONFIG_HOME", os.path.expanduser("~/.config"))
-                + "/copyparty"
-            )
 
         self.cfg = self.cfg.replace("\\", "/")
         try:

copyparty/__main__.py

@@ -23,7 +23,8 @@ from textwrap import dedent
 from .__init__ import E, WINDOWS, VT100, PY2, unicode
 from .__version__ import S_VERSION, S_BUILD_DT, CODENAME
 from .svchub import SvcHub
-from .util import py_desc, align_tab, IMPLICATIONS
+from .util import py_desc, align_tab, IMPLICATIONS, ansi_re
+from .authsrv import re_vol
 
 HAVE_SSL = True
 try:
@@ -66,8 +67,12 @@ class Dodge11874(RiceFormatter):
 def lprint(*a, **ka):
     global printed
 
-    printed += " ".join(unicode(x) for x in a) + ka.get("end", "\n")
+    txt = " ".join(unicode(x) for x in a) + ka.get("end", "\n")
-    print(*a, **ka)
+    printed += txt
+    if not VT100:
+        txt = ansi_re.sub("", txt)
+
+    print(txt, **ka)
 
 
 def warn(msg):
@@ -196,12 +201,18 @@ def run_argparse(argv, formatter):
         formatter_class=formatter,
         prog="copyparty",
         description="http file sharing hub v{} ({})".format(S_VERSION, S_BUILD_DT),
-        epilog=dedent(
+    )
-            """
+
+    sects = [
+        [
+            "accounts",
+            "accounts and volumes",
+            dedent(
+                """
             -a takes username:password,
-            -v takes src:dst:perm1:perm2:permN:cflag1:cflag2:cflagN:...
+            -v takes src:dst:perm1:perm2:permN:volflag1:volflag2:volflagN:...
                where "perm" is "accesslevels,username1,username2,..."
-               and "cflag" is config flags to set on this volume
+               and "volflag" is config flags to set on this volume
             
             list of accesslevels:
               "r" (read):   list folder contents, download files
@@ -209,11 +220,7 @@ def run_argparse(argv, formatter):
               "m" (move):   move files and folders; need "w" at destination
               "d" (delete): permanently delete files and folders
 
-            list of cflags:
+            too many volflags to list here, see the other sections
-              "c,nodupe" rejects existing files (instead of symlinking them)
-              "c,e2d" sets -e2d (all -e2* args can be set using ce2* cflags)
-              "c,d2t" disables metadata collection, overrides -e2t*
-              "c,d2d" disables all database stuff, overrides -e2*
 
             example:\033[35m
               -a ed:hunter2 -v .::r:rw,ed -v ../inc:dump:w:rw,ed:c,nodupe  \033[36m
@@ -230,29 +237,86 @@ def run_argparse(argv, formatter):
 
             consider the config file for more flexible account/volume management,
             including dynamic reload at runtime (and being more readable w)
+            """
+            ),
+        ],
+        [
+            "flags",
+            "list of volflags",
+            dedent(
+                """
+            volflags are appended to volume definitions, for example,
+            to create a write-only volume with the \033[33mnodupe\033[0m and \033[32mnosub\033[0m flags:
+              \033[35m-v /mnt/inc:/inc:w\033[33m:c,nodupe\033[32m:c,nosub
 
+            \033[0muploads, general:
+              \033[36mnodupe\033[35m rejects existing files (instead of symlinking them)
+              \033[36mnosub\033[35m forces all uploads into the top folder of the vfs
+              \033[36mgz\033[35m allows server-side gzip of uploads with ?gz (also c,xz)
+              \033[36mpk\033[35m forces server-side compression, optional arg: xz,9
+            
+            \033[0mupload rules:
+              \033[36mmaxn=250,600\033[35m max 250 uploads over 15min
+              \033[36mmaxb=1g,300\033[35m max 1 GiB over 5min (suffixes: b, k, m, g)
+              \033[36msz=1k-3m\033[35m allow filesizes between 1 KiB and 3MiB
+            
+            \033[0mupload rotation:
+            (moves all uploads into the specified folder structure)
+              \033[36mrotn=100,3\033[35m 3 levels of subfolders with 100 entries in each
+              \033[36mrotf=%Y-%m/%d-%H\033[35m date-formatted organizing
+              \033[36mlifetime=3600\033[35m uploads are deleted after 1 hour
+            
+            \033[0mdatabase, general:
+              \033[36me2d\033[35m sets -e2d (all -e2* args can be set using ce2* volflags)
+              \033[36md2t\033[35m disables metadata collection, overrides -e2t*
+              \033[36md2d\033[35m disables all database stuff, overrides -e2*
+              \033[36mdhash\033[35m disables file hashing on initial scans, also ehash
+              \033[36mhist=/tmp/cdb\033[35m puts thumbnails and indexes at that location
+              \033[36mscan=60\033[35m scan for new files every 60sec, same as --re-maxage
+            
+            \033[0mdatabase, audio tags:
+            "mte", "mth", "mtp", "mtm" all work the same as -mte, -mth, ...
+              \033[36mmtp=.bpm=f,audio-bpm.py\033[35m uses the "audio-bpm.py" program to
+                generate ".bpm" tags from uploads (f = overwrite tags)
+              \033[36mmtp=ahash,vhash=media-hash.py\033[35m collects two tags at once
+            \033[0m"""
+            ),
+        ],
+        [
+            "urlform",
+            "",
+            dedent(
+                """
             values for --urlform:
-              "stash" dumps the data to file and returns length + checksum
+              \033[36mstash\033[35m dumps the data to file and returns length + checksum
-              "save,get" dumps to file and returns the page like a GET
+              \033[36msave,get\033[35m dumps to file and returns the page like a GET
-              "print,get" prints the data in the log and returns GET
+              \033[36mprint,get\033[35m prints the data in the log and returns GET
               (leave out the ",get" to return an error instead)
-
+            """
-            values for --ls:
+            ),
-              "USR" is a user to browse as; * is anonymous, ** is all users
+        ],
-              "VOL" is a single volume to scan, default is * (all vols)
+        [
-              "FLAG" is flags;
+            "ls",
-                "v" in addition to realpaths, print usernames and vpaths
+            "volume inspection",
-                "ln" only prints symlinks leaving the volume mountpoint
+            dedent(
-                "p" exits 1 if any such symlinks are found
+                """
-                "r" resumes startup after the listing
+            \033[35m--ls USR,VOL,FLAGS
+              \033[36mUSR\033[0m is a user to browse as; * is anonymous, ** is all users
+              \033[36mVOL\033[0m is a single volume to scan, default is * (all vols)
+              \033[36mFLAG\033[0m is flags;
+                \033[36mv\033[0m in addition to realpaths, print usernames and vpaths
+                \033[36mln\033[0m only prints symlinks leaving the volume mountpoint
+                \033[36mp\033[0m exits 1 if any such symlinks are found
+                \033[36mr\033[0m resumes startup after the listing
             examples:
               --ls '**'          # list all files which are possible to read
               --ls '**,*,ln'     # check for dangerous symlinks
               --ls '**,*,ln,p,r' # check, then start normally if safe
-            \033[0m
             """
-        ),
+            ),
-    )
+        ],
+    ]
+
     # fmt: off
     u = unicode
     ap2 = ap.add_argument_group('general options')
@@ -264,9 +328,12 @@ def run_argparse(argv, formatter):
     ap2.add_argument("-ed", action="store_true", help="enable ?dots")
     ap2.add_argument("-emp", action="store_true", help="enable markdown plugins")
     ap2.add_argument("-mcr", metavar="SEC", type=int, default=60, help="md-editor mod-chk rate")
+    ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-forms; examples: [stash], [save,get]")
+
+    ap2 = ap.add_argument_group('upload options')
     ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads")
     ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="up2k min.size threshold (mswin-only)")
-    ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-forms; examples: [stash], [save,get]")
+    ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled")
 
     ap2 = ap.add_argument_group('network options')
     ap2.add_argument("-i", metavar="IP", type=u, default="0.0.0.0", help="ip to bind (comma-sep.)")
@@ -288,6 +355,7 @@ def run_argparse(argv, formatter):
     ap2.add_argument("-nih", action="store_true", help="no info hostname")
     ap2.add_argument("-nid", action="store_true", help="no info disk-usage")
     ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
+    ap2.add_argument("--no-lifetime", action="store_true", help="disable automatic deletion of uploads after a certain time (lifetime volflag)")
 
     ap2 = ap.add_argument_group('safety options')
     ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="scan all volumes; arguments USER,VOL,FLAGS; example [**,*,ln,p,r]")
@@ -319,25 +387,29 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age")
     ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat for")
 
-    ap2 = ap.add_argument_group('database options')
+    ap2 = ap.add_argument_group('general db options')
     ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
     ap2.add_argument("-e2ds", action="store_true", help="enable up2k db-scanner, sets -e2d")
     ap2.add_argument("-e2dsa", action="store_true", help="scan all folders (for search), sets -e2ds")
+    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs)")
+    ap2.add_argument("--no-hash", action="store_true", help="disable hashing during e2ds folder scans")
+    ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
+    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval, 0=off, can be set per-volume with the 'scan' volflag")
+    ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
+    
+    ap2 = ap.add_argument_group('metadata db options')
     ap2.add_argument("-e2t", action="store_true", help="enable metadata indexing")
     ap2.add_argument("-e2ts", action="store_true", help="enable metadata scanner, sets -e2t")
     ap2.add_argument("-e2tsr", action="store_true", help="rescan all metadata, sets -e2ts")
-    ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume state")
-    ap2.add_argument("--no-hash", action="store_true", help="disable hashing during e2ds folder scans")
     ap2.add_argument("--no-mutagen", action="store_true", help="use FFprobe for tags instead")
     ap2.add_argument("--no-mtag-mt", action="store_true", help="disable tag-read parallelism")
     ap2.add_argument("--no-mtag-ff", action="store_true", help="never use FFprobe as tag reader")
-    ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
-    ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval (0=off)")
     ap2.add_argument("-mtm", metavar="M=t,t,t", type=u, action="append", help="add/replace metadata mapping")
     ap2.add_argument("-mte", metavar="M,M,M", type=u, help="tags to index/display (comma-sep.)",
-        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,ac,vc,res,.fps")
+        default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,vc,ac,res,.fps,ahash,vhash")
+    ap2.add_argument("-mth", metavar="M,M,M", type=u, help="tags to hide by default (comma-sep.)",
+        default=".vq,.aq,vc,ac,res,.fps")
     ap2.add_argument("-mtp", metavar="M=[f,]bin", type=u, action="append", help="read tag M using bin")
-    ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
 
     ap2 = ap.add_argument_group('appearance options')
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
@@ -349,10 +421,22 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead")
     ap2.add_argument("--stackmon", metavar="P,S", type=u, help="write stacktrace to Path every S second")
     ap2.add_argument("--log-thrs", metavar="SEC", type=float, help="list active threads every SEC")
-    
-    return ap.parse_args(args=argv[1:])
     # fmt: on
 
+    ap2 = ap.add_argument_group("help sections")
+    for k, h, _ in sects:
+        ap2.add_argument("--help-" + k, action="store_true", help=h)
+
+    ret = ap.parse_args(args=argv[1:])
+    for k, h, t in sects:
+        k2 = "help_" + k.replace("-", "_")
+        if vars(ret)[k2]:
+            lprint("# {} help page".format(k))
+            lprint(t + "\033[0m")
+            sys.exit(0)
+
+    return ret
+
 
 def main(argv=None):
     time.strptime("19970815", "%Y%m%d")  # python#7980
@@ -391,13 +475,19 @@ def main(argv=None):
     nstrs = []
     anymod = False
     for ostr in al.v or []:
+        m = re_vol.match(ostr)
+        if not m:
+            # not our problem
+            nstrs.append(ostr)
+            continue
+
+        src, dst, perms = m.groups()
+        na = [src, dst]
         mod = False
-        oa = ostr.split(":")
+        for opt in perms.split(":"):
-        na = oa[:2]
-        for opt in oa[2:]:
             if re.match("c[^,]", opt):
                 mod = True
-                na.append("c," + opt[2:])
+                na.append("c," + opt[1:])
             elif re.sub("^[rwmd]*", "", opt) and "," not in opt:
                 mod = True
                 perm = opt[0]

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 12, 0)
+VERSION = (0, 13, 2)
-CODENAME = "fil\033[33med"
+CODENAME = "future-proof"
-BUILD_DT = (2021, 7, 28)
+BUILD_DT = (2021, 8, 12)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -5,15 +5,29 @@ import re
 import os
 import sys
 import stat
+import time
 import base64
 import hashlib
 import threading
+from datetime import datetime
 
 from .__init__ import WINDOWS
-from .util import IMPLICATIONS, uncyg, undot, absreal, Pebkac, fsdec, fsenc, statdir
+from .util import (
+    IMPLICATIONS,
+    uncyg,
+    undot,
+    unhumanize,
+    absreal,
+    Pebkac,
+    fsenc,
+    statdir,
+)
 from .bos import bos
 
 
+LEELOO_DALLAS = "leeloo_dallas"
+
+
 class AXS(object):
     def __init__(self, uread=None, uwrite=None, umove=None, udel=None):
         self.uread = {} if uread is None else {k: 1 for k in uread}
@@ -30,6 +44,156 @@ class AXS(object):
         )
 
 
+class Lim(object):
+    def __init__(self):
+        self.nups = {}  # num tracker
+        self.bups = {}  # byte tracker list
+        self.bupc = {}  # byte tracker cache
+
+        self.nosub = False  # disallow subdirectories
+
+        self.smin = None  # filesize min
+        self.smax = None  # filesize max
+
+        self.bwin = None  # bytes window
+        self.bmax = None  # bytes max
+        self.nwin = None  # num window
+        self.nmax = None  # num max
+
+        self.rotn = None  # rot num files
+        self.rotl = None  # rot depth
+        self.rotf = None  # rot datefmt
+        self.rot_re = None  # rotf check
+
+    def set_rotf(self, fmt):
+        self.rotf = fmt
+        r = re.escape(fmt).replace("%Y", "[0-9]{4}").replace("%j", "[0-9]{3}")
+        r = re.sub("%[mdHMSWU]", "[0-9]{2}", r)
+        self.rot_re = re.compile("(^|/)" + r + "$")
+
+    def all(self, ip, rem, sz, abspath):
+        self.chk_nup(ip)
+        self.chk_bup(ip)
+        self.chk_rem(rem)
+        if sz != -1:
+            self.chk_sz(sz)
+
+        ap2, vp2 = self.rot(abspath)
+        if abspath == ap2:
+            return ap2, rem
+
+        return ap2, ("{}/{}".format(rem, vp2) if rem else vp2)
+
+    def chk_sz(self, sz):
+        if self.smin is not None and sz < self.smin:
+            raise Pebkac(400, "file too small")
+
+        if self.smax is not None and sz > self.smax:
+            raise Pebkac(400, "file too big")
+
+    def chk_rem(self, rem):
+        if self.nosub and rem:
+            raise Pebkac(500, "no subdirectories allowed")
+
+    def rot(self, path):
+        if not self.rotf and not self.rotn:
+            return path, ""
+
+        if self.rotf:
+            path = path.rstrip("/\\")
+            if self.rot_re.search(path.replace("\\", "/")):
+                return path, ""
+
+            suf = datetime.utcnow().strftime(self.rotf)
+            if path:
+                path += "/"
+
+            return path + suf, suf
+
+        ret = self.dive(path, self.rotl)
+        if not ret:
+            raise Pebkac(500, "no available slots in volume")
+
+        d = ret[len(path) :].strip("/\\").replace("\\", "/")
+        return ret, d
+
+    def dive(self, path, lvs):
+        items = bos.listdir(path)
+
+        if not lvs:
+            # at leaf level
+            return None if len(items) >= self.rotn else ""
+
+        dirs = [int(x) for x in items if x and all(y in "1234567890" for y in x)]
+        dirs.sort()
+
+        if not dirs:
+            # no branches yet; make one
+            sub = os.path.join(path, "0")
+            bos.mkdir(sub)
+        else:
+            # try newest branch only
+            sub = os.path.join(path, str(dirs[-1]))
+
+        ret = self.dive(sub, lvs - 1)
+        if ret is not None:
+            return os.path.join(sub, ret)
+
+        if len(dirs) >= self.rotn:
+            # full branch or root
+            return None
+
+        # make a branch
+        sub = os.path.join(path, str(dirs[-1] + 1))
+        bos.mkdir(sub)
+        ret = self.dive(sub, lvs - 1)
+        if ret is None:
+            raise Pebkac(500, "rotation bug")
+
+        return os.path.join(sub, ret)
+
+    def nup(self, ip):
+        try:
+            self.nups[ip].append(time.time())
+        except:
+            self.nups[ip] = [time.time()]
+
+    def bup(self, ip, nbytes):
+        v = [time.time(), nbytes]
+        try:
+            self.bups[ip].append(v)
+            self.bupc[ip] += nbytes
+        except:
+            self.bups[ip] = [v]
+            self.bupc[ip] = nbytes
+
+    def chk_nup(self, ip):
+        if not self.nmax or ip not in self.nups:
+            return
+
+        nups = self.nups[ip]
+        cutoff = time.time() - self.nwin
+        while nups and nups[0] < cutoff:
+            nups.pop(0)
+
+        if len(nups) >= self.nmax:
+            raise Pebkac(429, "too many uploads")
+
+    def chk_bup(self, ip):
+        if not self.bmax or ip not in self.bups:
+            return
+
+        bups = self.bups[ip]
+        cutoff = time.time() - self.bwin
+        mark = self.bupc[ip]
+        while bups and bups[0][0] < cutoff:
+            mark -= bups.pop(0)[1]
+
+        self.bupc[ip] = mark
+        if mark >= self.bmax:
+            raise Pebkac(429, "ingress saturated")
+
+
 class VFS(object):
     """single level in the virtual fs"""
 
@@ -42,6 +206,7 @@ class VFS(object):
         self.nodes = {}  # child nodes
         self.histtab = None  # all realpath->histpath
         self.dbv = None  # closest full/non-jump parent
+        self.lim = None  # type: Lim  # upload limits; only set for dbv
 
         if realpath:
             self.histpath = os.path.join(realpath, ".hist")  # db / thumbcache
@@ -165,13 +330,14 @@ class VFS(object):
             [will_move, c.umove, "move"],
             [will_del, c.udel, "delete"],
         ]:
-            if req and (uname not in d and "*" not in d):
+            if req and (uname not in d and "*" not in d) and uname != LEELOO_DALLAS:
                 m = "you don't have {}-access for this location"
                 raise Pebkac(403, m.format(msg))
 
         return vn, rem
 
     def get_dbv(self, vrem):
+        # type: (str) -> tuple[VFS, str]
         dbv = self.dbv
         if not dbv:
             return self, vrem
@@ -310,6 +476,12 @@ class VFS(object):
                 yield f
 
 
+if WINDOWS:
+    re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
+else:
+    re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
+
+
 class AuthSrv(object):
     """verifies users against given paths"""
 
@@ -319,11 +491,6 @@ class AuthSrv(object):
         self.warn_anonwrite = warn_anonwrite
         self.line_ctr = 0
 
-        if WINDOWS:
-            self.re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
-        else:
-            self.re_vol = re.compile(r"^([^:]*):([^:]*):(.*)$")
-
         self.mutex = threading.Lock()
         self.reload()
 
@@ -390,6 +557,9 @@ class AuthSrv(object):
 
     def _read_vol_str(self, lvl, uname, axs, flags):
         # type: (str, str, AXS, any) -> None
+        if lvl.strip("crwmd"):
+            raise Exception("invalid volume flag: {},{}".format(lvl, uname))
+
         if lvl == "c":
             cval = True
             if "=" in uname:
@@ -401,17 +571,18 @@ class AuthSrv(object):
         if uname == "":
             uname = "*"
 
-        if "r" in lvl:
+        for un in uname.split(","):
-            axs.uread[uname] = 1
+            if "r" in lvl:
+                axs.uread[un] = 1
 
-        if "w" in lvl:
+            if "w" in lvl:
-            axs.uwrite[uname] = 1
+                axs.uwrite[un] = 1
 
-        if "m" in lvl:
+            if "m" in lvl:
-            axs.umove[uname] = 1
+                axs.umove[un] = 1
 
-        if "d" in lvl:
+            if "d" in lvl:
-            axs.udel[uname] = 1
+                axs.udel[un] = 1
 
     def _read_volflag(self, flags, name, value, is_list):
         if name not in ["mtp"]:
@@ -452,7 +623,7 @@ class AuthSrv(object):
             # list of src:dst:permset:permset:...
             # permset is <rwmd>[,username][,username] or <c>,<flag>[=args]
             for v_str in self.args.v:
-                m = self.re_vol.match(v_str)
+                m = re_vol.match(v_str)
                 if not m:
                     raise Exception("invalid -v argument: [{}]".format(v_str))
 
@@ -544,6 +715,9 @@ class AuthSrv(object):
             )
             raise Exception("invalid config")
 
+        if LEELOO_DALLAS in all_users:
+            raise Exception("sorry, reserved username: " + LEELOO_DALLAS)
+
         promote = []
         demote = []
         for vol in vfs.all_vols.values():
@@ -602,6 +776,51 @@ class AuthSrv(object):
 
         vfs.histtab = {v.realpath: v.histpath for v in vfs.all_vols.values()}
 
+        for vol in vfs.all_vols.values():
+            lim = Lim()
+            use = False
+
+            if vol.flags.get("nosub"):
+                use = True
+                lim.nosub = True
+
+            v = vol.flags.get("sz")
+            if v:
+                use = True
+                lim.smin, lim.smax = [unhumanize(x) for x in v.split("-")]
+
+            v = vol.flags.get("rotn")
+            if v:
+                use = True
+                lim.rotn, lim.rotl = [int(x) for x in v.split(",")]
+
+            v = vol.flags.get("rotf")
+            if v:
+                use = True
+                lim.set_rotf(v)
+
+            v = vol.flags.get("maxn")
+            if v:
+                use = True
+                lim.nmax, lim.nwin = [int(x) for x in v.split(",")]
+
+            v = vol.flags.get("maxb")
+            if v:
+                use = True
+                lim.bmax, lim.bwin = [unhumanize(x) for x in v.split(",")]
+
+            if use:
+                vol.lim = lim
+
+        for vol in vfs.all_vols.values():
+            if "pk" in vol.flags and "gz" not in vol.flags and "xz" not in vol.flags:
+                vol.flags["gz"] = False  # def.pk
+
+            if "scan" in vol.flags:
+                vol.flags["scan"] = int(vol.flags["scan"])
+            elif self.args.re_maxage:
+                vol.flags["scan"] = self.args.re_maxage
+
         all_mte = {}
         errors = False
         for vol in vfs.all_vols.values():
@@ -623,9 +842,11 @@ class AuthSrv(object):
                 if k1 in vol.flags:
                     vol.flags[k2] = True
 
-            # default tag-list if unset
+            # default tag cfgs if unset
             if "mte" not in vol.flags:
                 vol.flags["mte"] = self.args.mte
+            if "mth" not in vol.flags:
+                vol.flags["mth"] = self.args.mth
 
             # append parsers from argv to volume-flags
             self._read_volflag(vol.flags, "mtp", self.args.mtp, True)
@@ -795,7 +1016,7 @@ class AuthSrv(object):
 
                 atop = vn.realpath
                 g = vn.walk(
-                    "", "", [], u, True, [[True]], not self.args.no_scandir, False
+                    vn.vpath, "", [], u, [[True]], True, not self.args.no_scandir, False
                 )
                 for _, _, vpath, apath, files, _, _ in g:
                     fnames = [n[0] for n in files]

copyparty/broker_mpw.py

@@ -1,6 +1,5 @@
 # coding: utf-8
 from __future__ import print_function, unicode_literals
-from copyparty.authsrv import AuthSrv
 
 import sys
 import signal
@@ -9,6 +8,7 @@ import threading
 from .broker_util import ExceptionalQueue
 from .httpsrv import HttpSrv
 from .util import FAKE_MP
+from copyparty.authsrv import AuthSrv
 
 
 class MpWorker(object):

copyparty/httpcli.py

@@ -13,10 +13,15 @@ import ctypes
 from datetime import datetime
 import calendar
 
+try:
+    import lzma
+except:
+    pass
+
 from .__init__ import E, PY2, WINDOWS, ANYWIN, unicode
 from .util import *  # noqa  # pylint: disable=unused-wildcard-import
 from .bos import bos
-from .authsrv import AuthSrv
+from .authsrv import AuthSrv, Lim
 from .szip import StreamZip
 from .star import StreamTar
 
@@ -61,7 +66,10 @@ class HttpCli(object):
         a, b = m.groups()
         return "=\033[7m {} \033[27m{}".format(self.asrv.iacct[a], b)
 
-    def _check_nonfatal(self, ex):
+    def _check_nonfatal(self, ex, post):
+        if post:
+            return ex.code < 300
+
         return ex.code < 400 or ex.code in [404, 429]
 
     def _assert_safe_rem(self, rem):
@@ -103,8 +111,8 @@ class HttpCli(object):
             self.req = "[junk]"
             self.http_ver = "HTTP/1.1"
             # self.log("pebkac at httpcli.run #1: " + repr(ex))
-            self.keepalive = self._check_nonfatal(ex)
+            self.keepalive = False
-            self.loud_reply(unicode(ex), status=ex.code)
+            self.loud_reply(unicode(ex), status=ex.code, volsan=True)
             return self.keepalive
 
         # time.sleep(0.4)
@@ -177,9 +185,12 @@ class HttpCli(object):
                 if kc in cookies and ku not in uparam:
                     uparam[ku] = cookies[kc]
 
+        if len(uparam) > 10 or len(cookies) > 50:
+            raise Pebkac(400, "u wot m8")
+
         self.uparam = uparam
         self.cookies = cookies
-        self.vpath = unquotep(vpath)
+        self.vpath = unquotep(vpath)  # not query, so + means +
 
         pwd = uparam.get("pw")
         self.uname = self.asrv.iacct.get(pwd, "*")
@@ -213,18 +224,24 @@ class HttpCli(object):
             else:
                 raise Pebkac(400, 'invalid HTTP mode "{0}"'.format(self.mode))
 
-        except Pebkac as ex:
+        except Exception as ex:
+            pex = ex
+            if not hasattr(ex, "code"):
+                pex = Pebkac(500)
+
             try:
-                # self.log("pebkac at httpcli.run #2: " + repr(ex))
+                post = self.mode in ["POST", "PUT"] or "content-length" in self.headers
-                if not self._check_nonfatal(ex):
+                if not self._check_nonfatal(pex, post):
                     self.keepalive = False
 
-                self.log("{}\033[0m, {}".format(str(ex), self.vpath), 3)
+                msg = str(ex) if pex == ex else min_ex()
+                self.log("{}\033[0m, {}".format(msg, self.vpath), 3)
+
                 msg = "<pre>{}\r\nURL: {}\r\n".format(str(ex), self.vpath)
                 if self.hint:
                     msg += "hint: {}\r\n".format(self.hint)
 
-                self.reply(msg.encode("utf-8", "replace"), status=ex.code)
+                self.reply(msg.encode("utf-8", "replace"), status=pex.code, volsan=True)
                 return self.keepalive
             except Pebkac:
                 return False
@@ -257,8 +274,12 @@ class HttpCli(object):
         except:
             raise Pebkac(400, "client d/c while replying headers")
 
-    def reply(self, body, status=200, mime=None, headers=None):
+    def reply(self, body, status=200, mime=None, headers=None, volsan=False):
         # TODO something to reply with user-supplied values safely
+
+        if volsan:
+            body = vol_san(self.asrv.vfs.all_vols.values(), body)
+
         self.send_headers(len(body), status, mime, headers)
 
         try:
@@ -342,11 +363,36 @@ class HttpCli(object):
             static_path = os.path.join(E.mod, "web/", self.vpath[5:])
             return self.tx_file(static_path)
 
+        x = self.asrv.vfs.can_access(self.vpath, self.uname)
+        self.can_read, self.can_write, self.can_move, self.can_delete = x
+        if not self.can_read and not self.can_write:
+            if self.vpath:
+                self.log("inaccessible: [{}]".format(self.vpath))
+                raise Pebkac(404)
+
+            self.uparam["h"] = False
+
         if "tree" in self.uparam:
             return self.tx_tree()
 
-        if "stack" in self.uparam:
+        if "delete" in self.uparam:
-            return self.tx_stack()
+            return self.handle_rm()
+
+        if "move" in self.uparam:
+            return self.handle_mv()
+
+        if "scan" in self.uparam:
+            return self.scanvol()
+
+        if not self.vpath:
+            if "stack" in self.uparam:
+                return self.tx_stack()
+
+            if "ups" in self.uparam:
+                return self.tx_ups()
+
+            if "h" in self.uparam:
+                return self.tx_mounts()
 
         # conditional redirect to single volumes
         if self.vpath == "" and not self.ouparam:
@@ -362,28 +408,6 @@ class HttpCli(object):
                     self.redirect(vpath, flavor="redirecting to", use302=True)
                     return True
 
-        x = self.asrv.vfs.can_access(self.vpath, self.uname)
-        self.can_read, self.can_write, self.can_move, self.can_delete = x
-        if not self.can_read and not self.can_write:
-            if self.vpath:
-                self.log("inaccessible: [{}]".format(self.vpath))
-                raise Pebkac(404)
-
-            self.uparam = {"h": False}
-
-        if "delete" in self.uparam:
-            return self.handle_rm()
-
-        if "move" in self.uparam:
-            return self.handle_mv()
-
-        if "h" in self.uparam:
-            self.vpath = None
-            return self.tx_mounts()
-
-        if "scan" in self.uparam:
-            return self.scanvol()
-
         return self.tx_browser()
 
     def handle_options(self):
@@ -484,7 +508,11 @@ class HttpCli(object):
     def dump_to_file(self):
         reader, remains = self.get_body_reader()
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
+        lim = vfs.get_dbv(rem)[0].lim
         fdir = os.path.join(vfs.realpath, rem)
+        if lim:
+            fdir, rem = lim.all(self.ip, rem, remains, fdir)
+            bos.makedirs(fdir)
 
         addr = self.ip.replace(":", ".")
         fn = "put-{:.6f}-{}.bin".format(time.time(), addr)
@@ -492,13 +520,81 @@ class HttpCli(object):
         if self.args.nw:
             path = os.devnull
 
-        with open(fsenc(path), "wb", 512 * 1024) as f:
+        open_f = open
+        open_a = [fsenc(path), "wb", 512 * 1024]
+        open_ka = {}
+
+        # user-request || config-force
+        if ("gz" in vfs.flags or "xz" in vfs.flags) and (
+            "pk" in vfs.flags
+            or "pk" in self.uparam
+            or "gz" in self.uparam
+            or "xz" in self.uparam
+        ):
+            fb = {"gz": 9, "xz": 0}  # default/fallback level
+            lv = {}  # selected level
+            alg = None  # selected algo (gz=preferred)
+
+            # user-prefs first
+            if "gz" in self.uparam or "pk" in self.uparam:  # def.pk
+                alg = "gz"
+            if "xz" in self.uparam:
+                alg = "xz"
+            if alg:
+                v = self.uparam.get(alg)
+                lv[alg] = fb[alg] if v is None else int(v)
+
+            if alg not in vfs.flags:
+                alg = "gz" if "gz" in vfs.flags else "xz"
+
+            # then server overrides
+            pk = vfs.flags.get("pk")
+            if pk is not None:
+                # config-forced on
+                alg = alg or "gz"  # def.pk
+                try:
+                    # config-forced opts
+                    alg, lv = pk.split(",")
+                    lv[alg] = int(lv)
+                except:
+                    pass
+
+            lv[alg] = lv.get(alg) or fb.get(alg)
+
+            self.log("compressing with {} level {}".format(alg, lv.get(alg)))
+            if alg == "gz":
+                open_f = gzip.GzipFile
+                open_a = [fsenc(path), "wb", lv[alg], None, 0x5FEE6600]  # 2021-01-01
+            elif alg == "xz":
+                open_f = lzma.open
+                open_a = [fsenc(path), "wb"]
+                open_ka = {"preset": lv[alg]}
+            else:
+                self.log("fallthrough? thats a bug", 1)
+
+        with open_f(*open_a, **open_ka) as f:
             post_sz, _, sha_b64 = hashcopy(reader, f)
 
+        if lim:
+            lim.nup(self.ip)
+            lim.bup(self.ip, post_sz)
+            try:
+                lim.chk_sz(post_sz)
+            except:
+                bos.unlink(path)
+                raise
+
         if not self.args.nw:
             vfs, vrem = vfs.get_dbv(rem)
             self.conn.hsrv.broker.put(
-                False, "up2k.hash_file", vfs.realpath, vfs.flags, vrem, fn
+                False,
+                "up2k.hash_file",
+                vfs.realpath,
+                vfs.flags,
+                vrem,
+                fn,
+                self.ip,
+                time.time(),
             )
 
         return post_sz, sha_b64, remains, path
@@ -569,7 +665,7 @@ class HttpCli(object):
         try:
             remains = int(self.headers["content-length"])
         except:
-            raise Pebkac(400, "you must supply a content-length for JSON POST")
+            raise Pebkac(411)
 
         if remains > 1024 * 1024:
             raise Pebkac(413, "json 2big")
@@ -592,17 +688,17 @@ class HttpCli(object):
         if "srch" in self.uparam or "srch" in body:
             return self.handle_search(body)
 
+        if "delete" in self.uparam:
+            return self.handle_rm(body)
+
         # up2k-php compat
         for k in "chunkpit.php", "handshake.php":
             if self.vpath.endswith(k):
                 self.vpath = self.vpath[: -len(k)]
 
-        sub = None
         name = undot(body["name"])
         if "/" in name:
-            sub, name = name.rsplit("/", 1)
+            raise Pebkac(400, "your client is old; press CTRL-SHIFT-R and try again")
-            self.vpath = "/".join([self.vpath, sub]).strip("/")
-            body["name"] = name
 
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         dbv, vrem = vfs.get_dbv(rem)
@@ -613,7 +709,7 @@ class HttpCli(object):
         body["addr"] = self.ip
         body["vcfg"] = dbv.flags
 
-        if sub:
+        if rem:
             try:
                 dst = os.path.join(vfs.realpath, rem)
                 if not bos.path.isdir(dst):
@@ -633,9 +729,6 @@ class HttpCli(object):
 
         x = self.conn.hsrv.broker.put(True, "up2k.handle_json", body)
         ret = x.get()
-        if sub:
-            ret["name"] = "/".join([sub, ret["name"]])
-
         ret = json.dumps(ret)
         self.log(ret)
         self.reply(ret.encode("utf-8"), mime="application/json")
@@ -863,6 +956,11 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
+        lim = vfs.get_dbv(rem)[0].lim
+        fdir_base = os.path.join(vfs.realpath, rem)
+        if lim:
+            fdir_base, rem = lim.all(self.ip, rem, -1, fdir_base)
+
         files = []
         errmsg = ""
         t0 = time.time()
@@ -872,12 +970,9 @@ class HttpCli(object):
                     self.log("discarding incoming file without filename")
                     # fallthrough
 
+                fdir = fdir_base
+                fname = sanitize_fn(p_file, "", [".prologue.html", ".epilogue.html"])
                 if p_file and not nullwrite:
-                    fdir = os.path.join(vfs.realpath, rem)
-                    fname = sanitize_fn(
-                        p_file, "", [".prologue.html", ".epilogue.html"]
-                    )
-
                     if not bos.path.isdir(fdir):
                         raise Pebkac(404, "that folder does not exist")
 
@@ -888,25 +983,43 @@ class HttpCli(object):
                     fname = os.devnull
                     fdir = ""
 
+                if lim:
+                    lim.chk_bup(self.ip)
+                    lim.chk_nup(self.ip)
+                    if not nullwrite:
+                        bos.makedirs(fdir)
+
                 try:
                     with ren_open(fname, "wb", 512 * 1024, **open_args) as f:
                         f, fname = f["orz"]
-                        self.log("writing to {}/{}".format(fdir, fname))
+                        abspath = os.path.join(fdir, fname)
+                        self.log("writing to {}".format(abspath))
                         sz, sha512_hex, _ = hashcopy(p_data, f)
                         if sz == 0:
                             raise Pebkac(400, "empty files in post")
 
-                        files.append([sz, sha512_hex, p_file, fname])
+                    if lim:
-                        dbv, vrem = vfs.get_dbv(rem)
+                        lim.nup(self.ip)
-                        self.conn.hsrv.broker.put(
+                        lim.bup(self.ip, sz)
-                            False,
+                        try:
-                            "up2k.hash_file",
+                            lim.chk_sz(sz)
-                            dbv.realpath,
+                        except:
-                            dbv.flags,
+                            bos.unlink(abspath)
-                            vrem,
+                            raise
-                            fname,
+
-                        )
+                    files.append([sz, sha512_hex, p_file, fname])
-                        self.conn.nbyte += sz
+                    dbv, vrem = vfs.get_dbv(rem)
+                    self.conn.hsrv.broker.put(
+                        False,
+                        "up2k.hash_file",
+                        dbv.realpath,
+                        dbv.flags,
+                        vrem,
+                        fname,
+                        self.ip,
+                        time.time(),
+                    )
+                    self.conn.nbyte += sz
 
                 except Pebkac:
                     if fname != os.devnull:
@@ -925,7 +1038,7 @@ class HttpCli(object):
                     raise
 
         except Pebkac as ex:
-            errmsg = unicode(ex)
+            errmsg = volsan(self.asrv.vfs.all_vols.values(), unicode(ex))
 
         td = max(0.1, time.time() - t0)
         sz_total = sum(x[0] for x in files)
@@ -1004,6 +1117,20 @@ class HttpCli(object):
         vfs, rem = self.asrv.vfs.get(self.vpath, self.uname, False, True)
         self._assert_safe_rem(rem)
 
+        clen = int(self.headers.get("content-length", -1))
+        if clen == -1:
+            raise Pebkac(411)
+
+        rp, fn = vsplit(rem)
+        fp = os.path.join(vfs.realpath, rp)
+        lim = vfs.get_dbv(rem)[0].lim
+        if lim:
+            fp, rp = lim.all(self.ip, rp, clen, fp)
+            bos.makedirs(fp)
+
+        fp = os.path.join(fp, fn)
+        rem = "{}/{}".format(rp, fn).strip("/")
+
         if not rem.endswith(".md"):
             raise Pebkac(400, "only markdown pls")
 
@@ -1015,7 +1142,6 @@ class HttpCli(object):
             self.reply(response.encode("utf-8"))
             return True
 
-        fp = os.path.join(vfs.realpath, rem)
         srv_lastmod = srv_lastmod3 = -1
         try:
             st = bos.stat(fp)
@@ -1069,6 +1195,15 @@ class HttpCli(object):
         with open(fsenc(fp), "wb", 512 * 1024) as f:
             sz, sha512, _ = hashcopy(p_data, f)
 
+        if lim:
+            lim.nup(self.ip)
+            lim.bup(self.ip, sz)
+            try:
+                lim.chk_sz(sz)
+            except:
+                bos.unlink(fp)
+                raise
+
         new_lastmod = bos.stat(fp).st_mtime
         new_lastmod3 = int(new_lastmod * 1000)
         sha512 = sha512[:56]
@@ -1291,11 +1426,9 @@ class HttpCli(object):
         else:
             fn = self.headers.get("host", "hey")
 
-        afn = "".join(
+        safe = (string.ascii_letters + string.digits).replace("%", "")
-            [x if x in (string.ascii_letters + string.digits) else "_" for x in fn]
+        afn = "".join([x if x in safe.replace('"', "") else "_" for x in fn])
-        )
+        bascii = unicode(safe).encode("utf-8")
-
-        bascii = unicode(string.ascii_letters + string.digits).encode("utf-8")
         ufn = fn.encode("utf-8", "xmlcharrefreplace")
         if PY2:
             ufn = [unicode(x) if x in bascii else "%{:02x}".format(ord(x)) for x in ufn]
@@ -1310,6 +1443,7 @@ class HttpCli(object):
 
         cdis = "attachment; filename=\"{}.{}\"; filename*=UTF-8''{}.{}"
         cdis = cdis.format(afn, fmt, ufn, fmt)
+        self.log(cdis)
         self.send_headers(None, mime=mime, headers={"Content-Disposition": cdis})
 
         fgen = vn.zipgen(rem, items, self.uname, dots, not self.args.no_scandir)
@@ -1542,14 +1676,52 @@ class HttpCli(object):
         ret["a"] = dirs
         return ret
 
-    def handle_rm(self):
+    def tx_ups(self):
-        if not self.can_delete:
+        if not self.args.unpost:
+            raise Pebkac(400, "the unpost feature was disabled by server config")
+
+        filt = self.uparam.get("filter")
+        lm = "ups [{}]".format(filt)
+        self.log(lm)
+
+        ret = []
+        t0 = time.time()
+        idx = self.conn.get_u2idx()
+        lim = time.time() - self.args.unpost
+        for vol in self.asrv.vfs.all_vols.values():
+            cur = idx.get_cur(vol.realpath)
+            if not cur:
+                continue
+
+            q = "select sz, rd, fn, at from up where ip=? and at>?"
+            for sz, rd, fn, at in cur.execute(q, (self.ip, lim)):
+                vp = "/" + "/".join([rd, fn]).strip("/")
+                if filt and filt not in vp:
+                    continue
+
+                ret.append({"vp": vp, "sz": sz, "at": at})
+                if len(ret) > 3000:
+                    ret.sort(key=lambda x: x["at"], reverse=True)
+                    ret = ret[:2000]
+
+        ret.sort(key=lambda x: x["at"], reverse=True)
+        ret = ret[:2000]
+
+        jtxt = json.dumps(ret, indent=2, sort_keys=True).encode("utf-8", "replace")
+        self.log("{} #{} {:.2f}sec".format(lm, len(ret), time.time() - t0))
+        self.reply(jtxt, mime="application/json")
+
+    def handle_rm(self, req=None):
+        if not req and not self.can_delete:
             raise Pebkac(403, "not allowed for user " + self.uname)
 
         if self.args.no_del:
             raise Pebkac(403, "disabled by argv")
 
-        x = self.conn.hsrv.broker.put(True, "up2k.handle_rm", self.uname, self.vpath)
+        if not req:
+            req = [self.vpath]
+
+        x = self.conn.hsrv.broker.put(True, "up2k.handle_rm", self.uname, self.ip, req)
         self.loud_reply(x.get())
 
     def handle_mv(self):
@@ -1564,6 +1736,9 @@ class HttpCli(object):
         if not dst:
             raise Pebkac(400, "need dst vpath")
 
+        # x-www-form-urlencoded (url query part) uses
+        # either + or %20 for 0x20 so handle both
+        dst = unquotep(dst.replace("+", " "))
         x = self.conn.hsrv.broker.put(
             True, "up2k.handle_mv", self.uname, self.vpath, dst
         )
@@ -1696,12 +1871,13 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": json.dumps(perms),
             "taglist": [],
-            "tag_order": [],
+            "def_hcols": [],
             "have_up2k_idx": ("e2d" in vn.flags),
             "have_tags_idx": ("e2t" in vn.flags),
             "have_mv": (not self.args.no_mv),
             "have_del": (not self.args.no_del),
             "have_zip": (not self.args.no_zip),
+            "have_unpost": (self.args.unpost > 0),
             "have_b_u": (self.can_write and self.uparam.get("b") == "u"),
             "url_suf": url_suf,
             "logues": logues,
@@ -1892,8 +2068,8 @@ class HttpCli(object):
         j2a["logues"] = logues
         j2a["taglist"] = taglist
 
-        if "mte" in vn.flags:
+        if "mth" in vn.flags:
-            j2a["tag_order"] = json.dumps(vn.flags["mte"].split(","))
+            j2a["def_hcols"] = vn.flags["mth"].split(",")
 
         if self.args.css_browser:
             j2a["css"] = self.args.css_browser

copyparty/httpsrv.py

@@ -174,25 +174,26 @@ class HttpSrv(object):
         now = time.time()
 
         if now - (self.tp_time or now) > 300:
+            m = "httpserver threadpool died: tpt {:.2f}, now {:.2f}, nthr {}, ncli {}"
+            self.log(self.name, m.format(self.tp_time, now, self.tp_nthr, self.ncli), 1)
+            self.tp_time = None
             self.tp_q = None
 
-        if self.tp_q:
+        with self.mutex:
-            self.tp_q.put((sck, addr))
+            self.ncli += 1
-            with self.mutex:
+            if self.tp_q:
-                self.ncli += 1
                 self.tp_time = self.tp_time or now
-                self.tp_ncli = max(self.tp_ncli, self.ncli + 1)
+                self.tp_ncli = max(self.tp_ncli, self.ncli)
                 if self.tp_nthr < self.ncli + 4:
                     self.start_threads(8)
-            return
+
+                self.tp_q.put((sck, addr))
+                return
 
         if not self.args.no_htp:
             m = "looks like the httpserver threadpool died; please make an issue on github and tell me the story of how you pulled that off, thanks and dog bless\n"
             self.log(self.name, m, 1)
 
-        with self.mutex:
-            self.ncli += 1
-
         thr = threading.Thread(
             target=self.thr_client,
             args=(sck, addr),

copyparty/mtag.py

@@ -434,7 +434,15 @@ class MTag(object):
             try:
                 v = getattr(md.info, attr)
             except:
-                continue
+                if k != "ac":
+                    continue
+
+                try:
+                    v = str(md.info).split(".")[1]
+                    if v.startswith("ogg"):
+                        v = v[3:]
+                except:
+                    continue
 
             if not v:
                 continue

copyparty/sutil.py

@@ -18,8 +18,7 @@ def errdesc(errors):
         tf_path = tf.name
         tf.write("\r\n".join(report).encode("utf-8", "replace"))
 
-    dt = datetime.utcfromtimestamp(time.time())
+    dt = datetime.utcnow().strftime("%Y-%m%d-%H%M%S")
-    dt = dt.strftime("%Y-%m%d-%H%M%S")
 
     bos.chmod(tf_path, 0o444)
     return {

copyparty/svchub.py

@@ -14,7 +14,7 @@ from datetime import datetime, timedelta
 import calendar
 
 from .__init__ import E, PY2, WINDOWS, ANYWIN, MACOS, VT100, unicode
-from .util import mp, start_log_thrs, start_stackmon, min_ex
+from .util import mp, start_log_thrs, start_stackmon, min_ex, ansi_re
 from .authsrv import AuthSrv
 from .tcpsrv import TcpSrv
 from .up2k import Up2k
@@ -41,7 +41,6 @@ class SvcHub(object):
         self.stop_cond = threading.Condition()
         self.httpsrv_up = 0
 
-        self.ansi_re = re.compile("\033\\[[^m]*m")
         self.log_mutex = threading.Lock()
         self.next_day = 0
 
@@ -111,7 +110,7 @@ class SvcHub(object):
         thr.start()
 
     def _logname(self):
-        dt = datetime.utcfromtimestamp(time.time())
+        dt = datetime.utcnow()
         fn = self.args.lo
         for fs in "YmdHMS":
             fs = "%" + fs
@@ -244,8 +243,7 @@ class SvcHub(object):
             return
 
         with self.log_mutex:
-            ts = datetime.utcfromtimestamp(time.time())
+            ts = datetime.utcnow().strftime("%Y-%m%d-%H%M%S.%f")[:-3]
-            ts = ts.strftime("%Y-%m%d-%H%M%S.%f")[:-3]
             self.logf.write("@{} [{}] {}\n".format(ts, src, msg))
 
             now = time.time()
@@ -257,7 +255,7 @@ class SvcHub(object):
             self.logf.close()
             self._setup_logfile("")
 
-        dt = datetime.utcfromtimestamp(time.time())
+        dt = datetime.utcnow()
 
         # unix timestamp of next 00:00:00 (leap-seconds safe)
         day_now = dt.day
@@ -280,9 +278,9 @@ class SvcHub(object):
             if not VT100:
                 fmt = "{} {:21} {}\n"
                 if "\033" in msg:
-                    msg = self.ansi_re.sub("", msg)
+                    msg = ansi_re.sub("", msg)
                 if "\033" in src:
-                    src = self.ansi_re.sub("", src)
+                    src = ansi_re.sub("", src)
             elif c:
                 if isinstance(c, int):
                     msg = "\033[3{}m{}".format(c, msg)

copyparty/th_cli.py

@@ -26,6 +26,9 @@ class ThumbCli(object):
         if is_vid and self.args.no_vthumb:
             return None
 
+        if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg"]:
+            return os.path.join(ptop, rem)
+
         if fmt == "j" and self.args.th_no_jpg:
             fmt = "w"
 

copyparty/th_srv.py

@@ -205,8 +205,8 @@ class ThumbSrv(object):
                 try:
                     fun(abspath, tpath)
                 except:
-                    msg = "{} failed on {}\n{}"
+                    msg = "{} could not create thumbnail of {}\n{}"
-                    self.log(msg.format(fun.__name__, abspath, min_ex()), 3)
+                    self.log(msg.format(fun.__name__, abspath, min_ex()), "1;30")
                     with open(tpath, "wb") as _:
                         pass
 
@@ -286,8 +286,9 @@ class ThumbSrv(object):
         cmd += seek
         cmd += [
             b"-i", fsenc(abspath),
+            b"-map", b"0:v:0",
             b"-vf", scale,
-            b"-vframes", b"1",
+            b"-frames:v", b"1",
         ]
         # fmt: on
 
@@ -308,8 +309,9 @@ class ThumbSrv(object):
 
         ret, sout, serr = runcmd(cmd)
         if ret != 0:
-            msg = ["ff: {}".format(x) for x in serr.split("\n")]
+            m = "FFmpeg failed (probably a corrupt video file):\n"
-            self.log("FFmpeg failed:\n" + "\n".join(msg), c="1;30")
+            m += "\n".join(["ff: {}".format(x) for x in serr.split("\n")])
+            self.log(m, c="1;30")
             raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
 
     def poke(self, tdir):

copyparty/u2idx.py

@@ -88,7 +88,7 @@ class U2idx(object):
         is_date = False
         kw_key = ["(", ")", "and ", "or ", "not "]
         kw_val = ["==", "=", "!=", ">", ">=", "<", "<=", "like "]
-        ptn_mt = re.compile(r"^\.?[a-z]+$")
+        ptn_mt = re.compile(r"^\.?[a-z_-]+$")
         mt_ctr = 0
         mt_keycmp = "substr(up.w,1,16)"
         mt_keycmp2 = None
@@ -244,7 +244,7 @@ class U2idx(object):
             sret = []
             c = cur.execute(q, v)
             for hit in c:
-                w, ts, sz, rd, fn = hit
+                w, ts, sz, rd, fn, ip, at = hit
                 lim -= 1
                 if lim <= 0:
                     break

copyparty/up2k.py

@@ -36,7 +36,7 @@ from .util import (
     min_ex,
 )
 from .bos import bos
-from .authsrv import AuthSrv
+from .authsrv import AuthSrv, LEELOO_DALLAS
 from .mtag import MTag, MParser
 
 try:
@@ -45,7 +45,7 @@ try:
 except:
     HAVE_SQLITE3 = False
 
-DB_VER = 4
+DB_VER = 5
 
 
 class Up2k(object):
@@ -176,36 +176,71 @@ class Up2k(object):
         return None
 
     def _sched_rescan(self):
-        maxage = self.args.re_maxage
         volage = {}
         while True:
             time.sleep(self.args.re_int)
             now = time.time()
-            vpaths = list(sorted(self.asrv.vfs.all_vols.keys()))
             with self.mutex:
-                if maxage:
+                for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
-                    for vp in vpaths:
+                    maxage = vol.flags.get("scan")
-                        if vp not in volage:
+                    if not maxage:
-                            volage[vp] = now
+                        continue
 
-                        if now - volage[vp] >= maxage:
+                    if vp not in volage:
-                            self.need_rescan[vp] = 1
+                        volage[vp] = now
 
-                if not self.need_rescan:
+                    if now - volage[vp] >= maxage:
-                    continue
+                        self.need_rescan[vp] = 1
 
                 vols = list(sorted(self.need_rescan.keys()))
                 self.need_rescan = {}
 
-            err = self.rescan(self.asrv.vfs.all_vols, vols)
+            if vols:
-            if err:
+                err = self.rescan(self.asrv.vfs.all_vols, vols)
-                for v in vols:
+                if err:
-                    self.need_rescan[v] = True
+                    for v in vols:
+                        self.need_rescan[v] = True
 
+                    continue
+
+                for v in vols:
+                    volage[v] = now
+
+            if self.args.no_lifetime:
                 continue
 
-            for v in vols:
+            for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
-                volage[v] = now
+                lifetime = vol.flags.get("lifetime")
+                if not lifetime:
+                    continue
+
+                cur = self.cur.get(vol.realpath)
+                if not cur:
+                    continue
+
+                nrm = 0
+                deadline = time.time() - int(lifetime)
+                q = "select rd, fn from up where at > 0 and at < ? limit 100"
+                while True:
+                    with self.mutex:
+                        hits = cur.execute(q, (deadline,)).fetchall()
+
+                    if not hits:
+                        break
+
+                    for rd, fn in hits:
+                        if rd.startswith("//") or fn.startswith("//"):
+                            rd, fn = s3dec(rd, fn)
+
+                        fvp = "{}/{}".format(rd, fn).strip("/")
+                        if vp:
+                            fvp = "{}/{}".format(vp, fvp)
+
+                        self._handle_rm(LEELOO_DALLAS, None, fvp)
+                        nrm += 1
+
+                if nrm:
+                    self.log("{} files graduated in {}".format(nrm, vp))
 
     def _vis_job_progress(self, job):
         perc = 100 - (len(job["need"]) * 100.0 / len(job["hash"]))
@@ -522,7 +557,7 @@ class Up2k(object):
 
                     wark = up2k_wark_from_hashlist(self.salt, sz, hashes)
 
-                self.db_add(dbw[0], wark, rd, fn, lmod, sz)
+                self.db_add(dbw[0], wark, rd, fn, lmod, sz, "", 0)
                 dbw[1] += 1
                 ret += 1
                 td = time.time() - dbw[2]
@@ -537,8 +572,8 @@ class Up2k(object):
         rm = []
         nchecked = 0
         nfiles = next(cur.execute("select count(w) from up"))[0]
-        c = cur.execute("select * from up")
+        c = cur.execute("select rd, fn from up")
-        for dwark, dts, dsz, drd, dfn in c:
+        for drd, dfn in c:
             nchecked += 1
             if drd.startswith("//") or dfn.startswith("//"):
                 drd, dfn = s3dec(drd, dfn)
@@ -941,6 +976,15 @@ class Up2k(object):
         if not existed and ver is None:
             return self._create_db(db_path, cur)
 
+        if ver == 4:
+            try:
+                m = "creating backup before upgrade: "
+                cur = self._backup_db(db_path, cur, ver, m)
+                self._upgrade_v4(cur)
+                ver = 5
+            except:
+                self.log("WARN: failed to upgrade from v4", 3)
+
         if ver == DB_VER:
             try:
                 nfiles = next(cur.execute("select count(w) from up"))[0]
@@ -1011,9 +1055,10 @@ class Up2k(object):
             idx = r"create index up_w on up(w)"
 
         for cmd in [
-            r"create table up (w text, mt int, sz int, rd text, fn text)",
+            r"create table up (w text, mt int, sz int, rd text, fn text, ip text, at int)",
             r"create index up_rd on up(rd)",
             r"create index up_fn on up(fn)",
+            r"create index up_ip on up(ip)",
             idx,
             r"create table mt (w text, k text, v int)",
             r"create index mt_w on mt(w)",
@@ -1028,6 +1073,17 @@ class Up2k(object):
         self.log("created DB at {}".format(db_path))
         return cur
 
+    def _upgrade_v4(self, cur):
+        for cmd in [
+            r"alter table up add column ip text",
+            r"alter table up add column at int",
+            r"create index up_ip on up(ip)",
+            r"update kv set v=5 where k='sver'",
+        ]:
+            cur.execute(cmd)
+
+        cur.connection.commit()
+
     def handle_json(self, cj):
         with self.mutex:
             if not self.register_vpath(cj["ptop"], cj["vcfg"]):
@@ -1051,7 +1107,7 @@ class Up2k(object):
                     argv = (wark[:16], wark)
 
                 cur = cur.execute(q, argv)
-                for _, dtime, dsize, dp_dir, dp_fn in cur:
+                for _, dtime, dsize, dp_dir, dp_fn, ip, at in cur:
                     if dp_dir.startswith("//") or dp_fn.startswith("//"):
                         dp_dir, dp_fn = s3dec(dp_dir, dp_fn)
 
@@ -1065,6 +1121,8 @@ class Up2k(object):
                             "ptop": cj["ptop"],
                             "size": dsize,
                             "lmod": dtime,
+                            "addr": ip,
+                            "at": at,
                             "hash": [],
                             "need": [],
                         }
@@ -1119,11 +1177,22 @@ class Up2k(object):
                             self._symlink(src, dst)
 
                         if cur:
-                            a = [cj[x] for x in "prel name lmod size".split()]
+                            a = [cj[x] for x in "prel name lmod size addr".split()]
+                            a += [cj.get("at") or time.time()]
                             self.db_add(cur, wark, *a)
                             cur.connection.commit()
 
             if not job:
+                vfs = self.asrv.vfs.all_vols[cj["vtop"]]
+                if vfs.lim:
+                    ap1 = os.path.join(cj["ptop"], cj["prel"])
+                    ap2, cj["prel"] = vfs.lim.all(
+                        cj["addr"], cj["prel"], cj["size"], ap1
+                    )
+                    bos.makedirs(ap2)
+                    vfs.lim.nup(cj["addr"])
+                    vfs.lim.bup(cj["addr"], cj["size"])
+
                 job = {
                     "wark": wark,
                     "t0": now,
@@ -1154,8 +1223,11 @@ class Up2k(object):
 
                 self._new_upload(job)
 
+            purl = "/{}/".format("{}/{}".format(job["vtop"], job["prel"]).strip("/"))
+
             return {
                 "name": job["name"],
+                "purl": purl,
                 "size": job["size"],
                 "lmod": job["lmod"],
                 "hash": job["need"],
@@ -1206,7 +1278,7 @@ class Up2k(object):
                     hops = len(ndst[nc:]) - 1
                     lsrc = "../" * hops + "/".join(lsrc)
             os.symlink(fsenc(lsrc), fsenc(ldst))
-        except (AttributeError, OSError) as ex:
+        except Exception as ex:
             self.log("cannot symlink; creating copy: " + repr(ex))
             shutil.copy2(fsenc(src), fsenc(dst))
 
@@ -1266,20 +1338,21 @@ class Up2k(object):
                 a = [dst, job["size"], (int(time.time()), int(job["lmod"]))]
                 self.lastmod_q.put(a)
 
-            a = [job[x] for x in "ptop wark prel name lmod size".split()]
+            a = [job[x] for x in "ptop wark prel name lmod size addr".split()]
+            a += [job.get("at") or time.time()]
             if self.idx_wark(*a):
                 del self.registry[ptop][wark]
                 # in-memory registry is reserved for unfinished uploads
 
         return ret, dst
 
-    def idx_wark(self, ptop, wark, rd, fn, lmod, sz):
+    def idx_wark(self, ptop, wark, rd, fn, lmod, sz, ip, at):
         cur = self.cur.get(ptop)
         if not cur:
             return False
 
         self.db_rm(cur, rd, fn)
-        self.db_add(cur, wark, rd, fn, lmod, sz)
+        self.db_add(cur, wark, rd, fn, lmod, sz, ip, at)
         cur.connection.commit()
 
         if "e2t" in self.flags[ptop]:
@@ -1295,53 +1368,99 @@ class Up2k(object):
         except:
             db.execute(sql, s3enc(self.mem_cur, rd, fn))
 
-    def db_add(self, db, wark, rd, fn, ts, sz):
+    def db_add(self, db, wark, rd, fn, ts, sz, ip, at):
-        sql = "insert into up values (?,?,?,?,?)"
+        sql = "insert into up values (?,?,?,?,?,?,?)"
-        v = (wark, int(ts), sz, rd, fn)
+        v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
         try:
             db.execute(sql, v)
         except:
             rd, fn = s3enc(self.mem_cur, rd, fn)
-            v = (wark, int(ts), sz, rd, fn)
+            v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
             db.execute(sql, v)
 
-    def handle_rm(self, uname, vpath):
+    def handle_rm(self, uname, ip, vpaths):
-        permsets = [[True, False, False, True]]
+        n_files = 0
-        vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+        ok = {}
+        ng = {}
+        for vp in vpaths:
+            a, b, c = self._handle_rm(uname, ip, vp)
+            n_files += a
+            for k in b:
+                ok[k] = 1
+            for k in c:
+                ng[k] = 1
+
+        ng = {k: 1 for k in ng if k not in ok}
+        ok = len(ok)
+        ng = len(ng)
+
+        return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
+
+    def _handle_rm(self, uname, ip, vpath):
+        try:
+            permsets = [[True, False, False, True]]
+            vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+            unpost = False
+        except:
+            # unpost with missing permissions? try read+write and verify with db
+            if not self.args.unpost:
+                raise Pebkac(400, "the unpost feature was disabled by server config")
+
+            unpost = True
+            permsets = [[True, True]]
+            vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
+            _, _, _, _, dip, dat = self._find_from_vpath(vn.realpath, rem)
+
+            m = "you cannot delete this: "
+            if not dip:
+                m += "file not found"
+            elif dip != ip:
+                m += "not uploaded by (You)"
+            elif dat < time.time() - self.args.unpost:
+                m += "uploaded too long ago"
+            else:
+                m = None
+
+            if m:
+                raise Pebkac(400, m)
+
         ptop = vn.realpath
-        atop = vn.canonical(rem)
+        atop = vn.canonical(rem, False)
         adir, fn = os.path.split(atop)
         st = bos.lstat(atop)
         scandir = not self.args.no_scandir
         if stat.S_ISLNK(st.st_mode) or stat.S_ISREG(st.st_mode):
             dbv, vrem = self.asrv.vfs.get(vpath, uname, *permsets[0])
             dbv, vrem = dbv.get_dbv(vrem)
-            g = [[dbv, vrem, os.path.dirname(vpath), adir, [[fn, 0]], [], []]]
+            voldir = vsplit(vrem)[0]
+            vpath_dir = vsplit(vpath)[0]
+            g = [[dbv, voldir, vpath_dir, adir, [[fn, 0]], [], []]]
         else:
             g = vn.walk("", rem, [], uname, permsets, True, scandir, True)
+            if unpost:
+                raise Pebkac(400, "cannot unpost folders")
 
         n_files = 0
         for dbv, vrem, _, adir, files, rd, vd in g:
             for fn in [x[0] for x in files]:
                 n_files += 1
                 abspath = os.path.join(adir, fn)
-                vpath = "{}/{}".format(vrem, fn).strip("/")
+                volpath = "{}/{}".format(vrem, fn).strip("/")
+                vpath = "{}/{}".format(dbv.vpath, volpath).strip("/")
                 self.log("rm {}\n  {}".format(vpath, abspath))
-                _ = dbv.get(vrem, uname, *permsets[0])
+                _ = dbv.get(volpath, uname, *permsets[0])
                 with self.mutex:
                     try:
                         ptop = dbv.realpath
-                        cur, wark, _, _ = self._find_from_vpath(ptop, vrem)
+                        cur, wark, _, _, _, _ = self._find_from_vpath(ptop, volpath)
-                        self._forget_file(ptop, vpath, cur, wark)
+                        self._forget_file(ptop, volpath, cur, wark, True)
                     finally:
                         cur.connection.commit()
 
                 bos.unlink(abspath)
 
         rm = rmdirs(self.log_func, scandir, True, atop)
-        ok = len(rm[0])
+        return n_files, rm[0], rm[1]
-        ng = len(rm[1])
-        return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
 
     def handle_mv(self, uname, svp, dvp):
         svn, srem = self.asrv.vfs.get(svp, uname, True, False, True)
@@ -1351,9 +1470,10 @@ class Up2k(object):
         if not srem:
             raise Pebkac(400, "mv: cannot move a mountpoint")
 
-        st = bos.stat(sabs)
+        st = bos.lstat(sabs)
-        if stat.S_ISREG(st.st_mode):
+        if stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode):
-            return self._mv_file(uname, svp, dvp)
+            with self.mutex:
+                return self._mv_file(uname, svp, dvp)
 
         jail = svn.get_dbv(srem)[0]
         permsets = [[True, False, True]]
@@ -1378,7 +1498,8 @@ class Up2k(object):
                     raise Pebkac(500, "mv: bug at {}, top {}".format(svpf, svp))
 
                 dvpf = dvp + svpf[len(svp) :]
-                self._mv_file(uname, svpf, dvpf)
+                with self.mutex:
+                    self._mv_file(uname, svpf, dvpf)
 
         rmdirs(self.log_func, scandir, True, sabs)
         return "k"
@@ -1411,7 +1532,7 @@ class Up2k(object):
             self.need_rescan[dvn.vpath] = 1
             return "k"
 
-        c1, w, ftime, fsize = self._find_from_vpath(svn.realpath, srem)
+        c1, w, ftime, fsize, ip, at = self._find_from_vpath(svn.realpath, srem)
         c2 = self.cur.get(dvn.realpath)
 
         if ftime is None:
@@ -1420,15 +1541,15 @@ class Up2k(object):
             fsize = st.st_size
 
         if w:
-            if c2:
+            if c2 and c2 != c1:
                 self._copy_tags(c1, c2, w)
 
-            self._forget_file(svn.realpath, srem, c1, w)
+            self._forget_file(svn.realpath, srem, c1, w, c1 != c2)
             self._relink(w, svn.realpath, srem, dabs)
             c1.connection.commit()
 
             if c2:
-                self.db_add(c2, w, drd, dfn, ftime, fsize)
+                self.db_add(c2, w, drd, dfn, ftime, fsize, ip, at)
                 c2.connection.commit()
         else:
             self.log("not found in src db: [{}]".format(svp))
@@ -1452,7 +1573,7 @@ class Up2k(object):
             return None, None
 
         rd, fn = vsplit(vrem)
-        q = "select w, mt, sz from up where rd=? and fn=? limit 1"
+        q = "select w, mt, sz, ip, at from up where rd=? and fn=? limit 1"
         try:
             c = cur.execute(q, (rd, fn))
         except:
@@ -1460,20 +1581,24 @@ class Up2k(object):
 
         hit = c.fetchone()
         if hit:
-            wark, ftime, fsize = hit
+            wark, ftime, fsize, ip, at = hit
-            return cur, wark, ftime, fsize
+            return cur, wark, ftime, fsize, ip, at
-        return cur, None, None, None
+        return cur, None, None, None, None, None
 
-    def _forget_file(self, ptop, vrem, cur, wark):
+    def _forget_file(self, ptop, vrem, cur, wark, drop_tags):
         """forgets file in db, fixes symlinks, does not delete"""
         srd, sfn = vsplit(vrem)
         self.log("forgetting {}".format(vrem))
         if wark:
             self.log("found {} in db".format(wark))
-            self._relink(wark, ptop, vrem, None)
+            if drop_tags:
+                if self._relink(wark, ptop, vrem, None):
+                    drop_tags = False
+
+            if drop_tags:
+                q = "delete from mt where w=?"
+                cur.execute(q, (wark[:16],))
 
-            q = "delete from mt where w=?"
-            cur.execute(q, (wark[:16],))
             self.db_rm(cur, srd, sfn)
 
         reg = self.registry.get(ptop)
@@ -1510,7 +1635,7 @@ class Up2k(object):
                     self.log("found {} dupe: [{}] {}".format(wark, ptop, dvrem))
 
         if not dupes:
-            return
+            return 0
 
         full = {}
         links = {}
@@ -1526,7 +1651,7 @@ class Up2k(object):
             # deleting final remaining full copy; swap it with a symlink
             slabs = list(sorted(links.keys()))[0]
             ptop, rem = links.pop(slabs)
-            self.log("linkswap [{}] and [{}]".format(sabs, dabs))
+            self.log("linkswap [{}] and [{}]".format(sabs, slabs))
             bos.unlink(slabs)
             bos.rename(sabs, slabs)
             self._symlink(slabs, sabs, False)
@@ -1547,6 +1672,8 @@ class Up2k(object):
 
             self._symlink(dabs, alink, False)
 
+        return len(full) + len(links)
+
     def _get_wark(self, cj):
         if len(cj["name"]) > 1024 or len(cj["hash"]) > 512 * 1024:  # 16TiB
             raise Pebkac(400, "name or numchunks not according to spec")
@@ -1753,7 +1880,7 @@ class Up2k(object):
                 self.n_hashq -= 1
             # self.log("hashq {}".format(self.n_hashq))
 
-            ptop, rd, fn = self.hashq.get()
+            ptop, rd, fn, ip, at = self.hashq.get()
             # self.log("hashq {} pop {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
             if "e2d" not in self.flags[ptop]:
                 continue
@@ -1764,12 +1891,12 @@ class Up2k(object):
             hashes = self._hashlist_from_file(abspath)
             wark = up2k_wark_from_hashlist(self.salt, inf.st_size, hashes)
             with self.mutex:
-                self.idx_wark(ptop, wark, rd, fn, inf.st_mtime, inf.st_size)
+                self.idx_wark(ptop, wark, rd, fn, inf.st_mtime, inf.st_size, ip, at)
 
-    def hash_file(self, ptop, flags, rd, fn):
+    def hash_file(self, ptop, flags, rd, fn, ip, at):
         with self.mutex:
             self.register_vpath(ptop, flags)
-            self.hashq.put([ptop, rd, fn])
+            self.hashq.put([ptop, rd, fn, ip, at])
             self.n_hashq += 1
         # self.log("hashq {} push {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
 

copyparty/util.py

@@ -58,6 +58,9 @@ except:
         return struct.unpack(f.decode("ascii"), *a, **ka)
 
 
+ansi_re = re.compile("\033\\[[^m]*m")
+
+
 surrogateescape.register_surrogateescape()
 FS_ENCODING = sys.getfilesystemencoding()
 if WINDOWS and PY2:
@@ -77,6 +80,7 @@ HTTPCODE = {
     403: "Forbidden",
     404: "Not Found",
     405: "Method Not Allowed",
+    411: "Length Required",
     413: "Payload Too Large",
     416: "Requested Range Not Satisfiable",
     422: "Unprocessable Entity",
@@ -340,6 +344,15 @@ def log_thrs(log, ival, name):
         log(name, "\033[0m \033[33m".join(tv), 3)
 
 
+def vol_san(vols, txt):
+    for vol in vols:
+        txt = txt.replace(
+            vol.realpath.encode("utf-8"), vol.vpath.encode("utf-8")
+        )
+    
+    return txt
+
+
 def min_ex():
     et, ev, tb = sys.exc_info()
     tb = traceback.extract_tb(tb)
@@ -684,6 +697,17 @@ def humansize(sz, terse=False):
     return ret.replace("iB", "").replace(" ", "")
 
 
+def unhumanize(sz):
+    try:
+        return float(sz)
+    except:
+        pass
+
+    mul = sz[-1:].lower()
+    mul = {"k": 1024, "m": 1024 * 1024, "g": 1024 * 1024 * 1024}.get(mul, 1)
+    return float(sz[:-1]) * mul
+
+
 def get_spd(nbyte, t0, t=None):
     if t is None:
         t = time.time()
@@ -1063,6 +1087,9 @@ def statdir(logger, scandir, lstat, top):
 
 
 def rmdirs(logger, scandir, lstat, top):
+    if not os.path.exists(fsenc(top)) or not os.path.isdir(fsenc(top)):
+        top = os.path.dirname(top)
+
     dirs = statdir(logger, scandir, lstat, top)
     dirs = [x[0] for x in dirs if stat.S_ISDIR(x[1].st_mode)]
     dirs = [os.path.join(top, x) for x in dirs]

copyparty/web/baguettebox.js

@@ -22,7 +22,7 @@ window.baguetteBox = (function () {
             afterHide: null,
             onChange: null,
         },
-        overlay, slider, btnPrev, btnNext, btnHelp, btnVmode, btnClose,
+        overlay, slider, btnPrev, btnNext, btnHelp, btnSel, btnVmode, btnClose,
         currentGallery = [],
         currentIndex = 0,
         isOverlayVisible = false,
@@ -175,6 +175,7 @@ window.baguetteBox = (function () {
                 '<button id="bbox-next" class="bbox-btn" type="button" aria-label="Next">&gt;</button>' +
                 '<div id="bbox-btns">' +
                 '<button id="bbox-help" type="button">?</button>' +
+                '<button id="bbox-tsel" type="button">sel</button>' +
                 '<button id="bbox-vmode" type="button" tt="a"></button>' +
                 '<button id="bbox-close" type="button" aria-label="Close">X</button>' +
                 '</div></div>'
@@ -187,6 +188,7 @@ window.baguetteBox = (function () {
         btnPrev = ebi('bbox-prev');
         btnNext = ebi('bbox-next');
         btnHelp = ebi('bbox-help');
+        btnSel = ebi('bbox-tsel');
         btnVmode = ebi('bbox-vmode');
         btnClose = ebi('bbox-close');
         bindEvents();
@@ -203,6 +205,7 @@ window.baguetteBox = (function () {
             ['right, L', 'next file'],
             ['home', 'first file'],
             ['end', 'last file'],
+            ['S', 'toggle file selection'],
             ['space, P, K', 'video: play / pause'],
             ['U', 'video: seek 10sec back'],
             ['P', 'video: seek 10sec ahead'],
@@ -267,6 +270,8 @@ window.baguetteBox = (function () {
                     v.requestFullscreen();
             }
             catch (ex) { }
+        else if (k == "KeyS")
+            tglsel();
     }
 
     function setVmode() {
@@ -314,6 +319,40 @@ window.baguetteBox = (function () {
             tt.show.bind(this)();
     }
 
+    function tglsel() {
+        var thumb = currentGallery[currentIndex].imageElement,
+            name = vsplit(thumb.href)[1],
+            files = msel.getall();
+
+        for (var a = 0; a < files.length; a++)
+            if (vsplit(files[a].vp)[1] == name)
+                clmod(ebi(files[a].id).closest('tr'), 'sel', 't');
+
+        msel.selui();
+        selbg();
+    }
+
+    function selbg() {
+        var img = imagesElements[currentIndex].querySelector('img, video'),
+            thumb = currentGallery[currentIndex].imageElement,
+            name = vsplit(thumb.href)[1],
+            files = msel.getsel(),
+            sel = false;
+
+        for (var a = 0; a < files.length; a++)
+            if (vsplit(files[a].vp)[1] == name)
+                sel = true;
+
+        ebi('bbox-overlay').style.background = sel ?
+            'rgba(153,34,85,0.7)' : '';
+
+        img.style.boxShadow = sel ? '0 0 3em #f4a' : '';
+        img.style.borderRadius = sel ? '1em' : '';
+        btnSel.style.color = sel ? '#fff' : '';
+        btnSel.style.background = sel ? '#d48' : '';
+        btnSel.style.textShadow = sel ? '1px 1px 0 #b38' : '';
+    }
+
     function keyUpHandler(e) {
         if (e.ctrlKey || e.altKey || e.metaKey || e.isComposing)
             return;
@@ -348,6 +387,7 @@ window.baguetteBox = (function () {
         bind(btnClose, 'click', hideOverlay);
         bind(btnVmode, 'click', tglVmode);
         bind(btnHelp, 'click', halp);
+        bind(btnSel, 'click', tglsel);
         bind(slider, 'contextmenu', contextmenuHandler);
         bind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
         bind(overlay, 'touchmove', touchmoveHandler, passiveEvent);
@@ -362,6 +402,7 @@ window.baguetteBox = (function () {
         unbind(btnClose, 'click', hideOverlay);
         unbind(btnVmode, 'click', tglVmode);
         unbind(btnHelp, 'click', halp);
+        unbind(btnSel, 'click', tglsel);
         unbind(slider, 'contextmenu', contextmenuHandler);
         unbind(overlay, 'touchstart', touchstartHandler, nonPassiveEvent);
         unbind(overlay, 'touchmove', touchmoveHandler, passiveEvent);
@@ -679,6 +720,7 @@ window.baguetteBox = (function () {
             v.muted = vmute;
             v.loop = vloop;
         }
+        selbg();
         mp_ctl();
         setVmode();
     }

copyparty/web/browser.css

@@ -22,125 +22,9 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 pre, code, tt {
 	font-family: monospace, monospace;
 }
-
-
-
-#tt, #toast {
-	position: fixed;
-	max-width: 34em;
-	background: #222;
-	border: 0 solid #777;
-	box-shadow: 0 .2em .5em #222;
-	border-radius: .4em;
-	z-index: 9001;
-}
-#tt {
-	overflow: hidden;
-	margin-top: 1em;
-	padding: 0 1.3em;
-	height: 0;
-	opacity: .1;
-	transition: opacity 0.14s, height 0.14s, padding 0.14s;
-}
-#toast {
-	top: 1.4em;
-	right: -1em;
-	line-height: 1.5em;
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	transform: translateX(100%);
-	transition:
-		transform .4s cubic-bezier(.2, 1.2, .5, 1),
-		right .4s cubic-bezier(.2, 1.2, .5, 1);
-	text-shadow: 1px 1px 0 #000;
-	color: #fff;
-}
-#toastc {
-	display: inline-block;
-	position: absolute;
-	overflow: hidden;
-	left: 0;
-	width: 0;
-	opacity: 0;
-	padding: .3em 0;
-	margin: -.3em 0 0 0;
-	line-height: 1.5em;
-	color: #000;
-	border: none;
-	outline: none;
-	text-shadow: none;
-	border-radius: .5em 0 0 .5em;
-	transition: left .3s, width .3s, padding .3s, opacity .3s;
-}
-#toast.vis {
-	right: 1.3em;
-	transform: unset;
-}
-#toast.vis #toastc {
-	left: -2em;
-	width: .4em;
-	padding: .3em .8em;
-	opacity: 1;
-}
-#toast.inf {
-	background: #07a;
-	border-color: #0be;
-}
-#toast.inf #toastc {
-	background: #0be;
-}
-#toast.ok {
-	background: #4a0;
-	border-color: #8e4;
-}
-#toast.ok #toastc {
-	background: #8e4;
-}
-#toast.warn {
-	background: #970;
-	border-color: #fc0;
-}
-#toast.warn #toastc {
-	background: #fc0;
-}
-#toast.err {
-	background: #900;
-	border-color: #d06;
-}
-#toast.err #toastc {
-	background: #d06;
-}
-#tt.b {
-	padding: 0 2em;
-	border-radius: .5em;
-	box-shadow: 0 .2em 1em #000;
-}
-#tt.show {
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	height: auto;
-	opacity: 1;
-}
-#tt.show.b {
-	padding: 1.5em 2em;
-	border-width: .5em 0;
-}
-#tt code {
-	background: #3c3c3c;
-	padding: .1em .3em;
-	border-top: 1px solid #777;
-	border-radius: .3em;
-	line-height: 1.7em;
-}
-#tt em {
-	color: #f6a;
-}
 #path,
 #path * {
 	font-size: 1em;
@@ -807,6 +691,7 @@ input.eq_gain {
 #wrap {
 	margin-top: 2em;
 	min-height: 90vh;
+	padding-bottom: 5em;
 }
 #tree {
 	display: none;
@@ -952,7 +837,8 @@ input.eq_gain {
 	color: #300;
 	background: #fea;
 }
-.opwide {
+.opwide,
+#op_unpost {
 	max-width: none;
 	margin-right: 1.5em;
 }
@@ -1054,10 +940,79 @@ html.light #ggrid a:hover {
 	color: #015;
 	box-shadow: 0 .1em .5em #aaa;
 }
+#op_unpost {
+	padding: 1em;
+}
+#op_unpost td {
+	padding: .2em .4em;
+}
+#op_unpost a {
+	margin: 0;
+	padding: 0;
+}
+#rui {
+	position: fixed;
+	top: 0;
+	left: 0;
+	width: calc(100% - 2em);
+	height: auto;
+	overflow: auto;
+	max-height: calc(100% - 2em);
+	border-bottom: .5em solid #999;
+	box-shadow: 0 0 5em rgba(0,0,0,0.8);
+	background: #333;
+	padding: 1em;
+	z-index: 765;
+}
+html.light #rui {
+	color: #fff;
+}
+#rui div+div {
+	margin-top: 1em;
+}
+#rui table {
+	width: 100%;
+	border-collapse: collapse;
+}
+#rui td+td {
+	padding: .2em 0 .2em .5em;
+}
+#rn_vadv input {
+	font-family: monospace, monospace;
+}
+#rui td+td,
+#rui td input[type="text"] {
+	width: 100%;
+}
+#rn_f.m td:first-child {
+	white-space: nowrap;
+}
+#rn_f.m td+td {
+	width: 50%;
+}
+#rn_f .err td {
+	background: #c00;
+}
+#rn_f .err input[readonly] {
+	background: #600;
+	color: #fc0;
+}
+#rui input[readonly] {
+	color: #fff;
+	background: #444;
+	border: 1px solid #777;
+	padding: .2em .25em;
+}
+#rui h1 {
+	margin: 0 0 .3em 0;
+	padding: 0;
+	font-size: 1.5em;
+}
 #pvol,
 #barbuf,
 #barpos,
 #u2conf label,
+#rui label,
 #ops {
 	-webkit-user-select: none;
 	-moz-user-select: none;
@@ -1079,6 +1034,11 @@ html.light #ggrid a:hover {
 
 
 
+
+
+
+
+
 
 
 
@@ -1089,21 +1049,6 @@ html.light {
 	background: #eee;
 	text-shadow: none;
 }
-html.light #tt {
-	background: #fff;
-	border-color: #888 #000 #777 #000;
-}
-html.light #tt,
-html.light #toast {
-	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
-}
-html.light #tt code {
-	background: #060;
-	color: #fff;
-}
-html.light #tt em {
-	color: #d38;
-}
 html.light #ops,
 html.light .opbox,
 html.light #srch_form {
@@ -1320,6 +1265,30 @@ html.light #tree::-webkit-scrollbar {
 
 
 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/* bbox */
+
 #bbox-overlay {
 	display: none;
 	opacity: 0;
@@ -1437,6 +1406,7 @@ html.light #bbox-overlay figcaption a {
 	font-size: 1.4em;
 	line-height: 1.4em;
 	vertical-align: top;
+	font-variant: small-caps;
 }
 #bbox-overlay button:focus,
 #bbox-overlay button:hover {
@@ -1498,3 +1468,331 @@ html.light #bbox-overlay figcaption a {
 	0%, 100% {transform: scale(0)}
 	50% {transform: scale(1)}
 }
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+/* upload.css */
+
+#op_up2k {
+	padding: 0 1em 1em 1em;
+}
+#u2form {
+	position: absolute;
+	top: 0;
+	left: 0;
+	width: 2px;
+	height: 2px;
+	overflow: hidden;
+}
+#u2form input {
+	background: #444;
+	border: 0px solid #444;
+	outline: none;
+}
+#u2err.err {
+	color: #f87;
+	padding: .5em;
+}
+#u2err.msg {
+	color: #999;
+	padding: .5em;
+	font-size: .9em;
+}
+#u2btn {
+	color: #eee;
+	background: #555;
+	background: -moz-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	background: -webkit-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	background: linear-gradient(to bottom, #367 0%, #489 50%, #38788a 51%, #367 100%);
+	filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#489', endColorstr='#38788a', GradientType=0);
+	text-decoration: none;
+	line-height: 1.3em;
+	border: 1px solid #222;
+	border-radius: .4em;
+	text-align: center;
+	font-size: 1.5em;
+	margin: .5em auto;
+	padding: .8em 0;
+	width: 16em;
+	cursor: pointer;
+	box-shadow: .4em .4em 0 #111;
+}
+#op_up2k.srch #u2btn {
+	background: linear-gradient(to bottom, #ca3 0%, #fd8 50%, #fc6 51%, #b92 100%);
+	text-shadow: 1px 1px 1px #fc6;
+	color: #333;
+}
+#u2conf #u2btn {
+	margin: -1.5em 0;
+	padding: .8em 0;
+	width: 100%;
+	max-width: 12em;
+	display: inline-block;
+}
+#u2conf #u2btn_cw {
+	text-align: right;
+}
+#u2notbtn {
+	display: none;
+	text-align: center;
+	background: #333;
+	padding-top: 1em;
+}
+#u2notbtn * {
+	line-height: 1.3em;
+}
+#u2tab {
+	margin: 3em auto;
+	width: calc(100% - 2em);
+	max-width: 100em;
+}
+#op_up2k.srch #u2tab {
+	max-width: none;
+}
+#u2tab td {
+	border: 1px solid #ccc;
+	border-width: 0 0px 1px 0;
+	padding: .1em .3em;
+}
+#u2tab td:nth-child(2) {
+	width: 5em;
+	white-space: nowrap;
+}
+#u2tab td:nth-child(3) {
+	width: 40%;
+}
+#op_up2k.srch td.prog {
+	font-family: sans-serif;
+	font-size: 1em;
+	width: auto;
+}
+#u2tab tbody tr:hover td {
+	background: #222;
+}
+#u2cards {
+	padding: 1em 0 .3em 1em;
+	margin: 1.5em auto -2.5em auto;
+	white-space: nowrap;
+	text-align: center;
+	overflow: hidden;
+}
+#u2cards.w {
+	width: 45em;
+	text-align: left;
+}
+#u2cards a {
+	padding: .2em 1em;
+	border: 1px solid #777;
+	border-width: 0 0 1px 0;
+	background: linear-gradient(to bottom, #333, #222);
+}
+#u2cards a:first-child {
+	border-radius: .4em 0 0 0;
+}
+#u2cards a:last-child {
+	border-radius: 0 .4em 0 0;
+}
+#u2cards a.act {
+	padding-bottom: .5em;
+	border-width: 1px 1px .1em 1px;
+	border-radius: .3em .3em 0 0;
+	margin-left: -1px;
+	background: linear-gradient(to bottom, #464, #333 80%);
+	box-shadow: 0 -.17em .67em #280;
+	border-color: #7c5 #583 #333 #583;
+	position: relative;
+	color: #fd7;
+}
+#u2cards span {
+	color: #fff;
+}
+#u2conf {
+	margin: 1em auto;
+	width: 30em;
+}
+#u2conf.has_btn {
+	width: 48em;
+}
+#u2conf * {
+	text-align: center;
+	line-height: 1em;
+	margin: 0;
+	padding: 0;
+	border: none;
+	outline: none;
+}
+#u2conf .txtbox {
+	width: 3em;
+	color: #fff;
+	background: #444;
+	border: 1px solid #777;
+	font-size: 1.2em;
+	padding: .15em 0;
+	height: 1.05em;
+}
+#u2conf .txtbox.err {
+	background: #922;
+}
+#u2conf a {
+	color: #fff;
+	background: #c38;
+	text-decoration: none;
+	border-radius: .1em;
+	font-size: 1.5em;
+	padding: .1em 0;
+	margin: 0 -1px;
+	width: 1.5em;
+	height: 1em;
+	display: inline-block;
+	position: relative;
+	bottom: -0.08em;
+}
+#u2conf input+a {
+	background: #d80;
+}
+#u2conf label {
+	font-size: 1.6em;
+	width: 2em;
+	height: 1em;
+	padding: .4em 0;
+	display: block;
+	border-radius: .25em;
+}
+#u2conf input[type="checkbox"] {
+	position: relative;
+	opacity: .02;
+	top: 2em;
+}
+#u2conf input[type="checkbox"]+label {
+	position: relative;
+	background: #603;
+	border-bottom: .2em solid #a16;
+	box-shadow: 0 .1em .3em #a00 inset;
+}
+#u2conf input[type="checkbox"]:checked+label {
+	background: #6a1;
+	border-bottom: .2em solid #efa;
+	box-shadow: 0 .1em .5em #0c0;
+}
+#u2conf input[type="checkbox"]+label:hover {
+	box-shadow: 0 .1em .3em #fb0;
+	border-color: #fb0;
+}
+#op_up2k.srch #u2conf td:nth-child(1)>*,
+#op_up2k.srch #u2conf td:nth-child(2)>*,
+#op_up2k.srch #u2conf td:nth-child(3)>* {
+	background: #777;
+	border-color: #ccc;
+	box-shadow: none;
+	opacity: .2;
+}
+#u2foot {
+	color: #fff;
+	font-style: italic;
+}
+#u2foot .warn {
+	font-size: 1.3em;
+	padding: .5em .8em;
+	margin: 1em -.6em;
+	color: #f74;
+	background: #322;
+	border: 1px solid #633;
+	border-width: .1em 0;
+	text-align: center;
+}
+#u2foot .warn span {
+	color: #f86;
+}
+html.light #u2foot .warn {
+	color: #b00;
+	background: #fca;
+	border-color: #f70;
+}
+html.light #u2foot .warn span {
+	color: #930;
+}
+#u2foot span {
+	color: #999;
+	font-size: .9em;
+	font-weight: normal;
+}
+#u2footfoot {
+	margin-bottom: -1em;
+}
+.prog {
+	font-family: monospace, monospace;
+}
+#u2tab a>span {
+	font-weight: bold;
+	font-style: italic;
+	color: #fff;
+	padding-left: .2em;
+}
+#u2cleanup {
+	float: right;
+	margin-bottom: -.3em;
+}
+.fsearch_explain {
+	padding-left: .7em;
+	font-size: 1.1em;
+	line-height: 0;
+}
+
+
+
+
+
+html.light #u2btn {
+	box-shadow: .4em .4em 0 #ccc;
+}
+html.light #u2cards span {
+	color: #000;
+}
+html.light #u2cards a {
+	background: linear-gradient(to bottom, #eee, #fff);
+}
+html.light #u2cards a.act {
+	color: #037;
+	background: inherit;
+	box-shadow: 0 -.17em .67em #0ad;
+	border-color: #09c #05a #eee #05a;
+}
+html.light #u2conf .txtbox {
+	background: #fff;
+	color: #444;
+}
+html.light #u2conf .txtbox.err {
+	background: #f96;
+	color: #300;
+}
+html.light #op_up2k.srch #u2btn {
+	border-color: #a80;
+}
+html.light #u2foot {
+	color: #000;
+}
+html.light #u2tab tbody tr:hover td {
+	background: #fff;
+}

copyparty/web/browser.html

@@ -6,10 +6,10 @@
 	<title>⇆🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/browser.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="/.cpr/ui.css?_={{ ts }}">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/upload.css?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="/.cpr/browser.css?_={{ ts }}">
 	{%- if css %}
-	<link rel="stylesheet" type="text/css" media="screen" href="{{ css }}?_={{ ts }}">
+	<link rel="stylesheet" media="screen" href="{{ css }}?_={{ ts }}">
 	{%- endif %}
 </head>
 
@@ -59,6 +59,8 @@
 		</form>
 	</div>
 
+	<div id="op_unpost" class="opview opbox"></div>
+
 	<div id="op_up2k" class="opview"></div>
 
 	<div id="op_cfg" class="opview opbox opwide"></div>
@@ -123,11 +125,12 @@
 	<script>
 		var acct = "{{ acct }}",
 			perms = {{ perms }},
-			tag_order_cfg = {{ tag_order }},
+			def_hcols = {{ def_hcols|tojson }},
 			have_up2k_idx = {{ have_up2k_idx|tojson }},
 			have_tags_idx = {{ have_tags_idx|tojson }},
 			have_mv = {{ have_mv|tojson }},
 			have_del = {{ have_del|tojson }},
+			have_unpost = {{ have_unpost|tojson }},
 			have_zip = {{ have_zip|tojson }};
 	</script>
 	<script src="/.cpr/util.js?_={{ ts }}"></script>

copyparty/web/md.css

@@ -8,137 +8,6 @@ html, body {
 	font-family: sans-serif;
 	line-height: 1.5em;
 }
-
-
-
-#tt, #toast {
-	position: fixed;
-	max-width: 34em;
-	background: #222;
-	border: 0 solid #777;
-	box-shadow: 0 .2em .5em #222;
-	border-radius: .4em;
-	z-index: 9001;
-}
-#tt {
-	overflow: hidden;
-	margin-top: 1em;
-	padding: 0 1.3em;
-	height: 0;
-	opacity: .1;
-	transition: opacity 0.14s, height 0.14s, padding 0.14s;
-}
-#toast {
-	top: 1.4em;
-	right: -1em;
-	line-height: 1.5em;
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	transform: translateX(100%);
-	transition:
-		transform .4s cubic-bezier(.2, 1.2, .5, 1),
-		right .4s cubic-bezier(.2, 1.2, .5, 1);
-	text-shadow: 1px 1px 0 #000;
-	color: #fff;
-}
-#toastc {
-	display: inline-block;
-	position: absolute;
-	overflow: hidden;
-	left: 0;
-	width: 0;
-	opacity: 0;
-	padding: .3em 0;
-	margin: -.3em 0 0 0;
-	line-height: 1.5em;
-	color: #000;
-	border: none;
-	outline: none;
-	text-shadow: none;
-	border-radius: .5em 0 0 .5em;
-	transition: left .3s, width .3s, padding .3s, opacity .3s;
-}
-#toast.vis {
-	right: 1.3em;
-	transform: unset;
-}
-#toast.vis #toastc {
-	left: -2em;
-	width: .4em;
-	padding: .3em .8em;
-	opacity: 1;
-}
-#toast.inf {
-	background: #07a;
-	border-color: #0be;
-}
-#toast.inf #toastc {
-	background: #0be;
-}
-#toast.ok {
-	background: #4a0;
-	border-color: #8e4;
-}
-#toast.ok #toastc {
-	background: #8e4;
-}
-#toast.warn {
-	background: #970;
-	border-color: #fc0;
-}
-#toast.warn #toastc {
-	background: #fc0;
-}
-#toast.err {
-	background: #900;
-	border-color: #d06;
-}
-#toast.err #toastc {
-	background: #d06;
-}
-#tt.b {
-	padding: 0 2em;
-	border-radius: .5em;
-	box-shadow: 0 .2em 1em #000;
-}
-#tt.show {
-	padding: 1em 1.3em;
-	border-width: .4em 0;
-	height: auto;
-	opacity: 1;
-}
-#tt.show.b {
-	padding: 1.5em 2em;
-	border-width: .5em 0;
-}
-#tt code {
-	background: #3c3c3c;
-	padding: .1em .3em;
-	border-top: 1px solid #777;
-	border-radius: .3em;
-	line-height: 1.7em;
-}
-#tt em {
-	color: #f6a;
-}
-html.light #tt {
-	background: #fff;
-	border-color: #888 #000 #777 #000;
-}
-html.light #tt,
-html.light #toast {
-	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
-}
-html.light #tt code {
-	background: #060;
-	color: #fff;
-}
-html.light #tt em {
-	color: #d38;
-}
-
-
-
 #mtw {
 	display: none;
 }
@@ -146,6 +15,10 @@ html.light #tt em {
 	margin: 0 auto;
 	padding: 0 1.5em;
 }
+#toast {
+	bottom: auto;
+	top: 1.4em;
+}
 pre, code, a {
 	color: #480;
 	background: #f7f7f7;

copyparty/web/md.html

@@ -1,11 +1,12 @@
 <!DOCTYPE html><html><head>
 	<meta charset="utf-8">
-	<title>📝🎉 {{ title }}</title> <!-- 📜 -->
+	<title>📝🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/md.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/md.css?_={{ ts }}">
 	{%- if edit %}
-	<link href="/.cpr/md2.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/md2.css?_={{ ts }}">
 	{%- endif %}
 </head>
 <body>

copyparty/web/md.js

@@ -185,7 +185,7 @@ function md_plug_err(ex, js) {
     errbox.style.cssText = 'position:absolute;top:0;left:0;padding:1em .5em;background:#2b2b2b;color:#fc5'
     errbox.textContent = msg;
     errbox.onclick = function () {
-        alert('' + ex.stack);
+        modal.alert('<pre>' + ex.stack + '</pre>');
     };
     if (o) {
         errbox.appendChild(o);

copyparty/web/md2.js

@@ -285,15 +285,15 @@ function Modpoll() {
             console.log("modpoll diff |" + server_ref.length + "|, |" + server_now.length + "|");
             this.modpoll.disabled = true;
             var msg = [
-                "The document has changed on the server.<br />" +
+                "The document has changed on the server.",
                 "The changes will NOT be loaded into your editor automatically.",
-
+                "",
-                "Press F5 or CTRL-R to refresh the page,<br />" +
+                "Press F5 or CTRL-R to refresh the page,",
                 "replacing your document with the server copy.",
-
+                "",
                 "You can close this message to ignore and contnue."
             ];
-            return toast.warn(0, "<p>" + msg.join('</p>\n<p>') + '</p>');
+            return toast.warn(0, msg.join('\n'));
         }
 
         console.log('modpoll eq');
@@ -326,26 +326,32 @@ function save(e) {
         return toast.inf(2, "no changes");
 
     var force = (save_cls.indexOf('force-save') >= 0);
-    if (force && !confirm('confirm that you wish to lose the changes made on the server since you opened this document'))
+    function save2() {
-        return toast.inf(3, 'aborted');
+        var txt = dom_src.value,
+            fd = new FormData();
 
-    var txt = dom_src.value;
+        fd.append("act", "tput");
+        fd.append("lastmod", (force ? -1 : last_modified));
+        fd.append("body", txt);
 
-    var fd = new FormData();
+        var url = (document.location + '').split('?')[0];
-    fd.append("act", "tput");
+        var xhr = new XMLHttpRequest();
-    fd.append("lastmod", (force ? -1 : last_modified));
+        xhr.open('POST', url, true);
-    fd.append("body", txt);
+        xhr.responseType = 'text';
+        xhr.onreadystatechange = save_cb;
+        xhr.btn = save_btn;
+        xhr.txt = txt;
 
-    var url = (document.location + '').split('?')[0];
+        modpoll.skip_one = true;  // skip one iteration while we save
-    var xhr = new XMLHttpRequest();
+        xhr.send(fd);
-    xhr.open('POST', url, true);
+    }
-    xhr.responseType = 'text';
-    xhr.onreadystatechange = save_cb;
-    xhr.btn = save_btn;
-    xhr.txt = txt;
 
-    modpoll.skip_one = true;  // skip one iteration while we save
+    if (!force)
-    xhr.send(fd);
+        save2();
+    else
+        modal.confirm('confirm that you wish to lose the changes made on the server since you opened this document', save2, function () {
+            toast.inf(3, 'aborted');
+        });
 }
 
 function save_cb() {
@@ -353,14 +359,14 @@ function save_cb() {
         return;
 
     if (this.status !== 200)
-        return alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var r;
     try {
         r = JSON.parse(this.responseText);
     }
     catch (ex) {
-        return alert('Failed to parse reply from server:\n\n' + this.responseText);
+        return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
     }
 
     if (!r.ok) {
@@ -375,12 +381,10 @@ function save_cb() {
                 r.lastmod + ' lastmod on the server now,',
                 r.now + ' server time now,\n',
             ];
-            alert(msg.join('\n'));
+            return toast.err(0, msg.join('\n'));
         }
-        else {
+        else
-            alert('Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
+            return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
-        }
-        return;
     }
 
     this.btn.classList.remove('force-save');
@@ -407,10 +411,8 @@ function savechk_cb() {
     if (this.readyState != XMLHttpRequest.DONE)
         return;
 
-    if (this.status !== 200) {
+    if (this.status !== 200)
-        alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
-        return;
-    }
 
     var doc1 = this.txt.replace(/\r\n/g, "\n");
     var doc2 = this.responseText.replace(/\r\n/g, "\n");
@@ -423,12 +425,12 @@ function savechk_cb() {
             }, 100);
             return;
         }
-        alert(
+        modal.alert(
             'Error! The document on the server does not appear to have saved correctly (your editor contents and the server copy is not identical). Place the document on your clipboard for now and check the server logs for hints\n\n' +
             'Length: yours=' + doc1.length + ', server=' + doc2.length
         );
-        alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
+        modal.alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
-        alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
+        modal.alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
         return;
     }
 
@@ -865,12 +867,10 @@ function iter_uni(e) {
 function cfg_uni(e) {
     if (e) e.preventDefault();
 
-    var reply = prompt("unicode whitelist", esc_uni_whitelist);
+    modal.prompt("unicode whitelist", esc_uni_whitelist, function (reply) {
-    if (reply === null)
+        esc_uni_whitelist = reply;
-        return;
+        js_uni_whitelist = eval('\'' + esc_uni_whitelist + '\'');
-
+    }, null);
-    esc_uni_whitelist = reply;
-    js_uni_whitelist = eval('\'' + esc_uni_whitelist + '\'');
 }
 
 

copyparty/web/mde.css

@@ -18,6 +18,10 @@ html, body {
 	background: #f7f7f7;
 	color: #333;
 }
+#toast {
+	bottom: auto;
+	top: 1.4em;
+}
 #mn {
 	font-weight: normal;
 	margin: 1.3em 0 .7em 1em;

copyparty/web/mde.html

@@ -3,9 +3,10 @@
 	<title>📝🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/mde.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/ui.css?_={{ ts }}">
-	<link href="/.cpr/deps/mini-fa.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/mde.css?_={{ ts }}">
-	<link href="/.cpr/deps/easymde.css?_={{ ts }}" rel="stylesheet">
+	<link rel="stylesheet" href="/.cpr/deps/mini-fa.css?_={{ ts }}">
+	<link rel="stylesheet" href="/.cpr/deps/easymde.css?_={{ ts }}">
 </head>
 <body>
 	<div id="mw">

copyparty/web/mde.js

@@ -75,7 +75,7 @@ function set_jumpto() {
 }
 
 function jumpto(ev) {
-    var tgt = ev.target || ev.srcElement;
+    var tgt = ev.target;
     var ln = null;
     while (tgt && !ln) {
         ln = tgt.getAttribute('data-ln');
@@ -110,25 +110,31 @@ function save(mde) {
         return toast.inf(2, 'no changes');
 
     var force = save_btn.classList.contains('force-save');
-    if (force && !confirm('confirm that you wish to lose the changes made on the server since you opened this document'))
+    function save2() {
-        return toast.inf(3, 'aborted');
+        var txt = mde.value();
 
-    var txt = mde.value();
+        var fd = new FormData();
+        fd.append("act", "tput");
+        fd.append("lastmod", (force ? -1 : last_modified));
+        fd.append("body", txt);
 
-    var fd = new FormData();
+        var url = (document.location + '').split('?')[0];
-    fd.append("act", "tput");
+        var xhr = new XMLHttpRequest();
-    fd.append("lastmod", (force ? -1 : last_modified));
+        xhr.open('POST', url, true);
-    fd.append("body", txt);
+        xhr.responseType = 'text';
+        xhr.onreadystatechange = save_cb;
+        xhr.btn = save_btn;
+        xhr.mde = mde;
+        xhr.txt = txt;
+        xhr.send(fd);
+    }
 
-    var url = (document.location + '').split('?')[0];
+    if (!force)
-    var xhr = new XMLHttpRequest();
+        save2();
-    xhr.open('POST', url, true);
+    else
-    xhr.responseType = 'text';
+        modal.confirm('confirm that you wish to lose the changes made on the server since you opened this document', save2, function () {
-    xhr.onreadystatechange = save_cb;
+            toast.inf(3, 'aborted');
-    xhr.btn = save_btn;
+        });
-    xhr.mde = mde;
-    xhr.txt = txt;
-    xhr.send(fd);
 }
 
 function save_cb() {
@@ -136,14 +142,14 @@ function save_cb() {
         return;
 
     if (this.status !== 200)
-        return alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
 
     var r;
     try {
         r = JSON.parse(this.responseText);
     }
     catch (ex) {
-        return alert('Failed to parse reply from server:\n\n' + this.responseText);
+        return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
     }
 
     if (!r.ok) {
@@ -158,12 +164,10 @@ function save_cb() {
                 r.lastmod + ' lastmod on the server now,',
                 r.now + ' server time now,\n',
             ];
-            alert(msg.join('\n'));
+            return toast.err(0, msg.join('\n'));
         }
-        else {
+        else
-            alert('Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
+            return toast.err(0, 'Error! Save failed.  Maybe this JSON explains why:\n\n' + this.responseText);
-        }
-        return;
     }
 
     this.btn.classList.remove('force-save');
@@ -186,35 +190,23 @@ function save_chk() {
     if (this.readyState != XMLHttpRequest.DONE)
         return;
 
-    if (this.status !== 200) {
+    if (this.status !== 200)
-        alert('Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
+        return toast.err(0, 'Error!  The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
-        return;
-    }
 
     var doc1 = this.txt.replace(/\r\n/g, "\n");
     var doc2 = this.responseText.replace(/\r\n/g, "\n");
     if (doc1 != doc2) {
-        alert(
+        modal.alert(
             'Error! The document on the server does not appear to have saved correctly (your editor contents and the server copy is not identical). Place the document on your clipboard for now and check the server logs for hints\n\n' +
             'Length: yours=' + doc1.length + ', server=' + doc2.length
         );
-        alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
+        modal.alert('yours, ' + doc1.length + ' byte:\n[' + doc1 + ']');
-        alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
+        modal.alert('server, ' + doc2.length + ' byte:\n[' + doc2 + ']');
         return;
     }
 
     last_modified = this.lastmod;
     md_changed(this.mde, true);
 
-    var ok = mknod('div');
+    toast.ok(2, 'save OK' + (this.ntry ? '\nattempt ' + this.ntry : ''));
-    ok.setAttribute('style', 'font-size:6em;font-family:serif;font-weight:bold;color:#cf6;background:#444;border-radius:.3em;padding:.6em 0;position:fixed;top:30%;left:calc(50% - 2em);width:4em;text-align:center;z-index:9001;transition:opacity 0.2s ease-in-out;opacity:1');
-    ok.innerHTML = 'OK✔️';
-    var parent = ebi('m');
-    document.documentElement.appendChild(ok);
-    setTimeout(function () {
-        ok.style.opacity = 0;
-    }, 500);
-    setTimeout(function () {
-        ok.parentNode.removeChild(ok);
-    }, 750);
 }

copyparty/web/msg.css

@@ -16,9 +16,6 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-body {
-	padding-bottom: 5em;
-}
 #box {
 	padding: .5em 1em;
 	background: #2c2c2c;

copyparty/web/msg.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/msg.css?_={{ ts }}">
+    <link rel="stylesheet" media="screen" href="/.cpr/msg.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/splash.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/splash.css?_={{ ts }}">
+    <link rel="stylesheet" media="screen" href="/.cpr/splash.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/ui.css

@@ -0,0 +1,203 @@
+#tt, #toast {
+	position: fixed;
+	max-width: 34em;
+	background: #222;
+	border: 0 solid #777;
+	box-shadow: 0 .2em .5em #222;
+	border-radius: .4em;
+	z-index: 9001;
+}
+#tt {
+	overflow: hidden;
+	margin-top: 1em;
+	padding: 0 1.3em;
+	height: 0;
+	opacity: .1;
+	transition: opacity 0.14s, height 0.14s, padding 0.14s;
+}
+#toast {
+	bottom: 5em;
+	right: -1em;
+	line-height: 1.5em;
+	padding: 1em 1.3em;
+	border-width: .4em 0;
+	transform: translateX(100%);
+	transition:
+		transform .4s cubic-bezier(.2, 1.2, .5, 1),
+		right .4s cubic-bezier(.2, 1.2, .5, 1);
+	text-shadow: 1px 1px 0 #000;
+	color: #fff;
+}
+#toastc {
+	display: inline-block;
+	position: absolute;
+	overflow: hidden;
+	left: 0;
+	width: 0;
+	opacity: 0;
+	padding: .3em 0;
+	margin: -.3em 0 0 0;
+	line-height: 1.5em;
+	color: #000;
+	border: none;
+	outline: none;
+	text-shadow: none;
+	border-radius: .5em 0 0 .5em;
+	transition: left .3s, width .3s, padding .3s, opacity .3s;
+}
+#toast pre {
+	margin: 0;
+}
+#toast.vis {
+	right: 1.3em;
+	transform: unset;
+}
+#toast.vis #toastc {
+	left: -2em;
+	width: .4em;
+	padding: .3em .8em;
+	opacity: 1;
+}
+#toast.inf {
+	background: #07a;
+	border-color: #0be;
+}
+#toast.inf #toastc {
+	background: #0be;
+}
+#toast.ok {
+	background: #380;
+	border-color: #8e4;
+}
+#toast.ok #toastc {
+	background: #8e4;
+}
+#toast.warn {
+	background: #970;
+	border-color: #fc0;
+}
+#toast.warn #toastc {
+	background: #fc0;
+}
+#toast.err {
+	background: #900;
+	border-color: #d06;
+}
+#toast.err #toastc {
+	background: #d06;
+}
+#tt.b {
+	padding: 0 2em;
+	border-radius: .5em;
+	box-shadow: 0 .2em 1em #000;
+}
+#tt.show {
+	padding: 1em 1.3em;
+	border-width: .4em 0;
+	height: auto;
+	opacity: 1;
+}
+#tt.show.b {
+	padding: 1.5em 2em;
+	border-width: .5em 0;
+}
+#tt code {
+	background: #3c3c3c;
+	padding: .1em .3em;
+	border-top: 1px solid #777;
+	border-radius: .3em;
+	line-height: 1.7em;
+}
+#tt em {
+	color: #f6a;
+}
+html.light #tt {
+	background: #fff;
+	border-color: #888 #000 #777 #000;
+}
+html.light #tt,
+html.light #toast {
+	box-shadow: 0 .3em 1em rgba(0,0,0,0.4);
+}
+html.light #tt code {
+	background: #060;
+	color: #fff;
+}
+html.light #tt em {
+	color: #d38;
+}
+#modal {
+	position: fixed;
+    overflow: auto;
+	top: 0;
+	left: 0;
+	right: 0;
+	bottom: 0;
+	width: 100%;
+	height: 100%;
+	z-index: 9001;
+	background: rgba(64,64,64,0.6);
+}
+#modal>table {
+    width: 100%;
+    height: 100%;
+}
+#modal td {
+	text-align: center;
+}
+#modalc {
+	position: relative;
+	display: inline-block;
+	background: #f7f7f7;
+	color: #333;
+	text-shadow: none;
+	text-align: left;
+	margin: 3em;
+	padding: 1em 1.1em;
+	border-radius: .6em;
+    box-shadow: 0 .3em 3em rgba(0,0,0,0.5);
+	max-width: 50em;
+	max-height: 30em;
+	overflow: auto;
+}
+@media (min-width: 40em) {
+    #modalc {
+        min-width: 30em;
+    }
+}
+#modalb {
+	position: sticky;
+	text-align: right;
+	padding-top: 1em;
+	bottom: 0;
+	right: 0;
+}
+#modalb a {
+	color: #000;
+	background: #ccc;
+	display: inline-block;
+	border-radius: .3em;
+	padding: .5em 1em;
+	outline: none;
+	border: none;
+}
+#modalb a:focus,
+#modalb a:hover {
+	background: #06d;
+	color: #fff;
+}
+#modalb a+a {
+	margin-left: .5em;
+}
+#modali {
+	display: block;
+	width: calc(100% - 1.25em);
+    margin: 1em -.1em 0 -.1em;
+	padding: .5em;
+	outline: none;
+	border: .25em solid #ccc;
+	border-radius: .4em;
+}
+#modali:focus {
+	border-color: #06d;
+}

copyparty/web/up2k.js

@@ -97,13 +97,11 @@ function up2k_flagbus() {
         }
     };
     var do_take = function (now) {
-        //dbg('*', 'do_take');
         tx(now, "have");
         flag.owner = [flag.id, now];
         flag.ours = true;
     };
     var do_want = function (now) {
-        //dbg('*', 'do_want');
         tx(now, "want");
     };
     flag.take = function (now) {
@@ -135,15 +133,16 @@ function up2k_flagbus() {
 
 
 function U2pvis(act, btns) {
-    this.act = act;
+    var r = this;
-    this.ctr = { "ok": 0, "ng": 0, "bz": 0, "q": 0 };
+    r.act = act;
-    this.tab = [];
+    r.ctr = { "ok": 0, "ng": 0, "bz": 0, "q": 0 };
-    this.head = 0;
+    r.tab = [];
-    this.tail = -1;
+    r.head = 0;
-    this.wsz = 3;
+    r.tail = -1;
+    r.wsz = 3;
 
-    this.addfile = function (entry, sz, draw) {
+    r.addfile = function (entry, sz, draw) {
-        this.tab.push({
+        r.tab.push({
             "hn": entry[0],
             "ht": entry[1],
             "hp": entry[2],
@@ -155,34 +154,34 @@ function U2pvis(act, btns) {
             "bd": 0,  // bytes done
             "bd0": 0  // upload start
         });
-        this.ctr["q"]++;
+        r.ctr["q"]++;
         if (!draw)
             return;
 
-        this.drawcard("q");
+        r.drawcard("q");
-        if (this.act == "q") {
+        if (r.act == "q") {
-            this.addrow(this.tab.length - 1);
+            r.addrow(r.tab.length - 1);
         }
-        if (this.act == "bz") {
+        if (r.act == "bz") {
-            this.bzw();
+            r.bzw();
         }
     };
 
-    this.is_act = function (card) {
+    r.is_act = function (card) {
-        if (this.act == "done")
+        if (r.act == "done")
             return card == "ok" || card == "ng";
 
-        return this.act == card;
+        return r.act == card;
     }
 
-    this.seth = function (nfile, field, html) {
+    r.seth = function (nfile, field, html) {
-        var fo = this.tab[nfile];
+        var fo = r.tab[nfile];
         field = ['hn', 'ht', 'hp'][field];
         if (fo[field] === html)
             return;
 
         fo[field] = html;
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}{1}'.format(nfile, field.slice(1)));
@@ -193,26 +192,26 @@ function U2pvis(act, btns) {
         }
     };
 
-    this.setab = function (nfile, nblocks) {
+    r.setab = function (nfile, nblocks) {
         var t = [];
         for (var a = 0; a < nblocks; a++)
             t.push(0);
 
-        this.tab[nfile].cb = t;
+        r.tab[nfile].cb = t;
     };
 
-    this.setat = function (nfile, blocktab) {
+    r.setat = function (nfile, blocktab) {
-        this.tab[nfile].cb = blocktab;
+        r.tab[nfile].cb = blocktab;
 
         var bd = 0;
         for (var a = 0; a < blocktab.length; a++)
             bd += blocktab[a];
 
-        this.tab[nfile].bd = bd;
+        r.tab[nfile].bd = bd;
-        this.tab[nfile].bd0 = bd;
+        r.tab[nfile].bd0 = bd;
     };
 
-    this.perc = function (bd, bd0, sz, t0) {
+    r.perc = function (bd, bd0, sz, t0) {
         var td = Date.now() - t0,
             p = bd * 100.0 / sz,
             nb = bd - bd0,
@@ -222,15 +221,15 @@ function U2pvis(act, btns) {
         return [p, s2ms(eta), spd / (1024 * 1024)];
     };
 
-    this.hashed = function (fobj) {
+    r.hashed = function (fobj) {
-        var fo = this.tab[fobj.n],
+        var fo = r.tab[fobj.n],
             nb = fo.bt * (++fo.nh / fo.cb.length),
-            p = this.perc(nb, 0, fobj.size, fobj.t_hashing);
+            p = r.perc(nb, 0, fobj.size, fobj.t_hashing);
 
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
             p[0].toFixed(2), p[1], p[2].toFixed(2)
         );
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}p'.format(fobj.n)),
@@ -241,31 +240,31 @@ function U2pvis(act, btns) {
         obj.style.background = 'linear-gradient(90deg, #025, #06a ' + o1 + '%, #09d ' + o2 + '%, #333 ' + o3 + '%, #333 99%, #777)';
     };
 
-    this.prog = function (fobj, nchunk, cbd) {
+    r.prog = function (fobj, nchunk, cbd) {
-        var fo = this.tab[fobj.n],
+        var fo = r.tab[fobj.n],
             delta = cbd - fo.cb[nchunk];
 
         fo.cb[nchunk] = cbd;
         fo.bd += delta;
 
-        var p = this.perc(fo.bd, fo.bd0, fo.bt, fobj.t_uploading);
+        var p = r.perc(fo.bd, fo.bd0, fo.bt, fobj.t_uploading);
         fo.hp = '{0}%, {1}, {2} MB/s'.format(
             p[0].toFixed(2), p[1], p[2].toFixed(2)
         );
 
-        if (!this.is_act(fo.in))
+        if (!r.is_act(fo.in))
             return;
 
         var obj = ebi('f{0}p'.format(fobj.n)),
             o1 = p[0] - 2, o2 = p[0] - 0.1, o3 = p[0];
 
-        if (!obj) { //} || true) {
+        if (!obj) {
             var msg = [
-                "act", this.act,
+                "act", r.act,
                 "in", fo.in,
-                "is_act", this.is_act(fo.in),
+                "is_act", r.is_act(fo.in),
-                "head", this.head,
+                "head", r.head,
-                "tail", this.tail,
+                "tail", r.tail,
                 "nfile", fobj.n,
                 "name", fobj.name,
                 "sz", fobj.size,
@@ -283,12 +282,12 @@ function U2pvis(act, btns) {
             for (var a = 0, aa = ds.length; a < aa; a++) {
                 var id = ds[a].parentNode.getAttribute('id').slice(1, -1);
                 console.log("dom %d/%d = [%s] in(%s) is_act(%s) %s",
-                    a, aa, id, this.tab[id].in, this.is_act(fo.in), ds[a].textContent);
+                    a, aa, id, r.tab[id].in, r.is_act(fo.in), ds[a].textContent);
             }
 
-            for (var a = 0, aa = this.tab.length; a < aa; a++)
+            for (var a = 0, aa = r.tab.length; a < aa; a++)
-                if (this.is_act(this.tab[a].in))
+                if (r.is_act(r.tab[a].in))
-                    console.log("tab %d/%d = sz %s", a, aa, this.tab[a].bt);
+                    console.log("tab %d/%d = sz %s", a, aa, r.tab[a].bt);
 
             throw new Error('see console');
         }
@@ -298,27 +297,27 @@ function U2pvis(act, btns) {
         obj.style.background = 'linear-gradient(90deg, #050, #270 ' + o1 + '%, #4b0 ' + o2 + '%, #333 ' + o3 + '%, #333 99%, #777)';
     };
 
-    this.move = function (nfile, newcat) {
+    r.move = function (nfile, newcat) {
-        var fo = this.tab[nfile],
+        var fo = r.tab[nfile],
             oldcat = fo.in,
-            bz_act = this.act == "bz";
+            bz_act = r.act == "bz";
 
         if (oldcat == newcat)
             return;
 
         fo.in = newcat;
-        this.ctr[oldcat]--;
+        r.ctr[oldcat]--;
-        this.ctr[newcat]++;
+        r.ctr[newcat]++;
-        this.drawcard(oldcat);
+        r.drawcard(oldcat);
-        this.drawcard(newcat);
+        r.drawcard(newcat);
-        if (this.is_act(newcat)) {
+        if (r.is_act(newcat)) {
-            this.tail = Math.max(this.tail, nfile + 1);
+            r.tail = Math.max(r.tail, nfile + 1);
             if (!ebi('f' + nfile))
-                this.addrow(nfile);
+                r.addrow(nfile);
         }
-        else if (this.is_act(oldcat)) {
+        else if (r.is_act(oldcat)) {
-            while (this.head < Math.min(this.tab.length, this.tail) && this.precard[this.tab[this.head].in])
+            while (r.head < Math.min(r.tab.length, r.tail) && r.precard[r.tab[r.head].in])
-                this.head++;
+                r.head++;
 
             if (!bz_act) {
                 var tr = ebi("f" + nfile);
@@ -328,10 +327,10 @@ function U2pvis(act, btns) {
         else return;
 
         if (bz_act)
-            this.bzw();
+            r.bzw();
     };
 
-    this.bzw = function () {
+    r.bzw = function () {
         var first = QS('#u2tab>tbody>tr:first-child');
         if (!first)
             return;
@@ -340,93 +339,93 @@ function U2pvis(act, btns) {
         first = parseInt(first.getAttribute('id').slice(1));
         last = parseInt(last.getAttribute('id').slice(1));
 
-        while (this.head - first > this.wsz) {
+        while (r.head - first > r.wsz) {
             var obj = ebi('f' + (first++));
             if (obj)
                 obj.parentNode.removeChild(obj);
         }
-        while (last - this.tail < this.wsz && last < this.tab.length - 2) {
+        while (last - r.tail < r.wsz && last < r.tab.length - 2) {
             var obj = ebi('f' + (++last));
             if (!obj)
-                this.addrow(last);
+                r.addrow(last);
         }
     };
 
-    this.drawcard = function (cat) {
+    r.drawcard = function (cat) {
         var cards = QSA('#u2cards>a>span');
 
         if (cat == "q") {
-            cards[4].innerHTML = this.ctr[cat];
+            cards[4].innerHTML = r.ctr[cat];
             return;
         }
         if (cat == "bz") {
-            cards[3].innerHTML = this.ctr[cat];
+            cards[3].innerHTML = r.ctr[cat];
             return;
         }
 
-        cards[2].innerHTML = this.ctr["ok"] + this.ctr["ng"];
+        cards[2].innerHTML = r.ctr["ok"] + r.ctr["ng"];
 
         if (cat == "ng") {
-            cards[1].innerHTML = this.ctr[cat];
+            cards[1].innerHTML = r.ctr[cat];
         }
         if (cat == "ok") {
-            cards[0].innerHTML = this.ctr[cat];
+            cards[0].innerHTML = r.ctr[cat];
         }
     };
 
-    this.changecard = function (card) {
+    r.changecard = function (card) {
-        this.act = card;
+        r.act = card;
-        this.precard = has(["ok", "ng", "done"], this.act) ? {} : this.act == "bz" ? { "ok": 1, "ng": 1 } : { "ok": 1, "ng": 1, "bz": 1 };
+        r.precard = has(["ok", "ng", "done"], r.act) ? {} : r.act == "bz" ? { "ok": 1, "ng": 1 } : { "ok": 1, "ng": 1, "bz": 1 };
-        this.postcard = has(["ok", "ng", "done"], this.act) ? { "bz": 1, "q": 1 } : this.act == "bz" ? { "q": 1 } : {};
+        r.postcard = has(["ok", "ng", "done"], r.act) ? { "bz": 1, "q": 1 } : r.act == "bz" ? { "q": 1 } : {};
-        this.head = -1;
+        r.head = -1;
-        this.tail = -1;
+        r.tail = -1;
         var html = [];
-        for (var a = 0; a < this.tab.length; a++) {
+        for (var a = 0; a < r.tab.length; a++) {
-            var rt = this.tab[a].in;
+            var rt = r.tab[a].in;
-            if (this.is_act(rt)) {
+            if (r.is_act(rt)) {
-                html.push(this.genrow(a, true));
+                html.push(r.genrow(a, true));
 
-                this.tail = a;
+                r.tail = a;
-                if (this.head == -1)
+                if (r.head == -1)
-                    this.head = a;
+                    r.head = a;
             }
         }
-        if (this.head == -1) {
+        if (r.head == -1) {
-            for (var a = 0; a < this.tab.length; a++) {
+            for (var a = 0; a < r.tab.length; a++) {
-                var rt = this.tab[a].in;
+                var rt = r.tab[a].in;
-                if (this.precard[rt]) {
+                if (r.precard[rt]) {
-                    this.head = a + 1;
+                    r.head = a + 1;
-                    this.tail = a;
+                    r.tail = a;
                 }
-                else if (this.postcard[rt]) {
+                else if (r.postcard[rt]) {
-                    this.head = a;
+                    r.head = a;
-                    this.tail = a - 1;
+                    r.tail = a - 1;
                     break;
                 }
             }
         }
 
-        if (this.head < 0)
+        if (r.head < 0)
-            this.head = 0;
+            r.head = 0;
 
         if (card == "bz") {
-            for (var a = this.head - 1; a >= this.head - this.wsz && a >= 0; a--) {
+            for (var a = r.head - 1; a >= r.head - r.wsz && a >= 0; a--) {
-                html.unshift(this.genrow(a, true).replace(/><td>/, "><td>a "));
+                html.unshift(r.genrow(a, true).replace(/><td>/, "><td>a "));
             }
-            for (var a = this.tail + 1; a <= this.tail + this.wsz && a < this.tab.length; a++) {
+            for (var a = r.tail + 1; a <= r.tail + r.wsz && a < r.tab.length; a++) {
-                html.push(this.genrow(a, true).replace(/><td>/, "><td>b "));
+                html.push(r.genrow(a, true).replace(/><td>/, "><td>b "));
             }
         }
         ebi('u2tab').tBodies[0].innerHTML = html.join('\n');
     };
 
-    this.genrow = function (nfile, as_html) {
+    r.genrow = function (nfile, as_html) {
-        var r = this.tab[nfile],
+        var row = r.tab[nfile],
             td1 = '<td id="f' + nfile,
             td = '</td>' + td1,
-            ret = td1 + 'n">' + r.hn +
+            ret = td1 + 'n">' + row.hn +
-                td + 't">' + r.ht +
+                td + 't">' + row.ht +
-                td + 'p" class="prog">' + r.hp + '</td>';
+                td + 'p" class="prog">' + row.hp + '</td>';
 
         if (as_html)
             return '<tr id="f' + nfile + '">' + ret + '</tr>';
@@ -437,40 +436,50 @@ function U2pvis(act, btns) {
         return obj;
     };
 
-    this.addrow = function (nfile) {
+    r.addrow = function (nfile) {
-        var tr = this.genrow(nfile);
+        var tr = r.genrow(nfile);
         ebi('u2tab').tBodies[0].appendChild(tr);
     };
 
-    var that = this;
     btns = QSA(btns + '>a[act]');
     for (var a = 0; a < btns.length; a++) {
         btns[a].onclick = function (e) {
             ev(e);
             var newtab = this.getAttribute('act');
-            for (var b = 0; b < btns.length; b++) {
+            function go() {
-                btns[b].className = (
+                for (var b = 0; b < btns.length; b++) {
-                    btns[b].getAttribute('act') == newtab) ? 'act' : '';
+                    btns[b].className = (
+                        btns[b].getAttribute('act') == newtab) ? 'act' : '';
+                }
+                r.changecard(newtab);
             }
-            that.changecard(newtab);
+            var nf = r.ctr[newtab];
+            if (nf === undefined)
+                nf = r.ctr["ok"] + r.ctr["ng"];
+
+            if (nf < 9000)
+                return go();
+
+            modal.confirm('about to show ' + nf + ' files\n\nthis may crash your browser, are you sure?', go, null);
         };
     }
 
-    this.changecard(this.act);
+    r.changecard(r.act);
 }
 
 
-function fsearch_explain(e) {
+function fsearch_explain(n) {
-    ev(e);
+    if (n)
-    if (!has(perms, 'write'))
+        return toast.inf(60, 'your access to this folder is Read-Only\n\n' + (acct == '*' ? 'you are currently not logged in' : 'you are currently logged in as "' + acct + '"'));
-        return alert('your access to this folder is Read-Only\n\n' + (acct == '*' ? 'you are currently not logged in' : 'you are currently logged in as ' + acct));
 
-    alert('you are currently in file-search mode\n\nswitch to upload-mode by clicking the green magnifying glass (next to the big yellow search button), and then refresh\n\nsorry');
+    if (bcfg_get('fsearch', false))
+        return toast.inf(60, 'you are currently in file-search mode\n\nswitch to upload-mode by clicking the green magnifying glass (next to the big yellow search button), and then refresh\n\nsorry');
+
+    return toast.inf(60, 'refresh the page and try again, it should work now');
 }
 
 
 function up2k_init(subtle) {
-    // show modal message
     function showmodal(msg) {
         ebi('u2notbtn').innerHTML = msg;
         ebi('u2btn').style.display = 'none';
@@ -478,7 +487,6 @@ function up2k_init(subtle) {
         ebi('u2conf').style.opacity = '0.5';
     }
 
-    // hide modal message
     function unmodal() {
         ebi('u2notbtn').style.display = 'none';
         ebi('u2btn').style.display = 'block';
@@ -495,7 +503,6 @@ function up2k_init(subtle) {
         // chrome<37 firefox<34 edge<12 opera<24 safari<7
         shame = 'your browser is impressively ancient';
 
-    // upload ui hidden by default, clicking the header shows it
     var got_deps = false;
     function init_deps() {
         if (!got_deps && !subtle && !window.asmCrypto) {
@@ -512,11 +519,9 @@ function up2k_init(subtle) {
         got_deps = true;
     }
 
-    // show uploader if the user only has write-access
     if (perms.length && !has(perms, 'read'))
         goto('up2k');
 
-    // shows or clears a message in the basic uploader ui
     function setmsg(msg, type) {
         if (msg !== undefined) {
             ebi('u2err').setAttribute('class', type);
@@ -534,13 +539,11 @@ function up2k_init(subtle) {
         }
     }
 
-    // switches to the basic uploader with msg as error message
     function un2k(msg) {
         setmsg(msg, 'err');
         return false;
     }
 
-    // handle user intent to use the basic uploader instead
     ebi('u2nope').onclick = function (e) {
         ev(e);
         setmsg(suggest_up2k, 'msg');
@@ -642,7 +645,7 @@ function up2k_init(subtle) {
         else files = e.target.files;
 
         if (!files || !files.length)
-            return alert('no files selected??');
+            return toast.err(0, 'no files selected??');
 
         more_one_file();
         var bad_files = [],
@@ -710,9 +713,9 @@ function up2k_init(subtle) {
                 for (var a = 0; a < Math.min(20, missing.length); a++)
                     msg.push(missing[a]);
 
-                alert(msg.join('\n-- '));
+                return modal.alert(msg.join('\n-- '), function () {
-                dirs = [];
+                    read_dirs(rd, [], [], good, bad, spins);
-                pf = [];
+                });
             }
             spins = 0;
         }
@@ -757,7 +760,6 @@ function up2k_init(subtle) {
                 }
                 ngot += 1;
             });
-            // console.log("ngot: " + ngot);
             if (!ngot) {
                 dirs.shift();
                 rd = null;
@@ -777,16 +779,22 @@ function up2k_init(subtle) {
             if (good_files.length - bad_files.length <= 1 && ANDROID)
                 msg += '\nFirefox-Android has a bug which prevents selecting multiple files. Try selecting one file at a time. For more info, see firefox bug 1456557';
 
-            alert(msg);
+            return modal.alert(msg, function () {
+                gotallfiles(good_files, []);
+            });
         }
 
         var msg = ['upload these ' + good_files.length + ' files?'];
         for (var a = 0, aa = Math.min(20, good_files.length); a < aa; a++)
             msg.push(good_files[a][1]);
 
-        if (ask_up && !fsearch && !confirm(msg.join('\n')))
+        if (ask_up && !fsearch)
-            return;
+            return modal.confirm(msg.join('\n'), function () { up_them(good_files); }, null);
 
+        up_them(good_files);
+    }
+
+    function up_them(good_files) {
         var seen = {},
             evpath = get_evpath(),
             draw_each = good_files.length < 50;
@@ -796,17 +804,25 @@ function up2k_init(subtle) {
 
         for (var a = 0; a < good_files.length; a++) {
             var fobj = good_files[a][0],
+                name = good_files[a][1],
+                fdir = '',
                 now = Date.now(),
-                lmod = fobj.lastModified || now;
+                lmod = fobj.lastModified || now,
+                ofs = name.lastIndexOf('/') + 1;
+
+            if (ofs) {
+                fdir = name.slice(0, ofs);
+                name = name.slice(ofs);
+            }
 
             var entry = {
                 "n": st.files.length,
                 "t0": now,
                 "fobj": fobj,
-                "name": good_files[a][1],
+                "name": name,
                 "size": fobj.size,
                 "lmod": lmod / 1000,
-                "purl": evpath,
+                "purl": fdir,
                 "done": false,
                 "hash": []
             },
@@ -907,25 +923,23 @@ function up2k_init(subtle) {
         }
         return handshakes_permitted() && 0 ==
             st.todo.handshake.length +
-            st.busy.handshake.length;
+            st.busy.handshake.length +
+            st.todo.upload.length +
+            st.busy.upload.length;
     }
 
     var tasker = (function () {
-        var tto = null,
+        var running = false,
-            running = false,
             was_busy = false;
 
         function defer() {
             running = false;
-            clearTimeout(tto);
-            tto = setTimeout(taskerd, 100);
         }
 
         function taskerd() {
             if (running)
                 return;
 
-            clearTimeout(tto);
             if (crashed)
                 return defer();
 
@@ -1016,7 +1030,7 @@ function up2k_init(subtle) {
                     return defer();
             }
         }
-        taskerd();
+        timer.add(taskerd, true);
         return taskerd;
     })();
 
@@ -1147,7 +1161,7 @@ function up2k_init(subtle) {
                     return tasker();
                 }
 
-                alert('y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
+                toast.err(0, 'y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
             };
             reader.readAsArrayBuffer(
                 bobslice.call(t.fobj, car, cdr));
@@ -1301,7 +1315,8 @@ function up2k_init(subtle) {
 
                     if (!response || !response.hits || !response.hits.length) {
                         smsg = '404';
-                        msg = 'not found on server <a href="#" onclick="fsearch_explain()" class="fsearch_explain">(explain)</a>';
+                        msg = ('not found on server <a href="#" onclick="fsearch_explain(' +
+                            (has(perms, 'write') ? '0' : '1') + ')" class="fsearch_explain">(explain)</a>');
                     }
                     else {
                         smsg = 'found';
@@ -1325,9 +1340,10 @@ function up2k_init(subtle) {
                     return;
                 }
 
-                if (response.name !== t.name) {
+                if (response.purl !== t.purl || response.name !== t.name) {
-                    // file exists; server renamed us
+                    // server renamed us (file exists / path restrictions)
-                    console.log("server-rename [" + t.name + "] to [" + response.name + "]");
+                    console.log("server-rename [" + t.purl + "] [" + t.name + "] to [" + response.purl + "] [" + response.name + "]");
+                    t.purl = response.purl;
                     t.name = response.name;
                     pvis.seth(t.n, 0, linksplit(t.purl + t.name).join(' '));
                 }
@@ -1347,7 +1363,7 @@ function up2k_init(subtle) {
                 for (var a = 0; a < missing.length; a++) {
                     var idx = t.hash.indexOf(missing[a]);
                     if (idx < 0)
-                        return alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
+                        return modal.alert('wtf negative index for hash "{0}" in task:\n{1}'.format(
                             missing[a], JSON.stringify(t)));
 
                     t.postlist.push(idx);
@@ -1436,7 +1452,7 @@ function up2k_init(subtle) {
                     tasker();
                     return;
                 }
-                alert("server broke; hs-err {0} on file [{1}]:\n".format(
+                toast.err(0, "server broke; hs-err {0} on file [{1}]:\n".format(
                     xhr.status, t.name) + (
                         (xhr.response && xhr.response.err) ||
                         (xhr.responseText && xhr.responseText) ||
@@ -1456,7 +1472,7 @@ function up2k_init(subtle) {
         if (fsearch)
             req.srch = 1;
 
-        xhr.open('POST', t.purl + 'handshake.php', true);
+        xhr.open('POST', t.purl, true);
         xhr.responseType = 'text';
         xhr.send(JSON.stringify(req));
     }
@@ -1496,7 +1512,7 @@ function up2k_init(subtle) {
                 console.log("ignoring dupe-segment error", t);
             }
             else {
-                alert("server broke; cu-err {0} on file [{1}]:\n".format(
+                toast.err(0, "server broke; cu-err {0} on file [{1}]:\n".format(
                     xhr.status, t.name) + (txt || "no further information"));
                 return;
             }
@@ -1524,7 +1540,7 @@ function up2k_init(subtle) {
                 console.log('chunkpit onerror, retrying', t);
                 do_send();
             };
-            xhr.open('POST', t.purl + 'chunkpit.php', true);
+            xhr.open('POST', t.purl, true);
             xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
             xhr.setRequestHeader("X-Up2k-Wark", t.wark);
             xhr.setRequestHeader('Content-Type', 'application/octet-stream');
@@ -1544,7 +1560,7 @@ function up2k_init(subtle) {
 
     function onresize(e) {
         var bar = ebi('ops'),
-            wpx = innerWidth,
+            wpx = window.innerWidth,
             fpx = parseInt(getComputedStyle(bar)['font-size']),
             wem = wpx * 1.0 / fpx,
             wide = wem > 54,
@@ -1773,3 +1789,14 @@ if (QS('#op_up2k.act'))
     goto_up2k();
 
 apply_perms(perms);
+
+
+(function () {
+    goto();
+    var op = sread('opmode');
+    if (op !== null && op !== '.')
+        try {
+            goto(op);
+        }
+        catch (ex) { }
+})();

copyparty/web/upload.css

@@ -1,302 +0,0 @@
-
-#op_up2k {
-	padding: 0 1em 1em 1em;
-}
-#u2form {
-	position: absolute;
-	top: 0;
-	left: 0;
-	width: 2px;
-	height: 2px;
-	overflow: hidden;
-}
-#u2form input {
-	background: #444;
-	border: 0px solid #444;
-	outline: none;
-}
-#u2err.err {
-	color: #f87;
-	padding: .5em;
-}
-#u2err.msg {
-	color: #999;
-	padding: .5em;
-	font-size: .9em;
-}
-#u2btn {
-	color: #eee;
-	background: #555;
-	background: -moz-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	background: -webkit-linear-gradient(top, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	background: linear-gradient(to bottom, #367 0%, #489 50%, #38788a 51%, #367 100%);
-	filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#489', endColorstr='#38788a', GradientType=0);
-	text-decoration: none;
-	line-height: 1.3em;
-	border: 1px solid #222;
-	border-radius: .4em;
-	text-align: center;
-	font-size: 1.5em;
-	margin: .5em auto;
-	padding: .8em 0;
-	width: 16em;
-	cursor: pointer;
-	box-shadow: .4em .4em 0 #111;
-}
-#op_up2k.srch #u2btn {
-	background: linear-gradient(to bottom, #ca3 0%, #fd8 50%, #fc6 51%, #b92 100%);
-	text-shadow: 1px 1px 1px #fc6;
-	color: #333;
-}
-#u2conf #u2btn {
-	margin: -1.5em 0;
-	padding: .8em 0;
-	width: 100%;
-	max-width: 12em;
-	display: inline-block;
-}
-#u2conf #u2btn_cw {
-	text-align: right;
-}
-#u2notbtn {
-	display: none;
-	text-align: center;
-	background: #333;
-	padding-top: 1em;
-}
-#u2notbtn * {
-	line-height: 1.3em;
-}
-#u2tab {
-	margin: 3em auto;
-	width: calc(100% - 2em);
-	max-width: 100em;
-}
-#op_up2k.srch #u2tab {
-	max-width: none;
-}
-#u2tab td {
-	border: 1px solid #ccc;
-	border-width: 0 0px 1px 0;
-	padding: .1em .3em;
-}
-#u2tab td:nth-child(2) {
-	width: 5em;
-	white-space: nowrap;
-}
-#u2tab td:nth-child(3) {
-	width: 40%;
-}
-#op_up2k.srch td.prog {
-	font-family: sans-serif;
-	font-size: 1em;
-	width: auto;
-}
-#u2tab tbody tr:hover td {
-	background: #222;
-}
-#u2cards {
-	padding: 1em 0 .3em 1em;
-	margin: 1.5em auto -2.5em auto;
-	white-space: nowrap;
-	text-align: center;
-	overflow: hidden;
-}
-#u2cards.w {
-	width: 45em;
-	text-align: left;
-}
-#u2cards a {
-	padding: .2em 1em;
-	border: 1px solid #777;
-	border-width: 0 0 1px 0;
-	background: linear-gradient(to bottom, #333, #222);
-}
-#u2cards a:first-child {
-	border-radius: .4em 0 0 0;
-}
-#u2cards a:last-child {
-	border-radius: 0 .4em 0 0;
-}
-#u2cards a.act {
-	padding-bottom: .5em;
-	border-width: 1px 1px .1em 1px;
-	border-radius: .3em .3em 0 0;
-	margin-left: -1px;
-	background: linear-gradient(to bottom, #464, #333 80%);
-	box-shadow: 0 -.17em .67em #280;
-	border-color: #7c5 #583 #333 #583;
-	position: relative;
-	color: #fd7;
-}
-#u2cards span {
-	color: #fff;
-}
-#u2conf {
-	margin: 1em auto;
-	width: 30em;
-}
-#u2conf.has_btn {
-	width: 48em;
-}
-#u2conf * {
-	text-align: center;
-	line-height: 1em;
-	margin: 0;
-	padding: 0;
-	border: none;
-	outline: none;
-}
-#u2conf .txtbox {
-	width: 3em;
-	color: #fff;
-	background: #444;
-	border: 1px solid #777;
-	font-size: 1.2em;
-	padding: .15em 0;
-	height: 1.05em;
-}
-#u2conf .txtbox.err {
-	background: #922;
-}
-#u2conf a {
-	color: #fff;
-	background: #c38;
-	text-decoration: none;
-	border-radius: .1em;
-	font-size: 1.5em;
-	padding: .1em 0;
-	margin: 0 -1px;
-	width: 1.5em;
-	height: 1em;
-	display: inline-block;
-	position: relative;
-	bottom: -0.08em;
-}
-#u2conf input+a {
-	background: #d80;
-}
-#u2conf label {
-	font-size: 1.6em;
-	width: 2em;
-	height: 1em;
-	padding: .4em 0;
-	display: block;
-	border-radius: .25em;
-}
-#u2conf input[type="checkbox"] {
-	position: relative;
-	opacity: .02;
-	top: 2em;
-}
-#u2conf input[type="checkbox"]+label {
-	position: relative;
-	background: #603;
-	border-bottom: .2em solid #a16;
-	box-shadow: 0 .1em .3em #a00 inset;
-}
-#u2conf input[type="checkbox"]:checked+label {
-	background: #6a1;
-	border-bottom: .2em solid #efa;
-	box-shadow: 0 .1em .5em #0c0;
-}
-#u2conf input[type="checkbox"]+label:hover {
-	box-shadow: 0 .1em .3em #fb0;
-	border-color: #fb0;
-}
-#op_up2k.srch #u2conf td:nth-child(1)>*,
-#op_up2k.srch #u2conf td:nth-child(2)>*,
-#op_up2k.srch #u2conf td:nth-child(3)>* {
-	background: #777;
-	border-color: #ccc;
-	box-shadow: none;
-	opacity: .2;
-}
-#u2foot {
-	color: #fff;
-	font-style: italic;
-}
-#u2foot .warn {
-	font-size: 1.3em;
-	padding: .5em .8em;
-	margin: 1em -.6em;
-	color: #f74;
-	background: #322;
-	border: 1px solid #633;
-	border-width: .1em 0;
-	text-align: center;
-}
-#u2foot .warn span {
-	color: #f86;
-}
-html.light #u2foot .warn {
-	color: #b00;
-	background: #fca;
-	border-color: #f70;
-}
-html.light #u2foot .warn span {
-	color: #930;
-}
-#u2foot span {
-	color: #999;
-	font-size: .9em;
-	font-weight: normal;
-}
-#u2footfoot {
-	margin-bottom: -1em;
-}
-.prog {
-	font-family: monospace, monospace;
-}
-#u2tab a>span {
-	font-weight: bold;
-	font-style: italic;
-	color: #fff;
-	padding-left: .2em;
-}
-#u2cleanup {
-	float: right;
-	margin-bottom: -.3em;
-}
-.fsearch_explain {
-	padding-left: .7em;
-	font-size: 1.1em;
-	line-height: 0;
-}
-
-
-
-
-
-html.light #u2btn {
-	box-shadow: .4em .4em 0 #ccc;
-}
-html.light #u2cards span {
-	color: #000;
-}
-html.light #u2cards a {
-	background: linear-gradient(to bottom, #eee, #fff);
-}
-html.light #u2cards a.act {
-	color: #037;
-	background: inherit;
-	box-shadow: 0 -.17em .67em #0ad;
-	border-color: #09c #05a #eee #05a;
-}
-html.light #u2conf .txtbox {
-	background: #fff;
-	color: #444;
-}
-html.light #u2conf .txtbox.err {
-	background: #f96;
-	color: #300;
-}
-html.light #op_up2k.srch #u2btn {
-	border-color: #a80;
-}
-html.light #u2foot {
-	color: #000;
-}
-html.light #u2tab tbody tr:hover td {
-	background: #fff;
-}

copyparty/web/util.js

@@ -140,10 +140,10 @@ function import_js(url, cb) {
     var script = mknod('script');
     script.type = 'text/javascript';
     script.src = url;
-
-    script.onreadystatechange = cb;
     script.onload = cb;
-
+    script.onerror = function () {
+        toast.err(0, 'Failed to load module:\n' + url);
+    };
     head.appendChild(script);
 }
 
@@ -179,84 +179,6 @@ function clmod(obj, cls, add) {
 }
 
 
-function sortfiles(nodes) {
-    var sopts = jread('fsort', [["href", 1, ""]]);
-
-    try {
-        var is_srch = false;
-        if (nodes[0]['rp']) {
-            is_srch = true;
-            for (var b = 0, bb = nodes.length; b < bb; b++)
-                nodes[b].ext = nodes[b].rp.split('.').pop();
-            for (var b = 0; b < sopts.length; b++)
-                if (sopts[b][0] == 'href')
-                    sopts[b][0] = 'rp';
-        }
-        for (var a = sopts.length - 1; a >= 0; a--) {
-            var name = sopts[a][0], rev = sopts[a][1], typ = sopts[a][2];
-            if (!name)
-                continue;
-
-            if (name == 'ts')
-                typ = 'int';
-
-            if (name.indexOf('tags/') === 0) {
-                name = name.slice(5);
-                for (var b = 0, bb = nodes.length; b < bb; b++)
-                    nodes[b]._sv = nodes[b].tags[name];
-            }
-            else {
-                for (var b = 0, bb = nodes.length; b < bb; b++) {
-                    var v = nodes[b][name];
-
-                    if ((v + '').indexOf('<a ') === 0)
-                        v = v.split('>')[1];
-                    else if (name == "href" && v) {
-                        if (v.slice(-1) == '/')
-                            v = '\t' + v;
-
-                        v = uricom_dec(v)[0]
-                    }
-
-                    nodes[b]._sv = v;
-                }
-            }
-
-            var onodes = nodes.map(function (x) { return x; });
-            nodes.sort(function (n1, n2) {
-                var v1 = n1._sv,
-                    v2 = n2._sv;
-
-                if (v1 === undefined) {
-                    if (v2 === undefined) {
-                        return onodes.indexOf(n1) - onodes.indexOf(n2);
-                    }
-                    return -1 * rev;
-                }
-                if (v2 === undefined) return 1 * rev;
-
-                var ret = rev * (typ == 'int' ? (v1 - v2) : (v1.localeCompare(v2)));
-                if (ret === 0)
-                    ret = onodes.indexOf(n1) - onodes.indexOf(n2);
-
-                return ret;
-            });
-        }
-        for (var b = 0, bb = nodes.length; b < bb; b++) {
-            delete nodes[b]._sv;
-            if (is_srch)
-                delete nodes[b].ext;
-        }
-    }
-    catch (ex) {
-        console.log("failed to apply sort config: " + ex);
-        console.log("resetting fsort " + sread('fsort'))
-        localStorage.removeItem('fsort');
-    }
-    return nodes;
-}
-
-
 function sortTable(table, col, cb) {
     var tb = table.tBodies[0],
         th = table.tHead.rows[0].cells,
@@ -398,6 +320,15 @@ function uricom_dec(txt) {
 }
 
 
+function uricom_adec(arr) {
+    var ret = [];
+    for (var a = 0; a < arr.length; a++)
+        ret.push(uricom_dec(arr[a])[0]);
+
+    return ret;
+}
+
+
 function get_evpath() {
     var ret = document.location.pathname;
 
@@ -546,6 +477,39 @@ function hist_replace(url) {
 }
 
 
+var timer = (function () {
+    var r = {};
+    r.q = [];
+    r.last = 0;
+
+    r.add = function (fun, run) {
+        r.rm(fun);
+        r.q.push(fun);
+
+        if (run)
+            fun();
+    };
+
+    r.rm = function (fun) {
+        apop(r.q, fun);
+    };
+
+    function impl() {
+        if (Date.now() - r.last < 69)
+            return;
+
+        var q = r.q.slice(0);
+        for (var a = 0; a < q.length; a++)
+            q[a]();
+
+        r.last = Date.now();
+    }
+    setInterval(impl, 100);
+
+    return r;
+})();
+
+
 var tt = (function () {
     var r = {
         "tt": mknod("div"),
@@ -647,14 +611,24 @@ var tt = (function () {
 })();
 
 
+function lf2br(txt) {
+    var html = '', hp = txt.split(/(?=<.?pre>)/i);
+    for (var a = 0; a < hp.length; a++)
+        html += hp[a].startsWith('<pre>') ? hp[a] :
+            hp[a].replace(/<br ?.?>\n/g, '\n').replace(/\n<br ?.?>/g, '\n').replace(/\n/g, '<br />\n');
+
+    return html;
+}
+
+
 var toast = (function () {
     var r = {},
         te = null,
-        visible = false,
         obj = mknod('div');
 
     obj.setAttribute('id', 'toast');
-    document.body.appendChild(obj);;
+    document.body.appendChild(obj);
+    r.visible = false;
 
     r.hide = function (e) {
         ev(e);
@@ -668,12 +642,7 @@ var toast = (function () {
         if (ms)
             te = setTimeout(r.hide, ms * 1000);
 
-        var html = '', hp = txt.split(/(?=<.?pre>)/i);
+        obj.innerHTML = '<a href="#" id="toastc">x</a>' + lf2br(txt);
-        for (var a = 0; a < hp.length; a++)
-            html += hp[a].startsWith('<pre>') ? hp[a] :
-                hp[a].replace(/<br ?.?>\n/g, '\n').replace(/\n<br ?.?>/g, '\n').replace(/\n/g, '<br />\n');
-
-        obj.innerHTML = '<a href="#" id="toastc">x</a>' + html;
         obj.className = cl;
         ms += obj.offsetWidth;
         obj.className += ' vis';
@@ -696,3 +665,113 @@ var toast = (function () {
 
     return r;
 })();
+
+
+var modal = (function () {
+    var r = {},
+        q = [],
+        o = null,
+        cb_ok = null,
+        cb_ng = null;
+
+    r.busy = false;
+
+    r.show = function (html) {
+        o = mknod('div');
+        o.setAttribute('id', 'modal');
+        o.innerHTML = '<table><tr><td><div id="modalc">' + html + '</div></td></tr></table>';
+        document.body.appendChild(o);
+        document.addEventListener('keydown', onkey);
+        r.busy = true;
+
+        var a = ebi('modal-ng');
+        if (a)
+            a.onclick = ng;
+
+        a = ebi('modal-ok');
+        a.onclick = ok;
+
+        (ebi('modali') || a).focus();
+    };
+
+    r.hide = function () {
+        o.parentNode.removeChild(o);
+        document.removeEventListener('keydown', onkey);
+        r.busy = false;
+        setTimeout(next, 50);
+    };
+    function ok(e) {
+        ev(e);
+        var v = ebi('modali');
+        v = v ? v.value : true;
+        r.hide();
+        if (cb_ok)
+            cb_ok(v);
+    }
+    function ng(e) {
+        ev(e);
+        r.hide();
+        if (cb_ng)
+            cb_ng(null);
+    }
+
+    function onkey(e) {
+        if (e.code == 'Enter') {
+            var a = ebi('modal-ng');
+            if (a && document.activeElement == a)
+                return ng();
+
+            return ok();
+        }
+
+        if (e.code == 'Escape')
+            return ng();
+    }
+
+    function next() {
+        if (!r.busy && q.length)
+            q.shift()();
+    }
+
+    r.alert = function (html, cb) {
+        q.push(function () {
+            _alert(lf2br(html), cb);
+        });
+        next();
+    };
+    function _alert(html, cb) {
+        cb_ok = cb_ng = cb;
+        html += '<div id="modalb"><a href="#" id="modal-ok">OK</a></div>';
+        r.show(html);
+    }
+
+    r.confirm = function (html, cok, cng) {
+        q.push(function () {
+            _confirm(lf2br(html), cok, cng);
+        });
+        next();
+    }
+    function _confirm(html, cok, cng) {
+        cb_ok = cok;
+        cb_ng = cng === undefined ? cok : null;
+        html += '<div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        r.show(html);
+    }
+
+    r.prompt = function (html, v, cok, cng) {
+        q.push(function () {
+            _prompt(lf2br(html), v, cok, cng);
+        });
+        next();
+    }
+    function _prompt(html, v, cok, cng) {
+        cb_ok = cok;
+        cb_ng = cng === undefined ? cok : null;
+        html += '<input id="modali" type="text" /><div id="modalb"><a href="#" id="modal-ok">OK</a><a href="#" id="modal-ng">Cancel</a></div>';
+        r.show(html);
+
+        ebi('modali').value = v || '';
+    }
+
+    return r;
+})();

docs/notes.sh

@@ -44,7 +44,7 @@ avg() { awk 'function pr(ncsz) {if (nsmp>0) {printf "%3s %s\n", csz, sum/nsmp} c
 dirs=("$HOME/vfs/ほげ" "$HOME/vfs/ほげ/ぴよ" "$HOME/vfs/$(printf \\xed\\x91)" "$HOME/vfs/$(printf \\xed\\x91/\\xed\\x92)")
 mkdir -p "${dirs[@]}"
 for dir in "${dirs[@]}"; do for fn in ふが "$(printf \\xed\\x93)" 'qwe,rty;asd fgh+jkl%zxc&vbn <qwe>"rty'"'"'uio&asd&nbsp;fgh'; do echo "$dir" > "$dir/$fn.html"; done; done
-
+# qw er+ty%20ui%%20op<as>df&gh&amp;jk#zx'cv"bn`m=qw*er^ty?ui@op,as.df-gh_jk
 
 ##
 ## upload mojibake
@@ -79,6 +79,10 @@ command -v gdate && date() { gdate "$@"; }; while true; do t=$(date +%s.%N); (ti
 # get all up2k search result URLs
 var t=[]; var b=document.location.href.split('#')[0].slice(0, -1); document.querySelectorAll('#u2tab .prog a').forEach((x) => {t.push(b+encodeURI(x.getAttribute("href")))}); console.log(t.join("\n"));
 
+# rename all selected songs to <leading-track-number> + <Title> + <extension>
+var sel=msel.getsel(), ci=find_file_col('Title')[0], re=[]; for (var a=0; a<sel.length; a++) { var url=sel[a].vp, tag=ebi(sel[a].id).closest('tr').querySelectorAll('td')[ci].textContent, name=uricom_dec(vsplit(url)[1])[0], m=/^([0-9]+[\. -]+)?.*(\.[^\.]+$)/.exec(name), name2=(m[1]||'')+tag+m[2], url2=vsplit(url)[0]+uricom_enc(name2,false); if (url!=url2) re.push([url, url2]); }
+console.log(JSON.stringify(re, null, '  '));
+function f() { if (!re.length) return treectl.goto(get_evpath()); var [u1,u2] = re.shift(); fetch(u1+'?move='+u2).then((rsp) => {if (rsp.ok) f(); }); }; f();
 
 ##
 ## bash oneliners

scripts/make-sfx.sh

@@ -2,6 +2,7 @@
 set -e
 echo
 
+help() { exec cat <<'EOF'
 
 # optional args:
 #
@@ -26,6 +27,8 @@ echo
 #
 # `no-dd` saves ~2k by removing the mouse cursor
 
+EOF
+}
 
 # port install gnutar findutils gsed coreutils
 gtar=$(command -v gtar || command -v gnutar) || true
@@ -34,6 +37,7 @@ gtar=$(command -v gtar || command -v gnutar) || true
 	sed()  { gsed  "$@"; }
 	find() { gfind "$@"; }
 	sort() { gsort "$@"; }
+	sha1sum() { shasum "$@"; }
 	unexpand() { gunexpand "$@"; }
 	command -v grealpath >/dev/null &&
 		realpath() { grealpath "$@"; }
@@ -72,6 +76,7 @@ while [ ! -z "$1" ]; do
 		no-cm)  no_cm=1  ; ;;
 		no-sh)  do_sh=   ; ;;
 		no-py)  do_py=   ; ;;
+		*)      help     ; ;;
 	esac
 	shift
 done
@@ -81,16 +86,23 @@ tmv() {
 	mv t "$1"
 }
 
+stamp=$(
+	for d in copyparty scripts; do
+		find $d -type f -printf '%TY-%Tm-%Td %TH:%TM:%TS %p\n'
+	done | sort | tail -n 1 | sha1sum | cut -c-16
+)
+
 rm -rf sfx/*
 mkdir -p sfx build
 cd sfx
 
-[ $repack ] && {
+tmpdir="$(
-	old="$(
+	printf '%s\n' "$TMPDIR" /tmp |
-		printf '%s\n' "$TMPDIR" /tmp |
+	awk '/./ {print; exit}'
-		awk '/./ {print; exit}'
+)"
-	)/pe-copyparty"
 
+[ $repack ] && {
+	old="$tmpdir/pe-copyparty"
 	echo "repack of files in $old"
 	cp -pR "$old/"*{dep-j2,copyparty} .
 }
@@ -172,12 +184,12 @@ mkdir -p ../dist
 sfx_out=../dist/copyparty-sfx
 
 echo cleanup
-find .. -name '*.pyc' -delete
+find -name '*.pyc' -delete
-find .. -name __pycache__ -delete
+find -name __pycache__ -delete
 
 # especially prevent osx from leaking your lan ip (wtf apple)
-find .. -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
+find -type f \( -name .DS_Store -or -name ._.DS_Store \) -delete
-find .. -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
+find -type f -name ._\* | while IFS= read -r f; do cmp <(printf '\x00\x05\x16') <(head -c 3 -- "$f") && rm -f -- "$f"; done
 
 echo use smol web deps
 rm -f copyparty/web/deps/*.full.* copyparty/web/dbg-* copyparty/web/Makefile
@@ -196,19 +208,24 @@ done
 	rm -rf copyparty/web/mde.* copyparty/web/deps/easymde*
 	echo h > copyparty/web/mde.html
 	f=copyparty/web/md.html
-	sed -r '/edit2">edit \(fancy/d' <$f >t && tmv "$f"
+	sed -r '/edit2">edit \(fancy/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
 	f=copyparty/web/md.css
-	sed -r '/scp\.woff2/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r '/scp\.woff2/d' <$f >t
+	tmv "$f"
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
-	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t && tmv "$f"
+	gzip -d "$f"
+	sed -r 's/(cursor: )url\([^)]+\), (pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: cursor/d' <$f >t
+	tmv "$f"
 }
 
 [ $repack ] ||
@@ -222,7 +239,8 @@ awk '/^LOREM_IPSUM_WORDS/{o=1;print "LOREM_IPSUM_WORDS = u\"a\"";next} !o; /"""/
 tmv "$f"
 
 # up2k goes from 28k to 22k laff
-echo entabbening
+awk 'BEGIN{gensub(//,"",1)}' </dev/null &&
+echo entabbening &&
 find | grep -E '\.css$' | while IFS= read -r f; do
 	awk '{
 		sub(/^[ \t]+/,"");
@@ -236,25 +254,48 @@ find | grep -E '\.css$' | while IFS= read -r f; do
 	' <$f | sed 's/;\}$/}/' >t
 	tmv "$f"
 done
+unexpand -h 2>/dev/null &&
 find | grep -E '\.(js|html)$' | while IFS= read -r f; do
 	unexpand -t 4 --first-only <"$f" >t
 	tmv "$f"
 done
 
-
 gzres() {
-command -v pigz &&
+	command -v pigz &&
-	pk='pigz -11 -J 34 -I 100' ||
+		pk='pigz -11 -I 256' ||
-	pk='gzip'
+		pk='gzip'
 
-echo "$pk"
+	echo "$pk"
-find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
+	find | grep -E '\.(js|css)$' | grep -vF /deps/ | while IFS= read -r f; do
-	echo -n .
+		echo -n .
-	$pk "$f"
+		$pk "$f"
-done
+	done
-echo
+	echo
+}
+
+
+zdir="$tmpdir/cpp-mksfx"
+[ -e "$zdir/$stamp" ] || rm -rf "$zdir"
+mkdir -p "$zdir"
+echo a > "$zdir/$stamp"
+nf=$(ls -1 "$zdir"/arc.* | wc -l)
+[ $nf -ge 2 ] && [ ! $repack ] && use_zdir=1 || use_zdir=
+
+[ $use_zdir ] || {
+	echo "$nf alts += 1"
+	gzres
+	[ $repack ] ||
+		tar -cf "$zdir/arc.$(date +%s)" copyparty/web/*.gz
+}
+[ $use_zdir ] && {
+	arcs=("$zdir"/arc.*)
+	arc="${arcs[$RANDOM % ${#arcs[@]} ] }"
+	echo "using $arc"
+	tar -xf "$arc"
+	for f in copyparty/web/*.gz; do
+		rm "${f%.*}"
+	done
 }
-gzres
 
 
 echo gen tarlist

scripts/sfx.py

@@ -364,7 +364,7 @@ def confirm(rv):
     except:
         pass
 
-    sys.exit(rv)
+    sys.exit(rv or 1)
 
 
 def run(tmp, j2):

scripts/test/smoketest.py

@@ -124,7 +124,7 @@ def tc1():
 
         arg = "{}:{}:{}".format(pd, ud, p, hp)
         if hp:
-            arg += ":chist=" + hp
+            arg += ":c,hist=" + hp
 
         args += ["-v", arg]
 

scripts/uncomment.py

@@ -65,9 +65,9 @@ def uncomment(fpath):
 
 
 def main():
-    print("uncommenting", end="")
+    print("uncommenting", end="", flush=True)
     for f in sys.argv[1:]:
-        print(".", end="")
+        print(".", end="", flush=True)
         uncomment(f)
 
     print("k")

setup.py

@@ -99,6 +99,7 @@ args = {
         "Programming Language :: Python :: 3.7",
         "Programming Language :: Python :: 3.8",
         "Programming Language :: Python :: 3.9",
+        "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: Implementation :: CPython",
         "Programming Language :: Python :: Implementation :: PyPy",
         "Environment :: Console",

tests/test_httpcli.py

@@ -31,6 +31,7 @@ class Cfg(Namespace):
             rproxy=0,
             ed=False,
             nw=False,
+            unpost=600,
             no_mv=False,
             no_del=False,
             no_zip=False,
@@ -38,10 +39,12 @@ class Cfg(Namespace):
             no_scandir=False,
             no_sendfile=True,
             no_rescan=True,
+            re_maxage=0,
             ihead=False,
             nih=True,
             mtp=[],
             mte="a",
+            mth="",
             hist=None,
             no_hash=False,
             css_browser=None,

tests/test_vfs.py

@@ -21,10 +21,12 @@ class Cfg(Namespace):
         ex2 = {
             "mtp": [],
             "mte": "a",
+            "mth": "",
             "hist": None,
             "no_hash": False,
             "css_browser": None,
             "no_voldump": True,
+            "re_maxage": 0,
             "rproxy": 0,
         }
         ex.update(ex2)