v0.13.14

README.md

@@ -8,21 +8,21 @@
 
 turn your phone or raspi into a portable file server with resumable uploads/downloads using *any* web browser
 
-* server runs on anything with `py2.7` or `py3.3+`
+* server only needs `py2.7` or `py3.3+`, all dependencies optional
 * browse/upload with IE4 / netscape4.0 on win3.11 (heh)
 * *resumable* uploads need `firefox 34+` / `chrome 41+` / `safari 7+` for full speed
 * code standard: `black`
 
-📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [thumbnails](#thumbnails) // [md-viewer](#markdown-viewer) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [ie4](#browser-support)
+📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer) // [ie4](#browser-support)
 
 
 ## readme toc
 
 * top
-    * [quickstart](#quickstart) - download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
+    * **[quickstart](#quickstart)** - download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
         * [on debian](#on-debian) - recommended additional steps on debian
     * [notes](#notes) - general notes
-    * [status](#status) - summary: all planned features work! now please enjoy the bloatening
+    * [status](#status) - feature summary
     * [testimonials](#testimonials) - small collection of user feedback
 * [bugs](#bugs)
     * [general bugs](#general-bugs)
@@ -30,30 +30,31 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 * [accounts and volumes](#accounts-and-volumes) - per-folder, per-user permissions
 * [the browser](#the-browser) - accessing a copyparty server using a web-browser
     * [tabs](#tabs) - the main tabs in the ui
-    * [hotkeys](#hotkeys) - the browser has the following hotkeys (always qwerty)
+    * [hotkeys](#hotkeys) - the browser has the following hotkeys
     * [navpane](#navpane) - switching between breadcrumbs or navpane
-    * [thumbnails](#thumbnails) - press `g` to toggle image/video thumbnails instead of the file listing
+    * [thumbnails](#thumbnails) - press `g` to toggle grid-view instead of the file listing
     * [zip downloads](#zip-downloads) - download folders (or file selections) as `zip` or `tar` files
     * [uploading](#uploading) - web-browsers can upload using `bup` and `up2k`
         * [file-search](#file-search) - drop files/folders into up2k to see if they exist on the server
         * [unpost](#unpost) - undo/delete accidental uploads
     * [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
-    * [batch rename](#batch-rename) - select some files and press F2 to bring up the rename UI
+    * [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
     * [markdown viewer](#markdown-viewer) - and there are *two* editors
     * [other tricks](#other-tricks)
     * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
 * [server config](#server-config)
     * [file indexing](#file-indexing)
-    * [upload rules](#upload-rules) - set upload rules using volume flags, some examples
+    * [upload rules](#upload-rules) - set upload rules using volume flags
     * [compress uploads](#compress-uploads) - files can be autocompressed on upload
-    * [database location](#database-location) - can be stored in-volume (default) or elsewhere
+    * [database location](#database-location) - in-volume (`.hist/up2k.db`, default) or somewhere else
     * [metadata from audio files](#metadata-from-audio-files) - set `-e2t` to index tags on upload
     * [file parser plugins](#file-parser-plugins) - provide custom parsers to index additional tags
     * [complete examples](#complete-examples)
 * [browser support](#browser-support) - TLDR: yes
 * [client examples](#client-examples) - interact with copyparty using non-browser clients
 * [up2k](#up2k) - quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
-* [performance](#performance) - defaults are good for most cases
+    * [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
+* [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
 * [dependencies](#dependencies) - mandatory deps
     * [optional dependencies](#optional-dependencies) - install these to enable bonus features
     * [install recommended deps](#install-recommended-deps)
@@ -71,12 +72,12 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 
 ## quickstart
 
-download [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) and you're all set!
+download **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** and you're all set!
 
-running the sfx without arguments (for example doubleclicking it on Windows) will give everyone full access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
+running the sfx without arguments (for example doubleclicking it on Windows) will give everyone read/write access to the current folder; see `-h` for help if you want [accounts and volumes](#accounts-and-volumes) etc
 
 some recommended options:
-* `-e2dsa` enables general file indexing, see [search configuration](#search-configuration)
+* `-e2dsa` enables general [file indexing](#file-indexing)
 * `-e2ts` enables audio metadata indexing (needs either FFprobe or Mutagen), see [optional dependencies](#optional-dependencies)
 * `-v /mnt/music:/music:r:rw,foo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, and read-write for user `foo`, password `bar`
   * replace `:r:rw,foo` with `:r,foo` to only make the folder readable by `foo` and nobody else
@@ -90,9 +91,7 @@ you may also want these, especially on servers:
 
 ### on debian
 
-recommended additional steps on debian
-
-enable audio metadata and thumbnails (from images and videos):
+recommended additional steps on debian  which enable audio metadata and thumbnails (from images and videos):
 
 * as root, run the following:  
   `apt install python3 python3-pip python3-dev ffmpeg`
@@ -114,12 +113,12 @@ browser-specific:
 * Android-Chrome: increase "parallel uploads" for higher speed (android bug)
 * Android-Firefox: takes a while to select files (their fix for ☝️)
 * Desktop-Firefox: ~~may use gigabytes of RAM if your files are massive~~ *seems to be OK now*
-* Desktop-Firefox: may stop you from deleting folders you've uploaded until you visit `about:memory` and click `Minimize memory usage`
+* Desktop-Firefox: may stop you from deleting files you've uploaded until you visit `about:memory` and click `Minimize memory usage`
 
 
 ## status
 
-summary: all planned features work! now please enjoy the bloatening
+feature summary
 
 * backend stuff
   * ☑ sanic multipart parser
@@ -137,7 +136,7 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ [folders as zip / tar files](#zip-downloads)
   * ☑ FUSE client (read-only)
 * browser
-  * ☑ navpane (directory tree sidebar)
+  * ☑ [navpane](#navpane) (directory tree sidebar)
   * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
   * ☑ audio player (with OS media controls)
   * ☑ image gallery with webm player
@@ -168,12 +167,11 @@ small collection of user feedback
 * Windows: python 3.7 and older cannot read tags with FFprobe, so use Mutagen or upgrade
 * Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
 * Windows: python 2.7 cannot handle filenames with mojibake
-* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions
+* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
 
 ## general bugs
 
 * all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
-* cannot mount something at `/d1/d2/d3` unless `d2` exists inside `d1`
 * probably more, pls let me know
 
 ## not my bugs
@@ -194,15 +192,16 @@ per-folder, per-user permissions
 * `-a usr:pwd` adds account `usr` with password `pwd`
 * `-v .::r` adds current-folder `.` as the webroot, `r`eadable by anyone
   * the syntax is `-v src:dst:perm:perm:...` so local-path, url-path, and one or more permissions to set
-  * when granting permissions to an account, the names are comma-separated: `-v .::r,usr1,usr2:rw,usr3,usr4`
+  * granting the same permissions to multiple accounts:  
+    `-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
 
 permissions:
 * `r` (read): browse folder contents, download files, download as zip/tar
-* `w` (write): upload files, move files *into* folder
-* `m` (move): move files/folders *from* folder
+* `w` (write): upload files, move files *into* this folder
+* `m` (move): move files/folders *from* this folder
 * `d` (delete): delete files/folders
 
-example:
+examples:
 * add accounts named u1, u2, u3 with passwords p1, p2, p3: `-a u1:p1 -a u2:p2 -a u3:p3`
 * make folder `/srv` the root of the filesystem, read-only by anyone: `-v /srv::r`
 * make folder `/mnt/music` available at `/music`, read-only for u1 and u2, read-write for u3: `-v /mnt/music:music:r,u1,u2:rw,u3`
@@ -235,15 +234,15 @@ the main tabs in the ui
 
 ## hotkeys
 
-the browser has the following hotkeys (always qwerty)
-* `B` toggle breadcrumbs / navpane
+the browser has the following hotkeys  (always qwerty)
+* `B` toggle breadcrumbs / [navpane](#navpane)
 * `I/K` prev/next folder
 * `M` parent folder (or unexpand current)
-* `G` toggle list / grid view
+* `G` toggle list / [grid view](#thumbnails)
 * `T` toggle thumbnails / icons
 * `ctrl-X` cut selected files/folders
 * `ctrl-V` paste
-* `F2` rename selected file/folder
+* `F2` [rename](#batch-rename) selected file/folder
 * when a file/folder is selected (in not-grid-view):
   * `Up/Down` move cursor
   * shift+`Up/Down` select and move cursor
@@ -270,7 +269,7 @@ the browser has the following hotkeys (always qwerty)
     * `M` mute
 * when the navpane is open:
   * `A/D` adjust tree width
-* in the grid view:
+* in the [grid view](#thumbnails):
   * `S` toggle multiselect
   * shift+`A/D` zoom
 * in the markdown editor:
@@ -288,12 +287,14 @@ switching between breadcrumbs or navpane
 
 click the `🌲` or pressing the `B` hotkey to toggle between breadcrumbs path (default), or a navpane (tree-browser sidebar thing)
 
-click `[-]` and `[+]` (or hotkeys `A`/`D`) to adjust the size, and the `[a]` toggles if the tree should widen dynamically as you go deeper or stay fixed-size
+* `[-]` and `[+]` (or hotkeys `A`/`D`) adjust the size
+* `[v]` jumps to the currently open folder
+* `[a]` toggles automatic widening as you go deeper
 
 
 ## thumbnails
 
-press `g` to toggle image/video thumbnails instead of the file listing
+press `g` to toggle grid-view instead of the file listing,  and `t` toggles icons / thumbnails
 
 ![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
 
@@ -308,7 +309,7 @@ in the grid/thumbnail view, if the audio player panel is open, songs will start
 
 download folders (or file selections) as `zip` or `tar` files
 
-select which type of archive you want in the browser settings tab:
+select which type of archive you want in the `[⚙️] config` tab:
 
 | name | url-suffix | description |
 |--|--|--|
@@ -355,18 +356,22 @@ see [up2k](#up2k) for details on how it works
 the up2k UI is the epitome of polished inutitive experiences:
 * "parallel uploads" specifies how many chunks to upload at the same time
 * `[🏃]` analysis of other files should continue while one is uploading
-* `[💭]` ask for confirmation before files are added to the list
+* `[💭]` ask for confirmation before files are added to the queue
 * `[💤]` sync uploading between other copyparty browser-tabs so only one is active
-* `[🔎]` switch between upload and file-search mode
+* `[🔎]` switch between upload and [file-search](#file-search) mode
 
 and then theres the tabs below it,
-* `[ok]` is uploads which completed successfully
-* `[ng]` is the uploads which failed / got rejected (already exists, ...)
+* `[ok]` is the files which completed successfully
+* `[ng]` is the ones that failed / got rejected (already exists, ...)
 * `[done]` shows a combined list of `[ok]` and `[ng]`, chronological order
 * `[busy]` files which are currently hashing, pending-upload, or uploading
   * plus up to 3 entries each from `[done]` and `[que]` for context
 * `[que]` is all the files that are still queued
 
+note that since up2k has to read each file twice, `[🎈 bup]` can be up to 2x faster in some extreme cases (huge files combined with internet connection faster than the read-speed of your HDD)
+
+if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
+
 
 ### file-search
 
@@ -381,10 +386,6 @@ files go into `[ok]` if they exist (and you get a link to where it is), otherwis
 
 adding the same file multiple times is blocked, so if you first search for a file and then decide to upload it, you have to click the `[cleanup]` button to discard `[done]` files (or just refresh the page)
 
-note that since up2k has to read the file twice, `[🎈 bup]` can be up to 2x faster in extreme cases (if your internet connection is faster than the read-speed of your HDD)
-
-up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
-
 
 ### unpost
 
@@ -399,12 +400,22 @@ you can unpost even if you don't have regular move/delete access, however only f
 
 cut/paste, rename, and delete files/folders (if you have permission)
 
+file selection: click somewhere on the line (not the link itsef), then:
+* `space` to toggle
+* `up/down` to move
+* `shift-up/down` to move-and-select
+* `ctrl-shift-up/down` to also scroll
+
+* cut: select some files and `ctrl-x`
+* paste: `ctrl-v` in another folder
+* rename: `F2`
+
 you can move files across browser tabs (cut in one tab, paste in another)
 
 
 ## batch rename
 
-select some files and press F2 to bring up the rename UI
+select some files and press `F2` to bring up the rename UI
 
 ![batch-rename-fs8](https://user-images.githubusercontent.com/241032/128434204-eb136680-3c07-4ec7-92e0-ae86af20c241.png)
 
@@ -464,6 +475,12 @@ and there are *two* editors
 
 * if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
 
+* click the bottom-left `π` to open a javascript prompt for debugging
+
+* files named `.prologue.html` / `.epilogue.html` will be rendered before/after directory listings unless `--no-logues`
+
+* files named `README.md` / `readme.md` will be rendered after directory listings unless `--no-readme` (but `.epilogue.html` takes precedence)
+
 
 ## searching
 
@@ -486,15 +503,15 @@ add the argument `-e2ts` to also scan/index tags from music files, which brings
 
 ## file indexing
 
-file indexing relies on two databases, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`). Configuration can be done through arguments, volume flags, or a mix of both.
+file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volume flags, or a mix of both.
 
 through arguments:
 * `-e2d` enables file indexing on upload
-* `-e2ds` scans writable folders for new files on startup
-* `-e2dsa` scans all mounted volumes (including readonly ones)
+* `-e2ds` also scans writable folders for new files on startup
+* `-e2dsa` also scans all mounted volumes (including readonly ones)
 * `-e2t` enables metadata indexing on upload
-* `-e2ts` scans for tags in all files that don't have tags yet
-* `-e2tsr` deletes all existing tags, does a full reindex
+* `-e2ts` also scans for tags in all files that don't have tags yet
+* `-e2tsr` also deletes all existing tags, doing a full reindex
 
 the same arguments can be set as volume flags, in addition to `d2d` and `d2t` for disabling:
 * `-v ~/music::r:c,e2dsa:c,e2tsr` does a full reindex of everything on startup
@@ -502,10 +519,11 @@ the same arguments can be set as volume flags, in addition to `d2d` and `d2t` fo
 * `-v ~/music::r:c,d2t` disables all `-e2t*` (tags), does not affect `-e2d*`
 
 note:
+* the parser currently can't handle `c,e2dsa,e2tsr` so you have to `c,e2dsa:c,e2tsr`
 * `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
 * the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
 
-you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
+to save some time, you can choose to only index filename/path/size/last-modified (and not the hash of the file contents) by setting `--no-hash` or the volume-flag `:c,dhash`, this has the following consequences:
 * initial indexing is way faster, especially when the volume is on a network disk
 * makes it impossible to [file-search](#file-search)
 * if someone uploads the same file contents, the upload will not be detected as a dupe, so it will not get symlinked or rejected
@@ -515,7 +533,7 @@ if you set `--no-hash`, you can enable hashing for specific volumes using flag `
 
 ## upload rules
 
-set upload rules using volume flags, some examples:
+set upload rules using volume flags,  some examples:
 
 * `:c,sz=1k-3m` sets allowed filesize between 1 KiB and 3 MiB inclusive (suffixes: b, k, m, g)
 * `:c,nosub` disallow uploading into subdirectories; goes well with `rotn` and `rotf`:
@@ -534,9 +552,7 @@ you can also set transaction limits which apply per-IP and per-volume, but these
 
 ## compress uploads
 
-files can be autocompressed on upload
-
-compression is either on user-request (if config allows) or forced by server-config
+files can be autocompressed on upload,  either on user-request (if config allows) or forced by server-config
 
 * volume flag `gz` allows gz compression
 * volume flag `xz` allows lzma compression
@@ -557,7 +573,7 @@ some examples,
 
 ## database location
 
-can be stored in-volume (default) or elsewhere
+in-volume (`.hist/up2k.db`, default) or somewhere else
 
 copyparty creates a subfolder named `.hist` inside each volume where it stores the database, thumbnails, and some other stuff
 
@@ -579,7 +595,7 @@ set `-e2t` to index tags on upload
 
 if you add/remove a tag from `mte` you will need to run with `-e2tsr` once to rebuild the database, otherwise only new files will be affected
 
-but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the settings pane
+but instead of using `-mte`, `-mth` is a better way to hide tags in the browser: these tags will not be displayed by default, but they still get indexed and become searchable, and users can choose to unhide them in the `[⚙️] config` pane
 
 `-mtm` can be used to add or redefine a metadata mapping, say you have media files with `foo` and `bar` tags and you want them to display as `qux` in the browser (preferring `foo` if both are present), then do `-mtm qux=foo,bar` and now you can `-mte artist,title,qux`
 
@@ -647,10 +663,10 @@ TLDR: yes
 | **= feature =** | ie6 | ie9  | ie10 | ie11 | ff 52 | c 49 | iOS | Andr |
 
 * internet explorer 6 to 8 behave the same
-* firefox 52 and chrome 49 are the last winxp versions
-* `*1` yes, but extremely slow (ie10: 1 MiB/s, ie11: 270 KiB/s)
+* firefox 52 and chrome 49 are the final winxp versions
+* `*1` yes, but extremely slow (ie10: `1 MiB/s`, ie11: `270 KiB/s`)
 * `*2` causes a full-page refresh on each navigation
-* `*3` using a wasm decoder which can sometimes get stuck and consumes a bit more power
+* `*3` using a wasm decoder which consumes a bit more power
 
 quick summary of more eccentric web-browsers trying to view a directory index:
 
@@ -694,6 +710,8 @@ copyparty returns a truncated sha512sum of your PUT/POST as base64; you can gene
     b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
     b512 <movie.mkv
 
+you can provide passwords using cookie 'cppwd=hunter2', as a url query `?pw=hunter2`, or with basic-authentication (either as the username or password)
+
 
 # up2k
 
@@ -710,12 +728,25 @@ quick outline of the up2k protocol, see [uploading](#uploading) for the web-clie
   * server writes chunks into place based on the hash
 * client does another handshake with the hashlist; server replies with OK or a list of chunks to reupload
 
+up2k has saved a few uploads from becoming corrupted in-transfer already; caught an android phone on wifi redhanded in wireshark with a bitflip, however bup with https would *probably* have noticed as well (thanks to tls also functioning as an integrity check)
+
+
+## why chunk-hashes
+
+a single sha512 would be better, right?
+
+this is due to `crypto.subtle` not providing a streaming api (or the option to seed the sha512 hasher with a starting hash)
+
+as a result, the hashes are much less useful than they could have been (search the server by sha512, provide the sha512 in the response http headers, ...)
+
+hashwasm would solve the streaming issue but reduces hashing speed for sha512 (xxh128 does 6 GiB/s), and it would make old browsers and [iphones](https://bugs.webkit.org/show_bug.cgi?id=228552) unsupported
+
 
 # performance
 
-defaults are good for most cases
+defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
 
-you can ignore the `cannot efficiently use multiple CPU cores` message, it's very unlikely to be a problem
+you can ignore the `cannot efficiently use multiple CPU cores` message, very unlikely to be a problem
 
 below are some tweaks roughly ordered by usefulness:
 
@@ -730,6 +761,24 @@ below are some tweaks roughly ordered by usefulness:
   ...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
 
 
+# security
+
+some notes on hardening
+
+on public copyparty instances with anonymous upload enabled:
+
+* users can upload html/css/js which will evaluate for other visitors in a few ways,
+  * unless `--no-readme` is set: by uploading/modifying a file named `readme.md`
+  * if `move` access is granted AND none of `--no-logues`, `--no-dot-mv`, `--no-dot-ren` is set: by uploading some .html file and renaming it to `.epilogue.html` (uploading it directly is blocked)
+
+
+## gotchas
+
+behavior that might be unexpected
+
+* users without read-access to a folder can still see the `.prologue.html` / `.epilogue.html` / `README.md` contents, for the purpose of showing a description on how to use the uploader for example
+
+
 # dependencies
 
 mandatory deps:
@@ -744,17 +793,11 @@ enable music tags:
 * either `mutagen` (fast, pure-python, skips a few tags, makes copyparty GPL? idk)
 * or `ffprobe` (20x slower, more accurate, possibly dangerous depending on your distro and users)
 
-enable thumbnails of images:
-* `Pillow` (requires py2.7 or py3.5+)
-
-enable thumbnails of videos:
-* `ffmpeg` and `ffprobe` somewhere in `$PATH`
-
-enable thumbnails of HEIF pictures:
-* `pyheif-pillow-opener` (requires Linux or a C compiler)
-
-enable thumbnails of AVIF pictures:
-* `pillow-avif-plugin`
+enable [thumbnails](#thumbnails) of...
+* **images:** `Pillow` (requires py2.7 or py3.5+)
+* **videos:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
+* **HEIF pictures:** `pyheif-pillow-opener` (requires Linux or a C compiler)
+* **AVIF pictures:** `pillow-avif-plugin`
 
 
 ## install recommended deps
@@ -791,8 +834,10 @@ if you don't need all the features, you can repack the sfx and save a bunch of s
 * `223k` after `./scripts/make-sfx.sh re no-ogv no-cm`
 
 the features you can opt to drop are
-* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files
-* `cm`/easymde, the "fancy" markdown editor
+* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files, saves ~192k
+* `cm`/easymde, the "fancy" markdown editor, saves ~92k
+* `fnt`, source-code-pro, the monospace font, saves ~9k
+* `dd`, the custom mouse cursor for the media player tray tab, saves ~2k
 
 for the `re`pack to work, first run one of the sfx'es once to unpack it
 
@@ -828,7 +873,7 @@ pip install black bandit pylint flake8  # vscode tooling
 
 ## just the sfx
 
-grab the web-dependencies from a previous sfx (unless you need to modify something in those):
+first grab the web-dependencies from a previous sfx (assuming you don't need to modify something in those):
 
 ```sh
 rm -rf copyparty/web/deps
@@ -854,8 +899,8 @@ in the `scripts` folder:
 
 * run `make -C deps-docker` to build all dependencies
 * `git tag v1.2.3 && git push origin --tags`
-* create github release with `make-tgz-release.sh`
 * upload to pypi with `make-pypi-release.(sh|bat)`
+* create github release with `make-tgz-release.sh`
 * create sfx with `make-sfx.sh`
 
 
@@ -864,7 +909,6 @@ in the `scripts` folder:
 roughly sorted by priority
 
 * hls framework for Someone Else to drop code into :^)
-* readme.md as epilogue
 
 
 ## discarded ideas

contrib/README.md

@@ -29,7 +29,8 @@ however if your copyparty is behind a reverse-proxy, you may want to use [`share
 
 # OS integration
 init-scripts to start copyparty as a service
-* [`systemd/copyparty.service`](systemd/copyparty.service)
+* [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
+* [`systemd/prisonparty.service`](systemd/prisonparty.service) runs the sfx in a chroot
 * [`openrc/copyparty`](openrc/copyparty)
 
 # Reverse-proxy

copyparty/__main__.py

@@ -360,6 +360,10 @@ def run_argparse(argv, formatter):
     ap2 = ap.add_argument_group('safety options')
     ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="scan all volumes; arguments USER,VOL,FLAGS; example [**,*,ln,p,r]")
     ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt")
+    ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
+    ap2.add_argument("--no-dot-ren", action="store_true", help="disallow renaming dotfiles; makes it impossible to make something a dotfile")
+    ap2.add_argument("--no-logues", action="store_true", help="disable rendering .prologue/.epilogue.html into directory listings")
+    ap2.add_argument("--no-readme", action="store_true", help="disable rendering readme.md into directory listings")
 
     ap2 = ap.add_argument_group('logging options')
     ap2.add_argument("-q", action="store_true", help="quiet")

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 13, 13)
+VERSION = (0, 13, 14)
 CODENAME = "future-proof"
-BUILD_DT = (2021, 9, 3)
+BUILD_DT = (2021, 9, 6)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/httpcli.py

@@ -1868,11 +1868,21 @@ class HttpCli(object):
             tpl = "browser2"
 
         logues = ["", ""]
-        for n, fn in enumerate([".prologue.html", ".epilogue.html"]):
-            fn = os.path.join(abspath, fn)
-            if bos.path.exists(fn):
-                with open(fsenc(fn), "rb") as f:
-                    logues[n] = f.read().decode("utf-8")
+        if not self.args.no_logues:
+            for n, fn in enumerate([".prologue.html", ".epilogue.html"]):
+                fn = os.path.join(abspath, fn)
+                if bos.path.exists(fn):
+                    with open(fsenc(fn), "rb") as f:
+                        logues[n] = f.read().decode("utf-8")
+
+        readme = ""
+        if not self.args.no_readme and not logues[1]:
+            for fn in ["README.md", "readme.md"]:
+                fn = os.path.join(abspath, fn)
+                if bos.path.exists(fn):
+                    with open(fsenc(fn), "rb") as f:
+                        readme = f.read().decode("utf-8")
+                        break
 
         ls_ret = {
             "dirs": [],
@@ -1882,6 +1892,7 @@ class HttpCli(object):
             "acct": self.uname,
             "perms": perms,
             "logues": logues,
+            "readme": readme,
         }
         j2a = {
             "vdir": quotep(self.vpath),
@@ -1900,6 +1911,7 @@ class HttpCli(object):
             "have_b_u": (self.can_write and self.uparam.get("b") == "u"),
             "url_suf": url_suf,
             "logues": logues,
+            "readme": readme,
             "title": html_escape(self.vpath, crlf=True),
             "srv_info": srv_info,
         }

copyparty/stolen/surrogateescape.py

@@ -1,3 +1,5 @@
+# coding: utf-8
+
 """
 This is Victor Stinner's pure-Python implementation of PEP 383: the "surrogateescape" error
 handler of Python 3.
@@ -171,7 +173,7 @@ FS_ENCODING = sys.getfilesystemencoding()
 
 if WINDOWS and not PY3:
     # py2 thinks win* is mbcs, probably a bug? anyways this works
-    FS_ENCODING = 'utf-8'
+    FS_ENCODING = "utf-8"
 
 
 # normalize the filesystem encoding name.

copyparty/up2k.py

@@ -1520,6 +1520,14 @@ class Up2k(object):
         dabs = dvn.canonical(drem)
         drd, dfn = vsplit(drem)
 
+        n1 = svp.split('/')[-1]
+        n2 = dvp.split('/')[-1]
+        if n1.startswith('.') or n2.startswith('.'):
+            if self.args.no_dot_mv:
+                raise Pebkac(400, "moving dotfiles was disabled by server config")
+            elif self.args.no_dot_ren and n1 != n2:
+                raise Pebkac(400, "renaming dotfiles was disabled by server config")
+
         if bos.path.exists(dabs):
             raise Pebkac(400, "mv2: target file exists")
 

copyparty/web/browser.css

@@ -2,10 +2,6 @@
 	--grid-sz: 10em;
 	--grid-ln: 3;
 }
-@font-face {
-	font-family: 'scp';
-	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
-}
 * {
 	line-height: 1.2em;
 }
@@ -177,6 +173,10 @@ a, #files tbody div a:last-child {
 #epi.logue {
 	margin: .8em 0;
 }
+.mdo,
+.mdo * {
+	line-height: 1.4em;
+}
 #srv_info {
 	color: #a73;
 	background: #333;
@@ -1542,7 +1542,8 @@ html.light #bbox-overlay figcaption a {
 
 #op_up2k {
 	padding: 0 1em 1em 1em;
-	min-height: 60em;
+	min-height: 0;
+	transition: min-height .2s;
 }
 #u2form {
 	position: absolute;

copyparty/web/browser.html

@@ -133,7 +133,8 @@
 			have_mv = {{ have_mv|tojson }},
 			have_del = {{ have_del|tojson }},
 			have_unpost = {{ have_unpost|tojson }},
-			have_zip = {{ have_zip|tojson }};
+			have_zip = {{ have_zip|tojson }},
+			readme = {{ readme|tojson }};
 	</script>
 	<script src="/.cpr/util.js?_={{ ts }}"></script>
 	<script src="/.cpr/browser.js?_={{ ts }}"></script>

copyparty/web/browser.js

@@ -140,9 +140,10 @@ ebi('op_cfg').innerHTML = (
 	'	<div>\n' +
 	'		<a id="tooltips" class="tgl btn" href="#" tt="◔ ◡ ◔">ℹ️ tooltips</a>\n' +
 	'		<a id="lightmode" class="tgl btn" href="#">☀️ lightmode</a>\n' +
-	'		<a id="dotfiles" class="tgl btn" href="#" tt="show hidden files (if server permits)">dotfiles</a>\n' +
 	'		<a id="griden" class="tgl btn" href="#" tt="toggle icons or list-view$NHotkey: G">田 the grid</a>\n' +
 	'		<a id="thumbs" class="tgl btn" href="#" tt="in icon view, toggle icons or thumbnails$NHotkey: T">🖼️ thumbs</a>\n' +
+	'		<a id="dotfiles" class="tgl btn" href="#" tt="show hidden files (if server permits)">dotfiles</a>\n' +
+	'		<a id="ireadme" class="tgl btn" href="#" tt="show README.md in folder listings">📜 readme</a>\n' +
 	'	</div>\n' +
 	'</div>\n' +
 	(have_zip ? (
@@ -2999,7 +3000,8 @@ var treectl = (function () {
 	var treectl = {
 		"hidden": true,
 		"ls_cb": null,
-		"dir_cb": tree_scrollto
+		"dir_cb": tree_scrollto,
+		"ireadme": bcfg_get('ireadme', true)
 	},
 		entreed = false,
 		fixedpos = false,
@@ -3320,6 +3322,12 @@ var treectl = (function () {
 		ebi('pro').innerHTML = res.logues ? res.logues[0] || "" : "";
 		ebi('epi').innerHTML = res.logues ? res.logues[1] || "" : "";
 
+		clmod(ebi('epi'), 'mdo');
+		if (res.readme)
+			setTimeout(function () {
+				show_readme(res.readme);
+			}, 10);
+
 		document.title = '⇆🎉 ' + uricom_dec(document.location.pathname.slice(1, -1))[0];
 
 		filecols.set_style();
@@ -3372,6 +3380,12 @@ var treectl = (function () {
 		treectl.goto(get_evpath());
 	}
 
+	function treadme(e) {
+		ev(e);
+		treectl.ireadme = !treectl.ireadme;
+		bcfg_set('ireadme', treectl.ireadme);
+	}
+
 	function dyntree(e) {
 		ev(e);
 		dyn = !dyn;
@@ -3393,6 +3407,7 @@ var treectl = (function () {
 	ebi('detree').onclick = treectl.detree;
 	ebi('visdir').onclick = tree_scrollto;
 	ebi('dotfiles').onclick = tdots;
+	ebi('ireadme').onclick = treadme;
 	ebi('dyntree').onclick = dyntree;
 	ebi('twig').onclick = scaletree;
 	ebi('twobytwo').onclick = scaletree;
@@ -3823,6 +3838,7 @@ var light;
 
 	function freshen() {
 		clmod(document.documentElement, "light", light);
+		clmod(document.documentElement, "dark", !light);
 		pbar.drawbuf();
 		pbar.drawpos();
 		vbar.draw();
@@ -4036,6 +4052,51 @@ var msel = (function () {
 })();
 
 
+function show_readme(md, url, depth) {
+	if (!treectl.ireadme)
+		return;
+
+	var div = ebi('epi'),
+		errmsg = 'cannot show README.md:\n\n',
+		now = window.location.href.replace(/\/?[?#].*/, "");
+
+	url = url || now;
+	if (url != now)
+		return;
+
+	if (!window['marked']) {
+		if (depth)
+			return toast.warn(10, errmsg + 'failed to load marked.js')
+
+		return import_js('/.cpr/deps/marked.js', function () {
+			show_readme(md, url, 1);
+		});
+	}
+
+	try {
+		clmod(div, 'mdo', 1);
+		div.innerHTML = marked(md, {
+			headerPrefix: 'md-',
+			breaks: true,
+			gfm: true
+		});
+		var links = QSA('#epi a');
+		for (var a = 0, aa = links.length; a < aa; a++) {
+			var href = links[a].getAttribute('href');
+			if (!href.startsWith('#'))
+				continue;
+
+			links[a].setAttribute('href', '#md-' + href.slice(1));
+		}
+	}
+	catch (ex) {
+		toast.warn(10, errmsg + ex);
+	}
+}
+if (readme)
+	show_readme(readme);
+
+
 (function () {
 	try {
 		var tr = ebi('files').tBodies[0].rows;

copyparty/web/md.css

@@ -1,7 +1,3 @@
-@font-face {
-	font-family: 'scp';
-	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
-}
 html, body {
 	color: #333;
 	background: #eee;
@@ -11,7 +7,7 @@ html, body {
 #repl {
 	position: absolute;
 	top: 0;
-	right: .2em;
+	right: .5em;
 	border: none;
 	color: inherit;
 	background: none;
@@ -27,122 +23,8 @@ html, body {
 	bottom: auto;
 	top: 1.4em;
 }
-pre, code, a {
-	color: #480;
-	background: #f7f7f7;
-	border: .07em solid #ddd;
-	border-radius: .2em;
-	padding: .1em .3em;
-	margin: 0 .1em;
-}
-code {
-	font-size: .96em;
-}
-pre, code, tt {
-	font-family: 'scp', monospace, monospace;
-	white-space: pre-wrap;
-	word-break: break-all;
-}
-pre {
-	counter-reset: precode;
-}
-pre code {
-	counter-increment: precode;
-	display: inline-block;
-	margin: 0 -.3em;
-	padding: .4em .5em;
-	border: none;
-	border-bottom: 1px solid #cdc;
-	min-width: calc(100% - .6em);
-	line-height: 1.1em;
-}
-pre code:last-child {
-	border-bottom: none;
-}
-pre code::before {
-	content: counter(precode);
-	-webkit-user-select: none;
-	-moz-user-select: none;
-	-ms-user-select: none;
-	user-select: none;
-	display: inline-block;
-	text-align: right;
-	font-size: .75em;
-	color: #48a;
-	width: 4em;
-	padding-right: 1.5em;
-	margin-left: -5.5em;
-}
-pre code:hover {
-	background: #fec;
-	color: #360;
-}
-h1, h2 {
-	line-height: 1.5em;
-}
-h1 {
-	font-size: 1.7em;
-	text-align: center;
-	border: 1em solid #777;
-	border-width: .05em 0;
-	margin: 3em 0;
-}
-h2 {
-	font-size: 1.5em;
-	font-weight: normal;
-	background: #f7f7f7;
-	border-top: .07em solid #fff;
-	border-bottom: .07em solid #bbb;
-	border-radius: .5em .5em 0 0;
-	padding-left: .4em;
-	margin-top: 3em;
-}
-h3 {
-	border-bottom: .1em solid #999;
-}
-h1 a, h3 a, h5 a,
-h2 a, h4 a, h6 a {
-	color: inherit;
-	display: block;
-	background: none;
-	border: none;
-	padding: 0;
-	margin: 0;
-}
-#mp ul,
-#mp ol {
-	border-left: .3em solid #ddd;
-}
-#m>ul,
-#m>ol {
-	border-color: #bbb;
-}
-#mp ul>li {
-	list-style-type: disc;
-}
-#mp ul>li,
-#mp ol>li {
-	margin: .7em 0;
-}
-strong {
-	color: #000;
-}
-p>em,
-li>em,
-td>em {
-	color: #c50;
-	padding: .1em;
-	border-bottom: .1em solid #bbb;
-}
-blockquote {
-	font-family: serif;
-	background: #f7f7f7;
-	border: .07em dashed #ccc;
-	padding: 0 2em;
-	margin: 1em 0;
-}
-small {
-	opacity: .8;
+a {
+	text-decoration: none;
 }
 #toc {
 	margin: 0 1em;
@@ -190,14 +72,6 @@ small {
 	color: #6b3;
 	text-shadow: .02em 0 0 #6b3;
 }
-table {
-	border-collapse: collapse;
-	margin: 1em 0;
-}
-th, td {
-	padding: .2em .5em;
-	border: .12em solid #aaa;
-}
 blink {
 	animation: blinker .7s cubic-bezier(.9, 0, .1, 1) infinite;
 }
@@ -210,6 +84,36 @@ blink {
 	}
 }
 
+
+.mdo pre {
+	counter-reset: precode;
+}
+.mdo pre code {
+	counter-increment: precode;
+	display: inline-block;
+	border: none;
+	border-bottom: 1px solid #cdc;
+	min-width: calc(100% - .6em);
+}
+.mdo pre code:last-child {
+	border-bottom: none;
+}
+.mdo pre code::before {
+	content: counter(precode);
+	-webkit-user-select: none;
+	-moz-user-select: none;
+	-ms-user-select: none;
+	user-select: none;
+	display: inline-block;
+	text-align: right;
+	font-size: .75em;
+	color: #48a;
+	width: 4em;
+	padding-right: 1.5em;
+	margin-left: -5.5em;
+}
+
+
 @media screen {
 	html, body {
 		margin: 0;
@@ -226,34 +130,6 @@ blink {
 	#mp {
 		max-width: 52em;
 		margin-bottom: 6em;
-		word-break: break-word;
-		overflow-wrap: break-word;
-		word-wrap: break-word; /*ie*/
-	}
-	a {
-		color: #fff;
-		background: #39b;
-		text-decoration: none;
-		padding: 0 .3em;
-		border: none;
-		border-bottom: .07em solid #079;
-	}
-	h2 {
-		color: #fff;
-		background: #555;
-		margin-top: 2em;
-		border-bottom: .22em solid #999;
-		border-top: none;
-	}
-	h1 {
-		color: #fff;
-		background: #444;
-		font-weight: normal;
-		border-top: .4em solid #fb0;
-		border-bottom: .4em solid #777;
-		border-radius: 0 1em 0 1em;
-		margin: 3em 0 1em 0;
-		padding: .5em 0;
 	}
 	#mn {
 		padding: 1.3em 0 .7em 1em;
@@ -306,6 +182,8 @@ blink {
 		color: #444;
 		background: none;
 		text-decoration: underline;
+		margin: 0 .1em;
+		padding: 0 .3em;
 		border: none;
 	}
 	#mh a:hover {
@@ -354,55 +232,6 @@ blink {
 	html.dark #toc li {
 		border-width: 0;
 	}
-	html.dark #mp a {
-		background: #057;
-	}
-	html.dark #mp h1 a, html.dark #mp h4 a,
-	html.dark #mp h2 a, html.dark #mp h5 a,
-	html.dark #mp h3 a, html.dark #mp h6 a {
-		color: inherit;
-		background: none;
-	}
-	html.dark pre,
-	html.dark code {
-		color: #8c0;
-		background: #1a1a1a;
-		border: .07em solid #333;
-	}
-	html.dark #mp ul,
-	html.dark #mp ol {
-		border-color: #444;
-	}
-	html.dark #m>ul,
-	html.dark #m>ol {
-		border-color: #555;
-	}
-	html.dark strong {
-		color: #fff;
-	}
-	html.dark p>em,
-	html.dark li>em,
-	html.dark td>em {
-		color: #f94;
-		border-color: #666;
-	}
-	html.dark h1 {
-		background: #383838;
-		border-top: .4em solid #b80;
-		border-bottom: .4em solid #4c4c4c;
-	}
-	html.dark h2 {
-		background: #444;
-		border-bottom: .22em solid #555;
-	}
-	html.dark td,
-	html.dark th {
-		border-color: #444;
-	}
-	html.dark blockquote {
-		background: #282828;
-		border: .07em dashed #444;
-	}
 	html.dark #mn a:not(:last-child)::after {
 		border-color: rgba(255,255,255,0.3);
 	}
@@ -508,7 +337,7 @@ blink {
 		mso-footer-margin: .6in;
 		mso-paper-source: 0;
 	}
-	a {
+	.mdo a {
 		color: #079;
 		text-decoration: none;
 		border-bottom: .07em solid #4ac;
@@ -541,18 +370,20 @@ blink {
 	a[ctr]::before {
 		content: attr(ctr) '. ';
 	}
-	h1 {
+	.mdo h1 {
 		margin: 2em 0;
 	}
-	h2 {
+	.mdo h2 {
 		margin: 2em 0 0 0;
 	}
-	h1, h2, h3 {
+	.mdo h1,
+	.mdo h2,
+	.mdo h3 {
 		page-break-inside: avoid;
 	}
-	h1::after,
-	h2::after,
-	h3::after {
+	.mdo h1::after,
+	.mdo h2::after,
+	.mdo h3::after {
 		content: 'orz';
 		color: transparent;
 		display: block;
@@ -560,20 +391,20 @@ blink {
 		padding: 4em 0 0 0;
 		margin: 0 0 -5em 0;
 	}
-	p {
+	.mdo p {
 		page-break-inside: avoid;
 	}
-	table {
+	.mdo table {
 		page-break-inside: auto;
 	}
-	tr {
+	.mdo tr {
 		page-break-inside: avoid;
 		page-break-after: auto;
 	}
-	thead {
+	.mdo thead {
 		display: table-header-group;
 	}
-	tfoot {
+	.mdo tfoot {
 		display: table-footer-group;
 	}
 	#mp a.vis::after {
@@ -581,31 +412,32 @@ blink {
 		border-bottom: 1px solid #bbb;
 		color: #444;
 	}
-	blockquote {
+	.mdo blockquote {
 		border-color: #555;
 	}
-	code {
+	.mdo code {
 		border-color: #bbb;
 	}
-	pre, pre code {
+	.mdo pre,
+	.mdo pre code {
 		border-color: #999;
 	}
-	pre code::before {
+	.mdo pre code::before {
 		color: #058;
 	}
 
 
 	
-	html.dark a {
+	html.dark .mdo a {
 		color: #000;
 	}
-	html.dark pre,
-	html.dark code {
+	html.dark .mdo pre,
+	html.dark .mdo code {
 		color: #240;
 	}
-	html.dark p>em,
-	html.dark li>em,
-	html.dark td>em {
+	html.dark .mdo p>em,
+	html.dark .mdo li>em,
+	html.dark .mdo td>em {
 		color: #940;
 	}
 }

copyparty/web/md.html

@@ -43,7 +43,7 @@
 				if you're still reading this, check that javascript is allowed
 			</div>
 		</div>
-		<div id="mp"></div>
+		<div id="mp" class="mdo"></div>
 	</div>
 	<a href="#" id="repl">π</a>
 	

copyparty/web/md.js

@@ -24,23 +24,6 @@ var dbg = function () { };
 var md_plug = {};
 
 
-function hesc(txt) {
-    return txt.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
-}
-
-
-function cls(dom, name, add) {
-    var re = new RegExp('(^| )' + name + '( |$)');
-    var lst = (dom.getAttribute('class') + '').replace(re, "$1$2").replace(/  /, "");
-    dom.setAttribute('class', lst + (add ? ' ' + name : ''));
-}
-
-
-function statify(obj) {
-    return JSON.parse(JSON.stringify(obj));
-}
-
-
 // dodge browser issues
 (function () {
     var ua = navigator.userAgent;
@@ -65,7 +48,7 @@ function statify(obj) {
         if (a > 0)
             loc.push(n[a]);
 
-        var dec = hesc(uricom_dec(n[a])[0]);
+        var dec = esc(uricom_dec(n[a])[0]);
 
         nav.push('<a href="/' + loc.join('/') + '">' + dec + '</a>');
     }
@@ -73,6 +56,26 @@ function statify(obj) {
 })();
 
 
+// image load handler
+var img_load = (function () {
+    var r = {};
+    r.callbacks = [];
+
+    function fire() {
+        for (var a = 0; a < r.callbacks.length; a++)
+            r.callbacks[a]();
+    }
+
+    var timeout = null;
+    r.done = function () {
+        clearTimeout(timeout);
+        timeout = setTimeout(fire, 500);
+    };
+
+    return r;
+})();
+
+
 // faster than replacing the entire html (chrome 1.8x, firefox 1.6x)
 function copydom(src, dst, lv) {
     var sc = src.childNodes,
@@ -356,6 +359,10 @@ function convert_markdown(md_text, dest_dom) {
 
     copydom(md_dom, dest_dom, 0);
 
+    var imgs = dest_dom.getElementsByTagName('img');
+    for (var a = 0, aa = imgs.length; a < aa; a++)
+        imgs[a].onload = img_load.done;
+
     if (ext && ext[0].render2)
         try {
             ext[0].render2(dest_dom);
@@ -490,6 +497,7 @@ function init_toc() {
 // "main" :p
 convert_markdown(dom_src.value, dom_pre);
 var toc = init_toc();
+img_load.callbacks = [toc.refresh];
 
 
 // scroll handler

copyparty/web/md2.css

@@ -50,7 +50,7 @@
 	outline: none;
 	padding: 0;
 	margin: 0;
-	font-family: 'consolas', monospace, monospace;
+	font-family: 'scp', monospace, monospace;
 	white-space: pre-wrap;
 	word-break: break-word;
 	overflow-wrap: break-word;

copyparty/web/md2.js

@@ -98,7 +98,7 @@ var draw_md = (function () {
         var src = dom_src.value;
         convert_markdown(src, dom_pre);
 
-        var lines = hesc(src).replace(/\r/g, "").split('\n');
+        var lines = esc(src).replace(/\r/g, "").split('\n');
         nlines = lines.length;
         var html = [];
         for (var a = 0; a < lines.length; a++)
@@ -108,7 +108,7 @@ var draw_md = (function () {
         map_src = genmap(dom_ref, map_src);
         map_pre = genmap(dom_pre, map_pre);
 
-        cls(ebi('save'), 'disabled', src == server_md);
+        clmod(ebi('save'), 'disabled', src == server_md);
 
         var t1 = Date.now();
         delay = t1 - t0 > 100 ? 25 : 1;
@@ -127,6 +127,12 @@ var draw_md = (function () {
 })();
 
 
+// discard TOC callback, just regen editor scroll map
+img_load.callbacks = [function () {
+    map_pre = genmap(dom_pre, map_pre);
+}];
+
+
 // resize handler
 redraw = (function () {
     function onresize() {
@@ -136,7 +142,6 @@ redraw = (function () {
         dom_ref.style.width = getComputedStyle(dom_src).offsetWidth + 'px';
         map_src = genmap(dom_ref, map_src);
         map_pre = genmap(dom_pre, map_pre);
-        dbg(document.body.clientWidth + 'x' + document.body.clientHeight);
     }
     function setsbs() {
         dom_wrap.setAttribute('class', '');
@@ -1086,9 +1091,9 @@ action_stack = (function () {
         ref = newtxt;
         dbg('undos(%d) redos(%d)', hist.un.length, hist.re.length);
         if (hist.un.length > 0)
-            dbg(statify(hist.un.slice(-1)[0]));
+            dbg(jcp(hist.un.slice(-1)[0]));
         if (hist.re.length > 0)
-            dbg(statify(hist.re.slice(-1)[0]));
+            dbg(jcp(hist.re.slice(-1)[0]));
     }
 
     return {

copyparty/web/mde.css

@@ -7,6 +7,8 @@ html .editor-toolbar>button.active { border-color: rgba(0,0,0,0.4); background:
 html .editor-toolbar>i.separator { border-left: 1px solid #ccc; }
 html .editor-toolbar.disabled-for-preview>button:not(.no-disable) { opacity: .35 }
 
+
+
 html {
 	line-height: 1.5em;
 }
@@ -31,6 +33,9 @@ html, body {
 	background: none;
 	text-decoration: none;
 }
+
+
+
 #mn {
 	font-weight: normal;
 	margin: 1.3em 0 .7em 1em;
@@ -72,148 +77,12 @@ html .editor-toolbar>button.disabled {
 html .editor-toolbar>button.save.force-save {
 	background: #f97;
 }
-
-
-
-
-
-/* copied from md.css for now */
-.mdo pre,
-.mdo code,
-.mdo a {
-	color: #480;
-	background: #f7f7f7;
-	border: .07em solid #ddd;
-	border-radius: .2em;
-	padding: .1em .3em;
-	margin: 0 .1em;
-}
-.mdo code {
-	font-size: .96em;
-}
-.mdo pre,
-.mdo code {
-	font-family: monospace, monospace;
-	white-space: pre-wrap;
-	word-break: break-all;
-}
-.mdo pre code {
-	display: block;
-	margin: 0 -.3em;
-	padding: .4em .5em;
-	line-height: 1.1em;
-}
-.mdo a {
-	color: #fff;
-	background: #39b;
-	text-decoration: none;
-	padding: 0 .3em;
-	border: none;
-	border-bottom: .07em solid #079;
-}
-.mdo h2 {
-	color: #fff;
-	background: #555;
-	margin-top: 2em;
-	border-bottom: .22em solid #999;
-	border-top: none;
-}
-.mdo h1 {
-	color: #fff;
-	background: #444;
-	font-weight: normal;
-	border-top: .4em solid #fb0;
-	border-bottom: .4em solid #777;
-	border-radius: 0 1em 0 1em;
-	margin: 3em 0 1em 0;
-	padding: .5em 0;
-}
-h1, h2 {
-	line-height: 1.5em;
-}
-h1 {
-	font-size: 1.7em;
-	text-align: center;
-	border: 1em solid #777;
-	border-width: .05em 0;
-	margin: 3em 0;
-}
-h2 {
-	font-size: 1.5em;
-	font-weight: normal;
-	background: #f7f7f7;
-	border-top: .07em solid #fff;
-	border-bottom: .07em solid #bbb;
-	border-radius: .5em .5em 0 0;
-	padding-left: .4em;
-	margin-top: 3em;
-}
-.mdo ul,
-.mdo ol {
-	border-left: .3em solid #ddd;
-}
-.mdo>ul,
-.mdo>ol {
-	border-color: #bbb;
-}
-.mdo ul>li {
-	list-style-type: disc;
-}
-.mdo ul>li,
-.mdo ol>li {
-	margin: .7em 0;
-}
-strong {
-	color: #000;
-}
-p>em,
-li>em,
-td>em {
-	color: #c50;
-	padding: .1em;
-	border-bottom: .1em solid #bbb;
-}
-blockquote {
-	font-family: serif;
-	background: #f7f7f7;
-	border: .07em dashed #ccc;
-	padding: 0 2em;
-	margin: 1em 0;
-}
-small {
-	opacity: .8;
-}
-table {
-	border-collapse: collapse;
-}
-td {
-	padding: .2em .5em;
-	border: .12em solid #aaa;
-}
-th {
-	border: .12em solid #aaa;
-}
-
-
-
-
-
-/* mde support */
-.mdo {
-	padding: 1em;
-	background: #f7f7f7;
-}
-html.dark .mdo {
-	background: #1c1c1c;
-}
 .CodeMirror {
 	background: #f7f7f7;
 }
 
 
 
-
-
 /* darkmode */
 html.dark .mdo,
 html.dark .CodeMirror {
@@ -237,55 +106,6 @@ html.dark .CodeMirror-selectedtext {
 	background: #246;
 	color: #fff;
 }
-html.dark .mdo a {
-	background: #057;
-}
-html.dark .mdo h1 a, html.dark .mdo h4 a,
-html.dark .mdo h2 a, html.dark .mdo h5 a,
-html.dark .mdo h3 a, html.dark .mdo h6 a {
-	color: inherit;
-	background: none;
-}
-html.dark pre,
-html.dark code {
-	color: #8c0;
-	background: #1a1a1a;
-	border: .07em solid #333;
-}
-html.dark .mdo ul,
-html.dark .mdo ol {
-	border-color: #444;
-}
-html.dark .mdo>ul,
-html.dark .mdo>ol {
-	border-color: #555;
-}
-html.dark strong {
-	color: #fff;
-}
-html.dark p>em,
-html.dark li>em,
-html.dark td>em {
-	color: #f94;
-	border-color: #666;
-}
-html.dark h1 {
-	background: #383838;
-	border-top: .4em solid #b80;
-	border-bottom: .4em solid #4c4c4c;
-}
-html.dark h2 {
-	background: #444;
-	border-bottom: .22em solid #555;
-}
-html.dark td,
-html.dark th {
-	border-color: #444;
-}
-html.dark blockquote {
-	background: #282828;
-	border: .07em dashed #444;
-}
 
 
 
@@ -321,4 +141,15 @@ html.dark .editor-toolbar>button.active {
 html.dark .editor-toolbar::after,
 html.dark .editor-toolbar::before {
 	background: none;
-}
+}
+
+
+
+/* ui.css overrides */
+.mdo {
+	padding: 1em;
+	background: #f7f7f7;
+}
+html.dark .mdo {
+	background: #1c1c1c;
+}

copyparty/web/ui.css

@@ -1,3 +1,7 @@
+@font-face {
+	font-family: 'scp';
+	src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(/.cpr/deps/scp.woff2) format('woff2');
+}
 html {
 	touch-action: manipulation;
 }
@@ -241,4 +245,198 @@ html.light #tt em {
 }
 #repl_pre {
 	max-width: 24em;
-}
+}
+
+
+
+
+
+
+
+
+
+
+
+.mdo pre,
+.mdo code,
+.mdo a {
+	color: #480;
+	background: #f7f7f7;
+	border: .07em solid #ddd;
+	border-radius: .2em;
+	padding: .1em .3em;
+	margin: 0 .1em;
+}
+.mdo pre,
+.mdo code,
+.mdo tt {
+	font-family: 'scp', monospace, monospace;
+	white-space: pre-wrap;
+	word-break: break-all;
+}
+.mdo code {
+	font-size: .96em;
+}
+.mdo h1,
+.mdo h2 {
+	line-height: 1.5em;
+}
+.mdo h1 {
+	font-size: 1.7em;
+	text-align: center;
+	border: 1em solid #777;
+	border-width: .05em 0;
+	margin: 3em 0;
+}
+.mdo h2 {
+	font-size: 1.5em;
+	font-weight: normal;
+	background: #f7f7f7;
+	border-top: .07em solid #fff;
+	border-bottom: .07em solid #bbb;
+	border-radius: .5em .5em 0 0;
+	padding-left: .4em;
+	margin-top: 3em;
+}
+.mdo h3 {
+	border-bottom: .1em solid #999;
+}
+.mdo h1 a, .mdo h3 a, .mdo h5 a,
+.mdo h2 a, .mdo h4 a, .mdo h6 a {
+	color: inherit;
+	display: block;
+	background: none;
+	border: none;
+	padding: 0;
+	margin: 0;
+}
+.mdo ul,
+.mdo ol {
+	border-left: .3em solid #ddd;
+}
+.mdo ul>li,
+.mdo ol>li {
+	margin: .7em 0;
+	list-style-type: disc;
+}
+.mdo strong {
+	color: #000;
+}
+.mdo p>em,
+.mdo li>em,
+.mdo td>em {
+	color: #c50;
+	padding: .1em;
+	border-bottom: .1em solid #bbb;
+}
+.mdo blockquote {
+	font-family: serif;
+	background: #f7f7f7;
+	border: .07em dashed #ccc;
+	padding: 0 2em;
+	margin: 1em 0;
+}
+.mdo small {
+	opacity: .8;
+}
+.mdo pre code {
+	display: block;
+	margin: 0 -.3em;
+	padding: .4em .5em;
+	line-height: 1.1em;
+}
+.mdo pre code:hover {
+	background: #fec;
+	color: #360;
+}
+.mdo table {
+	border-collapse: collapse;
+	margin: 1em 0;
+}
+.mdo th,
+.mdo td {
+	padding: .2em .5em;
+	border: .12em solid #aaa;
+}
+
+@media screen {
+	.mdo {
+		word-break: break-word;
+		overflow-wrap: break-word;
+		word-wrap: break-word; /*ie*/
+	}
+	html.light .mdo a,
+	.mdo a {
+		color: #fff;
+		background: #39b;
+		text-decoration: none;
+		padding: 0 .3em;
+		border: none;
+		border-bottom: .07em solid #079;
+	}
+	.mdo h1 {
+		color: #fff;
+		background: #444;
+		font-weight: normal;
+		border-top: .4em solid #fb0;
+		border-bottom: .4em solid #777;
+		border-radius: 0 1em 0 1em;
+		margin: 3em 0 1em 0;
+		padding: .5em 0;
+	}
+	.mdo h2 {
+		color: #fff;
+		background: #555;
+		margin-top: 2em;
+		border-bottom: .22em solid #999;
+		border-top: none;
+	}
+
+
+
+	html.dark .mdo a {
+		background: #057;
+	}
+	html.dark .mdo h1 a, html.dark .mdo h4 a,
+	html.dark .mdo h2 a, html.dark .mdo h5 a,
+	html.dark .mdo h3 a, html.dark .mdo h6 a {
+		color: inherit;
+		background: none;
+	}
+	html.dark .mdo pre,
+	html.dark .mdo code {
+		color: #8c0;
+		background: #1a1a1a;
+		border: .07em solid #333;
+	}
+	html.dark .mdo ul,
+	html.dark .mdo ol {
+		border-color: #444;
+	}
+	html.dark .mdo strong {
+		color: #fff;
+	}
+	html.dark .mdo p>em,
+	html.dark .mdo li>em,
+	html.dark .mdo td>em {
+		color: #f94;
+		border-color: #666;
+	}
+	html.dark .mdo h1 {
+		background: #383838;
+		border-top: .4em solid #b80;
+		border-bottom: .4em solid #4c4c4c;
+	}
+	html.dark .mdo h2 {
+		background: #444;
+		border-bottom: .22em solid #555;
+	}
+	html.dark .mdo td,
+	html.dark .mdo th {
+		border-color: #444;
+	}
+	html.dark .mdo blockquote {
+		background: #282828;
+		border: .07em dashed #444;
+	}
+}

copyparty/web/up2k.js

@@ -904,7 +904,7 @@ function up2k_init(subtle) {
     }
     ebi('u2cleanup').onclick = u2cleanup;
 
-    var etaref = 0, etaskip = 0;
+    var etaref = 0, etaskip = 0, op_minh = 0;
     function etafun() {
         var nhash = st.busy.head.length + st.busy.hash.length + st.todo.head.length + st.todo.hash.length,
             nsend = st.busy.upload.length + st.todo.upload.length,
@@ -912,8 +912,20 @@ function up2k_init(subtle) {
             td = (now - (etaref || now)) / 1000.0;
 
         etaref = now;
+        if (td > 1.2)
+            td = 0.05;
+
         //ebi('acc_info').innerHTML = humantime(st.time.busy) + ' ' + f2f(now / 1000, 1);
 
+        var op = ebi('op_up2k'),
+            uff = ebi('u2footfoot'),
+            minh = QS('#op_up2k.act') ? Math.max(op_minh, uff.offsetTop + uff.offsetHeight - op.offsetTop + 32) : 0;
+
+        if (minh > op_minh || !op_minh) {
+            op_minh = minh;
+            op.style.minHeight = op_minh + 'px';
+        }
+
         if (!nhash)
             ebi('u2etah').innerHTML = 'Done ({0}, {1} files)'.format(humansize(st.bytes.hashed), pvis.ctr["ok"] + pvis.ctr["ng"]);
 
@@ -1075,6 +1087,7 @@ function up2k_init(subtle) {
                             toast.err(t, '{0} {1}'.format(ks, tng));
 
                         timer.rm(etafun);
+                        op_minh = 0;
                     }
                     else {
                         timer.add(etafun, false);
@@ -1370,8 +1383,11 @@ function up2k_init(subtle) {
                 }
             }
             apop(st.busy.head, t);
-            if (!ok && !t.srch)
-                return push_t(st.todo.hash, t);
+            if (!ok && !t.srch) {
+                push_t(st.todo.hash, t);
+                tasker();
+                return;
+            }
 
             t.done = true;
             st.bytes.hashed += t.size;
@@ -1380,6 +1396,7 @@ function up2k_init(subtle) {
             pvis.seth(t.n, 1, ok ? 'YOLO' : '404');
             pvis.seth(t.n, 2, "turbo'd");
             pvis.move(t.n, ok ? 'ok' : 'ng');
+            tasker();
         };
         xhr.onload = function (e) {
             try { orz(e); } catch (ex) { vis_exh(ex + '', '', '', '', ex); }
@@ -1900,7 +1917,7 @@ function up2k_init(subtle) {
     if (parallel_uploads < 1)
         bumpthread(1);
 
-    return { "init_deps": init_deps, "set_fsearch": set_fsearch }
+    return { "init_deps": init_deps, "set_fsearch": set_fsearch, "ui": pvis }
 }
 
 

copyparty/web/util.js

@@ -842,14 +842,14 @@ var toast = (function () {
         r.visible = false;
     };
 
-    r.show = function (cl, ms, txt) {
+    r.show = function (cl, sec, txt) {
         clearTimeout(te);
-        if (ms)
-            te = setTimeout(r.hide, ms * 1000);
+        if (sec)
+            te = setTimeout(r.hide, sec * 1000);
 
         obj.innerHTML = '<a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
         obj.className = cl;
-        ms += obj.offsetWidth;
+        sec += obj.offsetWidth;
         obj.className += ' vis';
         ebi('toastc').onclick = r.hide;
         timer.add(scrollchk);
@@ -857,17 +857,17 @@ var toast = (function () {
         r.txt = txt;
     };
 
-    r.ok = function (ms, txt) {
-        r.show('ok', ms, txt);
+    r.ok = function (sec, txt) {
+        r.show('ok', sec, txt);
     };
-    r.inf = function (ms, txt) {
-        r.show('inf', ms, txt);
+    r.inf = function (sec, txt) {
+        r.show('inf', sec, txt);
     };
-    r.warn = function (ms, txt) {
-        r.show('warn', ms, txt);
+    r.warn = function (sec, txt) {
+        r.show('warn', sec, txt);
     };
-    r.err = function (ms, txt) {
-        r.show('err', ms, txt);
+    r.err = function (sec, txt) {
+        r.show('err', sec, txt);
     };
 
     return r;

docs/notes.sh

@@ -164,7 +164,7 @@ brew install python@2
 pip install virtualenv
 
 # readme toc
-cat README.md | awk 'function pr() { if (!h) {return}; if (/^ *[*!#]/||!s) {printf "%s\n",h;h=0;return}; if (/.../) {printf "%s - %s\n",h,$0;h=0}; };    /^#/{s=1;pr()} /^#* *(file indexing|install on android|dev env setup|just the sfx|complete release|optional gpl stuff)|`$/{s=0} /^#/{lv=length($1);sub(/[^ ]+ /,"");bab=$0;gsub(/ /,"-",bab); h=sprintf("%" ((lv-1)*4+1) "s [%s](#%s)", "*",$0,bab);next} !h{next} {sub(/:$/,"")} {pr()}' > toc; grep -E '^## readme toc' -B1000 -A2 <README.md >p1; grep -E '^## quickstart' -B2 -A999999 <README.md >p2; (cat p1; grep quickstart -A1000 <toc; cat p2) >README.md
+cat README.md | awk 'function pr() { if (!h) {return}; if (/^ *[*!#]/||!s) {printf "%s\n",h;h=0;return}; if (/.../) {printf "%s - %s\n",h,$0;h=0}; };    /^#/{s=1;pr()} /^#* *(file indexing|install on android|dev env setup|just the sfx|complete release|optional gpl stuff)|`$/{s=0} /^#/{lv=length($1);sub(/[^ ]+ /,"");bab=$0;gsub(/ /,"-",bab); h=sprintf("%" ((lv-1)*4+1) "s [%s](#%s)", "*",$0,bab);next} !h{next} {sub(/  .*/,"");sub(/[:,]$/,"")} {pr()}' > toc; grep -E '^## readme toc' -B1000 -A2 <README.md >p1; grep -E '^## quickstart' -B2 -A999999 <README.md >p2; (cat p1; grep quickstart -A1000 <toc; cat p2) >README.md
 
 # fix firefox phantom breakpoints,
 # suggestions from bugtracker, doesnt work (debugger is not attachable)

scripts/make-sfx.sh

@@ -16,14 +16,14 @@ help() { exec cat <<'EOF'
 #
 # `no-sh` makes just the python sfx, skips the sh/unix sfx
 #
-# `no-ogv` saves ~500k by removing the opus/vorbis audio codecs
+# `no-ogv` saves ~192k by removing the opus/vorbis audio codecs
 #   (only affects apple devices; everything else has native support)
 #
-# `no-cm` saves ~90k by removing easymde/codemirror
+# `no-cm` saves ~92k by removing easymde/codemirror
 #   (the fancy markdown editor)
 #
 # `no-fnt` saves ~9k by removing the source-code-pro font
-#   (mainly used my the markdown viewer/editor)
+#   (browsers will try to use 'Consolas' instead)
 #
 # `no-dd` saves ~2k by removing the mouse cursor
 
@@ -178,7 +178,7 @@ git describe --tags >/dev/null 2>/dev/null && {
 
 [ -z "$ver" ] && 
 	ver="$(awk '/^VERSION *= \(/ {
-		gsub(/[^0-9,]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
+		gsub(/[^0-9,a-g-]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
 
 ts=$(date -u +%s)
 hts=$(date -u +%Y-%m%d-%H%M%S) # --date=@$ts (thx osx)
@@ -217,16 +217,16 @@ done
 
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
-	f=copyparty/web/md.css
-	gzip -d "$f"
-	sed -r '/scp\.woff2/d' <$f >t
+	f=copyparty/web/ui.css
+	gzip -d "$f.gz" || true
+	sed -r "s/src:.*scp.*\)/src:local('Consolas')/" <$f >t
 	tmv "$f"
 }
 
 [ $no_dd ] && {
 	rm -rf copyparty/web/dd
 	f=copyparty/web/browser.css
-	gzip -d "$f"
+	gzip -d "$f.gz" || true
 	sed -r 's/(cursor: ?)url\([^)]+\), ?(pointer)/\1\2/; /[0-9]+% \{cursor:/d; /animation: ?cursor/d' <$f >t
 	tmv "$f"
 }
@@ -241,6 +241,12 @@ f=dep-j2/jinja2/constants.py
 awk '/^LOREM_IPSUM_WORDS/{o=1;print "LOREM_IPSUM_WORDS = u\"a\"";next} !o; /"""/{o=0}' <$f >t
 tmv "$f"
 
+grep -rLE '^#[^a-z]*coding: utf-8' dep-j2 |
+while IFS= read -r f; do
+	(echo "# coding: utf-8"; cat "$f") >t
+	tmv "$f"
+done
+
 # up2k goes from 28k to 22k laff
 awk 'BEGIN{gensub(//,"",1)}' </dev/null &&
 echo entabbening &&