v1.1.1

README.md

@@ -46,7 +46,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
     * [markdown viewer](#markdown-viewer) - and there are *two* editors
     * [other tricks](#other-tricks)
     * [searching](#searching) - search by size, date, path/name, mp3-tags, ...
-* [server config](#server-config)
+* [server config](#server-config) - using arguments or config files, or a mix of both
     * [file indexing](#file-indexing)
     * [upload rules](#upload-rules) - set upload rules using volume flags
     * [compress uploads](#compress-uploads) - files can be autocompressed on upload
@@ -60,11 +60,17 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 * [up2k](#up2k) - quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
     * [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
 * [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
+    * [client-side](#client-side) - when uploading files
 * [security](#security) - some notes on hardening
     * [gotchas](#gotchas) - behavior that might be unexpected
 * [recovering from crashes](#recovering-from-crashes)
     * [client crashes](#client-crashes)
         * [frefox wsod](#frefox-wsod) - firefox 87 can crash during uploads
+* [HTTP API](#HTTP-API)
+    * [read](#read)
+    * [write](#write)
+    * [admin](#admin)
+    * [general](#general)
 * [dependencies](#dependencies) - mandatory deps
     * [optional dependencies](#optional-dependencies) - install these to enable bonus features
     * [install recommended deps](#install-recommended-deps)
@@ -153,7 +159,7 @@ feature summary
 * browser
   * ☑ [navpane](#navpane) (directory tree sidebar)
   * ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
-  * ☑ audio player (with OS media controls)
+  * ☑ audio player (with OS media controls and opus transcoding)
   * ☑ image gallery with webm player
   * ☑ [thumbnails](#thumbnails)
     * ☑ ...of images using Pillow
@@ -218,6 +224,7 @@ some improvement ideas
 * Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
 * Windows: python 2.7 cannot handle filenames with mojibake
 * `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
+* `--th-ff-swr` may fix audio thumbnails on some FFmpeg versions
 
 ## general bugs
 
@@ -304,6 +311,7 @@ the browser has the following hotkeys  (always qwerty)
 * `B` toggle breadcrumbs / [navpane](#navpane)
 * `I/K` prev/next folder
 * `M` parent folder (or unexpand current)
+* `V` toggle folders / textfiles in the navpane
 * `G` toggle list / [grid view](#thumbnails)
 * `T` toggle thumbnails / icons
 * `ctrl-X` cut selected files/folders
@@ -315,6 +323,10 @@ the browser has the following hotkeys  (always qwerty)
   * ctrl+`Up/Down` move cursor and scroll viewport
   * `Space` toggle file selection
   * `Ctrl-A` toggle select all
+* when a textfile is open:
+  * `I/K` prev/next textfile
+  * `S` toggle selection of open file
+  * `M` close textfile
 * when playing audio:
   * `J/L` prev/next song
   * `U/O` skip 10sec back/forward
@@ -366,6 +378,8 @@ press `g` to toggle grid-view instead of the file listing,  and `t` toggles icon
 
 it does static images with Pillow and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
 
+audio files are covnerted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
+
 images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
 
 in the grid/thumbnail view, if the audio player panel is open, songs will start playing when clicked
@@ -421,6 +435,8 @@ see [up2k](#up2k) for details on how it works
 
 **protip:** you can avoid scaring away users with [docs/minimal-up2k.html](docs/minimal-up2k.html) which makes it look [much simpler](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
 
+**protip:** if you enable `favicon` in the `[⚙️] settings` tab (by typing something into the textbox), the icon in the browser tab will indicate upload progress
+
 the up2k UI is the epitome of polished inutitive experiences:
 * "parallel uploads" specifies how many chunks to upload at the same time
 * `[🏃]` analysis of other files should continue while one is uploading
@@ -576,6 +592,10 @@ add the argument `-e2ts` to also scan/index tags from music files, which brings
 
 # server config
 
+using arguments or config files, or a mix of both:
+* config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf)
+
+
 ## file indexing
 
 file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volume flags, or a mix of both.
@@ -860,6 +880,21 @@ below are some tweaks roughly ordered by usefulness:
   ...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
 
 
+## client-side
+
+when uploading files,
+
+* chrome is recommended, at least compared to firefox:
+  * up to 90% faster when hashing, especially on SSDs
+  * up to 40% faster when uploading over extremely fast internets
+  * but [up2k.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) can be 40% faster than chrome again
+
+* if you're cpu-bottlenecked, or the browser is maxing a cpu core:
+  * up to 30% faster uploads if you hide the upload status list by switching away from the `[🚀]` up2k ui-tab (or closing it)
+    * switching to another browser-tab also works, the favicon will update every 10 seconds in that case
+  * unlikely to be a problem, but can happen when uploding many small files, or your internet is too fast, or PC too slow
+
+
 # security
 
 some notes on hardening
@@ -903,6 +938,83 @@ however you can hit `F12` in the up2k tab and use the devtools to see how far yo
   `await fetch('/inc', {method:'PUT', body:JSON.stringify(ng,null,1)})`
 
 
+# HTTP API
+
+* table-column `params` = URL parameters; `?foo=bar&qux=...`
+* table-column `body` = POST payload
+* method `jPOST` = json post
+* method `mPOST` = multipart post
+* method `uPOST` = url-encoded post
+* `FILE` = conventional HTTP file upload entry (rfc1867 et al, filename in `Content-Disposition`)
+
+authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
+
+## read
+
+| method | params | result |
+|--|--|--|
+| GET | `?ls` | list files/folders at URL as JSON |
+| GET | `?ls&dots` | list files/folders at URL as JSON, including dotfiles |
+| GET | `?ls=t` | list files/folders at URL as plaintext |
+| GET | `?ls=v` | list files/folders at URL, terminal-formatted |
+| GET | `?b` | list files/folders at URL as simplified HTML |
+| GET | `?tree=.` | list one level of subdirectories inside URL |
+| GET | `?tree` | list one level of subdirectories for each level until URL |
+| GET | `?tar` | download everything below URL as a tar file |
+| GET | `?zip=utf-8` | download everything below URL as a zip file |
+| GET | `?ups` | show recent uploads from your IP |
+| GET | `?ups&filter=f` | ...where URL contains `f` |
+| GET | `?mime=foo` | specify return mimetype `foo` |
+| GET | `?raw` | get markdown file at URL as plaintext |
+| GET | `?txt` | get file at URL as plaintext |
+| GET | `?txt=iso-8859-1` | ...with specific charset |
+| GET | `?th` | get image/video at URL as thumbnail |
+| GET | `?th=opus` | convert audio file to 128kbps opus |
+
+| method | body | result |
+|--|--|--|
+| jPOST | `{"q":"foo"}` | do a server-wide search; see the `[🔎]` search tab `raw` field for syntax |
+
+| method | params | body | result |
+|--|--|--|--|
+| jPOST | `?tar` | `["foo","bar"]` | download folders `foo` and `bar` inside URL as a tar file |
+
+## write
+
+| method | params | result |
+|--|--|--|
+| GET | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
+
+| method | params | body | result |
+|--|--|--|--|
+| PUT | | (binary data) | upload into file at URL |
+| PUT | `?gz` | (binary data) | compress with gzip and write into file at URL |
+| PUT | `?xz` | (binary data) | compress with xz and write into file at URL |
+| mPOST | | `act=bput`, `f=FILE` | upload `FILE` into the folder at URL |
+| mPOST | `?j` | `act=bput`, `f=FILE` | ...and reply with json |
+| mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
+| GET | `?delete` | | delete URL recursively |
+| jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
+| uPOST | | `msg=foo` | send message `foo` into server log |
+| mPOST | | `act=tput`, `body=TEXT` | overwrite markdown document at URL |
+
+server behavior of `msg` can be reconfigured with `--urlform`
+
+## admin
+
+| method | params | result |
+|--|--|--|
+| GET | `?reload=cfg` | reload config files and rescan volumes |
+| GET | `?scan` | initiate a rescan of the volume which provides URL |
+| GET | `?stack` | show a stacktrace of all threads |
+
+## general
+
+| method | params | result |
+|--|--|--|
+| GET | `?pw=x` | logout |
+
+
 # dependencies
 
 mandatory deps:
@@ -919,7 +1031,7 @@ enable music tags:
 
 enable [thumbnails](#thumbnails) of...
 * **images:** `Pillow` (requires py2.7 or py3.5+)
-* **videos:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
+* **videos/audio:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
 * **HEIF pictures:** `pyheif-pillow-opener` (requires Linux or a C compiler)
 * **AVIF pictures:** `pillow-avif-plugin`
 
@@ -953,13 +1065,15 @@ pls note that `copyparty-sfx.sh` will fail if you rename `copyparty-sfx.py` to `
 reduce the size of an sfx by removing features
 
 if you don't need all the features, you can repack the sfx and save a bunch of space; all you need is an sfx and a copy of this repo (nothing else to download or build, except if you're on windows then you need msys2 or WSL)
-* `525k` size of original sfx.py as of v0.11.30
-* `315k` after `./scripts/make-sfx.sh re no-ogv`
-* `223k` after `./scripts/make-sfx.sh re no-ogv no-cm`
+* `584k` size of original sfx.py as of v1.1.0
+* `392k` after `./scripts/make-sfx.sh re no-ogv`
+* `310k` after `./scripts/make-sfx.sh re no-ogv no-cm`
+* `269k` after `./scripts/make-sfx.sh re no-ogv no-cm no-hl`
 
 the features you can opt to drop are
 * `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files, saves ~192k
-* `cm`/easymde, the "fancy" markdown editor, saves ~92k
+* `cm`/easymde, the "fancy" markdown editor, saves ~82k
+* `hl`, prism, the syntax hilighter, saves ~41k
 * `fnt`, source-code-pro, the monospace font, saves ~9k
 * `dd`, the custom mouse cursor for the media player tray tab, saves ~2k
 

bin/up2k.py

@@ -3,7 +3,7 @@ from __future__ import print_function, unicode_literals
 
 """
 up2k.py: upload to copyparty
-2021-10-29, v0.10, ed <irc.rizon.net>, MIT-Licensed
+2021-10-31, v0.11, ed <irc.rizon.net>, MIT-Licensed
 https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py
 
 - dependencies: requests
@@ -455,6 +455,7 @@ class Ctl(object):
         self.nbytes = nbytes
 
         if ar.td:
+            requests.packages.urllib3.disable_warnings()
             req_ses.verify = False
         if ar.te:
             req_ses.verify = ar.te

contrib/README.md

@@ -30,6 +30,7 @@ however if your copyparty is behind a reverse-proxy, you may want to use [`share
 # OS integration
 init-scripts to start copyparty as a service
 * [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
+* [`rc/copyparty`](rc/copyparty) runs sfx normally on freebsd, create a `copyparty` user
 * [`systemd/prisonparty.service`](systemd/prisonparty.service) runs the sfx in a chroot
 * [`openrc/copyparty`](openrc/copyparty)
 

contrib/rc/copyparty

@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# PROVIDE: copyparty
+# REQUIRE: networking
+# KEYWORD:
+
+. /etc/rc.subr
+
+name="copyparty"
+rcvar="copyparty_enable"
+copyparty_user="copyparty"
+copyparty_args="-e2dsa -v /storage:/storage:r" # change as you see fit
+copyparty_command="/usr/local/bin/python3.8 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
+pidfile="/var/run/copyparty/${name}.pid"
+command="/usr/sbin/daemon"
+command_args="-P ${pidfile} -r -f ${copyparty_command}"
+
+stop_postcmd="copyparty_shutdown"
+
+copyparty_shutdown()
+{
+        if [ -e "${pidfile}" ]; then
+                echo "Stopping supervising daemon."
+                kill -s TERM `cat ${pidfile}`
+        fi
+}
+
+load_rc_config $name
+: ${copyparty_enable:=no}
+
+run_rc_command "$1"

contrib/systemd/copyparty.service

@@ -20,11 +20,8 @@
 #   python disabling line-buffering, so messages are out-of-order:
 #   https://user-images.githubusercontent.com/241032/126040249-cb535cc7-c599-4931-a796-a5d9af691bad.png
 #
-# enable line-buffering for realtime logging (slight performance cost):
-#   modify ExecStart and prefix it with `/usr/bin/stdbuf -oL` like so:
-#   ExecStart=/usr/bin/stdbuf -oL /usr/bin/python3 [...]
-# but some systemd versions require this instead (higher performance cost):
-#   inside the [Service] block, add the following line:
+# if you remove -q to enable logging, you may also want to remove the
+#   following line to enable buffering (slightly better performance):
 #   Environment=PYTHONUNBUFFERED=x
 
 [Unit]
@@ -33,8 +30,10 @@ Description=copyparty file server
 [Service]
 Type=notify
 SyslogIdentifier=copyparty
-ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -p 80,443,3923 -v /mnt::rw
+Environment=PYTHONUNBUFFERED=x
+ExecReload=/bin/kill -s USR1 $MAINPID
 ExecStartPre=/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
+ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -p 80,443,3923 -v /mnt::rw
 
 [Install]
 WantedBy=multi-user.target

copyparty/__main__.py

@@ -186,6 +186,32 @@ def configure_ssl_ciphers(al):
         sys.exit(0)
 
 
+def args_from_cfg(cfg_path):
+    ret = []
+    skip = False
+    with open(cfg_path, "rb") as f:
+        for ln in [x.decode("utf-8").strip() for x in f]:
+            if not ln:
+                skip = False
+                continue
+
+            if ln.startswith("#"):
+                continue
+
+            if not ln.startswith("-"):
+                continue
+
+            if skip:
+                continue
+
+            try:
+                ret.extend(ln.split(" ", 1))
+            except:
+                ret.append(ln)
+
+    return ret
+
+
 def sighandler(sig=None, frame=None):
     msg = [""] * 5
     for th in threading.enumerate():
@@ -397,11 +423,13 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|\?th=[wj]$", help="dont log URLs matching")
 
     ap2 = ap.add_argument_group('admin panel options')
+    ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
     ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
     ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
 
     ap2 = ap.add_argument_group('thumbnail options')
     ap2.add_argument("--no-thumb", action="store_true", help="disable all thumbnails")
+    ap2.add_argument("--no-athumb", action="store_true", help="disable audio thumbnails (spectrograms)")
     ap2.add_argument("--no-vthumb", action="store_true", help="disable video thumbnails")
     ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res")
     ap2.add_argument("--th-mt", metavar="CORES", type=int, default=cores, help="num cpu cores to use for generating thumbnails")
@@ -409,11 +437,16 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
     ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
     ap2.add_argument("--th-ff-jpg", action="store_true", help="force jpg for video thumbs")
+    ap2.add_argument("--th-ff-swr", action="store_true", help="use swresample instead of soxr for audio thumbs")
     ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown")
     ap2.add_argument("--th-clean", metavar="SEC", type=int, default=43200, help="cleanup interval; 0=disabled")
     ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age")
     ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat for")
 
+    ap2 = ap.add_argument_group('transcoding options')
+    ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
+    ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete transcode output after SEC seconds")
+
     ap2 = ap.add_argument_group('general db options')
     ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
     ap2.add_argument("-e2ds", action="store_true", help="enable up2k db-scanner, sets -e2d")
@@ -441,6 +474,7 @@ def run_argparse(argv, formatter):
     ap2 = ap.add_argument_group('ui options')
     ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
     ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
+    ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
 
     ap2 = ap.add_argument_group('debug options')
     ap2.add_argument("--no-sendfile", action="store_true", help="disable sendfile")
@@ -483,7 +517,12 @@ def main(argv=None):
     if HAVE_SSL:
         ensure_cert()
 
-    deprecated = [["-e2s", "-e2ds"]]
+    for k, v in zip(argv, argv[1:]):
+        if k == "-c":
+            supp = args_from_cfg(v)
+            argv.extend(supp)
+
+    deprecated = []
     for dk, nk in deprecated:
         try:
             idx = argv.index(dk)

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 0, 14)
-CODENAME = "sufficient"
-BUILD_DT = (2021, 10, 30)
+VERSION = (1, 1, 1)
+CODENAME = "opus"
+BUILD_DT = (2021, 11, 8)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -546,6 +546,7 @@ class AuthSrv(object):
 
     def _parse_config_file(self, fd, acct, daxs, mflags, mount):
         # type: (any, str, dict[str, AXS], any, str) -> None
+        skip = False
         vol_src = None
         vol_dst = None
         self.line_ctr = 0
@@ -555,6 +556,11 @@ class AuthSrv(object):
                 vol_src = None
                 vol_dst = None
 
+            if skip:
+                if not ln:
+                    skip = False
+                continue
+
             if not ln or ln.startswith("#"):
                 continue
 
@@ -562,6 +568,8 @@ class AuthSrv(object):
                 if ln.startswith("u "):
                     u, p = ln[2:].split(":", 1)
                     acct[u] = p
+                elif ln.startswith("-"):
+                    skip = True  # argv
                 else:
                     vol_src = ln
                 continue
@@ -613,7 +621,7 @@ class AuthSrv(object):
         if uname == "":
             uname = "*"
 
-        for un in uname.split(","):
+        for un in uname.replace(",", " ").strip().split():
             if "r" in lvl:
                 axs.uread[un] = 1
 

copyparty/broker_mp.py

@@ -62,6 +62,11 @@ class BrokerMp(object):
 
             procs.pop()
 
+    def reload(self):
+        self.log("broker", "reloading")
+        for _, proc in enumerate(self.procs):
+            proc.q_pend.put([0, "reload", []])
+
     def collector(self, proc):
         """receive message from hub in other process"""
         while True:

copyparty/broker_mpw.py

@@ -29,7 +29,7 @@ class MpWorker(object):
         # we inherited signal_handler from parent,
         # replace it with something harmless
         if not FAKE_MP:
-            for sig in [signal.SIGINT, signal.SIGTERM]:
+            for sig in [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]:
                 signal.signal(sig, self.signal_handler)
 
         # starting to look like a good idea
@@ -69,6 +69,11 @@ class MpWorker(object):
                 sys.exit(0)
                 return
 
+            elif dest == "reload":
+                self.logw("mpw.asrv reloading")
+                self.asrv.reload()
+                self.logw("mpw.asrv reloaded")
+
             elif dest == "listen":
                 self.httpsrv.listen(args[0], args[1])
 

copyparty/broker_thr.py

@@ -21,10 +21,13 @@ class BrokerThr(object):
 
         # instantiate all services here (TODO: inheritance?)
         self.httpsrv = HttpSrv(self, None)
+        self.reload = self.noop
 
     def shutdown(self):
         # self.log("broker", "shutting down")
         self.httpsrv.shutdown()
+
+    def noop(self):
         pass
 
     def put(self, want_retval, dest, *args):

copyparty/httpcli.py

@@ -298,7 +298,7 @@ class HttpCli(object):
 
         # default to utf8 html if no content-type is set
         if not mime:
-            mime = self.out_headers.get("Content-Type", "text/html; charset=UTF-8")
+            mime = self.out_headers.get("Content-Type", "text/html; charset=utf-8")
 
         self.out_headers["Content-Type"] = mime
 
@@ -419,6 +419,9 @@ class HttpCli(object):
             return self.scanvol()
 
         if not self.vpath:
+            if "reload" in self.uparam:
+                return self.handle_reload()
+
             if "stack" in self.uparam:
                 return self.tx_stack()
 
@@ -1461,12 +1464,15 @@ class HttpCli(object):
         else:
             self.permit_caching()
 
+        if "txt" in self.uparam:
+            mime = "text/plain; charset={}".format(self.uparam["txt"] or "utf-8")
+        elif "mime" in self.uparam:
+            mime = self.uparam.get("mime")
+        else:
+            mime = guess_mime(req_path)
+
         self.out_headers["Accept-Ranges"] = "bytes"
-        self.send_headers(
-            length=upper - lower,
-            status=status,
-            mime=guess_mime(req_path),
-        )
+        self.send_headers(length=upper - lower, status=status, mime=mime)
 
         logmsg += unicode(status) + logtail
 
@@ -1711,7 +1717,7 @@ class HttpCli(object):
 
         vn, _ = self.asrv.vfs.get(self.vpath, self.uname, True, True)
 
-        args = [self.asrv.vfs.all_vols, [vn.vpath]]
+        args = [self.asrv.vfs.all_vols, [vn.vpath], False]
 
         x = self.conn.hsrv.broker.put(True, "up2k.rescan", *args)
         x = x.get()
@@ -1721,6 +1727,20 @@ class HttpCli(object):
 
         raise Pebkac(500, x)
 
+    def handle_reload(self):
+        act = self.uparam.get("reload")
+        if act != "cfg":
+            raise Pebkac(400, "only config files ('cfg') can be reloaded rn")
+
+        if not [x for x in self.wvol if x in self.rvol]:
+            raise Pebkac(403, "not allowed for user " + self.uname)
+
+        if self.args.no_reload:
+            raise Pebkac(403, "the reload feature is disabled in server config")
+
+        x = self.conn.hsrv.broker.put(True, "reload")
+        return self.redirect("", "?h", x.get(), "return to", False)
+
     def tx_stack(self):
         if not [x for x in self.wvol if x in self.rvol]:
             raise Pebkac(403, "not allowed for user " + self.uname)
@@ -1862,7 +1882,7 @@ class HttpCli(object):
         arg = self.uparam["ls"]
         if arg in ["v", "t", "txt"]:
             try:
-                biggest = max(ls["files"], key=itemgetter("sz"))["sz"]
+                biggest = max(ls["files"] + ls["dirs"], key=itemgetter("sz"))["sz"]
             except:
                 biggest = 0
 
@@ -1904,14 +1924,14 @@ class HttpCli(object):
                 for x in y
             ]
             ret = "\n".join(ret)
-            mime = "text/plain; encoding=utf-8"
+            mime = "text/plain; charset=utf-8"
         else:
             [x.pop(k) for k in ["name", "dt"] for y in [dirs, files] for x in y]
 
             ret = json.dumps(ls)
             mime = "application/json"
 
-        self.reply(ret.encode("utf-8", "replace"), mime=mime)
+        self.reply(ret.encode("utf-8", "replace") + b"\n", mime=mime)
         return True
 
     def tx_browser(self):
@@ -2072,6 +2092,7 @@ class HttpCli(object):
             "def_hcols": [],
             "have_up2k_idx": ("e2d" in vn.flags),
             "have_tags_idx": ("e2t" in vn.flags),
+            "have_acode": (not self.args.no_acode),
             "have_mv": (not self.args.no_mv),
             "have_del": (not self.args.no_del),
             "have_zip": (not self.args.no_zip),
@@ -2267,6 +2288,22 @@ class HttpCli(object):
             ls_ret["taglist"] = taglist
             return self.tx_ls(ls_ret)
 
+        doc = self.uparam.get("doc") if self.can_read else None
+        if doc:
+            doc = unquotep(doc.replace("+", " "))
+            j2a["docname"] = doc
+            if next((x for x in files if x["name"] == doc), None):
+                with open(os.path.join(abspath, doc), "rb") as f:
+                    doc = f.read().decode("utf-8", "replace")
+            else:
+                self.log("doc 404: [{}]".format(doc), c=6)
+                doc = "( textfile not found )"
+
+            j2a["doc"] = doc
+
+        if not self.conn.hsrv.prism:
+            j2a["no_prism"] = True
+
         for d in dirs:
             d["name"] += "/"
 
@@ -2275,6 +2312,7 @@ class HttpCli(object):
         j2a["files"] = dirs + files
         j2a["logues"] = logues
         j2a["taglist"] = taglist
+        j2a["txt_ext"] = self.args.textfiles.replace(",", " ")
 
         if "mth" in vn.flags:
             j2a["def_hcols"] = vn.flags["mth"].split(",")

copyparty/httpconn.py

@@ -39,7 +39,7 @@ class HttpConn(object):
         self.u2fh = hsrv.u2fh
 
         enth = HAVE_PIL and not self.args.no_thumb
-        self.thumbcli = ThumbCli(hsrv.broker) if enth else None
+        self.thumbcli = ThumbCli(hsrv) if enth else None
         self.ico = Ico(self.args)
 
         self.t0 = time.time()

copyparty/httpsrv.py

@@ -76,6 +76,7 @@ class HttpSrv(object):
             x: env.get_template(x + ".html")
             for x in ["splash", "browser", "browser2", "msg", "md", "mde"]
         }
+        self.prism = os.path.exists(os.path.join(E.mod, "web", "deps", "prism.js.gz"))
 
         cert_path = os.path.join(E.cfg, "cert.pem")
         if bos.path.exists(cert_path):

copyparty/svchub.py

@@ -18,6 +18,7 @@ from .authsrv import AuthSrv
 from .tcpsrv import TcpSrv
 from .up2k import Up2k
 from .th_srv import ThumbSrv, HAVE_PIL, HAVE_WEBP
+from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
 
 
 class SvcHub(object):
@@ -36,7 +37,9 @@ class SvcHub(object):
         self.argv = argv
         self.logf = None
         self.stop_req = False
+        self.reload_req = False
         self.stopping = False
+        self.reloading = False
         self.stop_cond = threading.Condition()
         self.retcode = 0
         self.httpsrv_up = 0
@@ -88,6 +91,18 @@ class SvcHub(object):
                     "thumb", msg.format(" " * 37, os.path.basename(sys.executable)), c=3
                 )
 
+        if not args.no_acode and args.no_thumb:
+            msg = "setting --no-acode because --no-thumb (sorry)"
+            self.log("thumb", msg, c=6)
+            args.no_acode = True
+
+        if not args.no_acode and (not HAVE_FFMPEG or not HAVE_FFPROBE):
+            msg = "setting --no-acode because either FFmpeg or FFprobe is not available"
+            self.log("thumb", msg, c=6)
+            args.no_acode = True
+
+        args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
+
         # decide which worker impl to use
         if self.check_mp_enable():
             from .broker_mp import BrokerMp as Broker
@@ -182,7 +197,11 @@ class SvcHub(object):
         thr.daemon = True
         thr.start()
 
-        for sig in [signal.SIGINT, signal.SIGTERM]:
+        sigs = [signal.SIGINT, signal.SIGTERM]
+        if not ANYWIN:
+            sigs.append(signal.SIGUSR1)
+
+        for sig in sigs:
             signal.signal(sig, self.signal_handler)
 
         # macos hangs after shutdown on sigterm with while-sleep,
@@ -206,18 +225,45 @@ class SvcHub(object):
         else:
             self.stop_thr()
 
+    def reload(self):
+        if self.reloading:
+            return "cannot reload; already in progress"
+
+        self.reloading = True
+        t = threading.Thread(target=self._reload)
+        t.daemon = True
+        t.start()
+        return "reload initiated"
+
+    def _reload(self):
+        self.log("root", "reload scheduled")
+        with self.up2k.mutex:
+            self.asrv.reload()
+            self.up2k.reload()
+            self.broker.reload()
+
+        self.reloading = False
+
     def stop_thr(self):
         while not self.stop_req:
             with self.stop_cond:
                 self.stop_cond.wait(9001)
 
+            if self.reload_req:
+                self.reload_req = False
+                self.reload()
+
         self.shutdown()
 
     def signal_handler(self, sig, frame):
         if self.stopping:
             return
 
-        self.stop_req = True
+        if sig == signal.SIGUSR1:
+            self.reload_req = True
+        else:
+            self.stop_req = True
+
         with self.stop_cond:
             self.stop_cond.notify_all()
 

copyparty/th_cli.py

@@ -4,28 +4,44 @@ from __future__ import print_function, unicode_literals
 import os
 
 from .util import Cooldown
-from .th_srv import thumb_path, THUMBABLE, FMT_FF
+from .th_srv import thumb_path, THUMBABLE, FMT_FFV, FMT_FFA
 from .bos import bos
 
 
 class ThumbCli(object):
-    def __init__(self, broker):
-        self.broker = broker
-        self.args = broker.args
-        self.asrv = broker.asrv
+    def __init__(self, hsrv):
+        self.broker = hsrv.broker
+        self.log_func = hsrv.log
+        self.args = hsrv.args
+        self.asrv = hsrv.asrv
 
         # cache on both sides for less broker spam
         self.cooldown = Cooldown(self.args.th_poke)
 
+    def log(self, msg, c=0):
+        self.log_func("thumbcli", msg, c)
+
     def get(self, ptop, rem, mtime, fmt):
         ext = rem.rsplit(".")[-1].lower()
         if ext not in THUMBABLE:
             return None
 
-        is_vid = ext in FMT_FF
+        is_vid = ext in FMT_FFV
         if is_vid and self.args.no_vthumb:
             return None
 
+        want_opus = fmt == "opus"
+        is_au = ext in FMT_FFA
+        if is_au:
+            if want_opus:
+                if self.args.no_acode:
+                    return None
+            else:
+                if self.args.no_athumb:
+                    return None
+        elif want_opus:
+            return None
+
         if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg"]:
             return os.path.join(ptop, rem)
 
@@ -33,10 +49,14 @@ class ThumbCli(object):
             fmt = "w"
 
         if fmt == "w":
-            if self.args.th_no_webp or (is_vid and self.args.th_ff_jpg):
+            if self.args.th_no_webp or ((is_vid or is_au) and self.args.th_ff_jpg):
                 fmt = "j"
 
-        histpath = self.asrv.vfs.histtab[ptop]
+        histpath = self.asrv.vfs.histtab.get(ptop)
+        if not histpath:
+            self.log("no histpath for [{}]".format(ptop))
+            return None
+
         tpath = thumb_path(histpath, rem, mtime, fmt)
         ret = None
         try:
@@ -53,6 +73,11 @@ class ThumbCli(object):
             if self.cooldown.poke(tdir):
                 self.broker.put(False, "thumbsrv.poke", tdir)
 
+            if want_opus:
+                # audio files expire individually
+                if self.cooldown.poke(tpath):
+                    self.broker.put(False, "thumbsrv.poke", tpath)
+
             return ret
 
         x = self.broker.put(True, "thumbsrv.get", ptop, rem, mtime, fmt)

copyparty/th_srv.py

@@ -10,7 +10,7 @@ import threading
 import subprocess as sp
 
 from .__init__ import PY2, unicode
-from .util import fsenc, vsplit, runcmd, Queue, Cooldown, BytesIO, min_ex
+from .util import fsenc, vsplit, statdir, runcmd, Queue, Cooldown, BytesIO, min_ex
 from .bos import bos
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
 
@@ -50,7 +50,8 @@ except:
 # https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
 # ffmpeg -formats
 FMT_PIL = "bmp dib gif icns ico jpg jpeg jp2 jpx pcx png pbm pgm ppm pnm sgi tga tif tiff webp xbm dds xpm"
-FMT_FF = "av1 asf avi flv m4v mkv mjpeg mjpg mpg mpeg mpg2 mpeg2 h264 avc mts h265 hevc mov 3gp mp4 ts mpegts nut ogv ogm rm vob webm wmv"
+FMT_FFV = "av1 asf avi flv m4v mkv mjpeg mjpg mpg mpeg mpg2 mpeg2 h264 avc mts h265 hevc mov 3gp mp4 ts mpegts nut ogv ogm rm vob webm wmv"
+FMT_FFA = "aac m4a ogg opus flac alac mp3 mp2 ac3 dts wma ra wav aif aiff au alaw ulaw mulaw amr gsm ape tak tta wv"
 
 if HAVE_HEIF:
     FMT_PIL += " heif heifs heic heics"
@@ -58,7 +59,9 @@ if HAVE_HEIF:
 if HAVE_AVIF:
     FMT_PIL += " avif avifs"
 
-FMT_PIL, FMT_FF = [{x: True for x in y.split(" ") if x} for y in [FMT_PIL, FMT_FF]]
+FMT_PIL, FMT_FFV, FMT_FFA = [
+    {x: True for x in y.split(" ") if x} for y in [FMT_PIL, FMT_FFV, FMT_FFA]
+]
 
 
 THUMBABLE = {}
@@ -67,7 +70,8 @@ if HAVE_PIL:
     THUMBABLE.update(FMT_PIL)
 
 if HAVE_FFMPEG and HAVE_FFPROBE:
-    THUMBABLE.update(FMT_FF)
+    THUMBABLE.update(FMT_FFV)
+    THUMBABLE.update(FMT_FFA)
 
 
 def thumb_path(histpath, rem, mtime, fmt):
@@ -86,9 +90,13 @@ def thumb_path(histpath, rem, mtime, fmt):
     h = hashlib.sha512(fsenc(fn)).digest()
     fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
 
-    return "{}/th/{}/{}.{:x}.{}".format(
-        histpath, rd, fn, int(mtime), "webp" if fmt == "w" else "jpg"
-    )
+    if fmt == "opus":
+        cat = "ac"
+    else:
+        fmt = "webp" if fmt == "w" else "jpg"
+        cat = "th"
+
+    return "{}/{}/{}/{}.{:x}.{}".format(histpath, cat, rd, fn, int(mtime), fmt)
 
 
 class ThumbSrv(object):
@@ -115,7 +123,8 @@ class ThumbSrv(object):
             t.daemon = True
             t.start()
 
-        if not self.args.no_vthumb and (not HAVE_FFMPEG or not HAVE_FFPROBE):
+        want_ff = not self.args.no_vthumb or not self.args.no_athumb
+        if want_ff and (not HAVE_FFMPEG or not HAVE_FFPROBE):
             missing = []
             if not HAVE_FFMPEG:
                 missing.append("FFmpeg")
@@ -123,7 +132,7 @@ class ThumbSrv(object):
             if not HAVE_FFPROBE:
                 missing.append("FFprobe")
 
-            msg = "cannot create video thumbnails because some of the required programs are not available: "
+            msg = "cannot create audio/video thumbnails because some of the required programs are not available: "
             msg += ", ".join(missing)
             self.log(msg, c=3)
 
@@ -145,7 +154,11 @@ class ThumbSrv(object):
             return not self.nthr
 
     def get(self, ptop, rem, mtime, fmt):
-        histpath = self.asrv.vfs.histtab[ptop]
+        histpath = self.asrv.vfs.histtab.get(ptop)
+        if not histpath:
+            self.log("no histpath for [{}]".format(ptop))
+            return None
+
         tpath = thumb_path(histpath, rem, mtime, fmt)
         abspath = os.path.join(ptop, rem)
         cond = threading.Condition(self.mutex)
@@ -181,6 +194,7 @@ class ThumbSrv(object):
         try:
             st = bos.stat(tpath)
             if st.st_size:
+                self.poke(tpath)
                 return tpath
         except:
             pass
@@ -199,8 +213,13 @@ class ThumbSrv(object):
             if not bos.path.exists(tpath):
                 if ext in FMT_PIL:
                     fun = self.conv_pil
-                elif ext in FMT_FF:
+                elif ext in FMT_FFV:
                     fun = self.conv_ffmpeg
+                elif ext in FMT_FFA:
+                    if tpath.endswith(".opus"):
+                        fun = self.conv_opus
+                    else:
+                        fun = self.conv_spec
 
             if fun:
                 try:
@@ -326,8 +345,10 @@ class ThumbSrv(object):
             ]
 
         cmd += [fsenc(tpath)]
-        # self.log((b" ".join(cmd)).decode("utf-8"))
+        self._run_ff(cmd)
 
+    def _run_ff(self, cmd):
+        # self.log((b" ".join(cmd)).decode("utf-8"))
         ret, sout, serr = runcmd(cmd)
         if ret != 0:
             m = "FFmpeg failed (probably a corrupt video file):\n"
@@ -335,16 +356,81 @@ class ThumbSrv(object):
             self.log(m, c="1;30")
             raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
 
+    def conv_spec(self, abspath, tpath):
+        ret, _ = ffprobe(abspath)
+        if "ac" not in ret:
+            raise Exception("not audio")
+
+        fc = "[0:a:0]aresample=48000{},showspectrumpic=s=640x512,crop=780:544:70:50[o]"
+
+        if self.args.th_ff_swr:
+            fco = ":filter_size=128:cutoff=0.877"
+        else:
+            fco = ":resampler=soxr"
+
+        fc = fc.format(fco)
+
+        # fmt: off
+        cmd = [
+            b"ffmpeg",
+            b"-nostdin",
+            b"-v", b"error",
+            b"-hide_banner",
+            b"-i", fsenc(abspath),
+            b"-filter_complex", fc.encode("utf-8"),
+            b"-map", b"[o]"
+        ]
+        # fmt: on
+
+        if tpath.endswith(".jpg"):
+            cmd += [
+                b"-q:v",
+                b"6",  # default=??
+            ]
+        else:
+            cmd += [
+                b"-q:v",
+                b"50",  # default=75
+                b"-compression_level:v",
+                b"6",  # default=4, 0=fast, 6=max
+            ]
+
+        cmd += [fsenc(tpath)]
+        self._run_ff(cmd)
+
+    def conv_opus(self, abspath, tpath):
+        if self.args.no_acode:
+            raise Exception("disabled in server config")
+
+        ret, _ = ffprobe(abspath)
+        if "ac" not in ret:
+            raise Exception("not audio")
+
+        # fmt: off
+        cmd = [
+            b"ffmpeg",
+            b"-nostdin",
+            b"-v", b"error",
+            b"-hide_banner",
+            b"-i", fsenc(abspath),
+            b"-map", b"0:a:0",
+            b"-c:a", b"libopus",
+            b"-b:a", b"128k",
+            fsenc(tpath)
+        ]
+        # fmt: on
+
+        self._run_ff(cmd)
+
     def poke(self, tdir):
         if not self.poke_cd.poke(tdir):
             return
 
         ts = int(time.time())
         try:
-            p1 = os.path.dirname(tdir)
-            p2 = os.path.dirname(p1)
-            for dp in [tdir, p1, p2]:
-                bos.utime(dp, (ts, ts))
+            for _ in range(4):
+                bos.utime(tdir, (ts, ts))
+                tdir = os.path.dirname(tdir)
         except:
             pass
 
@@ -364,25 +450,36 @@ class ThumbSrv(object):
             self.log("\033[Jcln ok; rm {} dirs".format(ndirs))
 
     def clean(self, histpath):
-        thumbpath = os.path.join(histpath, "th")
+        ret = 0
+        for cat in ["th", "ac"]:
+            ret += self._clean(histpath, cat, None)
+
+        return ret
+
+    def _clean(self, histpath, cat, thumbpath):
+        if not thumbpath:
+            thumbpath = os.path.join(histpath, cat)
+
         # self.log("cln {}".format(thumbpath))
-        maxage = self.args.th_maxage
+        exts = ["jpg", "webp"] if cat == "th" else ["opus"]
+        maxage = getattr(self.args, cat + "_maxage")
         now = time.time()
         prev_b64 = None
         prev_fp = None
         try:
-            ents = bos.listdir(thumbpath)
+            ents = statdir(self.log, not self.args.no_scandir, False, thumbpath)
+            ents = sorted(list(ents))
         except:
             return 0
 
         ndirs = 0
-        for f in sorted(ents):
+        for f, inf in ents:
             fp = os.path.join(thumbpath, f)
             cmp = fp.lower().replace("\\", "/")
 
             # "top" or b64 prefix/full (a folder)
             if len(f) <= 3 or len(f) == 24:
-                age = now - bos.path.getmtime(fp)
+                age = now - inf.st_mtime
                 if age > maxage:
                     with self.mutex:
                         safe = True
@@ -396,16 +493,15 @@ class ThumbSrv(object):
                             self.log("rm -rf [{}]".format(fp))
                             shutil.rmtree(fp, ignore_errors=True)
                 else:
-                    ndirs += self.clean(fp)
+                    self._clean(histpath, cat, fp)
+
                 continue
 
             # thumb file
             try:
                 b64, ts, ext = f.split(".")
-                if len(b64) != 24 or len(ts) != 8 or ext not in ["jpg", "webp"]:
+                if len(b64) != 24 or len(ts) != 8 or ext not in exts:
                     raise Exception()
-
-                ts = int(ts, 16)
             except:
                 if f != "dir.txt":
                     self.log("foreign file in thumbs dir: [{}]".format(fp), 1)
@@ -416,6 +512,10 @@ class ThumbSrv(object):
                 self.log("rm replaced [{}]".format(fp))
                 bos.unlink(prev_fp)
 
+            if cat != "th" and inf.st_mtime + maxage < now:
+                self.log("rm expired [{}]".format(fp))
+                bos.unlink(fp)
+
             prev_b64 = b64
             prev_fp = fp
 

copyparty/u2idx.py

@@ -67,7 +67,11 @@ class U2idx(object):
         if cur:
             return cur
 
-        histpath = self.asrv.vfs.histtab[ptop]
+        histpath = self.asrv.vfs.histtab.get(ptop)
+        if not histpath:
+            self.log("no histpath for [{}]".format(ptop))
+            return None
+
         db_path = os.path.join(histpath, "up2k.db")
         if not bos.path.exists(db_path):
             return None

copyparty/up2k.py

@@ -68,6 +68,7 @@ class Up2k(object):
         self.tagq = Queue()
         self.n_hashq = 0
         self.n_tagq = 0
+        self.gid = 0
         self.volstate = {}
         self.need_rescan = {}
         self.dupesched = {}
@@ -114,6 +115,12 @@ class Up2k(object):
         t.daemon = True
         t.start()
 
+    def reload(self):
+        self.gid += 1
+        self.log("reload #{} initiated".format(self.gid))
+        all_vols = self.asrv.vfs.all_vols
+        self.rescan(all_vols, list(all_vols.keys()), True)
+
     def deferred_init(self):
         all_vols = self.asrv.vfs.all_vols
         have_e2d = self.init_indexes(all_vols)
@@ -168,15 +175,15 @@ class Up2k(object):
         }
         return json.dumps(ret, indent=4)
 
-    def rescan(self, all_vols, scan_vols):
-        if hasattr(self, "pp"):
+    def rescan(self, all_vols, scan_vols, wait):
+        if not wait and hasattr(self, "pp"):
             return "cannot initiate; scan is already in progress"
 
         args = (all_vols, scan_vols)
         t = threading.Thread(
             target=self.init_indexes,
             args=args,
-            name="up2k-rescan-{}".format(scan_vols[0]),
+            name="up2k-rescan-{}".format(scan_vols[0] if scan_vols else "all"),
         )
         t.daemon = True
         t.start()
@@ -196,6 +203,10 @@ class Up2k(object):
             if now < cooldown:
                 continue
 
+            if hasattr(self, "pp"):
+                cooldown = now + 5
+                continue
+
             timeout = now + 9001
             with self.mutex:
                 for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
@@ -217,7 +228,7 @@ class Up2k(object):
 
             if vols:
                 cooldown = now + 10
-                err = self.rescan(self.asrv.vfs.all_vols, vols)
+                err = self.rescan(self.asrv.vfs.all_vols, vols, False)
                 if err:
                     for v in vols:
                         self.need_rescan[v] = True
@@ -299,6 +310,16 @@ class Up2k(object):
         return True, ret
 
     def init_indexes(self, all_vols, scan_vols=None):
+        gid = self.gid
+        while hasattr(self, "pp") and gid == self.gid:
+            time.sleep(0.1)
+
+        if gid != self.gid:
+            return
+
+        if gid:
+            self.log("reload #{} running".format(self.gid))
+
         self.pp = ProgressPrinter()
         vols = all_vols.values()
         t0 = time.time()
@@ -429,7 +450,11 @@ class Up2k(object):
         return have_e2d
 
     def register_vpath(self, ptop, flags):
-        histpath = self.asrv.vfs.histtab[ptop]
+        histpath = self.asrv.vfs.histtab.get(ptop)
+        if not histpath:
+            self.log("no histpath for [{}]".format(ptop))
+            return None
+
         db_path = os.path.join(histpath, "up2k.db")
         if ptop in self.registry:
             try:
@@ -797,10 +822,11 @@ class Up2k(object):
         return ret
 
     def _run_all_mtp(self):
+        gid = self.gid
         t0 = time.time()
         for ptop, flags in self.flags.items():
             if "mtp" in flags:
-                self._run_one_mtp(ptop)
+                self._run_one_mtp(ptop, gid)
 
         td = time.time() - t0
         msg = "mtp finished in {:.2f} sec ({})"
@@ -811,7 +837,10 @@ class Up2k(object):
             if "OFFLINE" not in self.volstate[k]:
                 self.volstate[k] = "online, idle"
 
-    def _run_one_mtp(self, ptop):
+    def _run_one_mtp(self, ptop, gid):
+        if gid != self.gid:
+            return
+
         entags = self.entags[ptop]
 
         parsers = {}
@@ -844,6 +873,9 @@ class Up2k(object):
         in_progress = {}
         while True:
             with self.mutex:
+                if gid != self.gid:
+                    break
+
                 q = "select w from mt where k = 't:mtp' limit ?"
                 warks = cur.execute(q, (batch_sz,)).fetchall()
                 warks = [x[0] for x in warks]
@@ -1960,7 +1992,8 @@ class Up2k(object):
         self.snap_prev = {}
         while True:
             time.sleep(self.snap_persist_interval)
-            self.do_snapshot()
+            if not hasattr(self, "pp"):
+                self.do_snapshot()
 
     def do_snapshot(self):
         with self.mutex:
@@ -1969,7 +2002,10 @@ class Up2k(object):
 
     def _snap_reg(self, ptop, reg):
         now = time.time()
-        histpath = self.asrv.vfs.histtab[ptop]
+        histpath = self.asrv.vfs.histtab.get(ptop)
+        if not histpath:
+            return
+
         rm = [x for x in reg.values() if now - x["poke"] > self.snap_discard_interval]
         if rm:
             m = "dropping {} abandoned uploads in {}".format(len(rm), ptop)

copyparty/util.py

@@ -100,10 +100,24 @@ IMPLICATIONS = [
 
 
 MIMES = {
-    "md": "text/plain; charset=UTF-8",
+    "md": "text/plain",
+    "txt": "text/plain",
+    "js": "text/javascript",
     "opus": "audio/ogg; codecs=opus",
-    "webp": "image/webp",
+    "mp3": "audio/mpeg",
+    "m4a": "audio/mp4",
+    "jpg": "image/jpeg",
 }
+for ln in """text css html csv
+application json wasm xml pdf rtf zip
+image webp jpeg png gif bmp
+audio aac ogg wav
+video webm mp4 mpeg
+font woff woff2 otf ttf
+""".splitlines():
+    k, vs = ln.split(" ", 1)
+    for v in vs.strip().split():
+        MIMES[v] = "{}/{}".format(k, v)
 
 
 REKOBO_KEY = {
@@ -1280,11 +1294,18 @@ def guess_mime(url, fallback="application/octet-stream"):
     except:
         return fallback
 
-    ret = MIMES.get(ext) or mimetypes.guess_type(url)[0] or fallback
+    ret = MIMES.get(ext)
+
+    if not ret:
+        x = mimetypes.guess_type(url)
+        ret = "application/{}".format(x[1]) if x[1] else x[0]
+
+    if not ret:
+        ret = fallback
 
     if ";" not in ret:
         if ret.startswith("text/") or ret.endswith("/javascript"):
-            ret += "; charset=UTF-8"
+            ret += "; charset=utf-8"
 
     return ret
 

copyparty/web/baguettebox.js

@@ -331,7 +331,7 @@ window.baguetteBox = (function () {
 
     function tglsel() {
         var thumb = currentGallery[currentIndex].imageElement,
-            name = vsplit(thumb.href)[1],
+            name = vsplit(thumb.href)[1].split('?')[0],
             files = msel.getall();
 
         for (var a = 0; a < files.length; a++)
@@ -345,7 +345,7 @@ window.baguetteBox = (function () {
     function selbg() {
         var img = vidimg(),
             thumb = currentGallery[currentIndex].imageElement,
-            name = vsplit(thumb.href)[1],
+            name = vsplit(thumb.href)[1].split('?')[0],
             files = msel.getsel(),
             sel = false;
 
@@ -530,9 +530,7 @@ window.baguetteBox = (function () {
             if (options.bodyClass && document.body.classList)
                 document.body.classList.remove(options.bodyClass);
 
-            var h = ebi('bbox-halp');
-            if (h)
-                h.parentNode.removeChild(h);
+            qsr('#bbox-halp');
 
             if (options.afterHide)
                 options.afterHide();
@@ -590,8 +588,7 @@ window.baguetteBox = (function () {
 
         image.addEventListener(is_vid ? 'loadedmetadata' : 'load', function () {
             // Remove loader element
-            var spinner = QS('#baguette-img-' + index + ' .bbox-spinner');
-            figure.removeChild(spinner);
+            qsr('#baguette-img-' + index + ' .bbox-spinner');
             if (!options.async && callback)
                 callback();
         });

copyparty/web/browser.css

@@ -23,7 +23,7 @@ html, body {
 	margin: 0;
 	padding: 0;
 }
-pre, code, tt {
+pre, code, tt, #doc, #doc>code {
 	font-family: 'scp', monospace, monospace;
 }
 #path,
@@ -31,9 +31,8 @@ pre, code, tt {
 	font-size: 1em;
 }
 #path {
-	color: #aca;
+	color: #ccc;
 	text-shadow: 1px 1px 0 #000;
-	font-variant: small-caps;
 	font-weight: normal;
 	display: inline-block;
 	padding: .35em .5em .2em .5em;
@@ -205,6 +204,12 @@ a, #files tbody div a:last-child {
 #repl {
 	padding: .33em;
 }
+#files a.doc {
+	color: #666;
+}
+#files a.doc.bri {
+	color: #f5a;
+}
 #files tbody a.play {
 	color: #e70;
 	padding: .2em;
@@ -668,6 +673,17 @@ input.eq_gain {
 	border-bottom: 1px solid #111;
 	overflow: hidden;
 }
+#treepar {
+	z-index: 1;
+	position: fixed;
+	left: -.75em;
+	width: calc(var(--nav-sz) - 0.5em);
+	border-bottom: 1px solid #444;
+	overflow: hidden;
+}
+#treepar.off {
+	display: none;
+}
 #tree, #treeh {
 	border-radius: 0 .3em 0 0;
 }
@@ -732,37 +748,46 @@ input.eq_gain {
 #tree li:last-child {
 	border-bottom: none;
 }
-#treeul a.hl {
+#tree ul a.sel {
+	background: #111;
+	box-shadow: -.8em 0 0 #c37 inset;
+	color: #fff;
+}
+#tree ul a.hl {
 	color: #400;
 	background: #fc4;
 	text-shadow: none;
 }
-#treeul a {
+#tree ul a.par {
+	color: #fff;
+}
+#tree ul a {
 	border-radius: .3em;
 	display: inline-block;
 }
-#treeul a+a {
+.ntree a+a {
 	width: calc(100% - 2em);
 	line-height: 1em;
 }
-#tree.nowrap #treeul li {
+#tree.nowrap li {
 	min-height: 1.4em;
 	white-space: nowrap;
 }
-#tree.nowrap #treeul a+a:hover {
+#tree.nowrap .ntree a+a:hover {
 	background: rgba(16, 16, 16, 0.67);
 	min-width: calc(var(--nav-sz) - 2em);
 	width: auto;
 }
-html.light #tree.nowrap #treeul a+a:hover {
+html.light #tree.nowrap .ntree a+a:hover {
 	background: rgba(255, 255, 255, 0.67);
 	color: #000;
 }
-#treeul a+a:hover {
+#docul a:hover,
+#tree .ntree a+a:hover {
 	background: #181818;
 	color: #fff;
 }
-#treeul a:first-child {
+.ntree a:first-child {
 	font-family: 'scp', monospace, monospace;
 	font-size: 1.2em;
 	line-height: 0;
@@ -845,11 +870,13 @@ html.light #tree.nowrap #treeul a+a:hover {
 	border-bottom: 1px solid #555;
 }
 #thumbs,
+#au_fullpre,
 #au_osd_cv,
 #u2tdate {
 	opacity: .3;
 }
 #griden.on+#thumbs,
+#au_preload.on+#au_fullpre,
 #au_os_ctl.on+#au_osd_cv,
 #u2turbo.on+#u2tdate {
 	opacity: 1;
@@ -857,30 +884,31 @@ html.light #tree.nowrap #treeul a+a:hover {
 #wraptree.on+#hovertree {
 	display: none;
 }
-#ghead {
+.ghead {
 	border-radius: .3em;
 	padding: .2em .5em;
 	line-height: 2.3em;
-	margin-bottom: 1em;
+	margin-bottom: 1.5em;
+}
+#ghead {
 	position: sticky;
 	top: -.3em;
 	z-index: 1;
 }
-html.light #ghead {
+html.light .ghead {
 	background: #f7f7f7;
 	border-color: #ddd;
 }
-#ghead .btn {
+.ghead .btn {
 	position: relative;
 	top: 0;
 }
-#ghead>span {
+.ghead>span {
 	white-space: pre;
 	padding-left: .3em;
 }
 #ggrid {
-	padding-top: .5em;
-	margin: 0 -.5em;
+	margin: -.2em -.5em;
 }
 #ggrid>a>span {
 	overflow: hidden;
@@ -989,6 +1017,53 @@ html.light #rui {
 	padding: 0;
 	font-size: 1.5em;
 }
+#doc {
+	background: none;
+	overflow: visible;
+	margin: -1em 0 .5em 0;
+	padding: 1em 0 1em 0;
+}
+#docul {
+	position: relative;
+}
+#docul li.bn {
+	text-align: center;
+	padding: .5em;
+}
+#doc.prism {
+	padding-left: 3em;
+}
+#doc>code {
+	background: none;
+	box-shadow: none;
+	z-index: 1;
+}
+#doc.mdo {
+	white-space: normal;
+	font-family: sans-serif;
+}
+#doc.prism * {
+	line-height: 1.5em;
+}
+#doc .line-highlight {
+	border-radius: .3em;
+	box-shadow: 0 0 .5em #333;
+	background: linear-gradient(90deg, #111, #222);
+}
+html.light #doc .line-highlight {
+	box-shadow: 0 0 .5em #ccc;
+	background: linear-gradient(90deg, #fff, #eee);
+}
+#docul li {
+	margin: 0;
+}
+#tree #docul a {
+	display: block;
+}
+#seldoc.sel {
+	color: #fff;
+	background: #925;
+}
 #pvol,
 #barbuf,
 #barpos,
@@ -1043,7 +1118,7 @@ html,
 .opbox,
 #path,
 #srch_form,
-#ghead {
+.ghead {
 	background: #2b2b2b;
 	border: 1px solid #333;
 	box-shadow: 0 0 .3em #111;
@@ -1052,7 +1127,8 @@ html,
 	background: #282828;
 }
 #tree,
-#treeh {
+#treeh,
+#treepar {
 	background: #2b2b2b;
 }
 #wtoggle,
@@ -1110,12 +1186,15 @@ html.light #ops,
 html.light .opbox,
 html.light #path,
 html.light #srch_form,
-html.light #ghead,
+html.light .ghead,
 html.light #u2etas {
 	background: #f7f7f7;
 	box-shadow: 0 0 .3em #ccc;
 	border-color: #f7f7f7;
 }
+html.light #wrap.doc {
+	background: #f7f7f7;
+}
 html.light #ops a.act {
 	box-shadow: 0 .2em .2em #ccc;
 	background: #fff;
@@ -1154,21 +1233,25 @@ html.light #acc_info {
 html.light #srv_info span {
 	color: #777;
 }
-html.light #treeul a+a {
+html.light #tree .ntree a+a {
 	background: inherit;
 	color: #06a;
 }
-html.light #treeul a.hl {
+html.light #tree ul a.hl {
 	background: #07a;
 	color: #fff;
 }
-html.light #treeul a.hl:hover {
+html.light #tree ul a.par {
+	color: #000;
+}
+html.light #tree ul a.hl:hover {
 	background: #059;
 }
-html.light #tree li {
+html.light #tree li,
+html.light #tree #treepar {
 	border-color: #f7f7f7 #fff #ddd #fff;
 }
-html.light #treeul a:hover {
+html.light #tree ul a:hover {
 	background: #fff;
 }
 html.light #tree ul {
@@ -1216,6 +1299,12 @@ html.light #files tbody a.play {
 html.light #files tbody a.play.act {
 	color: #90c;
 }
+html.light #files a.doc {
+	color: #bbb;
+}
+html.light #files a.doc.bri {
+	color: #d38;
+}
 html.light #files tr.play td {
 	background: #fc5;
 	border-color: #eb1;
@@ -1283,6 +1372,7 @@ html.light #files td div span {
 	color: #000;
 }
 html.light #path {
+	color: #777;
 	background: #f7f7f7;
 	text-shadow: none;
 	box-shadow: 0 0 .3em #bbb;
@@ -1300,13 +1390,15 @@ html.light #path a:hover {
 html.light #files tbody div a {
 	color: #d38;
 }
+html.light #docul a:hover,
 html.light #files a:hover,
 html.light #files tr.sel a:hover {
 	color: #000;
 	background: #fff;
 	text-decoration: underline;
 }
-html.light #treeh {
+html.light #treeh,
+html.light #treepar {
 	background: #f7f7f7;
 	border-color: #ddd;
 }

copyparty/web/browser.html

@@ -76,6 +76,12 @@
 
 <div id="wrap">
 
+	{%- if doc %}
+	<div id="bdoc"><pre>{{ doc|e }}</pre></div>
+	{%- else %}
+	<div id="bdoc"></div>
+	{%- endif %}
+
 	<div id="pro" class="logue">{{ logues[0] }}</div>
 
 	<table id="files">
@@ -130,10 +136,13 @@
 			def_hcols = {{ def_hcols|tojson }},
 			have_up2k_idx = {{ have_up2k_idx|tojson }},
 			have_tags_idx = {{ have_tags_idx|tojson }},
+			have_acode = {{ have_acode|tojson }},
 			have_mv = {{ have_mv|tojson }},
 			have_del = {{ have_del|tojson }},
 			have_unpost = {{ have_unpost|tojson }},
 			have_zip = {{ have_zip|tojson }},
+			txt_ext = "{{ txt_ext }}",
+			{% if no_prism %}no_prism = 1,{% endif %}
 			readme = {{ readme|tojson }};
 
 		document.documentElement.setAttribute("class", localStorage.lightmode == 1 ? "light" : "dark");

copyparty/web/md.js

@@ -164,10 +164,7 @@ function copydom(src, dst, lv) {
 
 
 function md_plug_err(ex, js) {
-    var errbox = ebi('md_errbox');
-    if (errbox)
-        errbox.parentNode.removeChild(errbox);
-
+    qsr('#md_errbox');
     if (!ex)
         return;
 
@@ -183,7 +180,7 @@ function md_plug_err(ex, js) {
             o.textContent = lns[ln - 1];
         }
     }
-    errbox = mknod('div');
+    var errbox = mknod('div');
     errbox.setAttribute('id', 'md_errbox');
     errbox.style.cssText = 'position:absolute;top:0;left:0;padding:1em .5em;background:#2b2b2b;color:#fc5'
     errbox.textContent = msg;
@@ -381,8 +378,7 @@ function convert_markdown(md_text, dest_dom) {
 
 
 function init_toc() {
-    var loader = ebi('ml');
-    loader.parentNode.removeChild(loader);
+    qsr('#ml');
 
     var anchors = [];  // list of toc entries, complex objects
     var anchor = null; // current toc node

copyparty/web/mde.js

@@ -65,8 +65,7 @@ var mde = (function () {
     mde.codemirror.on("change", function () {
         md_changed(mde);
     });
-    var loader = ebi('ml');
-    loader.parentNode.removeChild(loader);
+    qsr('#ml');
     return mde;
 })();
 

copyparty/web/splash.css

@@ -29,15 +29,17 @@ a {
 	border-radius: .2em;
 	padding: .2em .8em;
 }
+a+a {
+	margin-left: .5em;
+}
 .refresh,
 .logout {
 	float: right;
-	margin-top: -.2em;
+	margin: -.2em 0 0 .5em;
 }
 .logout {
 	color: #c04;
 	border-color: #c7a;
-	margin-right: .5em;
 }
 #repl {
 	border: none;
@@ -52,6 +54,7 @@ table {
 .vols th {
 	padding: .3em .6em;
 	text-align: left;
+	white-space: nowrap;
 }
 .num {
 	border-right: 1px solid #bbb;

copyparty/web/splash.html

@@ -49,7 +49,8 @@
             </table>
         </td></tr></table>
         <div class="btns">
-            <a href="/?stack">dump stack</a>
+            <a href="/?stack" tt="shows the state of all active threads">dump stack</a>
+            <a href="/?reload=cfg" tt="reload config files (accounts/volumes/volflags),$Nand rescan all e2ds volumes">reload cfg</a>
         </div>
         {%- endif %}
 
@@ -88,5 +89,6 @@ if (localStorage.lightmode != 1)
 
 </script>
 <script src="/.cpr/util.js?_={{ ts }}"></script>
+<script>tt.init();</script>
 </body>
 </html>

copyparty/web/up2k.js

@@ -332,8 +332,7 @@ function U2pvis(act, btns) {
                 r.head++;
 
             if (!bz_act) {
-                var tr = ebi("f" + nfile);
-                tr.parentNode.removeChild(tr);
+                qsr("#f" + nfile);
             }
         }
         else return;
@@ -352,9 +351,7 @@ function U2pvis(act, btns) {
         last = parseInt(last.getAttribute('id').slice(1));
 
         while (r.head - first > r.wsz) {
-            var obj = ebi('f' + (first++));
-            if (obj)
-                obj.parentNode.removeChild(obj);
+            qsr('#f' + (first++));
         }
         while (last - r.tail < r.wsz && last < r.tab.length - 2) {
             var obj = ebi('f' + (++last));
@@ -1075,11 +1072,9 @@ function up2k_init(subtle) {
         for (var a = 0; a < st.files.length; a++) {
             var t = st.files[a];
             if (t.done && t.name) {
-                var tr = ebi('f' + t.n);
-                if (!tr)
+                if (!qsr('#f' + t.n))
                     continue;
 
-                tr.parentNode.removeChild(tr);
                 t.name = undefined;
             }
         }

copyparty/web/util.js

@@ -18,6 +18,15 @@ var ebi = document.getElementById.bind(document),
     mknod = document.createElement.bind(document);
 
 
+function qsr(sel) {
+    var el = QS(sel);
+    if (el)
+        el.parentNode.removeChild(el);
+
+    return el;
+}
+
+
 // error handler for mobile devices
 function esc(txt) {
     return txt.replace(/[&"<>]/g, function (c) {
@@ -163,7 +172,6 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
     catch (e) {
         document.body.innerHTML = html.join('\n');
     }
-    throw 'fatal_err';
 }
 function ignex(all) {
     var o = ebi('exbox');
@@ -727,7 +735,7 @@ function hist_replace(url) {
 
 function sethash(hv) {
     if (window.history && history.replaceState) {
-        hist_replace(document.location.pathname + '#' + hv);
+        hist_replace(document.location.pathname + document.location.search + '#' + hv);
     }
     else {
         document.location.hash = hv;

docs/README.md

@@ -29,7 +29,7 @@ point `--css-browser` to one of these by URL:
 * notes on using rclone as a fuse client/server
 
 ## [`example.conf`](example.conf)
-* example config file for `-c` (supports accounts, volumes, and volume-flags)
+* example config file for `-c`
 
 
 

docs/example.conf

@@ -1,3 +1,10 @@
+# append some arguments to the commandline;
+# the first space in a line counts as a separator,
+# any additional spaces are part of the value
+-e2dsa
+-e2ts
+-i 127.0.0.1
+
 # create users:
 # u username:password
 u ed:123
@@ -24,7 +31,8 @@ rw ed
 r k
 rw ed
 
-# this does the same thing:
+# this does the same thing,
+# and will cause an error on startup since /priv is already taken:
 ./priv
 /priv
 r ed k

docs/notes.sh

@@ -38,6 +38,13 @@ para() { for s in 1 2 3 4 5 6 7 8 12 16 24 32 48 64; do echo $s; for r in {1..4}
 avg() { awk 'function pr(ncsz) {if (nsmp>0) {printf "%3s %s\n", csz, sum/nsmp} csz=$1;sum=0;nsmp=0} {sub(/\r$/,"")} /^[0-9]+$/ {pr($1);next} / MiB/ {sub(/ MiB.*/,"");sub(/.* /,"");sum+=$1;nsmp++} END {pr(0)}' "$1"; }
 
 
+##
+## time between first and last upload
+
+python3 -um copyparty -nw -v srv::rw -i 127.0.0.1 2>&1 | tee log 
+cat log | awk '!/"purl"/{next} {s=$1;sub(/[^m]+m/,"");gsub(/:/," ");t=60*(60*$1+$2)+$3} !a{a=t;sa=s} {b=t;sb=s} END {print b-a,sa,sb}'
+
+
 ##
 ## bad filenames
 

scripts/copyparty-repack.sh

@@ -10,14 +10,41 @@ set -e
 #  (and those are usually linux so bash is good inaff)
 #   (but that said this even has macos support)
 #
-# bundle will look like:
-# -rwxr-xr-x  0 ed ed  183808 Nov 19 00:43 copyparty
-# -rw-r--r--  0 ed ed  491318 Nov 19 00:40 copyparty-extras/copyparty-0.5.4.tar.gz
-# -rwxr-xr-x  0 ed ed   30254 Nov 17 23:58 copyparty-extras/copyparty-fuse.py
-# -rwxr-xr-x  0 ed ed  481403 Nov 19 00:40 copyparty-extras/sfx-full/copyparty-sfx.sh
-# -rwxr-xr-x  0 ed ed  506043 Nov 19 00:40 copyparty-extras/sfx-full/copyparty-sfx.py
-# -rwxr-xr-x  0 ed ed  167699 Nov 19 00:43 copyparty-extras/sfx-lite/copyparty-sfx.sh
-# -rwxr-xr-x  0 ed ed  183808 Nov 19 00:43 copyparty-extras/sfx-lite/copyparty-sfx.py
+# output summary (filesizes and contents):
+#
+# 535672  copyparty-extras/sfx-full/copyparty-sfx.sh
+# 550760  copyparty-extras/sfx-full/copyparty-sfx.py
+#           `- original unmodified sfx from github
+#
+# 572923  copyparty-extras/sfx-full/copyparty-sfx-gz.py
+#           `- unmodified but recompressed from bzip2 to gzip
+#
+# 341792  copyparty-extras/sfx-ent/copyparty-sfx.sh
+# 353975  copyparty-extras/sfx-ent/copyparty-sfx.py
+# 376934  copyparty-extras/sfx-ent/copyparty-sfx-gz.py
+#           `- removed iOS ogg/opus/vorbis audio decoder,
+#              removed the audio tray mouse cursor,
+#              "enterprise edition"
+#
+# 259288  copyparty-extras/sfx-lite/copyparty-sfx.sh
+# 270004  copyparty-extras/sfx-lite/copyparty-sfx.py
+# 293159  copyparty-extras/sfx-lite/copyparty-sfx-gz.py
+#           `- also removed the codemirror markdown editor
+#              and the text-viewer syntax hilighting,
+#              only essential features remaining
+#
+# 646297  copyparty-extras/copyparty-1.0.14.tar.gz
+#   4823  copyparty-extras/copyparty-repack.sh
+#           `- source files from github
+#
+#  23663  copyparty-extras/up2k.py
+#           `- standalone utility to upload or search for files
+#
+#  32280  copyparty-extras/copyparty-fuse.py
+#           `- standalone to mount a URL as a local read-only filesystem
+#
+# 270004  copyparty
+#           `- minimal binary, same as sfx-lite/copyparty-sfx.py
 
 
 command -v gnutar && tar() { gnutar "$@"; }
@@ -54,6 +81,7 @@ cache="$od/.copyparty-repack.cache"
 		# fallback to awk (sorry)
 		awk -F\" '/"browser_download_url".*(\.tar\.gz|-sfx\.)/ {print$4}'
 	) |
+	grep -E '(sfx\.(sh|py)|tar\.gz)$' |
 	tee /dev/stderr |
 	tr -d '\r' | tr '\n' '\0' |
 	xargs -0 bash -c 'dl_files "$@"' _
@@ -64,7 +92,7 @@ cache="$od/.copyparty-repack.cache"
 
 # move src into copyparty-extras/,
 # move sfx into copyparty-extras/sfx-full/
-mkdir -p copyparty-extras/sfx-{full,lite}
+mkdir -p copyparty-extras/sfx-{full,ent,lite}
 mv copyparty-sfx.* copyparty-extras/sfx-full/
 mv copyparty-*.tar.gz copyparty-extras/
 
@@ -112,14 +140,17 @@ repack() {
 }
 
 repack sfx-full "re gz no-sh"
-repack sfx-lite "re no-ogv no-cm"
-repack sfx-lite "re no-ogv no-cm gz no-sh"
+repack sfx-ent  "re no-dd no-ogv"
+repack sfx-ent  "re no-dd no-ogv gz no-sh"
+repack sfx-lite "re no-dd no-ogv no-cm no-hl"
+repack sfx-lite "re no-dd no-ogv no-cm no-hl gz no-sh"
 
 
-# move fuse client into copyparty-extras/,
+# move fuse and up2k clients into copyparty-extras/,
 # copy lite-sfx.py to ./copyparty,
 # delete extracted source code
 ( cd copyparty-extras/
+mv copyparty-*/bin/up2k.py .
 mv copyparty-*/bin/copyparty-fuse.py .
 cp -pv sfx-lite/copyparty-sfx.py ../copyparty
 rm -rf copyparty-{0..9}*.*.*{0..9}

scripts/deps-docker/Dockerfile

@@ -45,6 +45,12 @@ RUN     mkdir -p /z/dist/no-pk \
         && tar -xf zopfli.tgz
 
 
+# todo
+# https://cdn.jsdelivr.net/gh/highlightjs/cdn-release/build/highlight.min.js
+# https://cdn.jsdelivr.net/gh/highlightjs/cdn-release/build/styles/default.min.css
+# https://prismjs.com/download.html#themes=prism-funky&languages=markup+css+clike+javascript+autohotkey+bash+basic+batch+c+csharp+cpp+cmake+diff+docker+go+ini+java+json+kotlin+latex+less+lisp+lua+makefile+objectivec+perl+powershell+python+r+jsx+ruby+rust+sass+scss+sql+swift+systemd+toml+typescript+vbnet+verilog+vhdl+yaml&plugins=line-highlight+line-numbers+autolinker
+
+
 # build fonttools (which needs zopfli)
 RUN     tar -xf zopfli.tgz \
         && cd zopfli* \

scripts/make-sfx.sh

@@ -19,9 +19,11 @@ help() { exec cat <<'EOF'
 # `no-ogv` saves ~192k by removing the opus/vorbis audio codecs
 #   (only affects apple devices; everything else has native support)
 #
-# `no-cm` saves ~92k by removing easymde/codemirror
+# `no-cm` saves ~82k by removing easymde/codemirror
 #   (the fancy markdown editor)
 #
+# `no-hl` saves ~41k by removing syntax hilighting in the text viewer
+#
 # `no-fnt` saves ~9k by removing the source-code-pro font
 #   (browsers will try to use 'Consolas' instead)
 #
@@ -75,6 +77,7 @@ while [ ! -z "$1" ]; do
 		gz)     use_gz=1 ; ;;
 		no-ogv) no_ogv=1 ; ;;
 		no-fnt) no_fnt=1 ; ;;
+		no-hl)  no_hl=1  ; ;;
 		no-dd)  no_dd=1  ; ;;
 		no-cm)  no_cm=1  ; ;;
 		no-sh)  do_sh=   ; ;;
@@ -226,6 +229,9 @@ rm have
 	tmv "$f"
 }
 
+[ $no_hl ] &&
+	rm -rf copyparty/web/deps/prism*
+
 [ $no_fnt ] && {
 	rm -f copyparty/web/deps/scp.woff2
 	f=copyparty/web/ui.css

scripts/sfx.ls

@@ -57,6 +57,9 @@ copyparty/web/deps/ogv-demuxer-ogg-wasm.js,
 copyparty/web/deps/ogv-demuxer-ogg-wasm.wasm,
 copyparty/web/deps/ogv-worker-audio.js,
 copyparty/web/deps/ogv.js,
+copyparty/web/deps/prism.js,
+copyparty/web/deps/prism.css,
+copyparty/web/deps/prismd.css,
 copyparty/web/deps/scp.woff2,
 copyparty/web/deps/sha512.ac.js,
 copyparty/web/deps/sha512.hw.js,

tests/test_httpcli.py

@@ -47,12 +47,13 @@ class Cfg(Namespace):
             mtp=[],
             mte="a",
             mth="",
+            textfiles="",
             hist=None,
             no_idx=None,
             no_hash=None,
             js_browser=None,
             css_browser=None,
-            **{k: False for k in "e2d e2ds e2dsa e2t e2ts e2tsr".split()}
+            **{k: False for k in "e2d e2ds e2dsa e2t e2ts e2tsr no_acode".split()}
         )
 
 

tests/test_vfs.py

@@ -17,7 +17,8 @@ from copyparty import util
 
 class Cfg(Namespace):
     def __init__(self, a=None, v=None, c=None):
-        ex = {k: False for k in "nw e2d e2ds e2dsa e2t e2ts e2tsr".split()}
+        ex = "nw e2d e2ds e2dsa e2t e2ts e2tsr no_logues no_readme no_acode"
+        ex = {k: False for k in ex.split()}
         ex2 = {
             "mtp": [],
             "mte": "a",
@@ -28,8 +29,6 @@ class Cfg(Namespace):
             "js_browser": None,
             "css_browser": None,
             "no_voldump": True,
-            "no_logues": False,
-            "no_readme": False,
             "re_maxage": 0,
             "rproxy": 0,
         }

tests/util.py

@@ -113,6 +113,7 @@ class VSock(object):
 class VHttpSrv(object):
     def __init__(self):
         self.broker = NullBroker()
+        self.prism = None
 
         aliases = ["splash", "browser", "browser2", "msg", "md", "mde"]
         self.j2 = {x: J2_FILES for x in aliases}