v0.11.30

fix ogvjs paused-seek

README.md

@@ -20,6 +20,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
 
 * top
     * [quickstart](#quickstart)
+        * [on debian](#on-debian)
     * [notes](#notes)
     * [status](#status)
 * [bugs](#bugs)
@@ -68,6 +69,7 @@ some recommended options:
 * `-e2dsa` enables general file indexing, see [search configuration](#search-configuration)
 * `-e2ts` enables audio metadata indexing (needs either FFprobe or mutagen), see [optional dependencies](#optional-dependencies)
 * `-v /mnt/music:/music:r:afoo -a foo:bar` shares `/mnt/music` as `/music`, `r`eadable by anyone, with user `foo` as `a`dmin (read/write), password `bar`
+  * the syntax is `-v src:dst:perm:perm:...` so local-path, url-path, and one or more permissions to set
   * replace `:r:afoo` with `:rfoo` to only make the folder readable by `foo` and nobody else
   * in addition to `r`ead and `a`dmin, `w`rite makes a folder write-only, so cannot list/access files in it
 * `--ls '**,*,ln,p,r'` to crash on startup if any of the volumes contain a symlink which point outside the volume, as that could give users unintended access
@@ -77,6 +79,19 @@ you may also want these, especially on servers:
 * [contrib/nginx/copyparty.conf](contrib/nginx/copyparty.conf) to reverse-proxy behind nginx (for better https)
 
 
+### on debian
+
+recommended steps to enable audio metadata and thumbnails (from images and videos):
+
+* as root, run the following:  
+  `apt install python3 python3-pip python3-dev ffmpeg`
+
+* then, as the user which will be running copyparty (so hopefully not root), run this:  
+  `python3 -m pip install --user -U Pillow pillow-avif-plugin`
+
+(skipped `pyheif-pillow-opener` because apparently debian is too old to build it)
+
+
 ## notes
 
 general:
@@ -111,7 +126,7 @@ summary: all planned features work! now please enjoy the bloatening
   * ☑ FUSE client (read-only)
 * browser
   * ☑ tree-view
-  * ☑ audio player
+  * ☑ audio player (with OS media controls)
   * ☑ thumbnails
     * ☑ images using Pillow
     * ☑ videos using FFmpeg
@@ -139,6 +154,8 @@ summary: all planned features work! now please enjoy the bloatening
 
 * all volumes must exist / be available on startup; up2k (mtp especially) gets funky otherwise
 * cannot mount something at `/d1/d2/d3` unless `d2` exists inside `d1`
+* dupe files will not have metadata (audio tags etc) displayed in the file listing
+  * because they don't get `up` entries in the db (probably best fix) and `tx_browser` does not `lstat`
 * probably more, pls let me know
 
 ## not my bugs
@@ -168,25 +185,28 @@ summary: all planned features work! now please enjoy the bloatening
 ## hotkeys
 
 the browser has the following hotkeys
+* `B` toggle breadcrumbs / directory tree
 * `I/K` prev/next folder
-* `P` parent folder
+* `M` parent folder
 * `G` toggle list / grid view
 * `T` toggle thumbnails / icons
 * when playing audio:
   * `0..9` jump to 10%..90%
   * `U/O` skip 10sec back/forward
   * `J/L` prev/next song
-  * `M` play/pause (also starts playing the folder)
+  * `P` play/pause (also starts playing the folder)
+* when tree-sidebar is open:
+  * `A/D` adjust tree width
 * in the grid view:
   * `S` toggle multiselect
-  * `A/D` zoom
+  * shift+`A/D` zoom
 
 
 ## tree-mode
 
-by default there's a breadcrumbs path; you can replace this with a tree-browser sidebar thing by clicking the 🌲
+by default there's a breadcrumbs path; you can replace this with a tree-browser sidebar thing by clicking the `🌲` or pressing the `B` hotkey
 
-click `[-]` and `[+]` to adjust the size, and the `[a]` toggles if the tree should widen dynamically as you go deeper or stay fixed-size
+click `[-]` and `[+]` (or hotkeys `A`/`D`) to adjust the size, and the `[a]` toggles if the tree should widen dynamically as you go deeper or stay fixed-size
 
 
 ## thumbnails
@@ -197,6 +217,8 @@ it does static images with Pillow and uses FFmpeg for video files, so you may wa
 
 images named `folder.jpg` and `folder.png` become the thumbnail of the folder they're in
 
+in the grid/thumbnail view, if the audio player panel is open, songs will start playing when clicked
+
 
 ## zip downloads
 
@@ -280,6 +302,8 @@ up2k has saved a few uploads from becoming corrupted in-transfer already; caught
 
 * you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
 
+* if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
+
 
 # searching
 
@@ -591,13 +615,14 @@ in the `scripts` folder:
 roughly sorted by priority
 
 * readme.md as epilogue
-* single sha512 across all up2k chunks? maybe
 * reduce up2k roundtrips
   * start from a chunk index and just go
   * terminate client on bad data
 
 discarded ideas
 
+* single sha512 across all up2k chunks?
+  * crypto.subtle cannot into streaming, would have to use hashwasm, expensive
 * separate sqlite table per tag
   * performance fixed by skipping some indexes (`+mt.k`)
 * audio fingerprinting

bin/README.md

@@ -48,15 +48,16 @@ you could replace winfsp with [dokan](https://github.com/dokan-dev/dokany/releas
 
 
 # [`dbtool.py`](dbtool.py)
-upgrade utility which can show db info and help transfer data between databases, for example when a new version of copyparty recommends to wipe the DB and reindex because it now collects additional metadata during analysis, but you have some really expensive `-mtp` parsers and want to copy over the tags from the old db
+upgrade utility which can show db info and help transfer data between databases, for example when a new version of copyparty is incompatible with the old DB and automatically rebuilds the DB from scratch, but you have some really expensive `-mtp` parsers and want to copy over the tags from the old db
 
-for that example (upgrading to v0.11.0), first move the old db aside, launch copyparty, let it rebuild the db until the point where it starts running mtp (colored messages as it adds the mtp tags), then CTRL-C and patch in the old mtp tags from the old db instead
+for that example (upgrading to v0.11.20), first launch the new version of copyparty like usual, let it make a backup of the old db and rebuild the new db until the point where it starts running mtp (colored messages as it adds the mtp tags), that's when you hit CTRL-C and patch in the old mtp tags from the old db instead
 
 so assuming you have `-mtp` parsers to provide the tags `key` and `.bpm`:
 
 ```
-~/bin/dbtool.py -ls up2k.db
-~/bin/dbtool.py -src up2k.db.v0.10.22 up2k.db -cmp
-~/bin/dbtool.py -src up2k.db.v0.10.22 up2k.db -rm-mtp-flag -copy key
-~/bin/dbtool.py -src up2k.db.v0.10.22 up2k.db -rm-mtp-flag -copy .bpm -vac
+cd /mnt/nas/music/.hist
+~/src/copyparty/bin/dbtool.py -ls up2k.db
+~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -cmp
+~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy key
+~/src/copyparty/bin/dbtool.py -src up2k.*.v3 up2k.db -rm-mtp-flag -copy .bpm -vac
 ```

bin/mtag/audio-bpm.py

@@ -60,7 +60,7 @@ def main():
     try:
         det(tf)
     except:
-        pass
+        pass  # mute
     finally:
         os.unlink(tf)
 

bin/mtag/audio-key-slicing.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python
+
+import re
+import os
+import sys
+import tempfile
+import subprocess as sp
+
+import keyfinder
+
+from copyparty.util import fsenc
+
+"""
+dep: github/mixxxdj/libkeyfinder
+dep: pypi/keyfinder
+dep: ffmpeg
+
+note: this is a janky edition of the regular audio-key.py,
+  slicing the files at 20sec intervals and keeping 5sec from each,
+  surprisingly accurate but still garbage (446 ok, 69 bad, 13% miss)
+
+  it is fast tho
+"""
+
+
+def get_duration():
+    # TODO provide ffprobe tags to mtp as json
+
+    # fmt: off
+    dur = sp.check_output([
+        "ffprobe",
+        "-hide_banner",
+        "-v", "fatal",
+        "-show_streams",
+        "-show_format",
+        fsenc(sys.argv[1])
+    ])
+    # fmt: on
+
+    dur = dur.decode("ascii", "replace").split("\n")
+    dur = [x.split("=")[1] for x in dur if x.startswith("duration=")]
+    dur = [float(x) for x in dur if re.match(r"^[0-9\.,]+$", x)]
+    return list(sorted(dur))[-1] if dur else None
+
+
+def get_segs(dur):
+    # keep first 5s of each 20s,
+    # keep entire last segment
+    ofs = 0
+    segs = []
+    while True:
+        seg = [ofs, 5]
+        segs.append(seg)
+        if dur - ofs < 20:
+            seg[-1] = int(dur - seg[0])
+            break
+
+        ofs += 20
+
+    return segs
+
+
+def slice(tf):
+    dur = get_duration()
+    dur = min(dur, 600)  # max 10min
+    segs = get_segs(dur)
+
+    # fmt: off
+    cmd = [
+        "ffmpeg",
+        "-nostdin",
+        "-hide_banner",
+        "-v", "fatal",
+        "-y"
+    ]
+
+    for seg in segs:
+        cmd.extend([
+            "-ss", str(seg[0]),
+            "-i", fsenc(sys.argv[1])
+        ])
+    
+    filt = ""
+    for n, seg in enumerate(segs):
+        filt += "[{}:a:0]atrim=duration={}[a{}]; ".format(n, seg[1], n)
+    
+    prev = "a0"
+    for n in range(1, len(segs)):
+        nxt = "b{}".format(n)
+        filt += "[{}][a{}]acrossfade=d=0.5[{}]; ".format(prev, n, nxt)
+        prev = nxt
+
+    cmd.extend([
+        "-filter_complex", filt[:-2],
+        "-map", "[{}]".format(nxt),
+        "-sample_fmt", "s16",
+        tf
+    ])
+    # fmt: on
+
+    # print(cmd)
+    sp.check_call(cmd)
+
+
+def det(tf):
+    slice(tf)
+    print(keyfinder.key(tf).camelot())
+
+
+def main():
+    with tempfile.NamedTemporaryFile(suffix=".flac", delete=False) as f:
+        f.write(b"h")
+        tf = f.name
+
+    try:
+        det(tf)
+    finally:
+        os.unlink(tf)
+        pass
+
+
+if __name__ == "__main__":
+    main()

bin/mtag/audio-key.py

@@ -1,18 +1,54 @@
 #!/usr/bin/env python
 
+import os
 import sys
+import tempfile
+import subprocess as sp
 import keyfinder
 
+from copyparty.util import fsenc
+
 """
 dep: github/mixxxdj/libkeyfinder
 dep: pypi/keyfinder
 dep: ffmpeg
-
-note: cannot fsenc
 """
 
 
-try:
-    print(keyfinder.key(sys.argv[1]).camelot())
-except:
-    pass
+# tried trimming the first/last 5th, bad idea,
+# misdetects 9a law field (Sphere Caliber) as 10b,
+# obvious when mixing 9a ghostly parapara ship
+
+
+def det(tf):
+    # fmt: off
+    sp.check_call([
+        "ffmpeg",
+        "-nostdin",
+        "-hide_banner",
+        "-v", "fatal",
+        "-y", "-i", fsenc(sys.argv[1]),
+        "-t", "300",
+        "-sample_fmt", "s16",
+        tf
+    ])
+    # fmt: on
+
+    print(keyfinder.key(tf).camelot())
+
+
+def main():
+    with tempfile.NamedTemporaryFile(suffix=".flac", delete=False) as f:
+        f.write(b"h")
+        tf = f.name
+
+    try:
+        det(tf)
+    except:
+        pass  # mute
+    finally:
+        os.unlink(tf)
+
+
+if __name__ == "__main__":
+    main()

copyparty/__main__.py

@@ -305,6 +305,7 @@ def run_argparse(argv, formatter):
     ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown")
     ap2.add_argument("--th-clean", metavar="SEC", type=int, default=43200, help="cleanup interval")
     ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age")
+    ap2.add_argument("--th-covers", metavar="N,N", type=str, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat for")
 
     ap2 = ap.add_argument_group('database options')
     ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
@@ -410,7 +411,7 @@ def main(argv=None):
             + "  (if you crash with codec errors then that is why)"
         )
 
-    if WINDOWS and sys.version_info < (3, 6):
+    if sys.version_info < (3, 6):
         al.no_scandir = True
 
     # signal.signal(signal.SIGINT, sighandler)

copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (0, 11, 20)
+VERSION = (0, 11, 30)
 CODENAME = "the grid"
-BUILD_DT = (2021, 6, 20)
+BUILD_DT = (2021, 7, 1)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

copyparty/authsrv.py

@@ -693,6 +693,11 @@ class AuthSrv(object):
             self.user = user
             self.iuser = {v: k for k, v in user.items()}
 
+            self.re_pwd = None
+            pwds = [re.escape(x) for x in self.iuser.keys()]
+            if pwds:
+                self.re_pwd = re.compile("=(" + "|".join(pwds) + ")([]&; ]|$)")
+
         # import pprint
         # pprint.pprint({"usr": user, "rd": mread, "wr": mwrite, "mnt": mount})
 

copyparty/httpcli.py

@@ -10,7 +10,6 @@ import json
 import string
 import socket
 import ctypes
-import traceback
 from datetime import datetime
 import calendar
 
@@ -50,12 +49,21 @@ class HttpCli(object):
         self.tls = hasattr(self.s, "cipher")
 
         self.bufsz = 1024 * 32
+        self.hint = None
         self.absolute_urls = False
         self.out_headers = {"Access-Control-Allow-Origin": "*"}
 
     def log(self, msg, c=0):
+        ptn = self.asrv.re_pwd
+        if ptn and ptn.search(msg):
+            msg = ptn.sub(self.unpwd, msg)
+
         self.log_func(self.log_src, msg, c)
 
+    def unpwd(self, m):
+        a, b = m.groups()
+        return "=\033[7m {} \033[27m{}".format(self.asrv.iuser[a], b)
+
     def _check_nonfatal(self, ex):
         return ex.code < 400 or ex.code in [404, 429]
 
@@ -64,14 +72,19 @@ class HttpCli(object):
         if rem.startswith("/") or rem.startswith("../") or "/../" in rem:
             raise Exception("that was close")
 
-    def j2(self, name, **kwargs):
+    def j2(self, name, **ka):
         tpl = self.conn.hsrv.j2[name]
-        return tpl.render(**kwargs) if kwargs else tpl
+        if ka:
+            ka["ts"] = self.conn.hsrv.cachebuster()
+            return tpl.render(**ka)
+
+        return tpl
 
     def run(self):
         """returns true if connection can be reused"""
         self.keepalive = False
         self.headers = {}
+        self.hint = None
         try:
             headerlines = read_header(self.sr)
             if not headerlines:
@@ -130,6 +143,9 @@ class HttpCli(object):
                 if v is not None:
                     self.log("[H] {}: \033[33m[{}]".format(k, v), 6)
 
+        if "&" in self.req and "?" not in self.req:
+            self.hint = "did you mean '?' instead of '&'"
+
         # split req into vpath + uparam
         uparam = {}
         if "?" not in self.req:
@@ -169,6 +185,9 @@ class HttpCli(object):
         self.rvol, self.wvol, self.avol = [[], [], []]
         self.asrv.vfs.user_tree(self.uname, self.rvol, self.wvol, self.avol)
 
+        if pwd and "pw" in self.ouparam and pwd != cookies.get("cppwd"):
+            self.out_headers["Set-Cookie"] = self.get_pwd_cookie(pwd)[0]
+
         ua = self.headers.get("user-agent", "")
         self.is_rclone = ua.startswith("rclone/")
         if self.is_rclone:
@@ -199,6 +218,9 @@ class HttpCli(object):
 
                 self.log("{}\033[0m, {}".format(str(ex), self.vpath), 3)
                 msg = "<pre>{}\r\nURL: {}\r\n".format(str(ex), self.vpath)
+                if self.hint:
+                    msg += "hint: {}\r\n".format(self.hint)
+
                 self.reply(msg.encode("utf-8", "replace"), status=ex.code)
                 return self.keepalive
             except Pebkac:
@@ -481,7 +503,7 @@ class HttpCli(object):
 
         spd1 = get_spd(nbytes, self.t0)
         spd2 = get_spd(self.conn.nbyte, self.conn.t0)
-        return spd1 + " " + spd2
+        return "{} {} n{}".format(spd1, spd2, self.conn.nreq)
 
     def handle_post_multipart(self):
         self.parser = MultipartParser(self.log, self.sr, self.headers)
@@ -623,7 +645,7 @@ class HttpCli(object):
             penalty = 0.7
             t_idle = t0 - idx.p_end
             if idx.p_dur > 0.7 and t_idle < penalty:
-                m = "rate-limit ({:.1f} sec), cost {:.2f}, idle {:.2f}"
+                m = "rate-limit {:.1f} sec, cost {:.2f}, idle {:.2f}"
                 raise Pebkac(429, m.format(penalty, idx.p_dur, t_idle))
 
         if "srch" in body:
@@ -743,6 +765,12 @@ class HttpCli(object):
         pwd = self.parser.require("cppwd", 64)
         self.parser.drop()
 
+        ck, msg = self.get_pwd_cookie(pwd)
+        html = self.j2("msg", h1=msg, h2='<a href="/">ack</a>', redir="/")
+        self.reply(html.encode("utf-8"), headers={"Set-Cookie": ck})
+        return True
+
+    def get_pwd_cookie(self, pwd):
         if pwd in self.asrv.iuser:
             msg = "login ok"
             dt = datetime.utcfromtimestamp(time.time() + 60 * 60 * 24 * 365)
@@ -753,9 +781,7 @@ class HttpCli(object):
             exp = "Fri, 15 Aug 1997 01:00:00 GMT"
 
         ck = "cppwd={}; Path=/; Expires={}; SameSite=Lax".format(pwd, exp)
-        html = self.j2("msg", h1=msg, h2='<a href="/">ack</a>', redir="/")
-        self.reply(html.encode("utf-8"), headers={"Set-Cookie": ck})
-        return True
+        return [ck, msg]
 
     def handle_mkdir(self):
         new_dir = self.parser.require("name", 512)
@@ -1305,7 +1331,7 @@ class HttpCli(object):
             ext = "folder"
             exact = True
 
-        bad = re.compile(r"[](){}/[]|^[0-9_-]*$")
+        bad = re.compile(r"[](){}/ []|^[0-9_-]*$")
         n = ext.split(".")[::-1]
         if not exact:
             n = n[:-1]
@@ -1361,6 +1387,7 @@ class HttpCli(object):
             "md_plug": "true" if self.args.emp else "false",
             "md_chk_rate": self.args.mcr,
             "md": boundary,
+            "ts": self.conn.hsrv.cachebuster(),
         }
         html = template.render(**targs).encode("utf-8", "replace")
         html = html.split(boundary.encode("utf-8"))
@@ -1540,7 +1567,7 @@ class HttpCli(object):
             th_fmt = self.uparam.get("th")
             if th_fmt is not None:
                 if is_dir:
-                    for fn in ["folder.png", "folder.jpg"]:
+                    for fn in self.args.th_covers.split(","):
                         fp = os.path.join(abspath, fn)
                         if os.path.exists(fp):
                             vrem = "{}/{}".format(vrem.rstrip("/"), fn)
@@ -1604,7 +1631,6 @@ class HttpCli(object):
 
         url_suf = self.urlq()
         is_ls = "ls" in self.uparam
-        ts = ""  # "?{}".format(time.time())
 
         tpl = "browser"
         if "b" in self.uparam:
@@ -1629,7 +1655,6 @@ class HttpCli(object):
             "vdir": quotep(self.vpath),
             "vpnodes": vpnodes,
             "files": [],
-            "ts": ts,
             "perms": json.dumps(perms),
             "taglist": [],
             "tag_order": [],
@@ -1765,28 +1790,44 @@ class HttpCli(object):
             fn = f["name"]
             rd = f["rd"]
             del f["rd"]
-            if icur:
-                if vn != dbv:
-                    _, rd = vn.get_dbv(rd)
+            if not icur:
+                break
+
+            if vn != dbv:
+                _, rd = vn.get_dbv(rd)
+
+            q = "select w from up where rd = ? and fn = ?"
+            r = None
+            try:
+                r = icur.execute(q, (rd, fn)).fetchone()
+            except Exception as ex:
+                if "database is locked" in str(ex):
+                    break
 
-                q = "select w from up where rd = ? and fn = ?"
                 try:
-                    r = icur.execute(q, (rd, fn)).fetchone()
-                except:
                     args = s3enc(idx.mem_cur, rd, fn)
                     r = icur.execute(q, args).fetchone()
+                except:
+                    m = "tag list error, {}/{}\n{}"
+                    self.log(m.format(rd, fn, min_ex()))
+                    break
 
-                tags = {}
-                f["tags"] = tags
+            tags = {}
+            f["tags"] = tags
 
-                if not r:
-                    continue
+            if not r:
+                continue
 
-                w = r[0][:16]
-                q = "select k, v from mt where w = ? and k != 'x'"
+            w = r[0][:16]
+            q = "select k, v from mt where w = ? and k != 'x'"
+            try:
                 for k, v in icur.execute(q, (w,)):
                     taglist[k] = True
                     tags[k] = v
+            except:
+                m = "tag read error, {}/{} [{}]:\n{}"
+                self.log(m.format(rd, fn, w, min_ex()))
+                break
 
         if icur:
             taglist = [k for k in vn.flags.get("mte", "").split(",") if k in taglist]

copyparty/httpconn.py

@@ -3,7 +3,6 @@ from __future__ import print_function, unicode_literals
 
 import re
 import os
-import sys
 import time
 import socket
 
@@ -44,6 +43,7 @@ class HttpConn(object):
 
         self.t0 = time.time()
         self.stopping = False
+        self.nreq = 0
         self.nbyte = 0
         self.workload = 0
         self.u2idx = None
@@ -189,6 +189,7 @@ class HttpConn(object):
                 if self.workload >= 2 ** 31:
                     self.workload = 100
 
+            self.nreq += 1
             cli = HttpCli(self)
             if not cli.run():
                 return

copyparty/httpsrv.py

@@ -4,6 +4,8 @@ from __future__ import print_function, unicode_literals
 import os
 import sys
 import time
+import base64
+import struct
 import socket
 import threading
 
@@ -25,7 +27,6 @@ except ImportError:
     sys.exit(1)
 
 from .__init__ import E, MACOS
-from .authsrv import AuthSrv
 from .httpconn import HttpConn
 
 
@@ -48,6 +49,8 @@ class HttpSrv(object):
         self.clients = {}
         self.workload = 0
         self.workload_thr_alive = False
+        self.cb_ts = 0
+        self.cb_v = 0
 
         env = jinja2.Environment()
         env.loader = jinja2.FileSystemLoader(os.path.join(E.mod, "web"))
@@ -138,11 +141,12 @@ class HttpSrv(object):
                         "shut({}): {}".format(fno, ex),
                         c="1;30",
                     )
-                if ex.errno not in [10038, 10054, 107, 57, 9]:
+                if ex.errno not in [10038, 10054, 107, 57, 49, 9]:
                     # 10038 No longer considered a socket
                     # 10054 Foribly closed by remote
                     #   107 Transport endpoint not connected
                     #    57 Socket is not connected
+                    #    49 Can't assign requested address (wifi down)
                     #     9 Bad file descriptor
                     raise
             finally:
@@ -177,3 +181,25 @@ class HttpSrv(object):
                     self.clients[cli] = now
 
             self.workload = total
+
+    def cachebuster(self):
+        if time.time() - self.cb_ts < 1:
+            return self.cb_v
+
+        with self.mutex:
+            if time.time() - self.cb_ts < 1:
+                return self.cb_v
+
+            v = E.t0
+            try:
+                with os.scandir(os.path.join(E.mod, "web")) as dh:
+                    for fh in dh:
+                        inf = fh.stat(follow_symlinks=False)
+                        v = max(v, inf.st_mtime)
+            except:
+                pass
+
+            v = base64.urlsafe_b64encode(struct.pack(">xxL", int(v)))
+            self.cb_v = v.decode("ascii")[-4:]
+            self.cb_ts = time.time()
+            return self.cb_v

copyparty/mtag.py

@@ -115,6 +115,19 @@ def parse_ffprobe(txt):
     ret = {}  # processed
     md = {}  # raw tags
 
+    is_audio = fmt.get("format_name") in ["mp3", "ogg", "flac", "wav"]
+    if fmt.get("filename", "").split(".")[-1].lower() in ["m4a", "aac"]:
+        is_audio = True
+
+    # if audio file, ensure audio stream appears first
+    if (
+        is_audio
+        and len(streams) > 2
+        and streams[1].get("codec_type") != "audio"
+        and streams[2].get("codec_type") == "audio"
+    ):
+        streams = [fmt, streams[2], streams[1]] + streams[3:]
+
     have = {}
     for strm in streams:
         typ = strm.get("codec_type")
@@ -134,9 +147,7 @@ def parse_ffprobe(txt):
             ]
 
         if typ == "video":
-            if strm.get("DISPOSITION:attached_pic") == "1" or fmt.get(
-                "format_name"
-            ) in ["mp3", "ogg", "flac"]:
+            if strm.get("DISPOSITION:attached_pic") == "1" or is_audio:
                 continue
 
             kvm = [
@@ -180,7 +191,7 @@ def parse_ffprobe(txt):
 
             k = k[4:].strip()
             v = v.strip()
-            if k and v:
+            if k and v and k not in md:
                 md[k] = [v]
 
     for k in [".q", ".vq", ".aq"]:

copyparty/u2idx.py

@@ -26,7 +26,7 @@ class U2idx(object):
         self.timeout = self.args.srch_time
 
         if not HAVE_SQLITE3:
-            self.log("could not load sqlite3; searchign wqill be disabled")
+            self.log("your python does not have sqlite3; searching will be disabled")
             return
 
         self.cur = {}
@@ -57,6 +57,9 @@ class U2idx(object):
             raise Pebkac(500, min_ex())
 
     def get_cur(self, ptop):
+        if not HAVE_SQLITE3:
+            return None
+
         cur = self.cur.get(ptop)
         if cur:
             return cur
@@ -66,7 +69,7 @@ class U2idx(object):
         if not os.path.exists(db_path):
             return None
 
-        cur = sqlite3.connect(db_path).cursor()
+        cur = sqlite3.connect(db_path, 2).cursor()
         self.cur[ptop] = cur
         return cur
 

copyparty/up2k.py

@@ -653,7 +653,7 @@ class Up2k(object):
             try:
                 parser = MParser(parser)
             except:
-                self.log("invalid argument: " + parser, 1)
+                self.log("invalid argument (could not find program): " + parser, 1)
                 return
 
             for tag in entags:
@@ -901,7 +901,7 @@ class Up2k(object):
             except:
                 self.log("WARN: could not list files; DB corrupt?\n" + min_ex())
 
-        elif ver > DB_VER:
+        if (ver or 0) > DB_VER:
             m = "database is version {}, this copyparty only supports versions <= {}"
             raise Exception(m.format(ver, DB_VER))
 
@@ -1019,7 +1019,8 @@ class Up2k(object):
                                 break
                         except:
                             # missing; restart
-                            job = None
+                            if not self.args.nw:
+                                job = None
                             break
                 else:
                     # file contents match, but not the path
@@ -1089,6 +1090,9 @@ class Up2k(object):
             }
 
     def _untaken(self, fdir, fname, ts, ip):
+        if self.args.nw:
+            return fname
+
         # TODO broker which avoid this race and
         # provides a new filename if taken (same as bup)
         suffix = ".{:.6f}-{}".format(ts, ip)
@@ -1098,6 +1102,9 @@ class Up2k(object):
     def _symlink(self, src, dst):
         # TODO store this in linktab so we never delete src if there are links to it
         self.log("linking dupe:\n  {0}\n  {1}".format(src, dst))
+        if self.args.nw:
+            return
+
         try:
             lsrc = src
             ldst = dst
@@ -1175,6 +1182,10 @@ class Up2k(object):
             if ret > 0:
                 return ret, src
 
+            if self.args.nw:
+                # del self.registry[ptop][wark]
+                return ret, dst
+
             atomic_move(src, dst)
 
             if ANYWIN:
@@ -1284,6 +1295,10 @@ class Up2k(object):
         if self.args.dotpart:
             tnam = "." + tnam
 
+        if self.args.nw:
+            job["tnam"] = tnam
+            return
+
         suffix = ".{:.6f}-{}".format(job["t0"], job["addr"])
         with ren_open(tnam, "wb", fdir=pdir, suffix=suffix) as f:
             f, job["tnam"] = f["orz"]

copyparty/util.py

@@ -1030,7 +1030,13 @@ def guess_mime(url, fallback="application/octet-stream"):
     except:
         return fallback
 
-    return MIMES.get(ext) or mimetypes.guess_type(url)[0] or fallback
+    ret = MIMES.get(ext) or mimetypes.guess_type(url)[0] or fallback
+
+    if ";" not in ret:
+        if ret.startswith("text/") or ret.endswith("/javascript"):
+            ret += "; charset=UTF-8"
+    
+    return ret
 
 
 def runcmd(*argv):

copyparty/web/baguettebox.js

@@ -119,7 +119,7 @@ window.baguetteBox = (function () {
             var gallery = [];
             [].forEach.call(tagsNodeList, function (imageElement, imageIndex) {
                 var imageElementClickHandler = function (event) {
-                    if (event && event.ctrlKey)
+                    if (event && (event.ctrlKey || event.metaKey))
                         return true;
 
                     event.preventDefault ? event.preventDefault() : event.returnValue = false;

copyparty/web/browser.css

@@ -607,7 +607,7 @@ input.eq_gain {
 #srch_q {
 	white-space: pre;
 	color: #f80;
-	height: 1em;
+	min-height: 1em;
 	margin: .2em 0 -1em 1.6em;
 }
 #tq_raw {
@@ -811,10 +811,12 @@ input.eq_gain {
 	padding: 0;
 	border-bottom: 1px solid #555;
 }
-#thumbs {
+#thumbs,
+#au_osd_cv {
 	opacity: .3;
 }
-#griden.on+#thumbs {
+#griden.on+#thumbs,
+#au_os_ctl.on+#au_osd_cv {
 	opacity: 1;
 }
 #ghead {

copyparty/web/browser.html

@@ -6,10 +6,10 @@
 	<title>⇆🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.8">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/browser.css{{ ts }}">
-	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/upload.css{{ ts }}">
+	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/browser.css?_={{ ts }}">
+	<link rel="stylesheet" type="text/css" media="screen" href="/.cpr/upload.css?_={{ ts }}">
 	{%- if css %}
-	<link rel="stylesheet" type="text/css" media="screen" href="{{ css }}{{ ts }}">
+	<link rel="stylesheet" type="text/css" media="screen" href="{{ css }}?_={{ ts }}">
 	{%- endif %}
 </head>
 
@@ -110,7 +110,7 @@
 	
 	<div id="epi" class="logue">{{ logues[1] }}</div>
 
-	<h2><a href="?h">control-panel</a></h2>
+	<h2><a href="/?h">control-panel</a></h2>
 
 </div>
 
@@ -127,9 +127,9 @@
 			have_tags_idx = {{ have_tags_idx|tojson }},
 			have_zip = {{ have_zip|tojson }};
 	</script>
-	<script src="/.cpr/util.js{{ ts }}"></script>
-	<script src="/.cpr/browser.js{{ ts }}"></script>
-	<script src="/.cpr/up2k.js{{ ts }}"></script>
+	<script src="/.cpr/util.js?_={{ ts }}"></script>
+	<script src="/.cpr/browser.js?_={{ ts }}"></script>
+	<script src="/.cpr/up2k.js?_={{ ts }}"></script>
 </body>
 
 </html>

copyparty/web/browser.js

@@ -78,7 +78,7 @@ ebi('op_up2k').innerHTML = (
 	'	<tr>\n' +
 	'		<td>\n' +
 	'			<a href="#" id="nthread_sub">&ndash;</a><input\n' +
-	'				class="txtbox" id="nthread" value="2"/><a\n' +
+	'				class="txtbox" id="nthread" value="2" tt="pause uploads by setting it to 0"/><a\n' +
 	'				href="#" id="nthread_add">+</a><br />&nbsp;\n' +
 	'		</td>\n' +
 	'	</tr>\n' +
@@ -222,10 +222,14 @@ var have_webp = null;
 
 
 var mpl = (function () {
+	var have_mctl = 'mediaSession' in navigator && window.MediaMetadata;
+
 	ebi('op_player').innerHTML = (
 		'<div><h3>switches</h3><div>' +
 		'<a href="#" class="tgl btn" id="au_preload" tt="start loading the next song near the end for gapless playback">preload</a>' +
 		'<a href="#" class="tgl btn" id="au_npclip" tt="show buttons for clipboarding the currently playing song">/np clip</a>' +
+		'<a href="#" class="tgl btn" id="au_os_ctl" tt="os integration (media hotkeys / osd)">os-ctl</a>' +
+		'<a href="#" class="tgl btn" id="au_osd_cv" tt="show album cover in osd">osd-cv</a>' +
 		'</div></div>' +
 
 		'<div><h3>playback mode</h3><div id="pb_mode">' +
@@ -233,12 +237,18 @@ var mpl = (function () {
 		'<a href="#" class="tgl btn" tt="load the next folder and continue">📂 next-folder</a>' +
 		'</div></div>' +
 
+		'<div><h3>tint</h3><div>' +
+		'<input type="text" id="pb_tint" size="3" value="0" tt="background level (0-100) on the seekbar$Nto make buffering less distracting" />' +
+		'</div></div>' +
+
 		'<div><h3>audio equalizer</h3><div id="audio_eq"></div></div>');
 
 	var r = {
 		"pb_mode": sread('pb_mode') || 'loop-folder',
 		"preload": bcfg_get('au_preload', true),
-		"clip": bcfg_get('au_npclip', false)
+		"clip": bcfg_get('au_npclip', false),
+		"os_ctl": bcfg_get('au_os_ctl', have_mctl) && have_mctl,
+		"osd_cv": bcfg_get('au_osd_cv', true),
 	};
 
 	ebi('au_preload').onclick = function (e) {
@@ -254,6 +264,20 @@ var mpl = (function () {
 		clmod(ebi('wtoggle'), 'np', r.clip && mp.au);
 	};
 
+	ebi('au_os_ctl').onclick = function (e) {
+		ev(e);
+		r.os_ctl = !r.os_ctl && have_mctl;
+		bcfg_set('au_os_ctl', r.os_ctl);
+		if (!have_mctl)
+			alert('need firefox 82+ or chrome 73+');
+	};
+
+	ebi('au_osd_cv').onclick = function (e) {
+		ev(e);
+		r.osd_cv = !r.osd_cv;
+		bcfg_set('au_osd_cv', r.osd_cv);
+	};
+
 	function draw_pb_mode() {
 		var btns = QSA('#pb_mode>a');
 		for (var a = 0, aa = btns.length; a < aa; a++) {
@@ -270,20 +294,98 @@ var mpl = (function () {
 		draw_pb_mode();
 	}
 
+	function set_tint() {
+		var tint = icfg_get('pb_tint', 0);
+		if (!tint)
+			ebi('barbuf').style.removeProperty('background');
+		else
+			ebi('barbuf').style.background = 'rgba(126,163,75,' + (tint / 100.0) + ')';
+	}
+	ebi('pb_tint').oninput = function (e) {
+		swrite('pb_tint', this.value);
+		set_tint();
+	};
+	set_tint();
+
+	r.pp = function () {
+		if (!r.os_ctl)
+			return;
+
+		navigator.mediaSession.playbackState = mp.au && !mp.au.paused ? "playing" : "paused";
+	};
+
+	r.announce = function () {
+		if (!r.os_ctl)
+			return;
+
+		var np = get_np()[0],
+			fns = np.file.split(' - '),
+			artist = (np.circle ? np.circle + ' // ' : '') + (np.artist || (fns.length > 1 ? fns[0] : '')),
+			tags = {
+				title: np.title || fns.slice(-1)[0]
+			};
+
+		if (artist)
+			tags.artist = artist;
+
+		if (np.album)
+			tags.album = np.album;
+
+		if (r.osd_cv) {
+			var files = QSA("#files tr>td:nth-child(2)>a[id]"),
+				cover = null;
+
+			for (var a = 0, aa = files.length; a < aa; a++) {
+				if (/^(cover|folder)\.(jpe?g|png|gif)$/.test(files[a].textContent)) {
+					cover = files[a].getAttribute('href');
+					break;
+				}
+			}
+
+			if (cover) {
+				cover += (cover.indexOf('?') === -1 ? '?' : '&') + 'th=j';
+
+				var pwd = get_pwd();
+				if (pwd)
+					cover += '&pw=' + uricom_enc(pwd);
+
+				tags.artwork = [{ "src": cover, type: "image/jpeg" }];
+			}
+		}
+
+		navigator.mediaSession.metadata = new MediaMetadata(tags);
+		navigator.mediaSession.setActionHandler('play', playpause);
+		navigator.mediaSession.setActionHandler('pause', playpause);
+		navigator.mediaSession.setActionHandler('seekbackward', function () { seek_au_rel(-10); });
+		navigator.mediaSession.setActionHandler('seekforward', function () { seek_au_rel(10); });
+		navigator.mediaSession.setActionHandler('previoustrack', prev_song);
+		navigator.mediaSession.setActionHandler('nexttrack', next_song);
+		r.pp();
+	};
+
+	r.stop = function () {
+		if (!r.os_ctl || !navigator.mediaSession.metadata)
+			return;
+
+		navigator.mediaSession.metadata = null;
+		navigator.mediaSession.playbackState = "paused";
+	};
+
 	return r;
 })();
 
 
 // extract songs + add play column
 function MPlayer() {
-	this.id = Date.now();
-	this.au = null;
-	this.au_native = null;
-	this.au_native2 = null;
-	this.au_ogvjs = null;
-	this.au_ogvjs2 = null;
-	this.tracks = {};
-	this.order = [];
+	var r = this;
+	r.id = Date.now();
+	r.au = null;
+	r.au_native = null;
+	r.au_native2 = null;
+	r.au_ogvjs = null;
+	r.au_ogvjs2 = null;
+	r.tracks = {};
+	r.order = [];
 
 	var re_audio = /\.(opus|ogg|m4a|aac|mp3|wav|flac)$/i,
 		trs = QSA('#files tbody tr');
@@ -298,32 +400,33 @@ function MPlayer() {
 
 		if (m) {
 			var tid = link.getAttribute('id');
-			this.order.push(tid);
-			this.tracks[tid] = url;
+			r.order.push(tid);
+			r.tracks[tid] = url;
 			tds[0].innerHTML = '<a id="a' + tid + '" href="#a' + tid + '" class="play">play</a></td>';
 			ebi('a' + tid).onclick = ev_play;
 		}
 	}
 
-	this.vol = sread('vol');
-	if (this.vol !== null)
-		this.vol = parseFloat(this.vol);
+	r.vol = sread('vol');
+	if (r.vol !== null)
+		r.vol = parseFloat(r.vol);
 	else
-		this.vol = 0.5;
+		r.vol = 0.5;
 
-	this.expvol = function () {
-		return 0.5 * this.vol + 0.5 * this.vol * this.vol;
+	r.expvol = function (v) {
+		return 0.5 * v + 0.5 * v * v;
 	};
 
-	this.setvol = function (vol) {
-		this.vol = Math.max(Math.min(vol, 1), 0);
+	r.setvol = function (vol) {
+		r.vol = Math.max(Math.min(vol, 1), 0);
 		swrite('vol', vol);
+		r.stopfade(true);
 
-		if (this.au)
-			this.au.volume = this.expvol();
+		if (r.au)
+			r.au.volume = r.expvol(r.vol);
 	};
 
-	this.read_order = function () {
+	r.read_order = function () {
 		var order = [],
 			links = QSA('#files>tbody>tr>td:nth-child(1)>a');
 
@@ -334,24 +437,71 @@ function MPlayer() {
 
 			order.push(tid.slice(1));
 		}
-		this.order = order;
+		r.order = order;
 	};
 
-	this.preload = function (url) {
+	r.fdir = 0;
+	r.fvol = -1;
+	r.ftid = -1;
+	r.ftimer = null;
+	r.fade_in = function () {
+		r.fvol = 0;
+		r.fdir = 0.025;
+		if (r.au) {
+			r.ftid = r.au.tid;
+			r.au.play();
+			mpl.pp();
+			fader();
+		}
+	};
+	r.fade_out = function () {
+		r.fvol = r.vol;
+		r.fdir = -0.05;
+		r.ftid = r.au.tid;
+		fader();
+	};
+	r.stopfade = function (hard) {
+		clearTimeout(r.ftimer);
+		if (hard)
+			r.ftid = -1;
+	}
+	function fader() {
+		r.stopfade();
+		if (!r.au || r.au.tid !== r.ftid)
+			return;
+
+		var done = true;
+		r.fvol += r.fdir;
+		if (r.fvol < 0) {
+			r.fvol = 0;
+			r.au.pause();
+			mpl.pp();
+		}
+		else if (r.fvol > r.vol)
+			r.fvol = r.vol;
+		else
+			done = false;
+
+		r.au.volume = r.expvol(r.fvol);
+		if (!done)
+			setTimeout(fader, 10);
+	}
+
+	r.preload = function (url) {
 		var au = null;
 		if (need_ogv_for(url)) {
 			au = mp.au_ogvjs2;
 			if (!au && window['OGVPlayer']) {
 				au = new OGVPlayer();
 				au.preload = "auto";
-				this.au_ogvjs2 = au;
+				r.au_ogvjs2 = au;
 			}
 		} else {
 			au = mp.au_native2;
 			if (!au) {
 				au = new Audio();
 				au.preload = "auto";
-				this.au_native2 = au;
+				r.au_native2 = au;
 			}
 		}
 		if (au) {
@@ -365,39 +515,62 @@ var mp = new MPlayer();
 makeSortable(ebi('files'), mp.read_order.bind(mp));
 
 
+function get_np() {
+	var th = ebi('files').tHead.rows[0].cells,
+		tr = QS('#files tr.play').cells,
+		rv = [],
+		ra = [],
+		rt = {};
+
+	for (var a = 1, aa = th.length; a < aa; a++) {
+		var tv = tr[a].textContent,
+			tk = a == 1 ? 'file' : th[a].getAttribute('name').split('/').slice(-1)[0],
+			vis = th[a].className.indexOf('min') === -1;
+
+		if (!tv)
+			continue;
+
+		(vis ? rv : ra).push(tk);
+		rt[tk] = tv;
+	}
+	return [rt, rv, ra];
+};
+
+
 // toggle player widget
 var widget = (function () {
-	var ret = {},
+	var r = {},
 		widget = ebi('widget'),
 		wtico = ebi('wtico'),
 		nptxt = ebi('nptxt'),
 		npirc = ebi('npirc'),
 		touchmode = false,
-		side_open = false,
 		was_paused = true;
 
-	ret.open = function () {
-		if (side_open)
+	r.is_open = false;
+
+	r.open = function () {
+		if (r.is_open)
 			return false;
 
 		widget.className = 'open';
-		side_open = true;
+		r.is_open = true;
 		return true;
 	};
-	ret.close = function () {
-		if (!side_open)
+	r.close = function () {
+		if (!r.is_open)
 			return false;
 
 		widget.className = '';
-		side_open = false;
+		r.is_open = false;
 		return true;
 	};
-	ret.toggle = function (e) {
-		ret.open() || ret.close();
+	r.toggle = function (e) {
+		r.open() || r.close();
 		ev(e);
 		return false;
 	};
-	ret.paused = function (paused) {
+	r.paused = function (paused) {
 		if (was_paused != paused) {
 			was_paused = paused;
 			ebi('bplay').innerHTML = paused ? '▶' : '⏸';
@@ -405,28 +578,22 @@ var widget = (function () {
 	};
 	wtico.onclick = function (e) {
 		if (!touchmode)
-			ret.toggle(e);
+			r.toggle(e);
 
 		return false;
 	};
 	npirc.onclick = nptxt.onclick = function (e) {
 		ev(e);
-		var th = ebi('files').tHead.rows[0].cells,
-			tr = QS('#files tr.play').cells,
-			irc = this.getAttribute('id') == 'npirc',
+		var irc = this.getAttribute('id') == 'npirc',
 			ck = irc ? '06' : '',
 			cv = irc ? '07' : '',
-			m = ck + 'np: ';
+			m = ck + 'np: ',
+			npr = get_np(),
+			npk = npr[1],
+			np = npr[0];
 
-		for (var a = 1, aa = th.length; a < aa; a++) {
-			if (th[a].className.indexOf('min') !== -1)
-				continue;
-
-			var tv = tr[a].textContent,
-				tk = a == 1 ? '' : th[a].getAttribute('name').split('/').slice(-1)[0];
-
-			m += tk + '(' + cv + tv + ck + ') // ';
-		}
+		for (var a = 0; a < npk.length; a++)
+			m += (npk[a] == 'file' ? '' : npk[a]) + '(' + cv + np[npk[a]] + ck + ') // ';
 
 		m += '[' + cv + s2ms(mp.au.currentTime) + ck + '/' + cv + s2ms(mp.au.duration) + ck + ']';
 
@@ -442,7 +609,7 @@ var widget = (function () {
 			document.body.removeChild(o);
 		}, 500);
 	};
-	return ret;
+	return r;
 })();
 
 
@@ -488,12 +655,15 @@ var pbar = (function () {
 	}
 
 	r.drawbuf = function () {
+		var bc = r.buf,
+			bctx = bc.ctx;
+
+		bctx.clearRect(0, 0, bc.w, bc.h);
+
 		if (!mp.au)
 			return;
 
-		var bc = r.buf,
-			bctx = bc.ctx,
-			sm = bc.w * 1.0 / mp.au.duration,
+		var sm = bc.w * 1.0 / mp.au.duration,
 			gk = bc.h + '' + light;
 
 		if (gradh != gk) {
@@ -501,7 +671,6 @@ var pbar = (function () {
 			grad = glossy_grad(bc, 85, [35, 40, 37, 35], light ? [45, 56, 50, 45] : [42, 51, 47, 42]);
 		}
 		bctx.fillStyle = grad;
-		bctx.clearRect(0, 0, bc.w, bc.h);
 		for (var a = 0; a < mp.au.buffered.length; a++) {
 			var x1 = sm * mp.au.buffered.start(a),
 				x2 = sm * mp.au.buffered.end(a);
@@ -511,15 +680,17 @@ var pbar = (function () {
 	};
 
 	r.drawpos = function () {
+		var bc = r.buf,
+			pc = r.pos,
+			pctx = pc.ctx;
+
+		pctx.clearRect(0, 0, pc.w, pc.h);
+
 		if (!mp.au || isNaN(mp.au.duration) || isNaN(mp.au.currentTime))
 			return;  // not-init || unsupp-codec
 
-		var bc = r.buf,
-			pc = r.pos,
-			pctx = pc.ctx,
-			sm = bc.w * 1.0 / mp.au.duration;
+		var sm = bc.w * 1.0 / mp.au.duration;
 
-		pctx.clearRect(0, 0, pc.w, pc.h);
 		pctx.fillStyle = light ? 'rgba(0,64,0,0.15)' : 'rgba(204,255,128,0.15)';
 		for (var p = 1, mins = mp.au.duration / 10; p <= mins; p++)
 			pctx.fillRect(Math.floor(sm * p * 10), 0, 2, pc.h);
@@ -635,6 +806,11 @@ function seek_au_mul(mul) {
 		seek_au_sec(mp.au.duration * mul);
 }
 
+function seek_au_rel(sec) {
+	if (mp.au)
+		seek_au_sec(mp.au.currentTime + sec);
+}
+
 function seek_au_sec(seek) {
 	if (!mp.au)
 		return;
@@ -645,9 +821,8 @@ function seek_au_sec(seek) {
 
 	mp.au.currentTime = seek;
 
-	// ogv.js breaks on .play() during playback
-	if (mp.au === mp.au_native)
-		mp.au.play();
+	if (mp.au.paused)
+		mp.fade_in();
 
 	mpui.progress_updater();
 }
@@ -669,22 +844,29 @@ function next_song(e) {
 }
 function prev_song(e) {
 	ev(e);
+
+	if (mp.au && !mp.au.paused && mp.au.currentTime > 3)
+		return seek_au_sec(0);
+
 	return song_skip(-1);
 }
 
 
 function playpause(e) {
+	// must be event-chain
 	ev(e);
 	if (mp.au) {
 		if (mp.au.paused)
-			mp.au.play();
+			mp.fade_in();
 		else
-			mp.au.pause();
+			mp.fade_out();
 
 		mpui.progress_updater();
 	}
 	else
-		play(0);
+		play(0, true);
+
+	mpl.pp();
 };
 
 
@@ -693,9 +875,13 @@ function playpause(e) {
 	ebi('bplay').onclick = playpause;
 	ebi('bprev').onclick = prev_song;
 	ebi('bnext').onclick = next_song;
-	ebi('barpos').onclick = function (e) {
+
+	var bar = ebi('barpos');
+
+	bar.onclick = function (e) {
 		if (!mp.au) {
-			return play(0);
+			play(0, true);
+			return mp.fade_in();
 		}
 
 		var rect = pbar.buf.can.getBoundingClientRect(),
@@ -703,6 +889,19 @@ function playpause(e) {
 
 		seek_au_mul(x * 1.0 / rect.width);
 	};
+
+	if (!is_touch)
+		bar.onwheel = function (e) {
+			var dist = Math.sign(e.deltaY) * 10;
+			if (Math.abs(e.deltaY) < 30 && !e.deltaMode)
+				dist = e.deltaY;
+
+			if (!dist || !mp.au)
+				return true;
+
+			seek_au_rel(dist);
+			ev(e);
+		};
 })();
 
 
@@ -770,7 +969,12 @@ var mpui = (function () {
 // event from play button next to a file in the list
 function ev_play(e) {
 	ev(e);
-	play(this.getAttribute('id').slice(1));
+
+	var fade = !mp.au || mp.au.paused;
+	play(this.getAttribute('id').slice(1), true);
+	if (fade)
+		mp.fade_in();
+
 	return false;
 }
 
@@ -1009,13 +1213,18 @@ var audio_eq = (function () {
 
 
 // plays the tid'th audio file on the page
-function play(tid, seek, call_depth) {
+function play(tid, is_ev, seek, call_depth) {
 	if (mp.order.length == 0)
 		return console.log('no audio found wait what');
 
+	mp.stopfade(true);
+
 	var tn = tid;
-	if ((tn + '').indexOf('f-') === 0)
+	if ((tn + '').indexOf('f-') === 0) {
 		tn = mp.order.indexOf(tn);
+		if (tn < 0)
+			return;
+	}
 
 	if (tn >= mp.order.length) {
 		if (mpl.pb_mode == 'loop-folder') {
@@ -1054,7 +1263,7 @@ function play(tid, seek, call_depth) {
 		}
 		else if (window['OGVPlayer']) {
 			mp.au = mp.au_ogvjs = new OGVPlayer();
-			attempt_play = false;
+			attempt_play = is_ev;
 			mp.au.addEventListener('error', evau_error, true);
 			mp.au.addEventListener('progress', pbar.drawpos);
 			mp.au.addEventListener('ended', next_song);
@@ -1067,7 +1276,7 @@ function play(tid, seek, call_depth) {
 			show_modal('<h1>loading ogv.js</h1><h2>thanks apple</h2>');
 
 			import_js('/.cpr/deps/ogv.js', function () {
-				play(tid, seek, 1);
+				play(tid, false, seek, 1);
 			});
 
 			return;
@@ -1088,7 +1297,7 @@ function play(tid, seek, call_depth) {
 
 	mp.au.tid = tid;
 	mp.au.src = url + (url.indexOf('?') < 0 ? '?cache' : '&cache');
-	mp.au.volume = mp.expvol();
+	mp.au.volume = mp.expvol(mp.vol);
 	var oid = 'a' + tid;
 	setclass(oid, 'play act');
 	var trs = ebi('files').getElementsByTagName('tbody')[0].getElementsByTagName('tr');
@@ -1124,6 +1333,7 @@ function play(tid, seek, call_depth) {
 
 		mpui.progress_updater();
 		pbar.drawbuf();
+		mpl.announce();
 		return true;
 	}
 	catch (ex) {
@@ -1178,7 +1388,8 @@ function show_modal(html) {
 
 
 // hide fullscreen message
-function unblocked() {
+function unblocked(e) {
+	ev(e);
 	var dom = ebi('blocked');
 	if (dom)
 		dom.parentNode.removeChild(dom);
@@ -1193,26 +1404,25 @@ function autoplay_blocked(seek) {
 
 	var go = ebi('blk_go'),
 		na = ebi('blk_na'),
-		fn = mp.tracks[mp.au.tid].split(/\//).pop();
+		tid = mp.au.tid,
+		fn = mp.tracks[tid].split(/\//).pop();
 
 	fn = uricom_dec(fn.replace(/\+/g, ' '))[0];
 
 	go.textContent = 'Play "' + fn + '"';
 	go.onclick = function (e) {
-		if (e) e.preventDefault();
-		unblocked();
-		mp.au.play();
-		if (seek)
-			seek_au_sec(seek);
-		else
-			mpui.progress_updater();
+		unblocked(e);
+		// chrome 91 may permanently taint on a failed play()
+		// depending on win10 settings or something? idk
+		mp.au_native = mp.au_ogvjs = null;
+		play(tid, true, seek);
+		mp.fade_in();
 	};
 	na.onclick = unblocked;
 }
 
 
-// autoplay linked track
-(function () {
+function play_linked() {
 	var v = location.hash;
 	if (v && v.indexOf('#af-') === 0) {
 		var id = v.slice(2).split('&');
@@ -1226,9 +1436,9 @@ function autoplay_blocked(seek) {
 		if (!m)
 			return play(id[0]);
 
-		return play(id[0], parseInt(m[1] || 0) * 60 + parseInt(m[2] || 0));
+		return play(id[0], false, parseInt(m[1] || 0) * 60 + parseInt(m[2] || 0));
 	}
-})();
+};
 
 
 var thegrid = (function () {
@@ -1334,35 +1544,57 @@ var thegrid = (function () {
 	}
 	setsz();
 
-	function seltgl(e) {
-		if (e && e.ctrlKey)
+	function gclick(e) {
+		if (e && (e.ctrlKey || e.metaKey))
 			return true;
 
-		ev(e);
 		var oth = ebi(this.getAttribute('ref')),
-			td = oth.parentNode.nextSibling,
+			href = this.getAttribute('href'),
+			aplay = ebi('a' + oth.getAttribute('id')),
+			is_img = /\.(gif|jpe?g|png|webp)(\?|$)/i.test(href),
+			in_tree = null,
+			have_sel = QS('#files tr.sel'),
+			td = oth.closest('td').nextSibling,
 			tr = td.parentNode;
 
-		td.click();
-		this.setAttribute('class', tr.getAttribute('class'));
-	}
+		if (/\/(\?|$)/.test(href)) {
+			var ta = QSA('#treeul a.hl+ul>li>a+a'),
+				txt = oth.textContent.slice(0, -1);
 
-	function bgopen(e) {
+			for (var a = 0, aa = ta.length; a < aa; a++) {
+				if (ta[a].textContent == txt) {
+					in_tree = ta[a];
+					break;
+				}
+			}
+		}
+
+		if (r.sel) {
+			td.click();
+			this.setAttribute('class', tr.getAttribute('class'));
+		}
+		else if (widget.is_open && aplay)
+			aplay.click();
+
+		else if (in_tree && !have_sel)
+			in_tree.click();
+
+		else if (!is_img && have_sel)
+			window.open(href, '_blank');
+
+		else return true;
 		ev(e);
-		var url = this.getAttribute('href');
-		window.open(url, '_blank');
 	}
 
 	r.loadsel = function () {
 		if (r.dirty)
 			return;
 
-		var ths = QSA('#ggrid>a'),
-			have_sel = !!QS('#files tr.sel');
+		var ths = QSA('#ggrid>a');
 
 		for (var a = 0, aa = ths.length; a < aa; a++) {
-			ths[a].onclick = r.sel ? seltgl : have_sel ? bgopen : null;
-			ths[a].setAttribute('class', ebi(ths[a].getAttribute('ref')).parentNode.parentNode.getAttribute('class'));
+			var tr = ebi(ths[a].getAttribute('ref')).closest('tr');
+			ths[a].setAttribute('class', tr.getAttribute('class'));
 		}
 		var uns = QS('#ggrid a[ref="unsearch"]');
 		if (uns)
@@ -1393,6 +1625,8 @@ var thegrid = (function () {
 
 			if (r.thumbs) {
 				ihref += (ihref.indexOf('?') === -1 ? '?' : '&') + 'th=' + (have_webp ? 'w' : 'j');
+				if (href == "#")
+					ihref = '/.cpr/ico/⏏️';
 			}
 			else if (isdir) {
 				ihref = '/.cpr/ico/folder';
@@ -1420,6 +1654,11 @@ var thegrid = (function () {
 				ihref + '" /><span' + ac + '>' + ao.innerHTML + '</span></a>');
 		}
 		ebi('ggrid').innerHTML = html.join('\n');
+
+		var ths = QSA('#ggrid>a');
+		for (var a = 0, aa = ths.length; a < aa; a++)
+			ths[a].onclick = gclick;
+
 		r.dirty = false;
 		r.bagit();
 		r.loadsel();
@@ -1507,26 +1746,32 @@ document.onkeydown = function (e) {
 	if (!document.activeElement || document.activeElement != document.body && document.activeElement.nodeName.toLowerCase() != 'a')
 		return;
 
-	if (e.ctrlKey || e.altKey || e.shiftKey || e.metaKey || e.isComposing)
+	if (e.ctrlKey || e.altKey || e.metaKey || e.isComposing)
+		return;
+
+	var k = (e.code + ''), pos = -1, n;
+
+	if (e.shiftKey && k != 'KeyA' && k != 'KeyD')
 		return;
 
-	var k = (e.code + ''), pos = -1;
 	if (k.indexOf('Digit') === 0)
 		pos = parseInt(k.slice(-1)) * 0.1;
 
 	if (pos !== -1)
-		return seek_au_mul(pos);
+		return seek_au_mul(pos) || true;
 
-	var n = k == 'KeyJ' ? -1 : k == 'KeyL' ? 1 : 0;
-	if (n !== 0)
-		return song_skip(n);
+	if (k == 'KeyJ')
+		return prev_song() || true;
+
+	if (k == 'KeyL')
+		return next_song() || true;
 
 	if (k == 'KeyP')
-		return playpause();
+		return playpause() || true;
 
 	n = k == 'KeyU' ? -10 : k == 'KeyO' ? 10 : 0;
 	if (n !== 0)
-		return mp.au ? seek_au_sec(mp.au.currentTime + n) : true;
+		return seek_au_rel(n) || true;
 
 	n = k == 'KeyI' ? -1 : k == 'KeyK' ? 1 : 0;
 	if (n !== 0)
@@ -1536,7 +1781,6 @@ document.onkeydown = function (e) {
 		return tree_up();
 
 	if (k == 'KeyB')
-		//return treectl.hidden ? treectl.show() : treectl.hide();
 		return treectl.hidden ? treectl.entree() : treectl.detree();
 
 	if (k == 'KeyG')
@@ -1545,6 +1789,14 @@ document.onkeydown = function (e) {
 	if (k == 'KeyT')
 		return ebi('thumbs').click();
 
+	if (!treectl.hidden && (!e.shiftKey || !thegrid.en)) {
+		if (k == 'KeyA')
+			return QS('#twig').click();
+
+		if (k == 'KeyD')
+			return QS('#twobytwo').click();
+	}
+
 	if (thegrid.en) {
 		if (k == 'KeyS')
 			return ebi('gridsel').click();
@@ -1627,6 +1879,7 @@ document.onkeydown = function (e) {
 	}
 
 	var search_timeout,
+		defer_timeout,
 		search_in_progress = 0;
 
 	function ev_search_input() {
@@ -1641,9 +1894,29 @@ document.onkeydown = function (e) {
 		if (id != "q_raw")
 			encode_query();
 
-		clearTimeout(search_timeout);
-		if (Date.now() - search_in_progress > 30 * 1000)
+		set_vq();
+
+		clearTimeout(defer_timeout);
+		defer_timeout = setTimeout(try_search, 2000);
+		try_search();
+	}
+
+	function try_search() {
+		if (Date.now() - search_in_progress > 30 * 1000) {
+			clearTimeout(defer_timeout);
+			clearTimeout(search_timeout);
 			search_timeout = setTimeout(do_search, 200);
+		}
+	}
+
+	function set_vq() {
+		if (search_in_progress)
+			return;
+
+		var q = ebi('q_raw').value,
+			vq = ebi('files').getAttribute('q_raw');
+
+		srch_msg(false, (q == vq) ? '' : 'search results below are from a previous query:\n  ' + (vq ? vq : '(*)'));
 	}
 
 	function encode_query() {
@@ -1713,7 +1986,8 @@ document.onkeydown = function (e) {
 		xhr.setRequestHeader('Content-Type', 'text/plain');
 		xhr.onreadystatechange = xhr_search_results;
 		xhr.ts = Date.now();
-		xhr.send(JSON.stringify({ "q": ebi('q_raw').value }));
+		xhr.q_raw = ebi('q_raw').value;
+		xhr.send(JSON.stringify({ "q": xhr.q_raw }));
 	}
 
 	function xhr_search_results() {
@@ -1784,6 +2058,8 @@ document.onkeydown = function (e) {
 
 		ofiles.innerHTML = html.join('\n');
 		ofiles.setAttribute("ts", this.ts);
+		ofiles.setAttribute("q_raw", this.q_raw);
+		set_vq();
 		mukey.render();
 		reload_browser();
 		filecols.set_style(['File Name']);
@@ -1795,6 +2071,7 @@ document.onkeydown = function (e) {
 		ev(e);
 		treectl.show();
 		ebi('files').innerHTML = orig_html;
+		ebi('files').removeAttribute('q_raw');
 		orig_html = null;
 		msel.render();
 		reload_browser();
@@ -2013,6 +2290,9 @@ var treectl = (function () {
 	}
 
 	function treego(e) {
+		if (e && (e.ctrlKey || e.metaKey))
+			return true;
+
 		ev(e);
 		if (this.getAttribute('class') == 'hl' &&
 			this.previousSibling.textContent == '-') {
@@ -2743,8 +3023,10 @@ function reload_mp() {
 		mp.au.pause();
 		mp.au = null;
 	}
+	mpl.stop();
 	widget.close();
 	mp = new MPlayer();
+	setTimeout(pbar.onresize, 1);
 }
 
 
@@ -2788,3 +3070,4 @@ function reload_browser(not_mp) {
 reload_browser(true);
 mukey.render();
 msel.render();
+play_linked();

copyparty/web/browser2.html

@@ -54,7 +54,7 @@
 	<div>{{ logues[1] }}</div><br />
 	{%- endif %}
 	
-	<h2><a href="{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
+	<h2><a href="/{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
 
 </body>
 </html>

copyparty/web/md.html

@@ -3,9 +3,9 @@
 	<title>📝🎉 {{ title }}</title> <!-- 📜 -->
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/md.css" rel="stylesheet">
+	<link href="/.cpr/md.css?_={{ ts }}" rel="stylesheet">
 	{%- if edit %}
-	<link href="/.cpr/md2.css" rel="stylesheet">
+	<link href="/.cpr/md2.css?_={{ ts }}" rel="stylesheet">
 	{%- endif %}
 </head>
 <body>
@@ -146,10 +146,10 @@ var md_opt = {
 })();
 
 	</script>
-    <script src="/.cpr/util.js"></script>
-	<script src="/.cpr/deps/marked.js"></script>
-	<script src="/.cpr/md.js"></script>
+    <script src="/.cpr/util.js?_={{ ts }}"></script>
+	<script src="/.cpr/deps/marked.js?_={{ ts }}"></script>
+	<script src="/.cpr/md.js?_={{ ts }}"></script>
 	{%- if edit %}
-	<script src="/.cpr/md2.js"></script>
+	<script src="/.cpr/md2.js?_={{ ts }}"></script>
 	{%- endif %}
 </body></html>

copyparty/web/mde.html

@@ -3,9 +3,9 @@
 	<title>📝🎉 {{ title }}</title>
 	<meta http-equiv="X-UA-Compatible" content="IE=edge">
 	<meta name="viewport" content="width=device-width, initial-scale=0.7">
-	<link href="/.cpr/mde.css" rel="stylesheet">
-	<link href="/.cpr/deps/mini-fa.css" rel="stylesheet">
-	<link href="/.cpr/deps/easymde.css" rel="stylesheet">
+	<link href="/.cpr/mde.css?_={{ ts }}" rel="stylesheet">
+	<link href="/.cpr/deps/mini-fa.css?_={{ ts }}" rel="stylesheet">
+	<link href="/.cpr/deps/easymde.css?_={{ ts }}" rel="stylesheet">
 </head>
 <body>
 	<div id="mw">
@@ -43,7 +43,7 @@ var lightswitch = (function () {
 })();
 
 	</script>
-    <script src="/.cpr/util.js"></script>
-	<script src="/.cpr/deps/easymde.js"></script>
-	<script src="/.cpr/mde.js"></script>
+    <script src="/.cpr/util.js?_={{ ts }}"></script>
+	<script src="/.cpr/deps/easymde.js?_={{ ts }}"></script>
+	<script src="/.cpr/mde.js?_={{ ts }}"></script>
 </body></html>

copyparty/web/msg.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/msg.css">
+    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/msg.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/splash.html

@@ -6,7 +6,7 @@
     <title>copyparty</title>
     <meta http-equiv="X-UA-Compatible" content="IE=edge">
     <meta name="viewport" content="width=device-width, initial-scale=0.8">
-    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/splash.css">
+    <link rel="stylesheet" type="text/css" media="screen" href="/.cpr/splash.css?_={{ ts }}">
 </head>
 
 <body>

copyparty/web/up2k.js

@@ -1,7 +1,5 @@
 "use strict";
 
-window.onerror = vis_exh;
-
 
 function goto_up2k() {
     if (up2k === false)
@@ -16,17 +14,19 @@ function goto_up2k() {
 
 // chrome requires https to use crypto.subtle,
 // usually it's undefined but some chromes throw on invoke
-var up2k = null;
-var sha_js = window.WebAssembly ? 'hw' : 'ac';  // ff53,c57,sa11
+var up2k = null,
+    sha_js = window.WebAssembly ? 'hw' : 'ac',  // ff53,c57,sa11
+    m = 'will use ' + sha_js + ' instead of native sha512 due to';
+
 try {
     var cf = crypto.subtle || crypto.webkitSubtle;
     cf.digest('SHA-512', new Uint8Array(1)).then(
         function (x) { console.log('sha-ok'); up2k = up2k_init(cf); },
-        function (x) { console.log('sha-ng:', x); up2k = up2k_init(false); }
+        function (x) { console.log(m, x); up2k = up2k_init(false); }
     );
 }
 catch (ex) {
-    console.log('sha-na:', ex);
+    console.log(m, ex);
     try {
         up2k = up2k_init(false);
     }
@@ -142,7 +142,7 @@ function U2pvis(act, btns) {
     this.tail = -1;
     this.wsz = 3;
 
-    this.addfile = function (entry, sz) {
+    this.addfile = function (entry, sz, draw) {
         this.tab.push({
             "hn": entry[0],
             "ht": entry[1],
@@ -156,6 +156,9 @@ function U2pvis(act, btns) {
             "bd0": 0  // upload start
         });
         this.ctr["q"]++;
+        if (!draw)
+            return;
+
         this.drawcard("q");
         if (this.act == "q") {
             this.addrow(this.tab.length - 1);
@@ -256,6 +259,41 @@ function U2pvis(act, btns) {
         var obj = ebi('f{0}p'.format(fobj.n)),
             o1 = p[0] - 2, o2 = p[0] - 0.1, o3 = p[0];
 
+        if (!obj) { //} || true) {
+            var msg = [
+                "act", this.act,
+                "in", fo.in,
+                "is_act", this.is_act(fo.in),
+                "head", this.head,
+                "tail", this.tail,
+                "nfile", fobj.n,
+                "name", fobj.name,
+                "sz", fobj.size,
+                "bytesDelta", delta,
+                "bytesDone", fo.bd,
+            ],
+                m2 = '',
+                ds = QSA("#u2tab>tbody>tr>td:first-child>a:last-child");
+
+            for (var a = 0; a < msg.length; a += 2)
+                m2 += msg[a] + '=' + msg[a + 1] + ', ';
+
+            console.log(m2);
+
+            for (var a = 0, aa = ds.length; a < aa; a++) {
+                var id = ds[a].parentNode.getAttribute('id').slice(1, -1);
+                console.log("dom %d/%d = [%s] in(%s) is_act(%s) %s",
+                    a, aa, id, this.tab[id].in, this.is_act(fo.in), ds[a].textContent);
+            }
+
+            for (var a = 0, aa = this.tab.length; a < aa; a++)
+                if (this.is_act(this.tab[a].in))
+                    console.log("tab %d/%d = sz %s", a, aa, this.tab[a].bt);
+
+            console.log("a");
+            throw 42;
+        }
+
         obj.innerHTML = fo.hp;
         obj.style.color = '#fff';
         obj.style.background = 'linear-gradient(90deg, #050, #270 ' + o1 + '%, #4b0 ' + o2 + '%, #333 ' + o3 + '%, #333 99%, #777)';
@@ -276,12 +314,14 @@ function U2pvis(act, btns) {
         this.drawcard(oldcat);
         this.drawcard(newcat);
         if (this.is_act(newcat)) {
-            this.tail++;
+            this.tail = Math.max(this.tail, nfile + 1);
             if (!ebi('f' + nfile))
                 this.addrow(nfile);
         }
         else if (this.is_act(oldcat)) {
-            this.head++;
+            while (this.head < Math.min(this.tab.length, this.tail) && (this.head == nfile || !this.is_act(this.tab[this.head].in)))
+                this.head++;
+
             if (!bz_act) {
                 var tr = ebi("f" + nfile);
                 tr.parentNode.removeChild(tr);
@@ -350,8 +390,21 @@ function U2pvis(act, btns) {
             }
         }
         if (this.head == -1) {
-            this.head = this.tab.length;
-            this.tail = this.head - 1;
+            var precard = has(["ok", "ng", "done"], this.act) ? {} : this.act == "bz" ? { "ok": 1, "ng": 1 } : { "ok": 1, "ng": 1, "bz": 1 },
+                postcard = has(["ok", "ng", "done"], this.act) ? { "bz": 1, "q": 1 } : this.act == "bz" ? { "q": 1 } : {};
+
+            for (var a = 0; a < this.tab.length; a++) {
+                var rt = this.tab[a].in;
+                if (precard[rt]) {
+                    this.head = a + 1;
+                    this.tail = a;
+                }
+                else if (postcard[rt]) {
+                    this.head = a;
+                    this.tail = a - 1;
+                    break;
+                }
+            }
         }
         if (card == "bz") {
             for (var a = this.head - 1; a >= this.head - this.wsz && a >= 0; a--) {
@@ -598,14 +651,50 @@ function up2k_init(subtle) {
         }
     }
 
-    function read_dirs(rd, pf, dirs, good, bad) {
+    function rd_flatten(pf, dirs) {
+        var ret = jcp(pf);
+        for (var a = 0; a < dirs.length; a++)
+            ret.push(dirs.fullPath || '');
+
+        ret.sort();
+        return ret;
+    }
+
+    var rd_missing_ref = [];
+    function read_dirs(rd, pf, dirs, good, bad, spins) {
+        spins = spins || 0;
+        if (++spins == 5)
+            rd_missing_ref = rd_flatten(pf, dirs);
+
+        if (spins == 200) {
+            var missing = rd_flatten(pf, dirs),
+                match = rd_missing_ref.length == missing.length,
+                aa = match ? missing.length : 0;
+
+            missing.sort();
+            for (var a = 0; a < aa; a++)
+                if (rd_missing_ref[a] != missing[a])
+                    match = false;
+
+            if (match) {
+                var msg = ['directory iterator got stuck on the following {0} items; good chance your browser is about to spinlock:'.format(missing.length)];
+                for (var a = 0; a < Math.min(20, missing.length); a++)
+                    msg.push(missing[a]);
+
+                alert(msg.join('\n-- '));
+                dirs = [];
+                pf = [];
+            }
+            spins = 0;
+        }
+
         if (!dirs.length) {
             if (!pf.length)
                 return gotallfiles(good, bad);
 
             console.log("retry pf, " + pf.length);
             setTimeout(function () {
-                read_dirs(rd, pf, dirs, good, bad);
+                read_dirs(rd, pf, dirs, good, bad, spins);
             }, 50);
             return;
         }
@@ -645,7 +734,7 @@ function up2k_init(subtle) {
                 dirs.shift();
                 rd = null;
             }
-            return read_dirs(rd, pf, dirs, good, bad);
+            return read_dirs(rd, pf, dirs, good, bad, spins);
         });
     }
 
@@ -670,41 +759,50 @@ function up2k_init(subtle) {
         if (ask_up && !fsearch && !confirm(msg.join('\n')))
             return;
 
+        var seen = {},
+            evpath = get_evpath(),
+            draw_each = good_files.length < 50;
+
+        for (var a = 0; a < st.files.length; a++)
+            seen[st.files[a].name + '\n' + st.files[a].size] = 1;
+
         for (var a = 0; a < good_files.length; a++) {
             var fobj = good_files[a][0],
                 now = Date.now(),
                 lmod = fobj.lastModified || now;
 
             var entry = {
-                "n": parseInt(st.files.length.toString()),
+                "n": st.files.length,
                 "t0": now,
                 "fobj": fobj,
                 "name": good_files[a][1],
                 "size": fobj.size,
                 "lmod": lmod / 1000,
-                "purl": get_evpath(),
+                "purl": evpath,
                 "done": false,
                 "hash": []
-            };
+            },
+                key = entry.name + '\n' + entry.size;
 
-            var skip = false;
-            for (var b = 0; b < st.files.length; b++)
-                if (entry.name == st.files[b].name &&
-                    entry.size == st.files[b].size)
-                    skip = true;
-
-            if (skip)
+            if (seen[key])
                 continue;
 
+            seen[key] = 1;
+
             pvis.addfile([
                 fsearch ? esc(entry.name) : linksplit(
                     uricom_dec(entry.purl)[0] + entry.name).join(' '),
                 '📐 hash',
                 ''
-            ], fobj.size);
+            ], fobj.size, draw_each);
+
             st.files.push(entry);
             st.todo.hash.push(entry);
         }
+        if (!draw_each) {
+            pvis.drawcard("q");
+            pvis.changecard(pvis.act);
+        }
     }
     ebi('u2btn').addEventListener('drop', gotfile, false);
 
@@ -740,9 +838,17 @@ function up2k_init(subtle) {
 
     function handshakes_permitted() {
         var lim = multitask ? 1 : 0;
-        return lim >=
+
+        if (lim <
             st.todo.upload.length +
-            st.busy.upload.length;
+            st.busy.upload.length)
+            return false;
+
+        var cd = st.todo.handshake.length ? st.todo.handshake[0].cooldown : 0;
+        if (cd && cd - Date.now() > 0)
+            return false;
+
+        return true;
     }
 
     function hashing_permitted() {
@@ -773,7 +879,7 @@ function up2k_init(subtle) {
 
             clearTimeout(tto);
             running = true;
-            while (true) {
+            while (window['vis_exh']) {
                 var is_busy = 0 !=
                     st.todo.hash.length +
                     st.todo.handshake.length +
@@ -946,7 +1052,7 @@ function up2k_init(subtle) {
 
             bpend += cdr - car;
 
-            reader.onload = function (e) {
+            function orz(e) {
                 if (!min_filebuf && nch == 1) {
                     min_filebuf = 1;
                     var td = Date.now() - t0;
@@ -956,9 +1062,30 @@ function up2k_init(subtle) {
                     }
                 }
                 hash_calc(nch, e.target.result);
+            }
+            reader.onload = function (e) {
+                try { orz(e); } catch (ex) { vis_exh(ex + '', '', '', '', ex); }
             };
             reader.onerror = function () {
-                alert('y o u   b r o k e    i t\nerror: ' + reader.error);
+                var err = reader.error + '';
+                var handled = false;
+
+                if (err.indexOf('NotReadableError') !== -1 || // win10-chrome defender
+                    err.indexOf('NotFoundError') !== -1  // macos-firefox permissions
+                ) {
+                    pvis.seth(t.n, 1, 'OS-error');
+                    pvis.seth(t.n, 2, err);
+                    handled = true;
+                }
+
+                if (handled) {
+                    pvis.move(t.n, 'ng');
+                    st.busy.hash.splice(st.busy.hash.indexOf(t), 1);
+                    st.bytes.uploaded += t.size;
+                    return tasker();
+                }
+
+                alert('y o u   b r o k e    i t\nfile: ' + t.name + '\nerror: ' + err);
             };
             reader.readAsArrayBuffer(
                 bobslice.call(t.fobj, car, cdr));
@@ -1039,12 +1166,12 @@ function up2k_init(subtle) {
                 console.log('zombie handshake onerror,', t);
                 return;
             }
-            console.log('handshake onerror, retrying');
+            console.log('handshake onerror, retrying', t);
             st.busy.handshake.splice(st.busy.handshake.indexOf(t), 1);
             st.todo.handshake.unshift(t);
             tasker();
         };
-        xhr.onload = function (e) {
+        function orz(e) {
             if (t.busied != me) {
                 console.log('zombie handshake onload,', t);
                 return;
@@ -1084,6 +1211,7 @@ function up2k_init(subtle) {
 
                 if (response.name !== t.name) {
                     // file exists; server renamed us
+                    console.log("server-rename [" + t.name + "] to [" + response.name + "]");
                     t.name = response.name;
                     pvis.seth(t.n, 0, linksplit(t.purl + t.name).join(' '));
                 }
@@ -1155,6 +1283,15 @@ function up2k_init(subtle) {
                 if (rsp.indexOf('<pre>') === 0)
                     rsp = rsp.slice(5);
 
+                if (rsp.indexOf('rate-limit ') !== -1) {
+                    var penalty = rsp.replace(/.*rate-limit /, "").split(' ')[0];
+                    console.log("rate-limit: " + penalty);
+                    t.cooldown = Date.now() + parseFloat(penalty) * 1000;
+                    st.busy.handshake.splice(st.busy.handshake.indexOf(t), 1);
+                    st.todo.handshake.unshift(t);
+                    return;
+                }
+
                 st.bytes.uploaded += t.size;
                 if (rsp.indexOf('partial upload exists') !== -1 ||
                     rsp.indexOf('file already exists') !== -1) {
@@ -1179,6 +1316,9 @@ function up2k_init(subtle) {
                         (xhr.responseText && xhr.responseText) ||
                         "no further information"));
             }
+        }
+        xhr.onload = function (e) {
+            try { orz(e); } catch (ex) { vis_exh(ex + '', '', '', '', ex); }
         };
 
         var req = {
@@ -1219,40 +1359,56 @@ function up2k_init(subtle) {
         if (cdr >= t.size)
             cdr = t.size;
 
-        var xhr = new XMLHttpRequest();
-        xhr.upload.onprogress = function (xev) {
-            pvis.prog(t, npart, xev.loaded);
-        };
-        xhr.onload = function (xev) {
+        function orz(xhr) {
+            var txt = ((xhr.response && xhr.response.err) || xhr.responseText) + '';
             if (xhr.status == 200) {
                 pvis.prog(t, npart, cdr - car);
                 st.bytes.uploaded += cdr - car;
                 t.bytes_uploaded += cdr - car;
-                st.busy.upload.splice(st.busy.upload.indexOf(upt), 1);
-                t.postlist.splice(t.postlist.indexOf(npart), 1);
-                if (t.postlist.length == 0) {
-                    t.t4 = Date.now();
-                    pvis.seth(t.n, 1, 'verifying');
-                    st.todo.handshake.unshift(t);
-                }
-                tasker();
             }
-            else
+            else if (txt.indexOf('already got that') !== -1) {
+                console.log("ignoring dupe-segment error", t);
+            }
+            else {
                 alert("server broke; cu-err {0} on file [{1}]:\n".format(
-                    xhr.status, t.name) + (
-                        (xhr.response && xhr.response.err) ||
-                        (xhr.responseText && xhr.responseText) ||
-                        "no further information"));
-        };
-        xhr.open('POST', t.purl + 'chunkpit.php', true);
-        xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
-        xhr.setRequestHeader("X-Up2k-Wark", t.wark);
-        xhr.setRequestHeader('Content-Type', 'application/octet-stream');
-        if (xhr.overrideMimeType)
-            xhr.overrideMimeType('Content-Type', 'application/octet-stream');
+                    xhr.status, t.name) + (txt || "no further information"));
+                return;
+            }
+            st.busy.upload.splice(st.busy.upload.indexOf(upt), 1);
+            t.postlist.splice(t.postlist.indexOf(npart), 1);
+            if (t.postlist.length == 0) {
+                t.t4 = Date.now();
+                pvis.seth(t.n, 1, 'verifying');
+                st.todo.handshake.unshift(t);
+            }
+            tasker();
+        }
+        function do_send() {
+            var xhr = new XMLHttpRequest();
+            xhr.upload.onprogress = function (xev) {
+                pvis.prog(t, npart, xev.loaded);
+            };
+            xhr.onload = function (xev) {
+                try { orz(xhr); } catch (ex) { vis_exh(ex + '', '', '', '', ex); }
+            };
+            xhr.onerror = function (xev) {
+                if (!window['vis_exh'])
+                    return;
 
-        xhr.responseType = 'text';
-        xhr.send(bobslice.call(t.fobj, car, cdr));
+                console.log('chunkpit onerror, retrying', t);
+                do_send();
+            };
+            xhr.open('POST', t.purl + 'chunkpit.php', true);
+            xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
+            xhr.setRequestHeader("X-Up2k-Wark", t.wark);
+            xhr.setRequestHeader('Content-Type', 'application/octet-stream');
+            if (xhr.overrideMimeType)
+                xhr.overrideMimeType('Content-Type', 'application/octet-stream');
+
+            xhr.responseType = 'text';
+            xhr.send(bobslice.call(t.fobj, car, cdr));
+        }
+        do_send();
     }
 
     /////
@@ -1292,6 +1448,17 @@ function up2k_init(subtle) {
     }
     tt.init();
 
+    function bumpthread2(e) {
+        if (e.ctrlKey || e.altKey || e.metaKey || e.isComposing)
+            return;
+
+        if (e.code == 'ArrowUp')
+            bumpthread(1);
+
+        if (e.code == 'ArrowDown')
+            bumpthread(-1);
+    }
+
     function bumpthread(dir) {
         try {
             dir.stopPropagation();
@@ -1302,7 +1469,7 @@ function up2k_init(subtle) {
         if (dir.target) {
             clmod(obj, 'err', 1);
             var v = Math.floor(parseInt(obj.value));
-            if (v < 1 || v > 8 || v !== v)
+            if (v < 0 || v > 64 || v !== v)
                 return;
 
             parallel_uploads = v;
@@ -1313,11 +1480,11 @@ function up2k_init(subtle) {
 
         parallel_uploads += dir;
 
-        if (parallel_uploads < 1)
-            parallel_uploads = 1;
+        if (parallel_uploads < 0)
+            parallel_uploads = 0;
 
-        if (parallel_uploads > 8)
-            parallel_uploads = 8;
+        if (parallel_uploads > 16)
+            parallel_uploads = 16;
 
         obj.value = parallel_uploads;
         bumpthread({ "target": 1 })
@@ -1413,6 +1580,7 @@ function up2k_init(subtle) {
         bumpthread(-1);
     };
 
+    ebi('nthread').onkeydown = bumpthread2;
     ebi('nthread').addEventListener('input', bumpthread, false);
     ebi('multitask').addEventListener('click', tgl_multitask, false);
     ebi('ask_up').addEventListener('click', tgl_ask_up, false);
@@ -1426,7 +1594,10 @@ function up2k_init(subtle) {
         nodes[a].addEventListener('touchend', nop, false);
 
     set_fsearch();
-    bumpthread({ "target": 1 })
+    bumpthread({ "target": 1 });
+    if (parallel_uploads < 1)
+        bumpthread(1);
+
     return { "init_deps": init_deps, "set_fsearch": set_fsearch }
 }
 

copyparty/web/util.js

@@ -11,16 +11,6 @@ var is_touch = 'ontouchstart' in window,
 
 
 // error handler for mobile devices
-function hcroak(msg) {
-    document.body.innerHTML = msg;
-    window.onerror = undefined;
-    throw 'fatal_err';
-}
-function croak(msg) {
-    document.body.textContent = msg;
-    window.onerror = undefined;
-    throw msg;
-}
 function esc(txt) {
     return txt.replace(/[&"<>]/g, function (c) {
         return {
@@ -32,9 +22,12 @@ function esc(txt) {
     });
 }
 function vis_exh(msg, url, lineNo, columnNo, error) {
+    if (!window.onerror)
+        return;
+
     window.onerror = undefined;
     window['vis_exh'] = null;
-    var html = ['<h1>you hit a bug!</h1><p>please screenshot this error and send me a copy arigathanks gozaimuch (ed/irc.rizon.net or ed#2644)</p><p>',
+    var html = ['<h1>you hit a bug!</h1><p>please send me a screenshot arigathanks gozaimuch: <code>ed/irc.rizon.net</code> or <code>ed#2644</code><br />&nbsp; (and if you can, press F12 and include the "Console" tab in the screenshot too)</p><p>',
         esc(String(msg)), '</p><p>', esc(url + ' @' + lineNo + ':' + columnNo), '</p>'];
 
     if (error) {
@@ -44,9 +37,13 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
                 html.push('<h2>' + find[a] + '</h2>' +
                     esc(String(error[find[a]])).replace(/\n/g, '<br />\n'));
     }
-    document.body.style.fontSize = '0.8em';
-    document.body.style.padding = '0 1em 1em 1em';
-    hcroak(html.join('\n'));
+    document.body.innerHTML = html.join('\n');
+
+    var s = mknod('style');
+    s.innerHTML = 'body{background:#333;color:#ddd;font-family:sans-serif;font-size:0.8em;padding:0 1em 1em 1em} code{color:#bf7;background:#222;padding:.1em;margin:.2em;font-size:1.1em;font-family:monospace,monospace} *{line-height:1.5em}';
+    document.head.appendChild(s);
+
+    throw 'fatal_err';
 }
 
 
@@ -67,6 +64,9 @@ function ev(e) {
     if (e.stopPropagation)
         e.stopPropagation();
 
+    if (e.stopImmediatePropagation)
+        e.stopImmediatePropagation();
+
     e.returnValue = false;
     return e;
 }
@@ -359,6 +359,15 @@ function get_vpath() {
 }
 
 
+function get_pwd() {
+    var pwd = ('; ' + document.cookie).split('; cppwd=');
+    if (pwd.length < 2)
+        return null;
+
+    return pwd[1].split(';')[0];
+}
+
+
 function unix2iso(ts) {
     return new Date(ts * 1000).toISOString().replace("T", " ").slice(0, -5);
 }
@@ -380,6 +389,11 @@ function has(haystack, needle) {
 }
 
 
+function jcp(obj) {
+    return JSON.parse(JSON.stringify(obj));
+}
+
+
 function sread(key) {
     if (window.localStorage)
         return localStorage.getItem(key);

docs/browser-icons.css

@@ -15,11 +15,6 @@
 }
 #ggrid>a[href$="/"]:before {
 	content: '📂';
-    display: block;
-    position: absolute;
-    margin: -.1em -.4em;
-    text-shadow: 0 0 .1em #000;
-    font-size: 2em;
 }
 
 
@@ -27,8 +22,11 @@
 #ggrid>a:before {
     display: block;
     position: absolute;
-    margin: -.1em -.4em;
+	padding: .3em 0;
+    margin: -.4em;
     text-shadow: 0 0 .1em #000;
+	background: linear-gradient(135deg,rgba(255,255,255,0) 50%,rgba(255,255,255,0.2));
+	border-radius: .3em;
     font-size: 2em;
 }
 

docs/notes.sh

@@ -103,6 +103,15 @@ cat warks | while IFS= read -r x; do sqlite3 up2k.db "delete from mt where w = '
 # dump all dbs
 find -iname up2k.db | while IFS= read -r x; do sqlite3 "$x" 'select substr(w,1,12), rd, fn from up' | sed -r 's/\|/ \| /g' | while IFS= read -r y; do printf '%s | %s\n' "$x" "$y"; done; done
 
+# unschedule mtp scan for all files somewhere under "enc/"
+sqlite3 -readonly up2k.db 'select substr(up.w,1,16) from up inner join mt on mt.w = substr(up.w,1,16) where rd like "enc/%" and +mt.k = "t:mtp"' > keys; awk '{printf "delete from mt where w = \"%s\" and +k = \"t:mtp\";\n", $0}' <keys | tee /dev/stderr | sqlite3 up2k.db
+
+# compare metadata key "key" between two databases
+sqlite3 -readonly up2k.db.key-full 'select w, v from mt where k = "key" order by w' > k1; sqlite3 -readonly up2k.db 'select w, v from mt where k = "key" order by w' > k2; ok=0; ng=0; while IFS='|' read w k2; do k1="$(grep -E "^$w" k1 | sed -r 's/.*\|//')"; [ "$k1" = "$k2" ] && ok=$((ok+1)) || { ng=$((ng+1)); printf '%3s %3s  %s\n' "$k1" "$k2" "$(sqlite3 -readonly up2k.db.key-full "select * from up where substr(w,1,16) = '$w'" | sed -r 's/\|/ | /g')"; }; done < <(cat k2); echo "match $ok   diff $ng"
+
+# actually this is much better
+sqlite3 -readonly up2k.db.key-full 'select w, v from mt where k = "key" order by w' > k1; sqlite3 -readonly up2k.db 'select mt.w, mt.v, up.rd, up.fn from mt inner join up on mt.w = substr(up.w,1,16) where mt.k = "key" order by up.rd, up.fn' > k2; ok=0; ng=0; while IFS='|' read w k2 path; do k1="$(grep -E "^$w" k1 | sed -r 's/.*\|//')"; [ "$k1" = "$k2" ] && ok=$((ok+1)) || { ng=$((ng+1)); printf '%3s %3s  %s\n' "$k1" "$k2" "$path"; }; done < <(cat k2); echo "match $ok   diff $ng"
+
 
 ##
 ## media
@@ -157,7 +166,7 @@ dbg.asyncStore.pendingBreakpoints = {}
 about:config >> devtools.debugger.prefs-schema-version = -1
 
 # determine server version
-git reset --hard origin/HEAD && git log --format=format:"%H %ai %d" --decorate=full > /dev/shm/revs && cat /dev/shm/revs | while read -r rev extra; do (git reset --hard $rev >/dev/null 2>/dev/null && dsz=$(cat copyparty/web/{util,browser,up2k}.js 2>/dev/null | diff -wNarU0 - <(cat /mnt/Users/ed/Downloads/ref/{util,browser,up2k}.js) | wc -c) && printf '%s %6s %s\n' "$rev" $dsz "$extra") </dev/null; done                
+git pull; git reset --hard origin/HEAD && git log --format=format:"%H %ai %d" --decorate=full > ../revs && cat ../{util,browser}.js >../vr && cat ../revs | while read -r rev extra; do (git reset --hard $rev >/dev/null 2>/dev/null && dsz=$(cat copyparty/web/{util,browser}.js >../vg 2>/dev/null && diff -wNarU0 ../{vg,vr} | wc -c) && printf '%s %6s %s\n' "$rev" $dsz "$extra") </dev/null; done                
 
 
 ##
@@ -200,3 +209,4 @@ mk() { rm -rf /tmp/foo; sudo -u ed bash -c 'mkdir /tmp/foo; echo hi > /tmp/foo/b
 mk && t0="$(date)" && while true; do date -s "$(date '+ 1 hour')"; systemd-tmpfiles --clean; ls -1 /tmp | grep foo || break; done; echo "$t0"
 mk && sudo -u ed flock /tmp/foo sleep 40 & sleep 1; ps aux | grep -E 'sleep 40$' && t0="$(date)" && for n in {1..40}; do date -s "$(date '+ 1 day')"; systemd-tmpfiles --clean; ls -1 /tmp | grep foo || break; done; echo "$t0"
 mk && t0="$(date)" && for n in {1..40}; do date -s "$(date '+ 1 day')"; systemd-tmpfiles --clean; ls -1 /tmp | grep foo || break; tar -cf/dev/null /tmp/foo; done; echo "$t0"
+

scripts/sfx.py

@@ -6,10 +6,10 @@ import re, os, sys, time, shutil, signal, threading, tarfile, hashlib, platform,
 import subprocess as sp
 
 """
-run me with any version of python, i will unpack and run copyparty
+pls don't edit this file with a text editor,
+  it breaks the compressed stuff at the end
 
-(but please don't edit this file with a text editor
-  since that would probably corrupt the binary stuff at the end)
+run me with any version of python, i will unpack and run copyparty
 
 there's zero binaries! just plaintext python scripts all the way down
   so you can easily unpack the archive and inspect it for shady stuff

tests/util.py

@@ -108,6 +108,9 @@ class VHttpSrv(object):
         aliases = ["splash", "browser", "browser2", "msg", "md", "mde"]
         self.j2 = {x: J2_FILES for x in aliases}
 
+    def cachebuster(self):
+        return "a"
+
 
 class VHttpConn(object):
     def __init__(self, args, asrv, log, buf):
@@ -121,6 +124,7 @@ class VHttpConn(object):
         self.log_src = "a"
         self.lf_url = None
         self.hsrv = VHttpSrv()
+        self.nreq = 0
         self.nbyte = 0
         self.workload = 0
         self.ico = None