Compare commits

...

17 Commits

Author SHA1 Message Date
ed
c4ba123779 v1.9.21 2023-11-25 14:17:58 +00:00
ed
72e355eb2c prisonparty: prevent overlapping setup/teardown 2023-11-25 14:03:41 +00:00
ed
43d409a5d9 prisonparty accepts user/group names 2023-11-25 13:40:21 +00:00
ed
b1fffc2246 open textfiles inline in grid-view, closes #63;
also fix the Y hotkey (which converts all links in the list-view into
download links), making that apply to the grid-view as well
2023-11-25 13:09:12 +00:00
ed
edd3e53ab3 prisonparty: support zfs-ubuntu
* when bind-mounting, resolve any symlinks ($v/) and read target inode;
   for example merged /bin and /usr/bin
* add failsafe in case this test should break in new exciting ways;
   inspect `mount` for any instances of the jailed path
   (not /proc/mounts since that has funny space encoding)
* unmount in a while-loop because xargs freaks out if one of them fail
   * and systemd doesn't give us a /dev/stderr to write to anyways
2023-11-25 02:16:48 +00:00
ed
aa0b119031 update pkgs to 1.9.20 2023-11-21 23:44:56 +00:00
ed
eddce00765 v1.9.20 2023-11-21 23:25:41 +00:00
ed
6f4bde2111 fix infinite backspin on "previous track";
when playing the first track in a folder and hitting the previous track
button, it would keep switching through the previous folders inifinitely
2023-11-21 23:23:51 +00:00
ed
f3035e8869 clear load-more buttons upon navigation (thx icxes) 2023-11-21 22:53:46 +00:00
ed
a9730499c0 don't suggest loading more search results beyond server cap 2023-11-21 22:38:35 +00:00
ed
b66843efe2 reduce cpu priority of ffmpeg, hooks, parsers 2023-11-21 22:21:33 +00:00
ed
cc1aaea300 update pkgs to 1.9.19 2023-11-19 12:45:32 +00:00
ed
9ccc238799 v1.9.19 2023-11-19 12:29:19 +00:00
ed
8526ef9368 srch-dbg: handle jumpvols correctly 2023-11-19 11:35:13 +00:00
ed
3c36727d07 fix filekeys not appearing in up2k in world-writable vols 2023-11-19 11:19:08 +00:00
ed
ef33ce94cd filter shadowed files from search results (#61),
also adds optimization to stop opening cursors
when max results has already been hit
2023-11-19 11:04:36 +00:00
ed
d500baf5c5 update pkgs to 1.9.18 2023-11-18 21:16:10 +00:00
16 changed files with 251 additions and 69 deletions

View File

@@ -119,8 +119,8 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
enable thumbnails (images/audio/video), media indexing, and audio transcoding by installing some recommended deps: enable thumbnails (images/audio/video), media indexing, and audio transcoding by installing some recommended deps:
* **Alpine:** `apk add py3-pillow ffmpeg` * **Alpine:** `apk add py3-pillow ffmpeg`
* **Debian:** `apt install python3-pil ffmpeg` * **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
* **Fedora:** `dnf install python3-pillow ffmpeg` * **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg`
* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg` * **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
* **MacOS:** `port install py-Pillow ffmpeg` * **MacOS:** `port install py-Pillow ffmpeg`
* **MacOS** (alternative): `brew install pillow ffmpeg` * **MacOS** (alternative): `brew install pillow ffmpeg`
@@ -1366,7 +1366,7 @@ now [available on copr-pypi](https://copr.fedorainfracloud.org/coprs/g/copr/PyPI
```bash ```bash
dnf copr enable @copr/PyPI dnf copr enable @copr/PyPI
dnf install python3-copyparty # just a minimal install, or... dnf install python3-copyparty # just a minimal install, or...
dnf install python3-{copyparty,pillow,argon2-cffi,pyftpdlib,pyOpenSSL} ffmpeg-free # with recommended deps dnf install python3-{copyparty,pillow,argon2-cffi,pyftpdlib,pyOpenSSL} ffmpeg # with recommended deps
``` ```
this *may* also work on RHEL but [I'm not paying IBM to verify that](https://www.jeffgeerling.com/blog/2023/dear-red-hat-are-you-dumb) this *may* also work on RHEL but [I'm not paying IBM to verify that](https://www.jeffgeerling.com/blog/2023/dear-red-hat-are-you-dumb)

View File

@@ -12,13 +12,13 @@ done
help() { cat <<'EOF' help() { cat <<'EOF'
usage: usage:
./prisonparty.sh <ROOTDIR> <UID> <GID> [VOLDIR [VOLDIR...]] -- python3 copyparty-sfx.py [...] ./prisonparty.sh <ROOTDIR> <USER|UID> <GROUP|GID> [VOLDIR [VOLDIR...]] -- python3 copyparty-sfx.py [...]
example: example:
./prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt/nas/music -- python3 copyparty-sfx.py -v /mnt/nas/music::rwmd ./prisonparty.sh /var/lib/copyparty-jail cpp cpp /mnt/nas/music -- python3 copyparty-sfx.py -v /mnt/nas/music::rwmd
example for running straight from source (instead of using an sfx): example for running straight from source (instead of using an sfx):
PYTHONPATH=$PWD ./prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt/nas/music -- python3 -um copyparty -v /mnt/nas/music::rwmd PYTHONPATH=$PWD ./prisonparty.sh /var/lib/copyparty-jail cpp cpp /mnt/nas/music -- python3 -um copyparty -v /mnt/nas/music::rwmd
note that if you have python modules installed as --user (such as bpm/key detectors), note that if you have python modules installed as --user (such as bpm/key detectors),
you should add /home/foo/.local as a VOLDIR you should add /home/foo/.local as a VOLDIR
@@ -28,6 +28,16 @@ exit 1
} }
errs=
for c in awk chroot dirname getent lsof mknod mount realpath sed sort stat uniq; do
command -v $c >/dev/null || {
echo ERROR: command not found: $c
errs=1
}
done
[ $errs ] && exit 1
# read arguments # read arguments
trap help EXIT trap help EXIT
jail="$(realpath "$1")"; shift jail="$(realpath "$1")"; shift
@@ -58,11 +68,18 @@ cpp="$1"; shift
} }
trap - EXIT trap - EXIT
usr="$(getent passwd $uid | cut -d: -f1)"
[ "$usr" ] || { echo "ERROR invalid username/uid $uid"; exit 1; }
uid="$(getent passwd $uid | cut -d: -f3)"
grp="$(getent group $gid | cut -d: -f1)"
[ "$grp" ] || { echo "ERROR invalid groupname/gid $gid"; exit 1; }
gid="$(getent group $gid | cut -d: -f3)"
# debug/vis # debug/vis
echo echo
echo "chroot-dir = $jail" echo "chroot-dir = $jail"
echo "user:group = $uid:$gid" echo "user:group = $uid:$gid ($usr:$grp)"
echo " copyparty = $cpp" echo " copyparty = $cpp"
echo echo
printf '\033[33m%s\033[0m\n' "copyparty can access these folders and all their subdirectories:" printf '\033[33m%s\033[0m\n' "copyparty can access these folders and all their subdirectories:"
@@ -80,34 +97,39 @@ jail="${jail%/}"
# bind-mount system directories and volumes # bind-mount system directories and volumes
for a in {1..30}; do mkdir "$jail/.prisonlock" && break; sleep 0.1; done
printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | sed -r 's`/$``' | LC_ALL=C sort | uniq | printf '%s\n' "${sysdirs[@]}" "${vols[@]}" | sed -r 's`/$``' | LC_ALL=C sort | uniq |
while IFS= read -r v; do while IFS= read -r v; do
[ -e "$v" ] || { [ -e "$v" ] || {
printf '\033[1;31mfolder does not exist:\033[0m %s\n' "$v" printf '\033[1;31mfolder does not exist:\033[0m %s\n' "$v"
continue continue
} }
i1=$(stat -c%D.%i "$v" 2>/dev/null || echo a) i1=$(stat -c%D.%i "$v/" 2>/dev/null || echo a)
i2=$(stat -c%D.%i "$jail$v" 2>/dev/null || echo b) i2=$(stat -c%D.%i "$jail$v/" 2>/dev/null || echo b)
# echo "v [$v] i1 [$i1] i2 [$i2]"
[ $i1 = $i2 ] && continue [ $i1 = $i2 ] && continue
mount | grep -qF " $jail$v " && echo wtf $i1 $i2 $v && continue
mkdir -p "$jail$v" mkdir -p "$jail$v"
mount --bind "$v" "$jail$v" mount --bind "$v" "$jail$v"
done done
rmdir "$jail/.prisonlock" || true
cln() { cln() {
rv=$? trap - EXIT
wait -f -p rv $p || true wait -f -n $p && rv=0 || rv=$?
cd / cd /
echo "stopping chroot..." echo "stopping chroot..."
lsof "$jail" | grep -F "$jail" && for a in {1..30}; do mkdir "$jail/.prisonlock" && break; sleep 0.1; done
lsof "$jail" 2>/dev/null | grep -F "$jail" &&
echo "chroot is in use; will not unmount" || echo "chroot is in use; will not unmount" ||
{ {
mount | grep -F " on $jail" | mount | grep -F " on $jail" |
awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' | awk '{sub(/ type .*/,"");sub(/.* on /,"");print}' |
LC_ALL=C sort -r | tee /dev/stderr | tr '\n' '\0' | xargs -r0 umount LC_ALL=C sort -r | while IFS= read -r v; do
umount "$v" && echo "umount OK: $v"
done
} }
rmdir "$jail/.prisonlock" || true
exit $rv exit $rv
} }
trap cln EXIT trap cln EXIT
@@ -128,8 +150,8 @@ chmod 777 "$jail/tmp"
# run copyparty # run copyparty
export HOME=$(getent passwd $uid | cut -d: -f6) export HOME="$(getent passwd $uid | cut -d: -f6)"
export USER=$(getent passwd $uid | cut -d: -f1) export USER="$usr"
export LOGNAME="$USER" export LOGNAME="$USER"
#echo "pybin [$pybin]" #echo "pybin [$pybin]"
#echo "pyarg [$pyarg]" #echo "pyarg [$pyarg]"
@@ -137,5 +159,5 @@ export LOGNAME="$USER"
chroot --userspec=$uid:$gid "$jail" "$pybin" $pyarg "$cpp" "$@" & chroot --userspec=$uid:$gid "$jail" "$pybin" $pyarg "$cpp" "$@" &
p=$! p=$!
trap 'kill -USR1 $p' USR1 trap 'kill -USR1 $p' USR1
trap 'kill $p' INT TERM trap 'trap - INT TERM; kill $p' INT TERM
wait wait

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe> # Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty pkgname=copyparty
pkgver="1.9.17" pkgver="1.9.20"
pkgrel=1 pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, zeroconf, media indexer, thumbnails++" pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, zeroconf, media indexer, thumbnails++"
arch=("any") arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
) )
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz") source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" ) backup=("etc/${pkgname}.d/init" )
sha256sums=("73d66a9ff21caf45d8093829ba7de5b161fcd595ff91f8674795f426db86644c") sha256sums=("e0e267e222a22fc21491bd1a902318e095e90189ef8ece90a7aba000f730d279")
build() { build() {
cd "${srcdir}/${pkgname}-${pkgver}" cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,11 +1,11 @@
# this will start `/usr/bin/copyparty-sfx.py` # this will start `/usr/bin/copyparty-sfx.py`
# in a chroot, preventing accidental access elsewhere # in a chroot, preventing accidental access elsewhere,
# and read config from `/etc/copyparty.d/*.conf` # and read copyparty config from `/etc/copyparty.d/*.conf`
# #
# expose additional filesystem locations to copyparty # expose additional filesystem locations to copyparty
# by listing them between the last `1000` and `--` # by listing them between the last `cpp` and `--`
# #
# `1000 1000` = what user to run copyparty as # `cpp cpp` = user/group to run copyparty as; can be IDs (1000 1000)
# #
# unless you add -q to disable logging, you may want to remove the # unless you add -q to disable logging, you may want to remove the
# following line to allow buffering (slightly better performance): # following line to allow buffering (slightly better performance):
@@ -24,7 +24,9 @@ ExecReload=/bin/kill -s USR1 $MAINPID
ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf' ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
# run copyparty # run copyparty
ExecStart=/bin/bash /usr/bin/prisonparty /var/lib/copyparty-jail 1000 1000 /etc/copyparty.d -- \ ExecStart=/bin/bash /usr/bin/prisonparty /var/lib/copyparty-jail cpp cpp \
/etc/copyparty.d \
-- \
/usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init /usr/bin/python3 /usr/bin/copyparty -c /etc/copyparty.d/init
[Install] [Install]

View File

@@ -1,5 +1,5 @@
{ {
"url": "https://github.com/9001/copyparty/releases/download/v1.9.17/copyparty-sfx.py", "url": "https://github.com/9001/copyparty/releases/download/v1.9.20/copyparty-sfx.py",
"version": "1.9.17", "version": "1.9.20",
"hash": "sha256-YLl7hGWRDsFgxUvQ6hUbq+DWduhm2bs4FSZWs/AgvB0=" "hash": "sha256-KZIM5r/zi6+7I9MvYZXI9v+DQPROzg8ApGnPgEwx8BY="
} }

View File

@@ -1,5 +1,5 @@
# this will start `/usr/local/bin/copyparty-sfx.py` # this will start `/usr/local/bin/copyparty-sfx.py`
# in a chroot, preventing accidental access elsewhere # in a chroot, preventing accidental access elsewhere,
# and share '/mnt' with anonymous read+write # and share '/mnt' with anonymous read+write
# #
# installation: # installation:
@@ -7,9 +7,9 @@
# 2) cp -pv prisonparty.service /etc/systemd/system && systemctl enable --now prisonparty # 2) cp -pv prisonparty.service /etc/systemd/system && systemctl enable --now prisonparty
# #
# expose additional filesystem locations to copyparty # expose additional filesystem locations to copyparty
# by listing them between the last `1000` and `--` # by listing them between the last `cpp` and `--`
# #
# `1000 1000` = what user to run copyparty as # `cpp cpp` = user/group to run copyparty as; can be IDs (1000 1000)
# #
# you may want to: # you may want to:
# change '/mnt::rw' to another location or permission-set # change '/mnt::rw' to another location or permission-set
@@ -32,7 +32,9 @@ ExecReload=/bin/kill -s USR1 $MAINPID
ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf' ExecStartPre=+/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
# run copyparty # run copyparty
ExecStart=/bin/bash /usr/local/bin/prisonparty.sh /var/lib/copyparty-jail 1000 1000 /mnt -- \ ExecStart=/bin/bash /usr/local/bin/prisonparty.sh /var/lib/copyparty-jail cpp cpp \
/mnt \
-- \
/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw /usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
[Install] [Install]

View File

@@ -1228,6 +1228,7 @@ def add_debug(ap):
ap2.add_argument("--no-scandir", action="store_true", help="disable scandir; instead using listdir + stat on each file") ap2.add_argument("--no-scandir", action="store_true", help="disable scandir; instead using listdir + stat on each file")
ap2.add_argument("--no-fastboot", action="store_true", help="wait for up2k indexing before starting the httpd") ap2.add_argument("--no-fastboot", action="store_true", help="wait for up2k indexing before starting the httpd")
ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead") ap2.add_argument("--no-htp", action="store_true", help="disable httpserver threadpool, create threads as-needed instead")
ap2.add_argument("--srch-dbg", action="store_true", help="explain search processing, and do some extra expensive sanity checks")
ap2.add_argument("--rclone-mdns", action="store_true", help="use mdns-domain instead of server-ip on /?hc") ap2.add_argument("--rclone-mdns", action="store_true", help="use mdns-domain instead of server-ip on /?hc")
ap2.add_argument("--stackmon", metavar="P,S", type=u, help="write stacktrace to Path every S second, for example --stackmon=\033[32m./st/%%Y-%%m/%%d/%%H%%M.xz,60") ap2.add_argument("--stackmon", metavar="P,S", type=u, help="write stacktrace to Path every S second, for example --stackmon=\033[32m./st/%%Y-%%m/%%d/%%H%%M.xz,60")
ap2.add_argument("--log-thrs", metavar="SEC", type=float, help="list active threads every SEC") ap2.add_argument("--log-thrs", metavar="SEC", type=float, help="list active threads every SEC")

View File

@@ -1,8 +1,8 @@
# coding: utf-8 # coding: utf-8
VERSION = (1, 9, 18) VERSION = (1, 9, 21)
CODENAME = "prometheable" CODENAME = "prometheable"
BUILD_DT = (2023, 11, 18) BUILD_DT = (2023, 11, 25)
S_VERSION = ".".join(map(str, VERSION)) S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT) S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -9,14 +9,13 @@ import stat
import sys import sys
import time import time
from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
from pyftpdlib.authorizers import AuthenticationFailed, DummyAuthorizer from pyftpdlib.authorizers import AuthenticationFailed, DummyAuthorizer
from pyftpdlib.filesystems import AbstractedFS, FilesystemError from pyftpdlib.filesystems import AbstractedFS, FilesystemError
from pyftpdlib.handlers import FTPHandler from pyftpdlib.handlers import FTPHandler
from pyftpdlib.ioloop import IOLoop from pyftpdlib.ioloop import IOLoop
from pyftpdlib.servers import FTPServer from pyftpdlib.servers import FTPServer
from .__init__ import ANYWIN, PY2, TYPE_CHECKING, E
from .authsrv import VFS from .authsrv import VFS
from .bos import bos from .bos import bos
from .util import ( from .util import (

View File

@@ -118,7 +118,7 @@ def ffprobe(
b"--", b"--",
fsenc(abspath), fsenc(abspath),
] ]
rc, so, se = runcmd(cmd, timeout=timeout) rc, so, se = runcmd(cmd, timeout=timeout, nice=True)
retchk(rc, cmd, se) retchk(rc, cmd, se)
return parse_ffprobe(so) return parse_ffprobe(so)
@@ -562,6 +562,7 @@ class MTag(object):
args = { args = {
"env": env, "env": env,
"nice": True,
"timeout": parser.timeout, "timeout": parser.timeout,
"kill": parser.kill, "kill": parser.kill,
"capture": parser.capture, "capture": parser.capture,
@@ -572,11 +573,6 @@ class MTag(object):
zd.update(ret) zd.update(ret)
args["sin"] = json.dumps(zd).encode("utf-8", "replace") args["sin"] = json.dumps(zd).encode("utf-8", "replace")
if WINDOWS:
args["creationflags"] = 0x4000
else:
cmd = ["nice"] + cmd
bcmd = [sfsenc(x) for x in cmd[:-1]] + [fsenc(cmd[-1])] bcmd = [sfsenc(x) for x in cmd[:-1]] + [fsenc(cmd[-1])]
rc, v, err = runcmd(bcmd, **args) # type: ignore rc, v, err = runcmd(bcmd, **args) # type: ignore
retchk(rc, bcmd, err, self.log, 5, self.args.mtag_v) retchk(rc, bcmd, err, self.log, 5, self.args.mtag_v)

View File

@@ -468,7 +468,7 @@ class ThumbSrv(object):
def _run_ff(self, cmd: list[bytes], vn: VFS) -> None: def _run_ff(self, cmd: list[bytes], vn: VFS) -> None:
# self.log((b" ".join(cmd)).decode("utf-8")) # self.log((b" ".join(cmd)).decode("utf-8"))
ret, _, serr = runcmd(cmd, timeout=vn.flags["convt"]) ret, _, serr = runcmd(cmd, timeout=vn.flags["convt"], nice=True)
if not ret: if not ret:
return return

View File

@@ -9,6 +9,7 @@ import time
from operator import itemgetter from operator import itemgetter
from .__init__ import ANYWIN, TYPE_CHECKING, unicode from .__init__ import ANYWIN, TYPE_CHECKING, unicode
from .authsrv import LEELOO_DALLAS
from .bos import bos from .bos import bos
from .up2k import up2k_wark_from_hashlist from .up2k import up2k_wark_from_hashlist
from .util import ( from .util import (
@@ -20,6 +21,7 @@ from .util import (
min_ex, min_ex,
quotep, quotep,
s3dec, s3dec,
vjoin,
) )
if HAVE_SQLITE3: if HAVE_SQLITE3:
@@ -282,6 +284,11 @@ class U2idx(object):
have_mt: bool, have_mt: bool,
lim: int, lim: int,
) -> tuple[list[dict[str, Any]], list[str], bool]: ) -> tuple[list[dict[str, Any]], list[str], bool]:
if self.args.srch_dbg:
t = "searching across all %s volumes in which the user has 'r' (full read access):\n %s"
zs = "\n ".join(["/%s = %s" % (x[0], x[1]) for x in vols])
self.log(t % (len(vols), zs), 5)
done_flag: list[bool] = [] done_flag: list[bool] = []
self.active_id = "{:.6f}_{}".format( self.active_id = "{:.6f}_{}".format(
time.time(), threading.current_thread().ident time.time(), threading.current_thread().ident
@@ -300,13 +307,31 @@ class U2idx(object):
ret = [] ret = []
seen_rps: set[str] = set() seen_rps: set[str] = set()
lim = min(lim, int(self.args.srch_hits)) clamp = int(self.args.srch_hits)
if lim >= clamp:
lim = clamp
clamped = True
else:
clamped = False
taglist = {} taglist = {}
for (vtop, ptop, flags) in vols: for (vtop, ptop, flags) in vols:
if lim < 0:
break
cur = self.get_cur(ptop) cur = self.get_cur(ptop)
if not cur: if not cur:
continue continue
excl = []
for vp2 in self.asrv.vfs.all_vols.keys():
if vp2.startswith((vtop + "/").lstrip("/")) and vtop != vp2:
excl.append(vp2[len(vtop) :].lstrip("/"))
if self.args.srch_dbg:
t = "searching in volume /%s (%s), excludelist %s"
self.log(t % (vtop, ptop, excl), 5)
self.active_cur = cur self.active_cur = cur
vuv = [] vuv = []
@@ -327,6 +352,13 @@ class U2idx(object):
if rd.startswith("//") or fn.startswith("//"): if rd.startswith("//") or fn.startswith("//"):
rd, fn = s3dec(rd, fn) rd, fn = s3dec(rd, fn)
if rd in excl or any([x for x in excl if rd.startswith(x + "/")]):
if self.args.srch_dbg:
zs = vjoin(vjoin(vtop, rd), fn)
t = "database inconsistency in volume '/%s'; ignoring: %s"
self.log(t % (vtop, zs), 1)
continue
rp = quotep("/".join([x for x in [vtop, rd, fn] if x])) rp = quotep("/".join([x for x in [vtop, rd, fn] if x]))
if not dots and "/." in ("/" + rp): if not dots and "/." in ("/" + rp):
continue continue
@@ -355,6 +387,19 @@ class U2idx(object):
if lim < 0: if lim < 0:
break break
if self.args.srch_dbg:
t = "in volume '/%s': hit: %s"
self.log(t % (vtop, rp), 5)
zs = vjoin(vtop, rp)
chk_vn, _ = self.asrv.vfs.get(zs, LEELOO_DALLAS, True, False)
chk_vn = chk_vn.dbv or chk_vn
if chk_vn.vpath != vtop:
raise Exception(
"database inconsistency! in volume '/%s' (%s), found file [%s] which belongs to volume '/%s' (%s)"
% (vtop, ptop, zs, chk_vn.vpath, chk_vn.realpath)
)
seen_rps.add(rp) seen_rps.add(rp)
sret.append({"ts": int(ts), "sz": sz, "rp": rp + suf, "w": w[:16]}) sret.append({"ts": int(ts), "sz": sz, "rp": rp + suf, "w": w[:16]})
@@ -372,12 +417,16 @@ class U2idx(object):
ret.extend(sret) ret.extend(sret)
# print("[{}] {}".format(ptop, sret)) # print("[{}] {}".format(ptop, sret))
if self.args.srch_dbg:
t = "in volume '/%s': got %d hits, %d total so far"
self.log(t % (vtop, len(sret), len(ret)), 5)
done_flag.append(True) done_flag.append(True)
self.active_id = "" self.active_id = ""
ret.sort(key=itemgetter("rp")) ret.sort(key=itemgetter("rp"))
return ret, list(taglist.keys()), lim < 0 return ret, list(taglist.keys()), lim < 0 and not clamped
def terminator(self, identifier: str, done_flag: list[bool]) -> None: def terminator(self, identifier: str, done_flag: list[bool]) -> None:
for _ in range(self.timeout): for _ in range(self.timeout):

View File

@@ -2659,7 +2659,12 @@ class Up2k(object):
not ret["hash"] not ret["hash"]
and "fk" in vfs.flags and "fk" in vfs.flags
and not self.args.nw and not self.args.nw
and (cj["user"] in vfs.axs.uread or cj["user"] in vfs.axs.upget) and (
cj["user"] in vfs.axs.uread
or cj["user"] in vfs.axs.upget
or "*" in vfs.axs.uread
or "*" in vfs.axs.upget
)
): ):
alg = 2 if "fka" in vfs.flags else 1 alg = 2 if "fka" in vfs.flags else 1
ap = absreal(djoin(job["ptop"], job["prel"], job["name"])) ap = absreal(djoin(job["ptop"], job["prel"], job["name"]))

View File

@@ -2508,9 +2508,34 @@ def killtree(root: int) -> None:
pass pass
def _find_nice() -> str:
if WINDOWS:
return "" # use creationflags
try:
zs = shutil.which("nice")
if zs:
return zs
except:
pass
# busted PATHs and/or py2
for zs in ("/bin", "/sbin", "/usr/bin", "/usr/sbin"):
zs += "/nice"
if os.path.exists(zs):
return zs
return ""
NICES = _find_nice()
NICEB = NICES.encode("utf-8")
def runcmd( def runcmd(
argv: Union[list[bytes], list[str]], timeout: Optional[float] = None, **ka: Any argv: Union[list[bytes], list[str]], timeout: Optional[float] = None, **ka: Any
) -> tuple[int, str, str]: ) -> tuple[int, str, str]:
isbytes = isinstance(argv[0], (bytes, bytearray))
kill = ka.pop("kill", "t") # [t]ree [m]ain [n]one kill = ka.pop("kill", "t") # [t]ree [m]ain [n]one
capture = ka.pop("capture", 3) # 0=none 1=stdout 2=stderr 3=both capture = ka.pop("capture", 3) # 0=none 1=stdout 2=stderr 3=both
@@ -2524,13 +2549,22 @@ def runcmd(
berr: bytes berr: bytes
if ANYWIN: if ANYWIN:
if isinstance(argv[0], (bytes, bytearray)): if isbytes:
if argv[0] in CMD_EXEB: if argv[0] in CMD_EXEB:
argv[0] += b".exe" argv[0] += b".exe"
else: else:
if argv[0] in CMD_EXES: if argv[0] in CMD_EXES:
argv[0] += ".exe" argv[0] += ".exe"
if ka.pop("nice", None):
if WINDOWS:
ka["creationflags"] = 0x4000
elif NICEB:
if isbytes:
argv = [NICEB] + argv
else:
argv = [NICES] + argv
p = sp.Popen(argv, stdout=cout, stderr=cerr, **ka) p = sp.Popen(argv, stdout=cout, stderr=cerr, **ka)
if not timeout or PY2: if not timeout or PY2:
bout, berr = p.communicate(sin) bout, berr = p.communicate(sin)
@@ -2678,6 +2712,7 @@ def _parsehook(
sp_ka = { sp_ka = {
"env": env, "env": env,
"nice": True,
"timeout": tout, "timeout": tout,
"kill": kill, "kill": kill,
"capture": cap, "capture": cap,

View File

@@ -327,8 +327,8 @@ var Ls = {
"tv_xe1": "could not load textfile:\n\nerror ", "tv_xe1": "could not load textfile:\n\nerror ",
"tv_xe2": "404, file not found", "tv_xe2": "404, file not found",
"tv_lst": "list of textfiles in", "tv_lst": "list of textfiles in",
"tvt_close": "return to folder view$NHotkey: M\">❌ close", "tvt_close": "return to folder view$NHotkey: M (or Esc)\">❌ close",
"tvt_dl": "download this file\">💾 download", "tvt_dl": "download this file$NHotkey: Y\">💾 download",
"tvt_prev": "show previous document$NHotkey: i\">⬆ prev", "tvt_prev": "show previous document$NHotkey: i\">⬆ prev",
"tvt_next": "show next document$NHotkey: K\">⬇ next", "tvt_next": "show next document$NHotkey: K\">⬇ next",
"tvt_sel": "select file &nbsp; ( for cut / delete / ... )$NHotkey: S\">sel", "tvt_sel": "select file &nbsp; ( for cut / delete / ... )$NHotkey: S\">sel",
@@ -559,8 +559,9 @@ var Ls = {
"dokumentviser", "dokumentviser",
["I/K", "forr./neste fil"], ["I/K", "forr./neste fil"],
["M", "lukk tekstdokument"], ["M", "lukk tekstdokument"],
["E", "rediger tekstdokument"] ["E", "rediger tekstdokument"],
["S", "velg fil (for F2/ctrl-x/...)"] ["S", "velg fil (for F2/ctrl-x/...)"],
["Y", "last ned tekstfil"],
] ]
], ],
@@ -806,8 +807,8 @@ var Ls = {
"tv_xe1": "kunne ikke laste tekstfil:\n\nfeil ", "tv_xe1": "kunne ikke laste tekstfil:\n\nfeil ",
"tv_xe2": "404, Fil ikke funnet", "tv_xe2": "404, Fil ikke funnet",
"tv_lst": "tekstfiler i mappen", "tv_lst": "tekstfiler i mappen",
"tvt_close": "gå tilbake til mappen$NSnarvei: M\">❌ lukk", "tvt_close": "gå tilbake til mappen$NSnarvei: M (eller Esc)\">❌ lukk",
"tvt_dl": "last ned denne filen\">💾 last ned", "tvt_dl": "last ned denne filen$NSnarvei: Y\">💾 last ned",
"tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ forr.", "tvt_prev": "vis forrige dokument$NSnarvei: i\">⬆ forr.",
"tvt_next": "vis neste dokument$NSnarvei: K\">⬇ neste", "tvt_next": "vis neste dokument$NSnarvei: K\">⬇ neste",
"tvt_sel": "markér filen &nbsp; ( for utklipp / sletting / ... )$NSnarvei: S\">merk", "tvt_sel": "markér filen &nbsp; ( for utklipp / sletting / ... )$NSnarvei: S\">merk",
@@ -2243,7 +2244,7 @@ function song_skip(n, dirskip) {
return; return;
} }
if (tid) if (tid && !dirskip)
play(ofs + n); play(ofs + n);
else else
play(mp.order[n == -1 ? mp.order.length - 1 : 0]); play(mp.order[n == -1 ? mp.order.length - 1 : 0]);
@@ -2276,6 +2277,21 @@ function next_song_cmn(e) {
mpl.traversals = 0; mpl.traversals = 0;
t_fchg = 0; t_fchg = 0;
} }
function last_song(e) {
ev(e);
if (mp.order.length) {
mpl.traversals = 0;
return song_skip(-1, true);
}
if (mpl.traversals++ < 5) {
treectl.ls_cb = last_song;
return tree_neigh(-1);
}
toast.inf(10, L.mm_nof);
console.log("mm_nof2");
mpl.traversals = 0;
t_fchg = 0;
}
function prev_song(e) { function prev_song(e) {
ev(e); ev(e);
@@ -2920,7 +2936,7 @@ function play(tid, is_ev, seek) {
tn = mp.order.length - 1; tn = mp.order.length - 1;
} }
else if (mpl.pb_mode == 'next') { else if (mpl.pb_mode == 'next') {
treectl.ls_cb = prev_song; treectl.ls_cb = last_song;
return tree_neigh(-1); return tree_neigh(-1);
} }
} }
@@ -4118,7 +4134,10 @@ var showfile = (function () {
if (lang == 'md' && td.textContent != '-') if (lang == 'md' && td.textContent != '-')
continue; continue;
td.innerHTML = '<a href="#" class="doc bri" hl="' + link.id + '">-txt-</a>'; td.innerHTML = '<a href="#" id="t' +
link.id + '" class="doc bri" hl="' +
link.id + '">-txt-</a>';
td.getElementsByTagName('a')[0].setAttribute('href', '?doc=' + fn); td.getElementsByTagName('a')[0].setAttribute('href', '?doc=' + fn);
} }
r.mktree(); r.mktree();
@@ -4212,6 +4231,9 @@ var showfile = (function () {
el.textContent = txt; el.textContent = txt;
el.innerHTML = '<code>' + el.innerHTML + '</code>'; el.innerHTML = '<code>' + el.innerHTML + '</code>';
if (!window.no_prism) { if (!window.no_prism) {
if ((lang == 'conf' || lang == 'cfg') && ('\n' + txt).indexOf('\n# -*- mode: yaml -*-') + 1)
lang = 'yaml';
el.className = 'prism linkable-line-numbers line-numbers language-' + lang; el.className = 'prism linkable-line-numbers line-numbers language-' + lang;
if (!defer) if (!defer)
fun(el.firstChild); fun(el.firstChild);
@@ -4502,7 +4524,10 @@ var thegrid = (function () {
function gclick(e, dbl) { function gclick(e, dbl) {
var oth = ebi(this.getAttribute('ref')), var oth = ebi(this.getAttribute('ref')),
href = noq_href(this), href = noq_href(this),
aplay = ebi('a' + oth.getAttribute('id')), fid = oth.getAttribute('id'),
aplay = ebi('a' + fid),
atext = ebi('t' + fid),
is_txt = atext && showfile.getlang(href),
is_img = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i.test(href), is_img = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i.test(href),
is_dir = href.endsWith('/'), is_dir = href.endsWith('/'),
is_srch = !!ebi('unsearch'), is_srch = !!ebi('unsearch'),
@@ -4517,15 +4542,21 @@ var thegrid = (function () {
return r.loadsel(); return r.loadsel();
clmod(this, 'sel', clgot(tr, 'sel')); clmod(this, 'sel', clgot(tr, 'sel'));
} }
else if (widget.is_open && aplay)
aplay.click();
else if (in_tree && !have_sel) else if (in_tree && !have_sel)
in_tree.click(); in_tree.click();
else if (oth.hasAttribute('download'))
oth.click();
else if (widget.is_open && aplay)
aplay.click();
else if (is_dir && !have_sel) else if (is_dir && !have_sel)
treectl.reqls(href, true); treectl.reqls(href, true);
else if (is_txt && !has(['md', 'htm', 'html'], is_txt))
atext.click();
else if (!is_img && have_sel) else if (!is_img && have_sel)
window.open(href, '_blank'); window.open(href, '_blank');
@@ -5051,6 +5082,13 @@ document.onkeydown = function (e) {
return QS('#twobytwo').click(); return QS('#twobytwo').click();
} }
if (showfile.active()) {
if (k == 'KeyS')
showfile.tglsel();
if (k == 'KeyE' && ebi('editdoc').style.display != 'none')
ebi('editdoc').click();
}
if (thegrid.en) { if (thegrid.en) {
if (k == 'KeyS') if (k == 'KeyS')
return ebi('gridsel').click(); return ebi('gridsel').click();
@@ -5061,13 +5099,6 @@ document.onkeydown = function (e) {
if (k == 'KeyD') if (k == 'KeyD')
return QSA('#ghead a[z]')[1].click(); return QSA('#ghead a[z]')[1].click();
} }
if (showfile.active()) {
if (k == 'KeyS')
showfile.tglsel();
if (k == 'KeyE' && ebi('editdoc').style.display != 'none')
ebi('editdoc').click();
}
}; };
@@ -6062,7 +6093,7 @@ var treectl = (function () {
} }
if (tn.lead == '-') if (tn.lead == '-')
tn.lead = '<a href="?doc=' + bhref + tn.lead = '<a href="?doc=' + bhref + '" id="t' + id +
'" class="doc' + (lang ? ' bri' : '') + '" class="doc' + (lang ? ' bri' : '') +
'" hl="' + id + '" name="' + hname + '">-txt-</a>'; '" hl="' + id + '" name="' + hname + '">-txt-</a>';
@@ -6097,6 +6128,7 @@ var treectl = (function () {
setTimeout(r.tscroll, 100); setTimeout(r.tscroll, 100);
} }
} }
else ebi('lazy').innerHTML = '';
function asdf() { function asdf() {
showfile.mktree(); showfile.mktree();

View File

@@ -1,3 +1,42 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-1121-2325 `v1.9.20` nice
## new features
* expensive subprocesses (ffmpeg, parsers, hooks) will run with `nice` to reduce cpu priority
* ...so listening to flacs won't grind everything else to a halt
## bugfixes
* the "load more" search results button didn't disappear if you hit the serverside limit
* the "show all" button for huge folders didn't disappear when navigating into a smaller folder
* trying to play the previous track when you're already playing the first track in a folder would send you on a wild adventure
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-1119-1229 `v1.9.19` shadow filter
## bugfixes
* #61 Mk.II: filter search results to also handle this issue in volumes where reindexing is disabled, or (spoiler warning:) a bug in the directory indexer prevents shadowed files from being forgotten
* filekeys didn't always get included in the up2k UI for world-readable folders
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-1118-2106 `v1.9.18` cache invalidation
## bugfixes
* #61 search results could contain stale records from overlapping volumes:
* if volume `/foo` is indexed and then volume `/foo/bar` is later created, any files inside the `bar` subfolder would not become forgotten in `/foo`'s database until something in `/foo` changes, which could be never
* as a result, search results could show stale metadata from `/foo`'s database regarding files in `/foo/bar`
* fix this by dropping caches and reindexing if copyparty is started with a different list of volumes than last time
* #60 client error when ctrl-clicking search results
* icons for the close/more buttons in search results are now pillow-10.x compatible
## other changes
* `u2c.exe`: upgraded certifi to version `2023.11.17`
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-1111-1738 `v1.9.17` 11-11 # 2023-1111-1738 `v1.9.17` 11-11