Compare commits

...

24 Commits

Author SHA1 Message Date
ed
c0dacbc4dd v1.16.7 2024-12-23 00:05:49 +00:00
ed
dd6e9ea70c when idp is enabled, always daemon(up2k-rescan)
fixes a bug reported on discord;

1. run with `--idp-h-usr=iu -v=srv::A`
2. upload a file with up2k; this succeeds
3. announce an idp user: `curl -Hiu:a 127.1:3923`
4. upload another file; fails with "fs-reload"

the idp announce would `up2k.reload` which raises the
`reload_flag` and `rescan_cond`, but there is nothing
listening on `rescan_cond` because `have_e2d` was false

must assume e2d if idp is enabled, because `have_e2d` will
only be true if there are non-idp volumes with e2d enabled
2024-12-23 17:16:56 +00:00
ed
87598dcd7f recent-uploads: move rendering to js
* loads 50% faster, reducing server-load by 30%

* inhibits search engines from indexing it

* eyecandy (filter applies automatically on edit)
2024-12-20 23:52:03 +00:00
ed
3bb7b677f8 jinja optimizations 2024-12-20 16:34:17 +00:00
ed
988a7223f4 remove some footguns
in case someone writes a plugin which
expects certain params to be sanitized

note that because mojibake filenames are supported,
URLs and filepaths can still be absolutely bonkers

this fixes one known issue:
invalid rss-feed xml if ?pw contains special chars

...and somehow things now run 2% faster, idgi
2024-12-20 14:03:40 +00:00
ed
7f044372fa 18:17:14 +Mai | ed: volume bar is bad design
18:17:26  &ed | what's wrong with it
18:17:38 +Mai | that you don't know it's the volume bar before you try it
18:17:46  &ed | oh
18:17:48  &ed | yeah i guess
18:17:54 +Mai | especially when it's at 100
18:18:00  &ed | how do i fix it tho
18:19:50 +Mai | you could add an icon that's also a mute button (to not make it a useless icon)
18:22:38  &ed | i'll make the volume text always visible and include a speaker icon before it
18:23:53 +Mai | that is better at least
2024-12-19 18:49:51 +00:00
ed
552897abbc fix log colors on loss of ext.ip 2024-12-19 18:48:03 +00:00
ed
946a8c5baa u2c: fix windowtitle 2024-12-19 18:02:29 +00:00
ed
888b31aa92 update pkgs to 1.16.6 2024-12-19 01:08:34 +00:00
ed
e2dec2510f v1.16.6 2024-12-19 00:37:24 +00:00
ed
da5ad2ab9f warn on ambiguous comments in config files 2024-12-19 00:25:10 +00:00
ed
eaa4b04a22 list recent uploads
also makes the unpost lister 5x faster
2024-12-18 22:17:30 +01:00
ed
3051b13108 try to avoid printing mojibake in logs
unprintable and side-effect-inducing paths and names are hex-escaped,
preserving greppability and making log-parsing slightly more okay
2024-12-18 01:45:54 +01:00
ed
4c4e48bab7 improve dotfile handling; closes #126
when deleting a folder, any dotfiles/folders within would only
be deleted if the user had the dot-permission to see dotfiles;
this gave the confusing behavior of not removing the "empty"
folders after deleting them

fix this to only require the delete-permission, and always
delete the entire folder, including any dotfiles within

similar behavior would also apply to moves, renames, and copies;

fix moves and renames to only require the move-permission in
the source volume; dotfiles will now always be included,
regardless of whether the user does (or does not) have the
dot-permission in either the source and/or destination volumes

copying folders now also behaves more intuitively: if the user has
the dot-permission in the target volume, then dotfiles will only be
included from source folders where the user also has the dot-perm,
to prevent the user from seeing intentionally hidden files/folders
2024-12-17 22:47:34 +01:00
ed
01a3eb29cb ui: improve some eta/idle fields
cpanel db-idle-time indicator would glitch on 0.0s

upload windowtitle was %.2f seconds, but the value is int
2024-12-17 22:01:36 +01:00
ed
73f7249c5f decode and log request URLs; closes #125
as processing of a HTTP request begins (GET, HEAD, PUT, POST, ...),
the original query line is printed in its encoded form. This makes
debugging easier, since there is no ambiguity in how the client
phrased its request.

however, this results in very opaque logs for non-ascii languages;
basically a wall of percent-encoded characters. Avoid this issue
by printing an additional log-message if the URL contains `%`,
immediately below the original url-encoded entry.

also fix tests on macos, and an unrelated bad logmsg in up2k
2024-12-16 00:53:22 +01:00
ed
18c6559199 update pkgs to 1.16.5 2024-12-11 22:59:44 +00:00
ed
e66ece993f v1.16.5 2024-12-11 22:36:19 +00:00
ed
0686860624 connectpage nitpick + update dompurify 2024-12-11 22:24:31 +00:00
ed
24ce46b380 avoid chrome webworker OOM bug; closes #124
chrome (and chromium-based browsers) can OOM when:

* the OS is Windows, MacOS, or Android (but not Linux?)
* the website is hosted on a remote IP (not localhost)
* webworkers are used to read files

unfortunately this also applies to Android, which heavily relies
on webworkers to make read-speeds anywhere close to acceptable

as for android, there are diminishing returns with more than 4
webworkers (1=1x, 2=2.3x, 3=3.8x, 4=4.2x, 6=4.5x, 8=5.3x), and
limiting the number of workers to ensure at least one idle core
appears to sufficiently reduce the OOM probability

on desktop, webworkers are only necessary for hashwasm, so
limit the number of workers to 2 if crypto.subtle is available
and otherwise use the nproc-1 rule for hashwasm in workers

bug report: https://issues.chromium.org/issues/383568268
2024-12-11 22:11:54 +00:00
ed
a49bf81ff2 mdns: improve nic-ip changelog
if a NIC is brought up with several IPs,
it would only mention one of the new IPs in the logs

or if a PCIe bus crashes and all NICs drop dead,
it would only mention one of the IPs that disappeared

as both scenarios are oddly common, be more verbose
2024-12-10 00:36:58 +00:00
ed
64501fd7f1 hybrid IdP (check regular users too); closes #122
previously, when IdP was enabled, the password-based login would be
entirely disabled. This was a semi-conscious decision, based on the
assumption that you would always want to use IdP after enabling it.

it makes more sense to keep password-based login working as usual,
conditionally disengaging it for requests which contains a valid
IdP username header. This makes it possible to define fallback
users, or API-only users, and all similar escape hatches.
2024-12-08 17:18:20 +00:00
ed
db3c0b0907 nice 2024-12-07 22:24:13 +00:00
ed
edda117a7a update pkgs to 1.16.4 2024-12-07 01:10:50 +00:00
43 changed files with 861 additions and 308 deletions

View File

@@ -48,6 +48,7 @@ turn almost any device into a file server with resumable uploads/downloads using
* [shares](#shares) - share a file or folder by creating a temporary link
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [rss feeds](#rss-feeds) - monitor a folder with your RSS reader
* [recent uploads](#recent-uploads) - list all recent uploads
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
@@ -339,6 +340,9 @@ same order here too
* [Chrome issue 1352210](https://bugs.chromium.org/p/chromium/issues/detail?id=1352210) -- plaintext http may be faster at filehashing than https (but also extremely CPU-intensive)
* [Chrome issue 383568268](https://issues.chromium.org/issues/383568268) -- filereaders in webworkers can OOM / crash the browser-tab
* copyparty has a workaround which seems to work well enough
* [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files
* Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)
@@ -714,7 +718,7 @@ files go into `[ok]` if they exist (and you get a link to where it is), otherwis
### unpost
undo/delete accidental uploads
undo/delete accidental uploads using the `[🧯]` tab in the UI
![copyparty-unpost-fs8](https://user-images.githubusercontent.com/241032/129635368-3afa6634-c20f-418c-90dc-ec411f3b3897.png)
@@ -873,6 +877,17 @@ url parameters:
* uppercase = reverse-sort; `M` = oldest file first
## recent uploads
list all recent uploads by clicking "show recent uploads" in the controlpanel
will show uploader IP and upload-time if the visitor has the admin permission
* global-option `--ups-when` makes upload-time visible to all users, and not just admins
note that the [🧯 unpost](#unpost) feature is better suited for viewing *your own* recent uploads, as it includes the option to undo/delete them
## media player
plays almost every audio format there is (if the server has FFmpeg installed for on-demand transcoding)
@@ -1488,7 +1503,9 @@ replace copyparty passwords with oauth and such
you can disable the built-in password-based login system, and instead replace it with a separate piece of software (an identity provider) which will then handle authenticating / authorizing of users; this makes it possible to login with passkeys / fido2 / webauthn / yubikey / ldap / active directory / oauth / many other single-sign-on contraptions
a popular choice is [Authelia](https://www.authelia.com/) (config-file based), another one is [authentik](https://goauthentik.io/) (GUI-based, more complex)
* the regular config-defined users will be used as a fallback for requests which don't include a valid (trusted) IdP username header
some popular identity providers are [Authelia](https://www.authelia.com/) (config-file based) and [authentik](https://goauthentik.io/) (GUI-based, more complex)
there is a [docker-compose example](./docs/examples/docker/idp-authelia-traefik) which is hopefully a good starting point (alternatively see [./docs/idp.md](./docs/idp.md) if you're the DIY type)

View File

@@ -1103,7 +1103,7 @@ class Ctl(object):
nleft = self.nfiles - self.up_f
tail = "\033[K\033[u" if VT100 and not self.ar.ns else "\r"
t = "%s eta @ %s/s, %s, %d# left\033[K" % (self.eta, spd, sleft, nleft)
t = "%s eta @ %s/s, %s, %d# left" % (self.eta, spd, sleft, nleft)
if not self.hash_b:
t = " now hashing..."
eprint(txt + "\033]0;{0}\033\\\r{0}{1}".format(t, tail))

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.16.3"
pkgver="1.16.6"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" )
sha256sums=("82e79e4b27661b23986b0eba7bd38c42d5f8a2d2c278b755134ac08588a7f27e")
sha256sums=("29a119f7e238c44b0697e5858da8154d883a97ae20ecbb10393904406fa4fe06")
build() {
cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.16.3/copyparty-sfx.py",
"version": "1.16.3",
"hash": "sha256-O1yVIUJ/zy1kde4CHxeBEex2gT+HWUJ28nhHgonW6Qg="
"url": "https://github.com/9001/copyparty/releases/download/v1.16.6/copyparty-sfx.py",
"version": "1.16.6",
"hash": "sha256-gs2jSaXa0XbVbvpW1H4i/Vzovg68Usry0iHWfbddBCc="
}

View File

@@ -91,6 +91,9 @@ web/mde.html
web/mde.js
web/msg.css
web/msg.html
web/rups.css
web/rups.html
web/rups.js
web/shares.css
web/shares.html
web/shares.js

View File

@@ -1083,7 +1083,7 @@ def add_cert(ap, cert_path):
def add_auth(ap):
ses_db = os.path.join(E.cfg, "sessions.db")
ap2 = ap.add_argument_group('IdP / identity provider / user authentication options')
ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks and assume the request-header \033[33mHN\033[0m contains the username of the requesting user (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks if the request-header \033[33mHN\033[0m contains a username to associate the request with (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
@@ -1250,7 +1250,6 @@ def add_optouts(ap):
ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
ap2.add_argument("--no-up-list", action="store_true", help="don't show list of incoming files in controlpanel")
ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
@@ -1326,7 +1325,10 @@ def add_admin(ap):
ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
ap2.add_argument("--no-ups-page", action="store_true", help="disable ?ru (list of recent uploads)")
ap2.add_argument("--no-up-list", action="store_true", help="don't show list of incoming files in controlpanel")
ap2.add_argument("--dl-list", metavar="LVL", type=int, default=2, help="who can see active downloads in the controlpanel? [\033[32m0\033[0m]=nobody, [\033[32m1\033[0m]=admins, [\033[32m2\033[0m]=everyone")
ap2.add_argument("--ups-when", action="store_true", help="let everyone see upload timestamps on the ?ru page, not just admins")
def add_thumbnail(ap):
@@ -1505,6 +1507,7 @@ def add_debug(ap):
ap2.add_argument("--bf-nc", metavar="NUM", type=int, default=200, help="bak-flips: stop if there's more than \033[33mNUM\033[0m files at \033[33m--kf-dir\033[0m already; default: 6.3 GiB max (200*32M)")
ap2.add_argument("--bf-dir", metavar="PATH", type=u, default="bf", help="bak-flips: store corrupted chunks at \033[33mPATH\033[0m; default: folder named 'bf' wherever copyparty was started")
ap2.add_argument("--bf-log", metavar="PATH", type=u, default="", help="bak-flips: log corruption info to a textfile at \033[33mPATH\033[0m")
ap2.add_argument("--no-cfg-cmt-warn", action="store_true", help=argparse.SUPPRESS)
# fmt: on
@@ -1737,7 +1740,7 @@ def main(argv: Optional[list[str]] = None) -> None:
except:
lprint("\nfailed to disable quick-edit-mode:\n" + min_ex() + "\n")
if al.ansi:
if not al.ansi:
al.wintitle = ""
# propagate implications

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 16, 4)
VERSION = (1, 16, 7)
CODENAME = "COPYparty"
BUILD_DT = (2024, 12, 7)
BUILD_DT = (2024, 12, 23)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -212,7 +212,7 @@ class Lim(object):
df, du, err = get_df(abspath, True)
if err:
t = "failed to read disk space usage for [%s]: %s"
t = "failed to read disk space usage for %r: %s"
self.log(t % (abspath, err), 3)
self.dfv = 0xAAAAAAAAA # 42.6 GiB
else:
@@ -526,7 +526,7 @@ class VFS(object):
"""returns [vfsnode,fs_remainder] if user has the requested permissions"""
if relchk(vpath):
if self.log:
self.log("vfs", "invalid relpath [{}]".format(vpath))
self.log("vfs", "invalid relpath %r @%s" % (vpath, uname))
raise Pebkac(422)
cvpath = undot(vpath)
@@ -543,11 +543,11 @@ class VFS(object):
if req and uname not in d and uname != LEELOO_DALLAS:
if vpath != cvpath and vpath != "." and self.log:
ap = vn.canonical(rem)
t = "{} has no {} in [{}] => [{}] => [{}]"
self.log("vfs", t.format(uname, msg, vpath, cvpath, ap), 6)
t = "%s has no %s in %r => %r => %r"
self.log("vfs", t % (uname, msg, vpath, cvpath, ap), 6)
t = 'you don\'t have %s-access in "/%s" or below "/%s"'
raise Pebkac(err, t % (msg, cvpath, vn.vpath))
t = "you don't have %s-access in %r or below %r"
raise Pebkac(err, t % (msg, "/" + cvpath, "/" + vn.vpath))
return vn, rem
@@ -658,7 +658,7 @@ class VFS(object):
seen: list[str],
uname: str,
permsets: list[list[bool]],
wantdots: bool,
wantdots: int,
scandir: bool,
lstat: bool,
subvols: bool = True,
@@ -693,8 +693,8 @@ class VFS(object):
and fsroot in seen
):
if self.log:
t = "bailing from symlink loop,\n prev: {}\n curr: {}\n from: {}/{}"
self.log("vfs.walk", t.format(seen[-1], fsroot, self.vpath, rem), 3)
t = "bailing from symlink loop,\n prev: %r\n curr: %r\n from: %r / %r"
self.log("vfs.walk", t % (seen[-1], fsroot, self.vpath, rem), 3)
return
if "xdev" in self.flags or "xvol" in self.flags:
@@ -706,7 +706,7 @@ class VFS(object):
rm1.append(le)
_ = [vfs_ls.remove(x) for x in rm1] # type: ignore
dots_ok = wantdots and uname in dbv.axs.udot
dots_ok = wantdots and (wantdots == 2 or uname in dbv.axs.udot)
if not dots_ok:
vfs_ls = [x for x in vfs_ls if "/." not in "/" + x[0]]
@@ -760,7 +760,7 @@ class VFS(object):
# if single folder: the folder itself is the top-level item
folder = "" if flt or not wrap else (vpath.split("/")[-1].lstrip(".") or "top")
g = self.walk(folder, vrem, [], uname, [[True, False]], True, scandir, False)
g = self.walk(folder, vrem, [], uname, [[True, False]], 1, scandir, False)
for _, _, vpath, apath, files, rd, vd in g:
if flt:
files = [x for x in files if x[0] in flt]
@@ -818,8 +818,8 @@ class VFS(object):
if vdev != st.st_dev:
if self.log:
t = "xdev: {}[{}] => {}[{}]"
self.log("vfs", t.format(vdev, self.realpath, st.st_dev, ap), 3)
t = "xdev: %s[%r] => %s[%r]"
self.log("vfs", t % (vdev, self.realpath, st.st_dev, ap), 3)
return None
@@ -829,7 +829,7 @@ class VFS(object):
return vn
if self.log:
self.log("vfs", "xvol: [{}]".format(ap), 3)
self.log("vfs", "xvol: %r" % (ap,), 3)
return None
@@ -914,7 +914,7 @@ class AuthSrv(object):
self.idp_accs[uname] = gnames
t = "reinitializing due to new user from IdP: [%s:%s]"
t = "reinitializing due to new user from IdP: [%r:%r]"
self.log(t % (uname, gnames), 3)
if not broker:
@@ -1568,7 +1568,7 @@ class AuthSrv(object):
continue
if self.args.shr_v:
t = "loading %s share [%s] by [%s] => [%s]"
t = "loading %s share %r by %r => %r"
self.log(t % (s_pr, s_k, s_un, s_vp))
if s_pw:
@@ -1765,7 +1765,7 @@ class AuthSrv(object):
use = True
try:
_ = float(zs)
zs = "%sg" % (zs)
zs = "%sg" % (zs,)
except:
pass
lim.dfl = unhumanize(zs)
@@ -2181,11 +2181,11 @@ class AuthSrv(object):
if not self.args.no_voldump:
self.log(t)
if have_e2d:
if have_e2d or self.args.idp_h_usr:
t = self.chk_sqlite_threadsafe()
if t:
self.log("\n\033[{}\033[0m\n".format(t))
if have_e2d:
if not have_e2t:
t = "hint: enable multimedia indexing (artist/title/...) with argument -e2ts"
self.log(t, 6)
@@ -2538,7 +2538,7 @@ class AuthSrv(object):
return
elif self.args.chpw_v == 2:
t = "chpw: %d changed" % (len(uok))
t = "chpw: %d changed" % (len(uok),)
if urst:
t += ", \033[0munchanged:\033[35m %s" % (", ".join(list(urst)))
@@ -2696,7 +2696,7 @@ class AuthSrv(object):
[],
u,
[[True, False]],
True,
1,
not self.args.no_scandir,
False,
False,
@@ -3017,6 +3017,19 @@ def expand_config_file(
ret.append("#\033[36m closed{}\033[0m".format(ipath))
zsl = []
for ln in ret:
zs = ln.split(" #")[0]
if " #" in zs and zs.split("#")[0].strip():
zsl.append(ln)
if zsl and "no-cfg-cmt-warn" not in "\n".join(ret):
t = "\033[33mWARNING: there is less than two spaces before the # in the following config lines, so instead of assuming that this is a comment, the whole line will become part of the config value:\n\n>>> %s\n\nif you are familiar with this and would like to mute this warning, specify the global-option no-cfg-cmt-warn\n\033[0m"
t = t % ("\n>>> ".join(zsl),)
if log:
log(t)
else:
print(t, file=sys.stderr)
def upgrade_cfg_fmt(
log: Optional["NamedLogger"], args: argparse.Namespace, orig: list[str], cfg_fp: str

View File

@@ -42,14 +42,14 @@ class Fstab(object):
self.cache = {}
fs = "ext4"
msg = "failed to determine filesystem at [{}]; assuming {}\n{}"
msg = "failed to determine filesystem at %r; assuming %s\n%s"
if ANYWIN:
fs = "vfat"
try:
path = self._winpath(path)
except:
self.log(msg.format(path, fs, min_ex()), 3)
self.log(msg % (path, fs, min_ex()), 3)
return fs
path = undot(path)
@@ -61,11 +61,11 @@ class Fstab(object):
try:
fs = self.get_w32(path) if ANYWIN else self.get_unix(path)
except:
self.log(msg.format(path, fs, min_ex()), 3)
self.log(msg % (path, fs, min_ex()), 3)
fs = fs.lower()
self.cache[path] = fs
self.log("found {} at {}".format(fs, path))
self.log("found %s at %r" % (fs, path))
return fs
def _winpath(self, path: str) -> str:

View File

@@ -145,6 +145,14 @@ A_FILE = os.stat_result(
(0o644, -1, -1, 1, 1000, 1000, 8, 0x39230101, 0x39230101, 0x39230101)
)
RE_CC = re.compile(r"[\x00-\x1f]") # search always faster
RE_HSAFE = re.compile(r"[\x00-\x1f<>\"'&]") # search always much faster
RE_HOST = re.compile(r"[^][0-9a-zA-Z.:_-]") # search faster <=17ch
RE_MHOST = re.compile(r"^[][0-9a-zA-Z.:_-]+$") # match faster >=18ch
RE_K = re.compile(r"[^0-9a-zA-Z_-]") # search faster <=17ch
UPARAM_CC_OK = set("doc move tree".split())
class HttpCli(object):
"""
@@ -389,6 +397,15 @@ class HttpCli(object):
self.host = self.headers.get("x-forwarded-host") or self.host
trusted_xff = True
m = RE_HOST.search(self.host)
if m and self.host != self.args.name:
zs = self.host
t = "malicious user; illegal Host header; req(%r) host(%r) => %r"
self.log(t % (self.req, zs, zs[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, zs, "bad_host", "illegal Host header")
self.terse_reply(b"illegal Host header", 400)
return False
if self.is_banned():
return False
@@ -434,6 +451,16 @@ class HttpCli(object):
self.loud_reply(t, status=400)
return False
ptn_cc = RE_CC
m = ptn_cc.search(self.req)
if m:
zs = self.req
t = "malicious user; Cc in req0 %r => %r"
self.log(t % (zs, zs[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, zs, "cc_r0", "Cc in req0")
self.terse_reply(b"", 500)
return False
# split req into vpath + uparam
uparam = {}
if "?" not in self.req:
@@ -446,8 +473,8 @@ class HttpCli(object):
self.trailing_slash = vpath.endswith("/")
vpath = undot(vpath)
ptn = self.conn.hsrv.ptn_cc
k_safe = self.conn.hsrv.uparam_cc_ok
re_k = RE_K
k_safe = UPARAM_CC_OK
for k in arglist.split("&"):
if "=" in k:
k, zs = k.split("=", 1)
@@ -457,6 +484,14 @@ class HttpCli(object):
else:
sv = ""
m = re_k.search(k)
if m:
t = "malicious user; bad char in query key; req(%r) qk(%r) => %r"
self.log(t % (self.req, k, k[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, self.req, "bc_q", "illegal qkey")
self.terse_reply(b"", 500)
return False
k = k.lower()
uparam[k] = sv
@@ -464,23 +499,32 @@ class HttpCli(object):
continue
zs = "%s=%s" % (k, sv)
m = ptn.search(zs)
m = ptn_cc.search(zs)
if not m:
continue
hit = zs[m.span()[0] :]
t = "malicious user; Cc in query [{}] => [{!r}]"
self.log(t.format(self.req, hit), 1)
t = "malicious user; Cc in query; req(%r) qp(%r) => %r"
self.log(t % (self.req, zs, zs[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, self.req, "cc_q", "Cc in query")
self.terse_reply(b"", 500)
return False
if "k" in uparam:
m = RE_K.search(uparam["k"])
if m:
zs = uparam["k"]
t = "malicious user; illegal filekey; req(%r) k(%r) => %r"
self.log(t % (self.req, zs, zs[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, zs, "bad_k", "illegal filekey")
self.terse_reply(b"illegal filekey", 400)
return False
if self.is_vproxied:
if vpath.startswith(self.args.R):
vpath = vpath[len(self.args.R) + 1 :]
else:
t = "incorrect --rp-loc or webserver config; expected vpath starting with [{}] but got [{}]"
self.log(t.format(self.args.R, vpath), 1)
t = "incorrect --rp-loc or webserver config; expected vpath starting with %r but got %r"
self.log(t % (self.args.R, vpath), 1)
self.ouparam = uparam.copy()
@@ -518,7 +562,7 @@ class HttpCli(object):
return self.tx_qr()
if relchk(self.vpath) and (self.vpath != "*" or self.mode != "OPTIONS"):
self.log("invalid relpath [{}]".format(self.vpath))
self.log("illegal relpath; req(%r) => %r" % (self.req, "/" + self.vpath))
self.cbonk(self.conn.hsrv.gmal, self.req, "bad_vp", "invalid relpaths")
return self.tx_404() and self.keepalive
@@ -542,8 +586,14 @@ class HttpCli(object):
except:
pass
self.pw = uparam.get("pw") or self.headers.get("pw") or bauth or cookie_pw
self.uname = (
self.asrv.sesa.get(self.pw)
or self.asrv.iacct.get(self.asrv.ah.hash(self.pw))
or "*"
)
if self.args.idp_h_usr:
self.pw = ""
idp_usr = self.headers.get(self.args.idp_h_usr) or ""
if idp_usr:
idp_grp = (
@@ -588,20 +638,11 @@ class HttpCli(object):
idp_grp = ""
if idp_usr in self.asrv.vfs.aread:
self.pw = ""
self.uname = idp_usr
self.html_head += "<script>var is_idp=1</script>\n"
else:
self.log("unknown username: [%s]" % (idp_usr), 1)
self.uname = "*"
else:
self.uname = "*"
else:
self.pw = uparam.get("pw") or self.headers.get("pw") or bauth or cookie_pw
self.uname = (
self.asrv.sesa.get(self.pw)
or self.asrv.iacct.get(self.asrv.ah.hash(self.pw))
or "*"
)
self.log("unknown username: %r" % (idp_usr,), 1)
if self.args.ipu and self.uname == "*":
self.uname = self.conn.ipu_iu[self.conn.ipu_nm.map(self.ip)]
@@ -666,7 +707,7 @@ class HttpCli(object):
origin = self.headers.get("origin", "<?>")
proto = "https://" if self.is_https else "http://"
guess = "modifying" if (origin and host) else "stripping"
t = "cors-reject %s because request-header Origin='%s' does not match request-protocol '%s' and host '%s' based on request-header Host='%s' (note: if this request is not malicious, check if your reverse-proxy is accidentally %s request headers, in particular 'Origin', for example by running copyparty with --ihead='*' to show all request headers)"
t = "cors-reject %s because request-header Origin=%r does not match request-protocol %r and host %r based on request-header Host=%r (note: if this request is not malicious, check if your reverse-proxy is accidentally %s request headers, in particular 'Origin', for example by running copyparty with --ihead='*' to show all request headers)"
self.log(t % (self.mode, origin, proto, self.host, host, guess), 3)
raise Pebkac(403, "rejected by cors-check")
@@ -712,7 +753,7 @@ class HttpCli(object):
if pex.code != 404 or self.do_log:
self.log(
"http%d: %s\033[0m, %s" % (pex.code, msg, self.vpath),
"http%d: %s\033[0m, %r" % (pex.code, msg, "/" + self.vpath),
6 if em.startswith("client d/c ") else 3,
)
@@ -874,12 +915,12 @@ class HttpCli(object):
for k, zs in list(self.out_headers.items()) + self.out_headerlist:
response.append("%s: %s" % (k, zs))
ptn_cc = RE_CC
for zs in response:
m = self.conn.hsrv.ptn_cc.search(zs)
m = ptn_cc.search(zs)
if m:
hit = zs[m.span()[0] :]
t = "malicious user; Cc in out-hdr {!r} => [{!r}]"
self.log(t.format(zs, hit), 1)
t = "malicious user; Cc in out-hdr; req(%r) hdr(%r) => %r"
self.log(t % (self.req, zs, zs[m.span()[0] :]), 1)
self.cbonk(self.conn.hsrv.gmal, zs, "cc_hdr", "Cc in out-hdr")
raise Pebkac(999)
@@ -1005,7 +1046,7 @@ class HttpCli(object):
if not kv:
return ""
r = ["%s=%s" % (k, quotep(zs)) if zs else k for k, zs in kv.items()]
r = ["%s=%s" % (quotep(k), quotep(zs)) if zs else k for k, zs in kv.items()]
return "?" + "&amp;".join(r)
def ourlq(self) -> str:
@@ -1121,6 +1162,8 @@ class HttpCli(object):
logmsg += " [\033[36m" + rval + "\033[0m]"
self.log(logmsg)
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
# "embedded" resources
if self.vpath.startswith(".cpr"):
@@ -1155,8 +1198,8 @@ class HttpCli(object):
return self.tx_res(res_path)
if res_path != undot(res_path):
t = "malicious user; attempted path traversal [{}] => [{}]"
self.log(t.format(self.vpath, res_path), 1)
t = "malicious user; attempted path traversal; req(%r) vp(%r) => %r"
self.log(t % (self.req, "/" + self.vpath, res_path), 1)
self.cbonk(self.conn.hsrv.gmal, self.req, "trav", "path traversal")
self.tx_404()
@@ -1167,11 +1210,11 @@ class HttpCli(object):
return True
if not self.can_read and not self.can_write and not self.can_get:
t = "@{} has no access to [{}]"
t = "@%s has no access to %r"
if "on403" in self.vn.flags:
t += " (on403)"
self.log(t.format(self.uname, self.vpath))
self.log(t % (self.uname, "/" + self.vpath))
ret = self.on40x(self.vn.flags["on403"], self.vn, self.rem)
if ret == "true":
return True
@@ -1190,7 +1233,7 @@ class HttpCli(object):
if self.vpath:
ptn = self.args.nonsus_urls
if not ptn or not ptn.search(self.vpath):
self.log(t.format(self.uname, self.vpath))
self.log(t % (self.uname, "/" + self.vpath))
return self.tx_404(True)
@@ -1234,6 +1277,9 @@ class HttpCli(object):
if "dls" in self.uparam:
return self.tx_dls()
if "ru" in self.uparam:
return self.tx_rups()
if "h" in self.uparam:
return self.tx_mounts()
@@ -1296,8 +1342,8 @@ class HttpCli(object):
pw = self.ouparam.get("pw")
if pw:
q_pw = "?pw=%s" % (pw,)
a_pw = "&pw=%s" % (pw,)
q_pw = "?pw=%s" % (html_escape(pw, True, True),)
a_pw = "&pw=%s" % (html_escape(pw, True, True),)
for i in hits:
i["rp"] += a_pw if "?" in i["rp"] else q_pw
else:
@@ -1384,6 +1430,8 @@ class HttpCli(object):
def handle_propfind(self) -> bool:
if self.do_log:
self.log("PFIND %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if self.args.no_dav:
raise Pebkac(405, "WebDAV is disabled in server config")
@@ -1434,14 +1482,14 @@ class HttpCli(object):
if depth == "infinity":
# allow depth:0 from unmapped root, but require read-axs otherwise
if not self.can_read and (self.vpath or self.asrv.vfs.realpath):
t = "depth:infinity requires read-access in /%s"
t = t % (self.vpath,)
t = "depth:infinity requires read-access in %r"
t = t % ("/" + self.vpath,)
self.log(t, 3)
raise Pebkac(401, t)
if not stat.S_ISDIR(topdir["st"].st_mode):
t = "depth:infinity can only be used on folders; /%s is 0o%o"
t = t % (self.vpath, topdir["st"])
t = "depth:infinity can only be used on folders; %r is 0o%o"
t = t % ("/" + self.vpath, topdir["st"])
self.log(t, 3)
raise Pebkac(400, t)
@@ -1467,7 +1515,7 @@ class HttpCli(object):
elif depth == "0" or not stat.S_ISDIR(st.st_mode):
# propfind on a file; return as topdir
if not self.can_read and not self.can_get:
self.log("inaccessible: [%s]" % (self.vpath,))
self.log("inaccessible: %r" % ("/" + self.vpath,))
raise Pebkac(401, "authenticate")
elif depth == "1":
@@ -1494,7 +1542,7 @@ class HttpCli(object):
raise Pebkac(412, t.format(depth, t2))
if not self.can_read and not self.can_write and not fgen:
self.log("inaccessible: [%s]" % (self.vpath,))
self.log("inaccessible: %r" % ("/" + self.vpath,))
raise Pebkac(401, "authenticate")
fgen = itertools.chain([topdir], fgen)
@@ -1565,12 +1613,14 @@ class HttpCli(object):
def handle_proppatch(self) -> bool:
if self.do_log:
self.log("PPATCH %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if self.args.no_dav:
raise Pebkac(405, "WebDAV is disabled in server config")
if not self.can_write:
self.log("{} tried to proppatch [{}]".format(self.uname, self.vpath))
self.log("%s tried to proppatch %r" % (self.uname, "/" + self.vpath))
raise Pebkac(401, "authenticate")
from xml.etree import ElementTree as ET
@@ -1620,13 +1670,15 @@ class HttpCli(object):
def handle_lock(self) -> bool:
if self.do_log:
self.log("LOCK %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if self.args.no_dav:
raise Pebkac(405, "WebDAV is disabled in server config")
# win7+ deadlocks if we say no; just smile and nod
if not self.can_write and "Microsoft-WebDAV" not in self.ua:
self.log("{} tried to lock [{}]".format(self.uname, self.vpath))
self.log("%s tried to lock %r" % (self.uname, "/" + self.vpath))
raise Pebkac(401, "authenticate")
from xml.etree import ElementTree as ET
@@ -1655,7 +1707,7 @@ class HttpCli(object):
token = str(uuid.uuid4())
if not lk.find(r"./{DAV:}depth"):
if lk.find(r"./{DAV:}depth") is None:
depth = self.headers.get("depth", "infinity")
lk.append(mktnod("D:depth", depth))
@@ -1685,12 +1737,14 @@ class HttpCli(object):
def handle_unlock(self) -> bool:
if self.do_log:
self.log("UNLOCK %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if self.args.no_dav:
raise Pebkac(405, "WebDAV is disabled in server config")
if not self.can_write and "Microsoft-WebDAV" not in self.ua:
self.log("{} tried to lock [{}]".format(self.uname, self.vpath))
self.log("%s tried to lock %r" % (self.uname, "/" + self.vpath))
raise Pebkac(401, "authenticate")
self.send_headers(None, 204)
@@ -1702,6 +1756,8 @@ class HttpCli(object):
if self.do_log:
self.log("MKCOL %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
try:
return self._mkdir(self.vpath, True)
@@ -1753,6 +1809,8 @@ class HttpCli(object):
def handle_options(self) -> bool:
if self.do_log:
self.log("OPTIONS %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
oh = self.out_headers
oh["Allow"] = ", ".join(self.conn.hsrv.mallow)
@@ -1768,10 +1826,14 @@ class HttpCli(object):
def handle_delete(self) -> bool:
self.log("DELETE %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
return self.handle_rm([])
def handle_put(self) -> bool:
self.log("PUT %s @%s" % (self.req, self.uname))
self.log("PUT %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if not self.can_write:
t = "user %s does not have write-access under /%s"
@@ -1790,6 +1852,8 @@ class HttpCli(object):
def handle_post(self) -> bool:
self.log("POST %s @%s" % (self.req, self.uname))
if "%" in self.req:
self.log(" `-- %r" % (self.vpath,))
if self.headers.get("expect", "").lower() == "100-continue":
try:
@@ -1834,7 +1898,7 @@ class HttpCli(object):
if "save" in opt:
post_sz, _, _, _, path, _ = self.dump_to_file(False)
self.log("urlform: {} bytes, {}".format(post_sz, path))
self.log("urlform: %d bytes, %r" % (post_sz, path))
elif "print" in opt:
reader, _ = self.get_body_reader()
buf = b""
@@ -1845,8 +1909,8 @@ class HttpCli(object):
if buf:
orig = buf.decode("utf-8", "replace")
t = "urlform_raw {} @ {}\n {}\n"
self.log(t.format(len(orig), self.vpath, orig))
t = "urlform_raw %d @ %r\n %r\n"
self.log(t % (len(orig), "/" + self.vpath, orig))
try:
zb = unquote(buf.replace(b"+", b" "))
plain = zb.decode("utf-8", "replace")
@@ -1872,8 +1936,8 @@ class HttpCli(object):
plain,
)
t = "urlform_dec {} @ {}\n {}\n"
self.log(t.format(len(plain), self.vpath, plain))
t = "urlform_dec %d @ %r\n %r\n"
self.log(t % (len(plain), "/" + self.vpath, plain))
except Exception as ex:
self.log(repr(ex))
@@ -2123,7 +2187,7 @@ class HttpCli(object):
try:
ext = self.conn.hsrv.magician.ext(path)
except Exception as ex:
self.log("filetype detection failed for [{}]: {}".format(path, ex), 6)
self.log("filetype detection failed for %r: %s" % (path, ex), 6)
ext = None
if ext:
@@ -2223,8 +2287,8 @@ class HttpCli(object):
def handle_stash(self, is_put: bool) -> bool:
post_sz, sha_hex, sha_b64, remains, path, url = self.dump_to_file(is_put)
spd = self._spd(post_sz)
t = "{} wrote {}/{} bytes to {} # {}"
self.log(t.format(spd, post_sz, remains, path, sha_b64[:28])) # 21
t = "%s wrote %d/%d bytes to %r # %s"
self.log(t % (spd, post_sz, remains, path, sha_b64[:28])) # 21
ac = self.uparam.get(
"want", self.headers.get("accept", "").lower().split(";")[-1]
@@ -2254,7 +2318,7 @@ class HttpCli(object):
flags: dict[str, Any],
) -> None:
now = time.time()
t = "bad-chunk: %.3f %s %s %d %s %s %s"
t = "bad-chunk: %.3f %s %s %d %s %s %r"
t = t % (now, bad_sha, good_sha, ofs, self.ip, self.uname, ap)
self.log(t, 5)
@@ -2394,7 +2458,7 @@ class HttpCli(object):
body = json.loads(json_buf.decode(enc, "replace"))
try:
zds = {k: v for k, v in body.items()}
zds["hash"] = "%d chunks" % (len(body["hash"]))
zds["hash"] = "%d chunks" % (len(body["hash"]),)
except:
zds = body
t = "POST len=%d type=%s ip=%s user=%s req=%r json=%s"
@@ -2438,7 +2502,7 @@ class HttpCli(object):
if not bos.path.isdir(dst):
bos.makedirs(dst)
except OSError as ex:
self.log("makedirs failed [{}]".format(dst))
self.log("makedirs failed %r" % (dst,))
if not bos.path.isdir(dst):
if ex.errno == errno.EACCES:
raise Pebkac(500, "the server OS denied write-access")
@@ -2463,7 +2527,7 @@ class HttpCli(object):
# strip common suffix (uploader's folder structure)
vp_req, vp_vfs = vroots(self.vpath, vjoin(dbv.vpath, vrem))
if not ret["purl"].startswith(vp_vfs):
t = "share-mapping failed; req=[%s] dbv=[%s] vrem=[%s] n1=[%s] n2=[%s] purl=[%s]"
t = "share-mapping failed; req=%r dbv=%r vrem=%r n1=%r n2=%r purl=%r"
zt = (self.vpath, dbv.vpath, vrem, vp_req, vp_vfs, ret["purl"])
raise Pebkac(500, t % zt)
ret["purl"] = vp_req + ret["purl"][len(vp_vfs) :]
@@ -2512,13 +2576,13 @@ class HttpCli(object):
# search by query params
q = body["q"]
n = body.get("n", self.args.srch_hits)
self.log("qj: {} |{}|".format(q, n))
self.log("qj: %r |%d|" % (q, n))
hits, taglist, trunc = idx.search(self.uname, vols, q, n)
msg = len(hits)
idx.p_end = time.time()
idx.p_dur = idx.p_end - t0
self.log("q#: {} ({:.2f}s)".format(msg, idx.p_dur))
self.log("q#: %r (%.2fs)" % (msg, idx.p_dur))
order = []
for t in self.args.mte:
@@ -2629,7 +2693,7 @@ class HttpCli(object):
t = "your client is sending %d bytes which is too much (server expected %d bytes at most)"
raise Pebkac(400, t % (remains, maxsize))
t = "writing %s %s+%d #%d+%d %s"
t = "writing %r %s+%d #%d+%d %s"
chunkno = cstart0[0] // chunksize
zs = " ".join([chashes[0][:15]] + [x[:9] for x in chashes[1:]])
self.log(t % (path, cstart0, remains, chunkno, len(chashes), zs))
@@ -2741,7 +2805,7 @@ class HttpCli(object):
cinf = self.headers.get("x-up2k-stat", "")
spd = self._spd(postsize)
self.log("{:70} thank {}".format(spd, cinf))
self.log("%70s thank %r" % (spd, cinf))
self.reply(b"thank")
return True
@@ -2823,7 +2887,7 @@ class HttpCli(object):
logpwd = "%" + ub64enc(zb[:12]).decode("ascii")
if pwd != "x":
self.log("invalid password: {}".format(logpwd), 3)
self.log("invalid password: %r" % (logpwd,), 3)
self.cbonk(self.conn.hsrv.gpwd, pwd, "pw", "invalid passwords")
msg = "naw dude"
@@ -2859,7 +2923,7 @@ class HttpCli(object):
rem = sanitize_vpath(rem, "/")
fn = vfs.canonical(rem)
if not fn.startswith(vfs.realpath):
self.log("invalid mkdir [%s] [%s]" % (self.gctx, vpath), 1)
self.log("invalid mkdir %r %r" % (self.gctx, vpath), 1)
raise Pebkac(422)
if not nullwrite:
@@ -3025,9 +3089,9 @@ class HttpCli(object):
elif bos.path.exists(abspath):
try:
wunlink(self.log, abspath, vfs.flags)
t = "overwriting file with new upload: %s"
t = "overwriting file with new upload: %r"
except:
t = "toctou while deleting for ?replace: %s"
t = "toctou while deleting for ?replace: %r"
self.log(t % (abspath,))
else:
open_args = {}
@@ -3110,7 +3174,7 @@ class HttpCli(object):
f, tnam = ren_open(tnam, "wb", self.args.iobuf, **open_args)
try:
tabspath = os.path.join(fdir, tnam)
self.log("writing to {}".format(tabspath))
self.log("writing to %r" % (tabspath,))
sz, sha_hex, sha_b64 = copier(
p_data, f, hasher, max_sz, self.args.s_wr_slp
)
@@ -3295,7 +3359,7 @@ class HttpCli(object):
jmsg["files"].append(jpart)
vspd = self._spd(sz_total, False)
self.log("{} {}".format(vspd, msg))
self.log("%s %r" % (vspd, msg))
suf = ""
if not nullwrite and self.args.write_uplog:
@@ -3558,7 +3622,7 @@ class HttpCli(object):
if req == zs:
return True
t = "wrong dirkey, want %s, got %s\n vp: %s\n ap: %s"
t = "wrong dirkey, want %s, got %s\n vp: %r\n ap: %r"
self.log(t % (zs, req, self.req, ap), 6)
return False
@@ -3586,7 +3650,7 @@ class HttpCli(object):
if req == zs:
return True
t = "wrong filekey, want %s, got %s\n vp: %s\n ap: %s"
t = "wrong filekey, want %s, got %s\n vp: %r\n ap: %r"
self.log(t % (zs, req, self.req, ap), 6)
return False
@@ -3634,6 +3698,7 @@ class HttpCli(object):
return logues, readmes
def _expand(self, txt: str, phs: list[str]) -> str:
ptn_hsafe = RE_HSAFE
for ph in phs:
if ph.startswith("hdr."):
sv = str(self.headers.get(ph[4:], ""))
@@ -3648,10 +3713,10 @@ class HttpCli(object):
elif ph == "srv.htime":
sv = datetime.now(UTC).strftime("%Y-%m-%d, %H:%M:%S")
else:
self.log("unknown placeholder in server config: [%s]" % (ph), 3)
self.log("unknown placeholder in server config: [%s]" % (ph,), 3)
continue
sv = self.conn.hsrv.ptn_hsafe.sub("_", sv)
sv = ptn_hsafe.sub("_", sv)
txt = txt.replace("{{%s}}" % (ph,), sv)
return txt
@@ -3805,7 +3870,7 @@ class HttpCli(object):
self.pipes.set(req_path, job)
except Exception as ex:
if getattr(ex, "errno", 0) != errno.ENOENT:
self.log("will not pipe [%s]; %s" % (ap_data, ex), 6)
self.log("will not pipe %r; %s" % (ap_data, ex), 6)
ptop = None
#
@@ -4095,7 +4160,7 @@ class HttpCli(object):
if lower >= data_end:
if data_end:
t = "pipe: uploader is too slow; aborting download at %.2f MiB"
self.log(t % (data_end / M))
self.log(t % (data_end / M,))
raise Pebkac(416, "uploader is too slow")
raise Pebkac(416, "no data available yet; please retry in a bit")
@@ -4239,7 +4304,7 @@ class HttpCli(object):
cdis = "attachment; filename=\"{}.{}\"; filename*=UTF-8''{}.{}"
cdis = cdis.format(afn, ext, ufn, ext)
self.log(cdis)
self.log(repr(cdis))
self.send_headers(None, mime=mime, headers={"Content-Disposition": cdis})
fgen = vn.zipgen(
@@ -4902,9 +4967,9 @@ class HttpCli(object):
raise Pebkac(500, "sqlite3 not found on server; unpost is disabled")
raise Pebkac(500, "server busy, cannot unpost; please retry in a bit")
filt = self.uparam.get("filter") or ""
lm = "ups [{}]".format(filt)
self.log(lm)
zs = self.uparam.get("filter") or ""
filt = re.compile(zs, re.I) if zs else None
lm = "ups %r" % (zs,)
if self.args.shr and self.vpath.startswith(self.args.shr1):
shr_dbv, shr_vrem = self.vn.get_dbv(self.rem)
@@ -4945,13 +5010,18 @@ class HttpCli(object):
nfk, fk_alg = fk_vols.get(vol) or (0, 0)
q = "select sz, rd, fn, at from up where ip=? and at>?"
n = 2000
q = "select sz, rd, fn, at from up where ip=? and at>? order by at desc"
for sz, rd, fn, at in cur.execute(q, (self.ip, lim)):
vp = "/" + "/".join(x for x in [vol.vpath, rd, fn] if x)
if filt and filt not in vp:
if filt and not filt.search(vp):
continue
rv = {"vp": quotep(vp), "sz": sz, "at": at, "nfk": nfk}
n -= 1
if not n:
break
rv = {"vp": vp, "sz": sz, "at": at, "nfk": nfk}
if nfk:
rv["ap"] = vol.canonical(vjoin(rd, fn))
rv["fk_alg"] = fk_alg
@@ -4962,8 +5032,12 @@ class HttpCli(object):
ret = ret[:2000]
ret.sort(key=lambda x: x["at"], reverse=True) # type: ignore
n = 0
for rv in ret[:11000]:
if len(ret) > 2000:
ret = ret[:2000]
for rv in ret:
rv["vp"] = quotep(rv["vp"])
nfk = rv.pop("nfk")
if not nfk:
continue
@@ -4980,12 +5054,6 @@ class HttpCli(object):
)
rv["vp"] += "?k=" + fk[:nfk]
n += 1
if n > 2000:
break
ret = ret[:2000]
if shr_dbv:
# translate vpaths from share-target to share-url
# to satisfy access checks
@@ -5009,6 +5077,126 @@ class HttpCli(object):
self.reply(jtxt.encode("utf-8", "replace"), mime="application/json")
return True
def tx_rups(self) -> bool:
if self.args.no_ups_page:
raise Pebkac(500, "listing of recent uploads is disabled in server config")
idx = self.conn.get_u2idx()
if not idx or not hasattr(idx, "p_end"):
if not HAVE_SQLITE3:
raise Pebkac(500, "sqlite3 not found on server; recent-uploads n/a")
raise Pebkac(500, "server busy, cannot list recent uploads; please retry")
sfilt = self.uparam.get("filter") or ""
filt = re.compile(sfilt, re.I) if sfilt else None
lm = "ru %r" % (sfilt,)
self.log(lm)
ret: list[dict[str, Any]] = []
t0 = time.time()
allvols = [
x
for x in self.asrv.vfs.all_vols.values()
if "e2d" in x.flags and ("*" in x.axs.uread or self.uname in x.axs.uread)
]
fk_vols = {
vol: (vol.flags["fk"], 2 if "fka" in vol.flags else 1)
for vol in allvols
if "fk" in vol.flags and "*" not in vol.axs.uread
}
for vol in allvols:
cur = idx.get_cur(vol)
if not cur:
continue
nfk, fk_alg = fk_vols.get(vol) or (0, 0)
adm = "*" in vol.axs.uadmin or self.uname in vol.axs.uadmin
dots = "*" in vol.axs.udot or self.uname in vol.axs.udot
n = 1000
q = "select sz, rd, fn, ip, at from up where at>0 order by at desc"
for sz, rd, fn, ip, at in cur.execute(q):
vp = "/" + "/".join(x for x in [vol.vpath, rd, fn] if x)
if filt and not filt.search(vp):
continue
if not dots and "/." in vp:
continue
rv = {
"vp": vp,
"sz": sz,
"ip": ip,
"at": at,
"nfk": nfk,
"adm": adm,
}
if nfk:
rv["ap"] = vol.canonical(vjoin(rd, fn))
rv["fk_alg"] = fk_alg
ret.append(rv)
if len(ret) > 2000:
ret.sort(key=lambda x: x["at"], reverse=True) # type: ignore
ret = ret[:1000]
n -= 1
if not n:
break
ret.sort(key=lambda x: x["at"], reverse=True) # type: ignore
if len(ret) > 1000:
ret = ret[:1000]
for rv in ret:
rv["vp"] = quotep(rv["vp"])
nfk = rv.pop("nfk")
if not nfk:
continue
alg = rv.pop("fk_alg")
ap = rv.pop("ap")
try:
st = bos.stat(ap)
except:
continue
fk = self.gen_fk(
alg, self.args.fk_salt, ap, st.st_size, 0 if ANYWIN else st.st_ino
)
rv["vp"] += "?k=" + fk[:nfk]
if self.args.ups_when:
for rv in ret:
adm = rv.pop("adm")
if not adm:
rv["ip"] = "(You)" if rv["ip"] == self.ip else "(?)"
else:
for rv in ret:
adm = rv.pop("adm")
if not adm:
rv["ip"] = "(You)" if rv["ip"] == self.ip else "(?)"
rv["at"] = 0
if self.is_vproxied:
for v in ret:
v["vp"] = self.args.SR + v["vp"]
now = time.time()
self.log("%s #%d %.2fsec" % (lm, len(ret), now - t0))
ret2 = {"now": int(now), "filter": sfilt, "ups": ret}
jtxt = json.dumps(ret2, separators=(",\n", ": "))
if "j" in self.ouparam:
self.reply(jtxt.encode("utf-8", "replace"), mime="application/json")
return True
html = self.j2s("rups", this=self, v=jtxt)
self.reply(html.encode("utf-8"), status=200)
return True
def tx_shares(self) -> bool:
if self.uname == "*":
self.loud_reply("you're not logged in")
@@ -5714,7 +5902,7 @@ class HttpCli(object):
linf = stats.get(fn) or bos.lstat(fspath)
inf = bos.stat(fspath) if stat.S_ISLNK(linf.st_mode) else linf
except:
self.log("broken symlink: {}".format(repr(fspath)))
self.log("broken symlink: %r" % (fspath,))
continue
is_dir = stat.S_ISDIR(inf.st_mode)
@@ -5829,8 +6017,7 @@ class HttpCli(object):
erd_efn = s3enc(idx.mem_cur, rd, fn)
r = icur.execute(q, erd_efn)
except:
t = "tag read error, {}/{}\n{}"
self.log(t.format(rd, fn, min_ex()))
self.log("tag read error, %r / %r\n%s" % (rd, fn, min_ex()))
break
tags = {k: v for k, v in r}
@@ -5950,10 +6137,10 @@ class HttpCli(object):
if doc.lower().endswith(".md") and "exp" in vn.flags:
doctxt = self._expand(doctxt, vn.flags.get("exp_md") or [])
else:
self.log("doc 2big: [{}]".format(doc), c=6)
self.log("doc 2big: %r" % (doc,), 6)
doctxt = "( size of textfile exceeds serverside limit )"
else:
self.log("doc 404: [{}]".format(doc), c=6)
self.log("doc 404: %r" % (doc,), 6)
doctxt = "( textfile not found )"
if doctxt is not None:

View File

@@ -172,15 +172,16 @@ class HttpSrv(object):
env = jinja2.Environment()
env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
jn = [
"splash",
"shares",
"svcs",
"browser",
"browser2",
"msg",
"cf",
"md",
"mde",
"cf",
"msg",
"rups",
"shares",
"splash",
"svcs",
]
self.j2 = {x: env.get_template(x + ".html") for x in jn}
self.prism = has_resource(self.E, "web/deps/prism.js.gz")
@@ -194,10 +195,6 @@ class HttpSrv(object):
self.xff_nm = build_netmap(self.args.xff_src)
self.xff_lan = build_netmap("lan")
self.ptn_cc = re.compile(r"[\x00-\x1f]")
self.ptn_hsafe = re.compile(r"[\x00-\x1f<>\"'&]")
self.uparam_cc_ok = set("doc move tree".split())
self.mallow = "GET HEAD POST PUT DELETE OPTIONS".split()
if not self.args.no_dav:
zs = "PROPFIND PROPPATCH LOCK UNLOCK MKCOL COPY MOVE"

View File

@@ -194,7 +194,7 @@ def au_unpk(
except Exception as ex:
if ret:
t = "failed to decompress audio file [%s]: %r"
t = "failed to decompress audio file %r: %r"
log(t % (abspath, ex))
wunlink(log, ret, vn.flags if vn else VF_CAREFUL)
@@ -582,7 +582,7 @@ class MTag(object):
raise Exception()
except Exception as ex:
if self.args.mtag_v:
self.log("mutagen-err [{}] @ [{}]".format(ex, abspath), "90")
self.log("mutagen-err [%s] @ %r" % (ex, abspath), "90")
return self.get_ffprobe(abspath) if self.can_ffprobe else {}
@@ -699,8 +699,8 @@ class MTag(object):
ret[tag] = zj[tag]
except:
if self.args.mtag_v:
t = "mtag error: tagname {}, parser {}, file {} => {}"
self.log(t.format(tagname, parser.bin, abspath, min_ex()))
t = "mtag error: tagname %r, parser %r, file %r => %r"
self.log(t % (tagname, parser.bin, abspath, min_ex()), 6)
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)

View File

@@ -263,7 +263,7 @@ class SMB(object):
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
yeet("blocked by xbu server config: %r" % (vpath,))
ret = bos.open(ap, flags, *a, mode=chmod, **ka)
if wr:

View File

@@ -110,7 +110,7 @@ def errdesc(
report = ["copyparty failed to add the following files to the archive:", ""]
for fn, err in errors:
report.extend([" file: {}".format(fn), "error: {}".format(err), ""])
report.extend([" file: %r" % (fn,), "error: %s" % (err,), ""])
btxt = "\r\n".join(report).encode("utf-8", "replace")
btxt = vol_san(list(vfs.all_vols.values()), btxt)

View File

@@ -402,17 +402,17 @@ class TcpSrv(object):
if not netdevs:
continue
added = "nothing"
removed = "nothing"
add = []
rem = []
for k, v in netdevs.items():
if k not in self.netdevs:
added = "{} = {}".format(k, v)
add.append("\n\033[32m added %s = %s" % (k, v))
for k, v in self.netdevs.items():
if k not in netdevs:
removed = "{} = {}".format(k, v)
rem.append("\n\033[33mremoved %s = %s" % (k, v))
t = "network change detected:\n added {}\033[0;33m\nremoved {}"
self.log("tcpsrv", t.format(added, removed), 3)
t = "network change detected:%s%s"
self.log("tcpsrv", t % ("".join(add), "".join(rem)), 3)
self.netdevs = netdevs
self._distribute_netdevs()

View File

@@ -357,7 +357,7 @@ class Tftpd(object):
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
yeet("blocked by xbu server config: %r" % (vpath,))
if not self.args.tftp_nols and bos.path.isdir(ap):
return self._ls(vpath, "", 0, True)

View File

@@ -109,13 +109,13 @@ class ThumbCli(object):
fmt = sfmt
elif fmt[:1] == "p" and not is_au and not is_vid:
t = "cannot thumbnail [%s]: png only allowed for waveforms"
self.log(t % (rem), 6)
t = "cannot thumbnail %r: png only allowed for waveforms"
self.log(t % (rem,), 6)
return None
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)

View File

@@ -239,7 +239,7 @@ class ThumbSrv(object):
def get(self, ptop: str, rem: str, mtime: float, fmt: str) -> Optional[str]:
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
tpath = thumb_path(histpath, rem, mtime, fmt, self.fmt_ffa)
@@ -249,7 +249,7 @@ class ThumbSrv(object):
with self.mutex:
try:
self.busy[tpath].append(cond)
self.log("joined waiting room for %s" % (tpath,))
self.log("joined waiting room for %r" % (tpath,))
except:
thdir = os.path.dirname(tpath)
bos.makedirs(os.path.join(thdir, "w"))
@@ -266,11 +266,11 @@ class ThumbSrv(object):
allvols = list(self.asrv.vfs.all_vols.values())
vn = next((x for x in allvols if x.realpath == ptop), None)
if not vn:
self.log("ptop [{}] not in {}".format(ptop, allvols), 3)
self.log("ptop %r not in %s" % (ptop, allvols), 3)
vn = self.asrv.vfs.all_aps[0][1]
self.q.put((abspath, tpath, fmt, vn))
self.log("conv {} :{} \033[0m{}".format(tpath, fmt, abspath), c=6)
self.log("conv %r :%s \033[0m%r" % (tpath, fmt, abspath), 6)
while not self.stopping:
with self.mutex:
@@ -375,8 +375,8 @@ class ThumbSrv(object):
fun(ap_unpk, ttpath, fmt, vn)
break
except Exception as ex:
msg = "{} could not create thumbnail of {}\n{}"
msg = msg.format(fun.__name__, abspath, min_ex())
msg = "%s could not create thumbnail of %r\n%s"
msg = msg % (fun.__name__, abspath, min_ex())
c: Union[str, int] = 1 if "<Signals.SIG" in msg else "90"
self.log(msg, c)
if getattr(ex, "returncode", 0) != 321:

View File

@@ -136,7 +136,7 @@ class U2idx(object):
ptop = vn.realpath
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
db_path = os.path.join(histpath, "up2k.db")
@@ -151,7 +151,7 @@ class U2idx(object):
db = sqlite3.connect(uri, timeout=2, uri=True, check_same_thread=False)
cur = db.cursor()
cur.execute('pragma table_info("up")').fetchone()
self.log("ro: {}".format(db_path))
self.log("ro: %r" % (db_path,))
except:
self.log("could not open read-only: {}\n{}".format(uri, min_ex()))
# may not fail until the pragma so unset it
@@ -161,7 +161,7 @@ class U2idx(object):
# on windows, this steals the write-lock from up2k.deferred_init --
# seen on win 10.0.17763.2686, py 3.10.4, sqlite 3.37.2
cur = sqlite3.connect(db_path, timeout=2, check_same_thread=False).cursor()
self.log("opened {}".format(db_path))
self.log("opened %r" % (db_path,))
self.cur[ptop] = cur
return cur

View File

@@ -794,7 +794,7 @@ class Up2k(object):
if ccd != cd:
continue
self.log("xiu: {}# {}".format(len(wrfs), cmd))
self.log("xiu: %d# %r" % (len(wrfs), cmd))
runihook(self.log, cmd, vol, ups)
def _vis_job_progress(self, job: dict[str, Any]) -> str:
@@ -856,9 +856,9 @@ class Up2k(object):
self.iacct = self.asrv.iacct
self.grps = self.asrv.grps
have_e2d = self.args.idp_h_usr
vols = list(all_vols.values())
t0 = time.time()
have_e2d = False
if self.no_expr_idx:
modified = False
@@ -1060,7 +1060,7 @@ class Up2k(object):
"""mutex(main,reg) me"""
histpath = self.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
self.log("no histpath for %r" % (ptop,))
return None
db_path = os.path.join(histpath, "up2k.db")
@@ -1154,7 +1154,7 @@ class Up2k(object):
job["poke"] = time.time()
job["busy"] = {}
else:
self.log("ign deleted file in snap: [{}]".format(fp))
self.log("ign deleted file in snap: %r" % (fp,))
if not n4g:
rm.append(k)
continue
@@ -1386,12 +1386,12 @@ class Up2k(object):
xvol: bool,
) -> tuple[int, int, int]:
if xvol and not rcdir.startswith(top):
self.log("skip xvol: [{}] -> [{}]".format(cdir, rcdir), 6)
self.log("skip xvol: %r -> %r" % (cdir, rcdir), 6)
return 0, 0, 0
if rcdir in seen:
t = "bailing from symlink loop,\n prev: {}\n curr: {}\n from: {}"
self.log(t.format(seen[-1], rcdir, cdir), 3)
t = "bailing from symlink loop,\n prev: %r\n curr: %r\n from: %r"
self.log(t % (seen[-1], rcdir, cdir), 3)
return 0, 0, 0
# total-files-added, total-num-files, recursive-size
@@ -1447,7 +1447,7 @@ class Up2k(object):
and inf.st_dev != dev
and not (ANYWIN and bos.stat(rap).st_dev == dev)
):
self.log("skip xdev {}->{}: {}".format(dev, inf.st_dev, abspath), 6)
self.log("skip xdev %s->%s: %r" % (dev, inf.st_dev, abspath), 6)
continue
if abspath in excl or rap in excl:
unreg.append(rp)
@@ -1476,10 +1476,10 @@ class Up2k(object):
tnf += i2
rsz += i3
except:
t = "failed to index subdir [{}]:\n{}"
self.log(t.format(abspath, min_ex()), c=1)
t = "failed to index subdir %r:\n%s"
self.log(t % (abspath, min_ex()), 1)
elif not stat.S_ISREG(inf.st_mode):
self.log("skip type-0%o file [%s]" % (inf.st_mode, abspath))
self.log("skip type-0%o file %r" % (inf.st_mode, abspath))
else:
# self.log("file: {}".format(abspath))
if rp.endswith(".PARTIAL") and time.time() - lmod < 60:
@@ -1562,7 +1562,7 @@ class Up2k(object):
db.c.execute("insert into cv values (?,?,?)", (crd, cdn, cv))
db.n += 1
except Exception as ex:
self.log("cover {}/{} failed: {}".format(rd, cv, ex), 6)
self.log("cover %r/%r failed: %s" % (rd, cv, ex), 6)
seen_files = set([x[2] for x in files]) # for dropcheck
for sz, lmod, fn in files:
@@ -1584,9 +1584,9 @@ class Up2k(object):
self.pp.n -= 1
dw, dts, dsz, ip, at = in_db[0]
if len(in_db) > 1:
t = "WARN: multiple entries: [{}] => [{}] |{}|\n{}"
t = "WARN: multiple entries: %r => %r |%d|\n%r"
rep_db = "\n".join([repr(x) for x in in_db])
self.log(t.format(top, rp, len(in_db), rep_db))
self.log(t % (top, rp, len(in_db), rep_db))
dts = -1
if fat32 and abs(dts - lmod) == 1:
@@ -1595,10 +1595,8 @@ class Up2k(object):
if dts == lmod and dsz == sz and (nohash or dw[0] != "#" or not sz):
continue
t = "reindex [{}] => [{}] mtime({}/{}) size({}/{})".format(
top, rp, dts, lmod, dsz, sz
)
self.log(t)
t = "reindex %r => %r mtime(%s/%s) size(%s/%s)"
self.log(t % (top, rp, dts, lmod, dsz, sz))
self.db_rm(db.c, rd, fn, 0)
tfa += 1
db.n += 1
@@ -1614,14 +1612,14 @@ class Up2k(object):
wark = up2k_wark_from_metadata(self.salt, sz, lmod, rd, fn)
else:
if sz > 1024 * 1024:
self.log("file: {}".format(abspath))
self.log("file: %r" % (abspath,))
try:
hashes, _ = self._hashlist_from_file(
abspath, "a{}, ".format(self.pp.n)
)
except Exception as ex:
self.log("hash: {} @ [{}]".format(repr(ex), abspath))
self.log("hash: %r @ %r" % (ex, abspath))
continue
if not hashes:
@@ -1667,8 +1665,8 @@ class Up2k(object):
assert erd_erd # type: ignore # !rm
if n:
t = "forgetting {} shadowed autoindexed files in [{}] > [{}]"
self.log(t.format(n, top, sh_rd))
t = "forgetting %d shadowed autoindexed files in %r > %r"
self.log(t % (n, top, sh_rd))
q = "delete from dh where (d = ? or d like ?||'/%')"
db.c.execute(q, erd_erd)
@@ -1865,7 +1863,7 @@ class Up2k(object):
stl = bos.lstat(abspath)
st = bos.stat(abspath) if stat.S_ISLNK(stl.st_mode) else stl
except Exception as ex:
self.log("missing file: %s" % (abspath,), 3)
self.log("missing file: %r" % (abspath,), 3)
f404.append((drd, dfn, w))
continue
@@ -1876,12 +1874,12 @@ class Up2k(object):
w2 = up2k_wark_from_metadata(self.salt, sz2, mt2, rd, fn)
else:
if sz2 > 1024 * 1024 * 32:
self.log("file: {}".format(abspath))
self.log("file: %r" % (abspath,))
try:
hashes, _ = self._hashlist_from_file(abspath, pf)
except Exception as ex:
self.log("hash: {} @ [{}]".format(repr(ex), abspath))
self.log("hash: %r @ %r" % (ex, abspath))
continue
if not hashes:
@@ -1901,9 +1899,8 @@ class Up2k(object):
rewark.append((drd, dfn, w2, sz2, mt2))
t = "hash mismatch: {}\n db: {} ({} byte, {})\n fs: {} ({} byte, {})"
t = t.format(abspath, w, sz, mt, w2, sz2, mt2)
self.log(t, 1)
t = "hash mismatch: %r\n db: %s (%d byte, %d)\n fs: %s (%d byte, %d)"
self.log(t % (abspath, w, sz, mt, w2, sz2, mt2), 1)
if e2vp and (rewark or f404):
self.hub.retcode = 1
@@ -2451,7 +2448,7 @@ class Up2k(object):
q.task_done()
def _log_tag_err(self, parser: Any, abspath: str, ex: Any) -> None:
msg = "{} failed to read tags from {}:\n{}".format(parser, abspath, ex)
msg = "%s failed to read tags from %r:\n%s" % (parser, abspath, ex)
self.log(msg.lstrip(), c=1 if "<Signals.SIG" in msg else 3)
def _tagscan_file(
@@ -2991,11 +2988,11 @@ class Up2k(object):
job = rj
break
else:
self.log("asserting contents of %s" % (orig_ap,))
self.log("asserting contents of %r" % (orig_ap,))
hashes2, st = self._hashlist_from_file(orig_ap)
wark2 = up2k_wark_from_hashlist(self.salt, st.st_size, hashes2)
if dwark != wark2:
t = "will not dedup (fs index desync): fs=%s, db=%s, file: %s\n%s"
t = "will not dedup (fs index desync): fs=%s, db=%s, file: %r\n%s"
self.log(t % (wark2, dwark, orig_ap, rj))
lost.append(dupe[3:])
continue
@@ -3013,8 +3010,8 @@ class Up2k(object):
if lost:
c2 = None
for cur, dp_dir, dp_fn in lost:
t = "forgetting desynced db entry: /{}"
self.log(t.format(vjoin(vjoin(vfs.vpath, dp_dir), dp_fn)))
t = "forgetting desynced db entry: %r"
self.log(t % ("/" + vjoin(vjoin(vfs.vpath, dp_dir), dp_fn)))
self.db_rm(cur, dp_dir, dp_fn, cj["size"])
if c2 and c2 != cur:
c2.connection.commit()
@@ -3043,8 +3040,8 @@ class Up2k(object):
except:
# missing; restart
if not self.args.nw and not n4g:
t = "forgetting deleted partial upload at {}"
self.log(t.format(path))
t = "forgetting deleted partial upload at %r"
self.log(t % (path,))
del reg[wark]
break
@@ -3055,19 +3052,25 @@ class Up2k(object):
pass
elif st.st_size != rj["size"]:
t = "will not dedup (fs index desync): {}, size fs={} db={}, mtime fs={} db={}, file: {}\n{}"
t = t.format(
wark, st.st_size, rj["size"], st.st_mtime, rj["lmod"], path, rj
t = "will not dedup (fs index desync): %s, size fs=%d db=%d, mtime fs=%d db=%d, file: %r\n%s"
t = t % (
wark,
st.st_size,
rj["size"],
st.st_mtime,
rj["lmod"],
path,
rj,
)
self.log(t)
del reg[wark]
elif inc_ap != orig_ap and not data_ok and "done" in reg[wark]:
self.log("asserting contents of %s" % (orig_ap,))
self.log("asserting contents of %r" % (orig_ap,))
hashes2, _ = self._hashlist_from_file(orig_ap)
wark2 = up2k_wark_from_hashlist(self.salt, st.st_size, hashes2)
if wark != wark2:
t = "will not dedup (fs index desync): fs=%s, idx=%s, file: %s\n%s"
t = "will not dedup (fs index desync): fs=%s, idx=%s, file: %r\n%s"
self.log(t % (wark2, wark, orig_ap, rj))
del reg[wark]
@@ -3084,7 +3087,7 @@ class Up2k(object):
vsrc = djoin(job["vtop"], job["prel"], job["name"])
vsrc = vsrc.replace("\\", "/") # just for prints anyways
if "done" not in job:
self.log("unfinished:\n {0}\n {1}".format(src, dst))
self.log("unfinished:\n %r\n %r" % (src, dst))
err = "partial upload exists at a different location; please resume uploading here instead:\n"
err += "/" + quotep(vsrc) + " "
@@ -3101,7 +3104,7 @@ class Up2k(object):
raise Pebkac(422, err)
elif "nodupe" in vfs.flags:
self.log("dupe-reject:\n {0}\n {1}".format(src, dst))
self.log("dupe-reject:\n %r\n %r" % (src, dst))
err = "upload rejected, file already exists:\n"
err += "/" + quotep(vsrc) + " "
raise Pebkac(409, err)
@@ -3163,7 +3166,7 @@ class Up2k(object):
"",
)
if not hr:
t = "upload blocked by xbu server config: %s" % (dst,)
t = "upload blocked by xbu server config: %r" % (dst,)
self.log(t, 1)
raise Pebkac(403, t)
if hr.get("reloc"):
@@ -3304,7 +3307,7 @@ class Up2k(object):
times = (int(time.time()), int(cj["lmod"]))
bos.utime(ap, times, False)
self.log("touched %s from %d to %d" % (ap, job["lmod"], cj["lmod"]))
self.log("touched %r from %d to %d" % (ap, job["lmod"], cj["lmod"]))
except Exception as ex:
self.log("umod failed, %r" % (ex,), 3)
@@ -3319,7 +3322,7 @@ class Up2k(object):
fp = djoin(fdir, fname)
if job.get("replace") and bos.path.exists(fp):
self.log("replacing existing file at {}".format(fp))
self.log("replacing existing file at %r" % (fp,))
cur = None
ptop = job["ptop"]
vf = self.flags.get(ptop) or {}
@@ -3362,9 +3365,9 @@ class Up2k(object):
raise Exception(t % (src, fsrc, dst))
if verbose:
t = "linking dupe:\n point-to: {0}\n link-loc: {1}"
t = "linking dupe:\n point-to: {0!r}\n link-loc: {1!r}"
if fsrc:
t += "\n data-src: {2}"
t += "\n data-src: {2!r}"
self.log(t.format(src, dst, fsrc))
if self.args.nw:
@@ -3434,7 +3437,7 @@ class Up2k(object):
elif fsrc and bos.path.isfile(fsrc):
csrc = fsrc
else:
t = "BUG: no valid sources to link from! orig(%s) fsrc(%s) link(%s)"
t = "BUG: no valid sources to link from! orig(%r) fsrc(%r) link(%r)"
self.log(t, 1)
raise Exception(t % (src, fsrc, dst))
shutil.copy2(fsenc(csrc), fsenc(dst))
@@ -3612,15 +3615,12 @@ class Up2k(object):
atomic_move(self.log, src, dst, vflags)
times = (int(time.time()), int(job["lmod"]))
self.log(
"no more chunks, setting times {} ({}) on {}".format(
times, bos.path.getsize(dst), dst
)
)
t = "no more chunks, setting times %s (%d) on %r"
self.log(t % (times, bos.path.getsize(dst), dst))
try:
bos.utime(dst, times)
except:
self.log("failed to utime ({}, {})".format(dst, times))
self.log("failed to utime (%r, %s)" % (dst, times))
zs = "prel name lmod size ptop vtop wark dwrk host user addr"
z2 = [job[x] for x in zs.split()]
@@ -3937,7 +3937,7 @@ class Up2k(object):
if jrem == rem:
if job["ptop"] != ptop:
t = "job.ptop [%s] != vol.ptop [%s] ??"
raise Exception(t % (job["ptop"] != ptop))
raise Exception(t % (job["ptop"], ptop))
partial = vn.canonical(vjoin(job["prel"], job["tnam"]))
break
if partial:
@@ -3977,7 +3977,7 @@ class Up2k(object):
if is_dir:
# note: deletion inside shares would require a rewrite here;
# shares necessitate get_dbv which is incompatible with walk
g = vn0.walk("", rem0, [], uname, permsets, True, scandir, True)
g = vn0.walk("", rem0, [], uname, permsets, 2, scandir, True)
if unpost:
raise Pebkac(400, "cannot unpost folders")
elif stat.S_ISLNK(st.st_mode) or stat.S_ISREG(st.st_mode):
@@ -3985,7 +3985,7 @@ class Up2k(object):
vpath_dir = vsplit(vpath)[0]
g = [(vn, voldir, vpath_dir, adir, [(fn, 0)], [], {})] # type: ignore
else:
self.log("rm: skip type-0%o file [%s]" % (st.st_mode, atop))
self.log("rm: skip type-0%o file %r" % (st.st_mode, atop))
return 0, [], []
xbd = vn.flags.get("xbd")
@@ -4009,7 +4009,7 @@ class Up2k(object):
volpath = ("%s/%s" % (vrem, fn)).strip("/")
vpath = ("%s/%s" % (dbv.vpath, volpath)).strip("/")
self.log("rm %s\n %s" % (vpath, abspath))
self.log("rm %r\n %r" % (vpath, abspath))
if not unpost:
# recursion-only sanchk
_ = dbv.get(volpath, uname, *permsets[0])
@@ -4032,8 +4032,8 @@ class Up2k(object):
time.time(),
"",
):
t = "delete blocked by xbd server config: {}"
self.log(t.format(abspath), 1)
t = "delete blocked by xbd server config: %r"
self.log(t % (abspath,), 1)
continue
n_files += 1
@@ -4090,6 +4090,7 @@ class Up2k(object):
raise Pebkac(400, "cp: cannot copy parent into subfolder")
svn, srem = self.vfs.get(svp, uname, True, False)
dvn, drem = self.vfs.get(dvp, uname, False, True)
svn_dbv, _ = svn.get_dbv(srem)
sabs = svn.canonical(srem, False)
curs: set["sqlite3.Cursor"] = set()
@@ -4112,8 +4113,12 @@ class Up2k(object):
permsets = [[True, False]]
scandir = not self.args.no_scandir
# if user can see dotfiles in target volume, only include
# dots from source vols where user also has the dot perm
dots = 1 if uname in dvn.axs.udot else 2
# don't use svn_dbv; would skip subvols due to _ls `if not rem:`
g = svn.walk("", srem, [], uname, permsets, True, scandir, True)
g = svn.walk("", srem, [], uname, permsets, dots, scandir, True)
with self.mutex:
try:
for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4122,7 +4127,7 @@ class Up2k(object):
svpf = "/".join(x for x in [dbv.vpath, vrem, fn[0]] if x)
if not svpf.startswith(svp + "/"): # assert
self.log(min_ex(), 1)
t = "cp: bug at %s, top %s%s"
t = "cp: bug at %r, top %r%s"
raise Pebkac(500, t % (svpf, svp, SEESLOG))
dvpf = dvp + svpf[len(svp) :]
@@ -4162,7 +4167,7 @@ class Up2k(object):
except:
pass # broken symlink; keep as-is
elif not stat.S_ISREG(st.st_mode):
self.log("skipping type-0%o file [%s]" % (st.st_mode, sabs))
self.log("skipping type-0%o file %r" % (st.st_mode, sabs))
return ""
else:
is_link = False
@@ -4190,7 +4195,7 @@ class Up2k(object):
time.time(),
"",
):
t = "copy blocked by xbr server config: {}".format(svp)
t = "copy blocked by xbr server config: %r" % (svp,)
self.log(t, 1)
raise Pebkac(405, t)
@@ -4227,7 +4232,7 @@ class Up2k(object):
)
curs.add(c2)
else:
self.log("not found in src db: [{}]".format(svp))
self.log("not found in src db: %r" % (svp,))
try:
if is_link and st != stl:
@@ -4244,7 +4249,7 @@ class Up2k(object):
if ex.errno != errno.EXDEV:
raise
self.log("using plain copy (%s):\n %s\n %s" % (ex.strerror, sabs, dabs))
self.log("using plain copy (%s):\n %r\n %r" % (ex.strerror, sabs, dabs))
b1, b2 = fsenc(sabs), fsenc(dabs)
is_link = os.path.islink(b1) # due to _relink
try:
@@ -4325,13 +4330,13 @@ class Up2k(object):
scandir = not self.args.no_scandir
# following symlinks is too scary
g = svn.walk("", srem, [], uname, permsets, True, scandir, True)
g = svn.walk("", srem, [], uname, permsets, 2, scandir, True)
for dbv, vrem, _, atop, files, rd, vd in g:
if dbv != jail:
# fail early (prevent partial moves)
raise Pebkac(400, "mv: source folder contains other volumes")
g = svn.walk("", srem, [], uname, permsets, True, scandir, True)
g = svn.walk("", srem, [], uname, permsets, 2, scandir, True)
with self.mutex:
try:
for dbv, vrem, _, atop, files, rd, vd in g:
@@ -4344,7 +4349,7 @@ class Up2k(object):
svpf = "/".join(x for x in [dbv.vpath, vrem, fn[0]] if x)
if not svpf.startswith(svp + "/"): # assert
self.log(min_ex(), 1)
t = "mv: bug at %s, top %s%s"
t = "mv: bug at %r, top %r%s"
raise Pebkac(500, t % (svpf, svp, SEESLOG))
dvpf = dvp + svpf[len(svp) :]
@@ -4363,7 +4368,7 @@ class Up2k(object):
for ap in reversed(zsl):
if not ap.startswith(sabs):
self.log(min_ex(), 1)
t = "mv_d: bug at %s, top %s%s"
t = "mv_d: bug at %r, top %r%s"
raise Pebkac(500, t % (ap, sabs, SEESLOG))
rem = ap[len(sabs) :].replace(os.sep, "/").lstrip("/")
@@ -4434,7 +4439,7 @@ class Up2k(object):
time.time(),
"",
):
t = "move blocked by xbr server config: {}".format(svp)
t = "move blocked by xbr server config: %r" % (svp,)
self.log(t, 1)
raise Pebkac(405, t)
@@ -4444,8 +4449,8 @@ class Up2k(object):
if is_dirlink:
dlabs = absreal(sabs)
t = "moving symlink from [{}] to [{}], target [{}]"
self.log(t.format(sabs, dabs, dlabs))
t = "moving symlink from %r to %r, target %r"
self.log(t % (sabs, dabs, dlabs))
mt = bos.path.getmtime(sabs, False)
wunlink(self.log, sabs, svn.flags)
self._symlink(dlabs, dabs, dvn.flags, False, lmod=mt)
@@ -4517,7 +4522,7 @@ class Up2k(object):
)
curs.add(c2)
else:
self.log("not found in src db: [{}]".format(svp))
self.log("not found in src db: %r" % (svp,))
try:
if is_xvol and has_dupes:
@@ -4538,7 +4543,7 @@ class Up2k(object):
if ex.errno != errno.EXDEV:
raise
self.log("using copy+delete (%s):\n %s\n %s" % (ex.strerror, sabs, dabs))
self.log("using copy+delete (%s):\n %r\n %r" % (ex.strerror, sabs, dabs))
b1, b2 = fsenc(sabs), fsenc(dabs)
is_link = os.path.islink(b1) # due to _relink
try:
@@ -4646,7 +4651,7 @@ class Up2k(object):
"""
srd, sfn = vsplit(vrem)
has_dupes = False
self.log("forgetting {}".format(vrem))
self.log("forgetting %r" % (vrem,))
if wark and cur:
self.log("found {} in db".format(wark))
if drop_tags:
@@ -4722,7 +4727,7 @@ class Up2k(object):
dvrem = vjoin(rd, fn).strip("/")
if ptop != sptop or srem != dvrem:
dupes.append([ptop, dvrem])
self.log("found {} dupe: [{}] {}".format(wark, ptop, dvrem))
self.log("found %s dupe: %r %r" % (wark, ptop, dvrem))
if not dupes:
return 0
@@ -4735,7 +4740,7 @@ class Up2k(object):
d = links if bos.path.islink(ap) else full
d[ap] = (ptop, vp)
except:
self.log("relink: not found: [{}]".format(ap))
self.log("relink: not found: %r" % (ap,))
# self.log("full:\n" + "\n".join(" {:90}: {}".format(*x) for x in full.items()))
# self.log("links:\n" + "\n".join(" {:90}: {}".format(*x) for x in links.items()))
@@ -4743,7 +4748,7 @@ class Up2k(object):
# deleting final remaining full copy; swap it with a symlink
slabs = list(sorted(links.keys()))[0]
ptop, rem = links.pop(slabs)
self.log("linkswap [{}] and [{}]".format(sabs, slabs))
self.log("linkswap %r and %r" % (sabs, slabs))
mt = bos.path.getmtime(slabs, False)
flags = self.flags.get(ptop) or {}
atomic_move(self.log, sabs, slabs, flags)
@@ -4778,7 +4783,7 @@ class Up2k(object):
zs = absreal(alink)
if ldst != zs:
t = "relink because computed != actual destination:\n %s\n %s"
t = "relink because computed != actual destination:\n %r\n %r"
self.log(t % (ldst, zs), 3)
ldst = zs
faulty = True
@@ -4793,7 +4798,7 @@ class Up2k(object):
t = "relink because symlink verification failed: %s; %r"
self.log(t % (ex, ex), 3)
self.log("relinking [%s] to [%s]" % (alink, dabs))
self.log("relinking %r to %r" % (alink, dabs))
flags = self.flags.get(parts[0]) or {}
try:
lmod = bos.path.getmtime(alink, False)
@@ -4908,7 +4913,7 @@ class Up2k(object):
"",
)
if not hr:
t = "upload blocked by xbu server config: {}".format(vp_chk)
t = "upload blocked by xbu server config: %r" % (vp_chk,)
self.log(t, 1)
raise Pebkac(403, t)
if hr.get("reloc"):
@@ -4958,7 +4963,7 @@ class Up2k(object):
try:
sp.check_call(["fsutil", "sparse", "setflag", abspath])
except:
self.log("could not sparse [{}]".format(abspath), 3)
self.log("could not sparse %r" % (abspath,), 3)
relabel = True
sprs = False
@@ -5145,7 +5150,7 @@ class Up2k(object):
self._tag_file(cur, entags, wark, abspath, tags)
cur.connection.commit()
self.log("tagged {} ({}+{})".format(abspath, ntags1, len(tags) - ntags1))
self.log("tagged %r (%d+%d)" % (abspath, ntags1, len(tags) - ntags1))
def _hasher(self) -> None:
with self.hashq_mutex:
@@ -5164,7 +5169,7 @@ class Up2k(object):
if not self._hash_t(task) and self.stop:
return
except Exception as ex:
self.log("failed to hash %s: %s" % (task, ex), 1)
self.log("failed to hash %r: %s" % (task, ex), 1)
def _hash_t(
self, task: tuple[str, str, dict[str, Any], str, str, str, float, str, bool]
@@ -5176,7 +5181,7 @@ class Up2k(object):
return True
abspath = djoin(ptop, rd, fn)
self.log("hashing " + abspath)
self.log("hashing %r" % (abspath,))
inf = bos.stat(abspath)
if not inf.st_size:
wark = up2k_wark_from_metadata(
@@ -5269,7 +5274,7 @@ class Up2k(object):
x = pathmod(self.vfs, "", req_vp, {"vp": fvp, "fn": fn})
if not x:
t = "hook_fx(%s): failed to resolve %s based on %s"
t = "hook_fx(%s): failed to resolve %r based on %r"
self.log(t % (act, fvp, req_vp))
continue

View File

@@ -1024,7 +1024,7 @@ class ProgressPrinter(threading.Thread):
now = time.time()
if msg and now - tp > 10:
tp = now
self.log("progress: %s" % (msg,), 6)
self.log("progress: %r" % (msg,), 6)
if no_stdout:
continue
@@ -1626,7 +1626,7 @@ class MultipartParser(object):
(only the fallback non-js uploader relies on these filenames)
"""
for ln in read_header(self.sr, 2, 2592000):
self.log(ln)
self.log(repr(ln))
m = self.re_ctype.match(ln)
if m:
@@ -1917,11 +1917,11 @@ def gen_filekey_dbg(
if p2 != fspath:
raise Exception()
except:
t = "maybe wrong abspath for filekey;\norig: {}\nreal: {}"
log(t.format(fspath, p2), 1)
t = "maybe wrong abspath for filekey;\norig: %r\nreal: %r"
log(t % (fspath, p2), 1)
t = "fk({}) salt({}) size({}) inode({}) fspath({}) at({})"
log(t.format(ret[:8], salt, fsize, inode, fspath, ctx), 5)
t = "fk(%s) salt(%s) size(%d) inode(%d) fspath(%r) at(%s)"
log(t % (ret[:8], salt, fsize, inode, fspath, ctx), 5)
return ret
@@ -2277,7 +2277,7 @@ def log_reloc(
rem: str,
) -> None:
nap, nvp, nfn, (nvn, nrem) = pm
t = "reloc %s:\nold ap [%s]\nnew ap [%s\033[36m/%s\033[0m]\nold vp [%s]\nnew vp [%s\033[36m/%s\033[0m]\nold fn [%s]\nnew fn [%s]\nold vfs [%s]\nnew vfs [%s]\nold rem [%s]\nnew rem [%s]"
t = "reloc %s:\nold ap %r\nnew ap %r\033[36m/%r\033[0m\nold vp %r\nnew vp %r\033[36m/%r\033[0m\nold fn %r\nnew fn %r\nold vfs %r\nnew vfs %r\nold rem %r\nnew rem %r"
log(t % (re, ap, nap, nfn, vp, nvp, nfn, fn, nfn, vn.vpath, nvn.vpath, rem, nrem))
@@ -2448,7 +2448,7 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
try:
rc, so, se = runcmd([b"lsof", b"-R", fsenc(abspath)], timeout=45)
zs = (so.strip() + "\n" + se.strip()).strip()
log("lsof {} = {}\n{}".format(abspath, rc, zs), 3)
log("lsof %r = %s\n%s" % (abspath, rc, zs), 3)
except:
log("lsof failed; " + min_ex(), 3)
@@ -2484,17 +2484,17 @@ def _fs_mvrm(
for attempt in range(90210):
try:
if ino and os.stat(bsrc).st_ino != ino:
t = "src inode changed; aborting %s %s"
t = "src inode changed; aborting %s %r"
log(t % (act, src), 1)
return False
if (dst and not atomic) and os.path.exists(bdst):
t = "something appeared at dst; aborting rename [%s] ==> [%s]"
t = "something appeared at dst; aborting rename %r ==> %r"
log(t % (src, dst), 1)
return False
osfun(*args)
if attempt:
now = time.time()
t = "%sd in %.2f sec, attempt %d: %s"
t = "%sd in %.2f sec, attempt %d: %r"
log(t % (act, now - t0, attempt + 1, src))
return True
except OSError as ex:
@@ -2506,7 +2506,7 @@ def _fs_mvrm(
if not attempt:
if not PY2:
ino = os.stat(bsrc).st_ino
t = "%s failed (err.%d); retrying for %d sec: [%s]"
t = "%s failed (err.%d); retrying for %d sec: %r"
log(t % (act, ex.errno, maxtime + 0.99, src))
time.sleep(chill)
@@ -3535,7 +3535,7 @@ def runhook(
log, src, cmd, ap, vp, host, uname, perms, mt, sz, ip, at, txt
)
if log and args.hook_v:
log("hook(%s) [%s] => \033[32m%s" % (src, cmd, hr), 6)
log("hook(%s) %r => \033[32m%s" % (src, cmd, hr), 6)
if not hr:
return {}
for k, v in hr.items():

View File

@@ -131,7 +131,7 @@
<div id="widget"></div>
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
CGV1 = {{ cgv1 }},
CGV = {{ cgv|tojson }},
TS = "{{ ts }}",

View File

@@ -3064,6 +3064,7 @@ var vbar = (function () {
can = r.can.can;
ctx = r.can.ctx;
ctx.font = '.7em sans-serif';
ctx.fontVariantCaps = 'small-caps';
w = r.can.w;
h = r.can.h;
r.draw();
@@ -3089,18 +3090,18 @@ var vbar = (function () {
ctx.fillStyle = grad2; ctx.fillRect(0, 0, w, h);
ctx.fillStyle = grad1; ctx.fillRect(0, 0, w * mp.vol, h);
if (Date.now() - lastv > 1000)
return;
var vt = 'volume ' + Math.floor(mp.vol * 100),
tw = ctx.measureText(vt).width,
x = w * mp.vol - tw - 8,
li = dy;
var vt = Math.floor(mp.vol * 100),
tw = ctx.measureText(vt).width;
var li = dy;
if (mp.vol < 0.05)
if (mp.vol < 0.5) {
x += tw + 16;
li = !li;
}
ctx.fillStyle = li ? '#fff' : '#210';
ctx.fillText(vt, Math.max(4, w * mp.vol - tw - 8), h / 3 * 2);
ctx.fillText(vt, x, h / 3 * 2);
clearTimeout(untext);
untext = setTimeout(r.draw, 1000);

View File

@@ -128,9 +128,9 @@ write markdown (most html is 🙆 too)
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
last_modified = {{ lastmod }},
have_emp = {{ have_emp|tojson }},
have_emp = {{ "true" if have_emp else "false" }},
dfavico = "{{ favico }}";
var md_opt = {

View File

@@ -26,9 +26,9 @@
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR = "{{ r }}",
last_modified = {{ lastmod }},
have_emp = {{ have_emp|tojson }},
have_emp = {{ "true" if have_emp else "false" }},
dfavico = "{{ favico }}";
var md_opt = {

107
copyparty/web/rups.css Normal file
View File

@@ -0,0 +1,107 @@
html {
color: #333;
background: #f7f7f7;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
touch-action: manipulation;
}
#wrap {
margin: 2em auto;
padding: 0 1em 3em 1em;
line-height: 2.3em;
}
a {
color: #047;
background: #fff;
text-decoration: none;
border-bottom: 1px solid #8ab;
border-radius: .2em;
padding: .2em .6em;
margin: 0 .3em;
}
#wrap td a {
margin: 0;
line-height: 1em;
display: inline-block;
white-space: initial;
font-family: var(--font-main), sans-serif;
}
#repl {
border: none;
background: none;
color: inherit;
padding: 0;
position: fixed;
bottom: .25em;
left: .2em;
}
#wrap table {
border-collapse: collapse;
position: relative;
margin-top: 2em;
}
#wrap th {
top: -1px;
position: sticky;
background: #f7f7f7;
}
#wrap td {
font-family: var(--font-mono), monospace, monospace;
white-space: pre; /*date*/
overflow: hidden; /*ipv6*/
}
#wrap th:first-child,
#wrap td:first-child {
text-align: right;
}
#wrap td,
#wrap th {
text-align: left;
padding: .3em .6em;
max-width: 30vw;
}
#wrap tr:hover td {
background: #ddd;
box-shadow: 0 -1px 0 rgba(128, 128, 128, 0.5) inset;
}
#wrap th:first-child,
#wrap td:first-child {
border-radius: .5em 0 0 .5em;
}
#wrap th:last-child,
#wrap td:last-child {
border-radius: 0 .5em .5em 0;
}
html.z {
background: #222;
color: #ccc;
}
html.bz {
background: #11121d;
color: #bbd;
}
html.z a {
color: #fff;
background: #057;
border-color: #37a;
}
html.z input[type=text] {
color: #ddd;
background: #223;
border: none;
border-bottom: 1px solid #fc5;
border-radius: .2em;
padding: .2em .3em;
}
html.z #wrap th {
background: #222;
}
html.bz #wrap th {
background: #223;
}
html.z #wrap tr:hover td {
background: #000;
}

50
copyparty/web/rups.html Normal file
View File

@@ -0,0 +1,50 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="robots" content="noindex, nofollow">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/rups.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
</head>
<body>
<div id="wrap">
<a href="#" id="re">refresh</a>
<a href="{{ r }}/?h">control-panel</a>
&nbsp; Filter: <input type="text" id="filter" size="20" placeholder="documents/passwords" />
&nbsp; <span id="hits"></span>
<table id="tab"><thead><tr>
<th>size</th>
<th>who</th>
<th>when</th>
<th>age</th>
<th>dir</th>
<th>file</th>
</tr></thead><tbody id="tb"></tbody></table>
</div>
<a href="#" id="repl">π</a>
<script>
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script>var V={{ v }};</script>
<script src="{{ r }}/.cpr/rups.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

66
copyparty/web/rups.js Normal file
View File

@@ -0,0 +1,66 @@
function render() {
var ups = V.ups, now = V.now, html = [];
ebi('filter').value = V.filter;
ebi('hits').innerHTML = 'showing ' + ups.length + ' files';
for (var a = 0; a < ups.length; a++) {
var f = ups[a],
vsp = vsplit(f.vp.split('?')[0]),
dn = esc(uricom_dec(vsp[0])),
fn = esc(uricom_dec(vsp[1])),
at = f.at,
td = now - f.at,
ts = !at ? '(?)' : unix2iso(at),
sa = !at ? '(?)' : td > 60 ? shumantime(td) : (td + 's'),
sz = ('' + f.sz).replace(/\B(?=(\d{3})+(?!\d))/g, " ");
html.push('<tr><td>' + sz +
'</td><td>' + f.ip +
'</td><td>' + ts +
'</td><td>' + sa +
'</td><td><a href="' + vsp[0] + '">' + dn +
'</a></td><td><a href="' + f.vp + '">' + fn +
'</a></td></tr>');
}
if (!ups.length) {
var t = V.filter ? ' matching the filter' : '';
html = ['<tr><td colspan="6">there are no uploads' + t + '</td></tr>'];
}
ebi('tb').innerHTML = html.join('');
}
render();
var ti;
function ask(e) {
ev(e);
clearTimeout(ti);
ebi('hits').innerHTML = 'Loading...';
var xhr = new XHR(),
filter = unsmart(ebi('filter').value);
hist_replace(get_evpath().split('?')[0] + '?ru&filter=' + uricom_enc(filter));
xhr.onload = xhr.onerror = function () {
try {
V = JSON.parse(this.responseText)
}
catch (ex) {
ebi('tb').innerHTML = '<tr><td colspan="6">failed to decode server response as json: <pre>' + esc(this.responseText) + '</pre></td></tr>';
return;
}
render();
};
xhr.open('GET', SR + '/?ru&j&filter=' + uricom_enc(filter), true);
xhr.send();
}
ebi('re').onclick = ask;
ebi('filter').oninput = function () {
clearTimeout(ti);
ti = setTimeout(ask, 500);
ebi('hits').innerHTML = '...';
};
ebi('filter').onkeydown = function (e) {
if (('' + e.key).endsWith('Enter'))
ask();
};

View File

@@ -44,9 +44,10 @@ a {
bottom: .25em;
left: .2em;
}
table {
#wrap table {
border-collapse: collapse;
position: relative;
position: relative;
margin-top: 2em;
}
th {
top: -1px;
@@ -62,6 +63,14 @@ th {
#wrap td+td+td+td+td+td+td+td {
font-family: var(--font-mono), monospace, monospace;
}
#wrap th:first-child,
#wrap td:first-child {
border-radius: .5em 0 0 .5em;
}
#wrap th:last-child,
#wrap td:last-child {
border-radius: 0 .5em .5em 0;
}
@@ -81,3 +90,6 @@ html.bz {
color: #bbd;
background: #11121d;
}
html.bz th {
background: #223;
}

View File

@@ -6,6 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="robots" content="noindex, nofollow">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/shares.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
@@ -14,8 +15,8 @@
<body>
<div id="wrap">
<a id="a" href="{{ r }}/?shares" class="af">refresh</a>
<a id="a" href="{{ r }}/?h" class="af">control-panel</a>
<a href="{{ r }}/?shares">refresh</a>
<a href="{{ r }}/?h">control-panel</a>
<span>axs = perms (read,write,move,delet)</span>
<span>nf = numFiles (0=dir)</span>
@@ -58,9 +59,11 @@
{% if not rows %}
(you don't have any active shares btw)
{% endif %}
</div>
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
shr="{{ shr }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -157,6 +157,7 @@
<blockquote id="ad">enabling no304 will disable all caching; try this if k304 wasn't enough. This will waste a huge amount of network traffic!</blockquote></li>
{% endif %}
<li><a id="af" href="{{ r }}/?ru">show recent uploads</a></li>
<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
</ul>
@@ -167,7 +168,7 @@
{%- endif %}
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -38,6 +38,7 @@ var Ls = {
"ac1": "skru på no304",
"ad1": "no304 stopper all bruk av cache. Hvis ikke k304 var nok, prøv denne. Vil mangedoble dataforbruk!",
"ae1": "utgående:",
"af1": "vis nylig opplastede filer",
},
"eng": {
"d2": "shows the state of all active threads",
@@ -88,6 +89,7 @@ var Ls = {
"ac1": "开启 k304",
"ad1": "启用 no304 将禁用所有缓存;如果 k304 不够,可以尝试此选项。这将消耗大量的网络流量!", //m
"ae1": "正在下载:", //m
"af1": "显示最近上传的文件", //m
}
};

View File

@@ -53,7 +53,6 @@
{% if s %}
<li>running <code>rclone mount</code> on LAN (or just dont have valid certificates)? add <code>--no-check-certificate</code></li>
{% endif %}
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
@@ -137,7 +136,6 @@
{% if args.ftps %}
<li>running on LAN (or just dont have valid certificates)? add <code>no_check_certificate=true</code> to the config command</li>
{% endif %}
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
<p>if you want to use the native FTP client in windows instead (please dont), press <code>win+R</code> and run this command:</p>
@@ -235,7 +233,7 @@
<a href="#" id="repl">π</a>
<script>
var SR = {{ r|tojson }},
var SR="{{ r }}",
lang="{{ lang }}",
dfavico="{{ favico }}";

View File

@@ -695,8 +695,9 @@ function Donut(uc, st) {
}
if (++r.tc >= 10) {
var s = r.eta === null ? 'paused' : r.eta > 60 ? shumantime(r.eta) : (r.eta + 's');
wintitle("{0}%, {1}, #{2}, ".format(
f2f(v * 100 / t, 1), shumantime(r.eta), st.files.length - st.nfile.upload), true);
f2f(v * 100 / t, 1), s, st.files.length - st.nfile.upload), true);
r.tc = 0;
}
@@ -1359,7 +1360,15 @@ function up2k_init(subtle) {
draw_each = good_files.length < 50;
if (WebAssembly && !hws.length) {
for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
var nw = Math.min(navigator.hardwareConcurrency || 4, 16);
if (CHROME) {
// chrome-bug 383568268 // #124
nw = Math.max(1, (nw > 4 ? 4 : (nw - 1)));
nw = (subtle && !MOBILE && nw > 2) ? 2 : nw;
}
for (var a = 0; a < nw; a++)
hws.push(new Worker(SR + '/.cpr/w.hash.js?_=' + TS));
if (!subtle)

View File

@@ -886,8 +886,11 @@ if (window.Number && Number.isFinite)
function f2f(val, nd) {
// 10.toFixed(1) returns 10.00 for certain values of 10
if (!isNum(val))
val = 999;
if (!isNum(val)) {
val = parseFloat(val);
if (!isNum(val))
val = 999;
}
val = (val * Math.pow(10, nd)).toFixed(0).split('.')[0];
return nd ? (val.slice(0, -nd) || '0') + '.' + val.slice(-nd) : val;
}

View File

@@ -1,3 +1,81 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1219-0037 `v1.16.6` merry \x58mas
# ☃️🎄 **it is time** 🎅🎁
❄️❄️❄️ please [enjoy some appropriate music](https://a.ocv.me/pub/demo/music/.bonus/#af-55d4554d) (trust me on this one, you won't regret it) ❄️❄️❄️
## 🧪 new features
* [list of recent uploads](https://a.ocv.me/?ru) eaa4b04a
* new button in the controlpanel; can be disabled with `--no-ups-page`
* only users with the dot-permission can see dotfiles
* only admins can see uploader-ip and upload-times
* enable `--ups-when` to let all users see upload-times
* #125 log decoded request-URLs 73f7249c
* non-ascii filenames would make the accesslog a wall of `%E5%B9%BB%E6%83%B3%E9%83%B7` so print [the decoded URL](https://github.com/user-attachments/assets/9d411183-30f3-4cb2-a880-84cf18011183) in addition to the original one, which is left as-is for debugging purposes
## 🩹 bugfixes
* #126 improve dotfile handling 4c4e48ba
* was impossible to delete a folder which contained hidden files if the user did not have the permission to see hidden files
* would also affect moving, renaming, copying folders, in which case the dotfiles would not be carried over to the new location
* now, dotfiles are always deleted, and always moved/copied into a new destination, on the condition that this is safe -- if the user has the dotfile permission in the target loocation but not in the source location, the dotfiles will be left behind to avoid accidentally making then browsable
* ux: cosmetic eta/idle-timer fixes 01a3eb29
## 🔧 other changes
* warn on ambiguous comments in config files da5ad2ab
* avoid writing mojibake to the log 3051b131
* use `\x`-encoding for unprintable text
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1211-2236 `v1.16.5` 4chrome
## 🧪 new features
* #124 add workaround for a chrome bug (crash during upload) 24ce46b3
* chrome and chromium-based browsers could OOM
* https://issues.chromium.org/issues/383568268
* #122 "hybrid IdP", regular users can still auth while [IdP](https://github.com/9001/copyparty#identity-providers) is enabled 64501fd7
* previously, enabling IdP would entirely disable password-based login
* now, password-auth is attempted for requests without a valid IdP header
## 🩹 bugfixes
* the terminal window title would only change if `--no-ansi` was specified, which is exactly the opposite of what it should be (and now is) doing db3c0b09
## 🔧 other changes
* mDNS: better log messages when several IPs are added/removed a49bf81f
* webdeps: update dompurify 06868606
----
this release includes a build of [copyparty-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.16.5/copyparty-winpe64.exe) since the last one was [almost a year ago](https://github.com/9001/copyparty/releases/tag/v1.10.1)
* winpe64.exe is only for *very* specific usecases, you almost definitely *do not* want to download it, please just grab the regular [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) instead (works on all 64bit machines running win8 or newer)
* the only difference between winpe64.exe and [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) is that winpe64.exe works in the win7x64 PE (rescue-env), which makes it *almost* entirely useless, and every bit as dangerous to use as copyparty32.exe
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1207-0024 `v1.16.4` ux is hard
## 🧪 new features
* improve the upload ui so it explains how to abort an unfinished upload when someone uploads to the wrong folder by accident be6afe2d
* also reduces serverload slightly when cloning an incoming file to multiple destinations
* u2c (commandline uploader): windows improvements 91637800
* now supports globbing (filename wildcards) on windows
* progressbar in the windows taskbar (requires conemu or the "new windows terminal")
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-1204-0003 `v1.16.3` 120%

View File

@@ -143,6 +143,9 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?dls` | show active downloads (do this as admin) |
| GET | `?ups` | show recent uploads from your IP |
| GET | `?ups&filter=f` | ...where URL contains `f` |
| GET | `?ru` | show all recent uploads |
| GET | `?ru&filter=f` | ...where URL contains `f` |
| GET | `?ru&j` | ...as json |
| GET | `?mime=foo` | specify return mimetype `foo` |
| GET | `?v` | render markdown file at URL |
| GET | `?v` | open image/video/audio in mediaplayer |

View File

@@ -3,7 +3,7 @@ WORKDIR /z
ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
ver_hashwasm=4.12.0 \
ver_marked=4.3.0 \
ver_dompf=3.2.2 \
ver_dompf=3.2.3 \
ver_mde=2.18.0 \
ver_codemirror=5.65.18 \
ver_fontawesome=5.13.0 \

View File

@@ -1,7 +1,7 @@
f117016b1e6a7d7e745db30d3e67f1acf7957c443a0dd301b6c5e10b8368f2aa4db6be9782d2d3f84beadd139bfeef4982e40f21ca5d9065cb794eeb0e473e82 altgraph-0.17.4-py2.py3-none-any.whl
6a624018f30da375581d5751eca0080edbbe37f102f643f856279fcfded3a4379fd1b6fb0661cdb2e72bbbbc726ca714a1f5990cc348df365db62bc53e4c4503 Git-2.45.2-32-bit.exe
17ce52ba50692a9d964f57a23ac163fb74c77fdeb2ca988a6d439ae1fe91955ff43730c073af97a7b3223093ffea3479a996b9b50ee7fba0869247a56f74baa6 pefile-2023.2.7-py3-none-any.whl
749a473646c6d4c7939989649733d4c7699fd1c359c27046bf5bc9c070d1a4b8b986bbc65f60d7da725baf16dbfdd75a4c2f5bb8335f2cb5685073f5fee5c2d1 pywin32_ctypes-0.2.2-py3-none-any.whl
b297ff66ec50cf5a1abcf07d6ac949644c5150ba094ffac974c5d27c81574c3e97ed814a47547f4b03a4c83ea0fb8f026433fca06a3f08e32742dc5c024f3d07 pywin32_ctypes-0.2.3-py3-none-any.whl
085d39ef4426aa5f097fbc484595becc16e61ca23fc7da4d2a8bba540a3b82e789e390b176c7151bdc67d01735cce22b1562cdb2e31273225a2d3e275851a4ad setuptools-70.3.0-py3-none-any.whl
360a141928f4a7ec18a994602cbb28bbf8b5cc7c077a06ac76b54b12fa769ed95ca0333a5cf728923a8e0baeb5cc4d5e73e5b3de2666beb05eb477d8ae719093 upx-4.2.4-win32.zip
# win7
@@ -23,11 +23,11 @@ ac96786e5d35882e0c5b724794329c9125c2b86ae7847f17acfc49f0d294312c6afc1c3f248655de
# win10
0a2cd4cadf0395f0374974cd2bc2407e5cc65c111275acdffb6ecc5a2026eee9e1bb3da528b35c7f0ff4b64563a74857d5c2149051e281cc09ebd0d1968be9aa en-us_windows_10_enterprise_ltsc_2021_x64_dvd_d289cf96.iso
16cc0c58b5df6c7040893089f3eb29c074aed61d76dae6cd628d8a89a05f6223ac5d7f3f709a12417c147594a87a94cc808d1e04a6f1e407cc41f7c9f47790d1 virtio-win-0.1.248.iso
d1420c8417fad7888766dd26b9706a87c63e8f33dceeb8e26d0056d5127b0b3ed9272e44b4b761132d4b3320327252eab1696520488ca64c25958896b41f547b jinja2-3.1.4-py3-none-any.whl
8e6847bcde75a2736be0214731f834bc1b5854238d703351e68bc4e74d38404b212b8568565ae22c844189e466d3fbe6024836351cb69ffb1824131387644fef MarkupSafe-2.1.5-cp312-cp312-win_amd64.whl
18b9e8cfa682da51da1b682612652030bd7f10e4a1d5ea5220ab32bde734b0e6fe1c7dbd903ac37928c0171fd45d5ca602952054de40a4e55e9ed596279516b5 jinja2-3.1.5-py3-none-any.whl
6df21f0da408a89f6504417c7cdf9aaafe4ed88cfa13e9b8fa8414f604c0401f885a04bbad0484dc51a29284af5d1548e33c6cc6bfb9896d9992c1b1074f332d MarkupSafe-3.0.2-cp312-cp312-win_amd64.whl
8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e mutagen-1.47.0-py3-none-any.whl
0203ec2551c4836696cfab0b2c9fff603352f03fa36e7476e2e1ca7ec57a3a0c24bd791fcd92f342bf817f0887854d9f072e0271c643de4b313d8c9569ba8813 packaging-24.1-py3-none-any.whl
2be320b4191f208cdd6af183c77ba2cf460ea52164ee45ac3ff17d6dfa57acd9deff016636c2dd42a21f4f6af977d5f72df7dacf599bebcf41757272354d14c1 pillow-10.4.0-cp312-cp312-win_amd64.whl
896ddddbd4b85e86e0600cb65eb4c07fbc7f3802d47e7f660411e20b5500831469b97ed4770f25820f4e75cbfac40308da624fd86d4f62e578149d5c276a9cde pyinstaller-6.10.0-py3-none-win_amd64.whl
873781decaeef07f6a79b0ed8b9f35f3fa534a1ea0d866991e40278a10818fa5b60c70b0d5828971b045364f1099694cd1e5d5d60d480acb93fcfbfbced4a09e pyinstaller_hooks_contrib-2024.8-py3-none-any.whl
f0463895e9aee97f31a2003323de235fed1b26289766dc0837261e3f4a594a31162b69e9adbb0e9a31e2e2d4b5f25c762ed1669553df7dc89a8ba4f85d297873 pyinstaller-6.11.1-py3-none-win_amd64.whl
d550a0a14428386945533de2220c4c2e37c0c890fc51a600f626c6ca90a32d39572c121ec04c157ba3a8d6601cb021f8433d871b5c562a3d342c804fffec90c1 pyinstaller_hooks_contrib-2024.11-py3-none-any.whl
0f623c9ab52d050283e97a986ba626d86b04cd02fa7ffdf352740576940b142b264709abadb5d875c90f625b28103d7210b900e0d77f12c1c140108bd2a159aa python-3.12.8-amd64.exe

View File

@@ -105,6 +105,9 @@ copyparty/web/mde.html,
copyparty/web/mde.js,
copyparty/web/msg.css,
copyparty/web/msg.html,
copyparty/web/rups.css,
copyparty/web/rups.html,
copyparty/web/rups.js,
copyparty/web/shares.css,
copyparty/web/shares.html,
copyparty/web/shares.js,

View File

@@ -80,6 +80,7 @@ var tl_cpanel = {
"ac1": "enable no304",
"ad1": "enabling no304 will disable all caching; try this if k304 wasn't enough. This will waste a huge amount of network traffic!",
"ae1": "active downloads:",
"af1": "show recent uploads",
},
};

View File

@@ -115,6 +115,7 @@ var tl_cpanel = {{
"ac1": "enable no304",
"ad1": "enabling no304 will disable all caching; try this if k304 wasn't enough. This will waste a huge amount of network traffic!",
"ae1": "active downloads:",
"af1": "show recent uploads",
}},
}};

View File

@@ -33,14 +33,6 @@ def eprint(*a, **ka):
sys.stderr.flush()
if MACOS:
import posixpath
posixpath.islink = nah
os.path.islink = nah
# 25% faster; until any tests do symlink stuff
from copyparty.__main__ import init_E
from copyparty.broker_thr import BrokerThr
from copyparty.ico import Ico
@@ -283,8 +275,6 @@ class VHttpSrv(object):
self.gurl = Garda("")
self.u2idx = None
self.ptn_cc = re.compile(r"[\x00-\x1f]")
self.uparam_cc_ok = set("doc move tree".split())
def cachebuster(self):
return "a"