Compare commits

...

19 Commits

Author SHA1 Message Date
ed
1e22222c60 v1.7.0 2023-04-29 21:14:38 +00:00
ed
544e0549bc make xvol and xdev apply at runtime (closes #24):
* when accessing files inside an xdev volume, verify that the file
   exists on the same device/filesystem as the volume root

* when accessing files inside an xvol volume, verify that the file
   exists within any volume where the user has read access
2023-04-29 21:10:02 +00:00
ed
83178d0836 preserve empty folders (closes #23):
* when deleting files, do not cascade upwards through empty folders
* when moving folders, also move any empty folders inside

the only remaining action which autoremoves empty folders is
files getting deleted as they expire volume lifetimes

also prevents accidentally moving parent folders into subfolders
(even though that actually worked surprisingly well)
2023-04-29 11:30:43 +00:00
ed
c44f5f5701 nit 2023-04-29 09:44:46 +00:00
ed
138f5bc989 warn about android powersave settings on music interruption + fix eq on folder change 2023-04-29 09:31:53 +00:00
ed
e4759f86ef ftpd correctness:
* winscp mkdir failed because the folder-not-found error got repeated
* rmdir fails after all files in the folder have poofed; that's OK
* add --ftp4 as a precaution
2023-04-28 20:50:45 +00:00
ed
d71416437a show file selection summary 2023-04-27 19:33:52 +00:00
ed
a84c583b2c ok that wasn't enough 2023-04-27 19:06:35 +00:00
ed
cdacdccdb8 update pkgs to 1.6.15 2023-04-27 00:36:56 +00:00
ed
d3ccd3f174 v1.6.15 2023-04-26 23:00:55 +00:00
ed
cb6de0387d a bit faster 2023-04-26 19:56:27 +00:00
ed
abff40519d eyecandy: restore playback indicator on folder hop 2023-04-26 19:09:16 +00:00
ed
55c74ad164 30% faster folder listings (wtf...) 2023-04-26 18:55:53 +00:00
ed
673b4f7e23 option to show symlink's lastmod instead of deref;
mainly motivated by u2cli's folder syncing in turbo mode
which would un-turbo on most dupes due to wrong lastmod

disabled by default for regular http listings
(to avoid confusion in most regular usecases),
enable per-request with urlparam lt

enabled by default for single-level webdav listings
(because rclone hits the same issue as u2cli),
can be disabled with arg --dav-rt or volflag davrt

impossible to enable for recursive webdav listings
2023-04-26 18:54:21 +00:00
ed
d11e02da49 u2cli: avoid dns lookups while uploading 2023-04-26 18:46:42 +00:00
ed
8790f89e08 fix installing from source tarball 2023-04-26 18:40:47 +00:00
ed
33442026b8 try to discourage android from stopping playback...
...when continuing into the next folder

accidentally introduces a neat bonus feature where the music
no longer stops while you go looking for stuff to play next
2023-04-26 18:33:30 +00:00
ed
03193de6d0 socket read/write timeout 2023-04-24 20:04:22 +00:00
ed
8675ff40f3 update pkgs to 1.6.14 2023-04-24 07:52:12 +00:00
25 changed files with 628 additions and 172 deletions

1
.gitignore vendored
View File

@@ -37,6 +37,7 @@ up.*.txt
.hist/
scripts/docker/*.out
scripts/docker/*.err
/perf.*
# nix build output link
result

10
.vscode/launch.py vendored
View File

@@ -30,9 +30,17 @@ except:
argv = [os.path.expanduser(x) if x.startswith("~") else x for x in argv]
sfx = ""
if len(sys.argv) > 1 and os.path.isfile(sys.argv[1]):
sfx = sys.argv[1]
sys.argv = [sys.argv[0]] + sys.argv[2:]
argv += sys.argv[1:]
if re.search(" -j ?[0-9]", " ".join(argv)):
if sfx:
argv = [sys.executable, sfx] + argv
sp.check_call(argv)
elif re.search(" -j ?[0-9]", " ".join(argv)):
argv = [sys.executable, "-m", "copyparty"] + argv
sp.check_call(argv)
else:

View File

@@ -41,6 +41,7 @@ turn almost any device into a file server with resumable uploads/downloads using
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - bass boosted
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
* [markdown viewer](#markdown-viewer) - and there are *two* editors
* [other tricks](#other-tricks)
* [searching](#searching) - search by size, date, path/name, mp3-tags, ...
@@ -69,6 +70,8 @@ turn almost any device into a file server with resumable uploads/downloads using
* [themes](#themes)
* [complete examples](#complete-examples)
* [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
* [packages](#packages) - the party might be closer than you think
* [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
* [nix package](#nix-package) - `nix profile install github:9001/copyparty`
* [nixos module](#nixos-module)
* [browser support](#browser-support) - TLDR: yes
@@ -103,7 +106,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
* or install through pypi (python3 only): `python3 -m pip install --user -U copyparty`
* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
* or [install through nix](#nix-package), or [on NixOS](#nixos-module)
* or [install through nix](#nix-package), or [on NixOS](#nixos-module), or [on arch](#arch-package)
* or if you are on android, [install copyparty in termux](#install-on-android)
* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
* docker has all deps built-in, so skip this step:
@@ -275,6 +278,8 @@ server notes:
* [Firefox issue 1790500](https://bugzilla.mozilla.org/show_bug.cgi?id=1790500) -- entire browser can crash after uploading ~4000 small files
* Android: music playback randomly stops due to [battery usage settings](#fix-unreliable-playback-on-android)
* iPhones: the volume control doesn't work because [apple doesn't want it to](https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html#//apple_ref/doc/uid/TP40009523-CH5-SW11)
* *future workaround:* enable the equalizer, make it all-zero, and set a negative boost to reduce the volume
* "future" because `AudioContext` can't maintain a stable playback speed in the current iOS version (15.7), maybe one day...
@@ -701,6 +706,11 @@ can also boost the volume in general, or increase/decrease stereo width (like [c
has the convenient side-effect of reducing the pause between songs, so gapless albums play better with the eq enabled (just make it flat)
### fix unreliable playback on android
due to phone / app settings, android phones may randomly stop playing music when the power saver kicks in, especially at the end of an album -- you can fix it by [disabling power saving](https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png) in the [app settings](https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png) of the browser you use for music streaming (preferably a dedicated one)
## markdown viewer
and there are *two* editors
@@ -948,7 +958,11 @@ avoid traversing into other filesystems using `--xdev` / volflag `:c,xdev`, ski
and/or you can `--xvol` / `:c,xvol` to ignore all symlinks leaving the volume's top directory, but still allow bind-mounts pointing elsewhere
**NB: only affects the indexer** -- users can still access anything inside a volume, unless shadowed by another volume
* symlinks are permitted with `xvol` if they point into another volume where the user has the same level of access
these options will reduce performance; unlikely worst-case estimates are 14% reduction for directory listings, 35% for download-as-tar
as of copyparty v1.7.0 these options also prevent file access at runtime -- in previous versions it was just hints for the indexer
### periodic rescan
@@ -1176,6 +1190,16 @@ example webserver configs:
* [apache2 config](contrib/apache/copyparty.conf) -- location-based
# packages
the party might be closer than you think
## arch package
now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
## nix package
`nix profile install github:9001/copyparty`

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env python3
from __future__ import print_function, unicode_literals
S_VERSION = "1.6"
S_BUILD_DT = "2023-04-20"
S_VERSION = "1.8"
S_BUILD_DT = "2023-04-27"
"""
up2k.py: upload to copyparty
@@ -21,6 +21,7 @@ import math
import time
import atexit
import signal
import socket
import base64
import hashlib
import platform
@@ -58,6 +59,7 @@ PY2 = sys.version_info < (3,)
if PY2:
from Queue import Queue
from urllib import quote, unquote
from urlparse import urlsplit, urlunsplit
sys.dont_write_bytecode = True
bytes = str
@@ -65,6 +67,7 @@ else:
from queue import Queue
from urllib.parse import unquote_to_bytes as unquote
from urllib.parse import quote_from_bytes as quote
from urllib.parse import urlsplit, urlunsplit
unicode = str
@@ -337,6 +340,32 @@ class CTermsize(object):
ss = CTermsize()
def undns(url):
usp = urlsplit(url)
hn = usp.hostname
gai = None
eprint("resolving host [{0}] ...".format(hn), end="")
try:
gai = socket.getaddrinfo(hn, None)
hn = gai[0][4][0]
except KeyboardInterrupt:
raise
except:
t = "\n\033[31mfailed to resolve upload destination host;\033[0m\ngai={0}\n"
eprint(t.format(repr(gai)))
raise
if usp.port:
hn = "{0}:{1}".format(hn, usp.port)
if usp.username or usp.password:
hn = "{0}:{1}@{2}".format(usp.username, usp.password, hn)
usp = usp._replace(netloc=hn)
url = urlunsplit(usp)
eprint(" {0}".format(url))
return url
def _scd(err, top):
"""non-recursive listing of directory contents, along with stat() info"""
with os.scandir(top) as dh:
@@ -853,7 +882,7 @@ class Ctl(object):
print(" ls ~{0}".format(srd))
zb = self.ar.url.encode("utf-8")
zb += quotep(rd.replace(b"\\", b"/"))
r = req_ses.get(zb + b"?ls&dots", headers=headers)
r = req_ses.get(zb + b"?ls&lt&dots", headers=headers)
if not r:
raise Exception("HTTP {0}".format(r.status_code))
@@ -931,7 +960,7 @@ class Ctl(object):
upath = file.abs.decode("utf-8", "replace")
if not VT100:
upath = upath[4:]
upath = upath.lstrip("\\?")
hs, sprs = handshake(self.ar, file, search)
if search:
@@ -1073,6 +1102,7 @@ source file/folder selection uses rsync syntax, meaning that:
ap = app.add_argument_group("compatibility")
ap.add_argument("--cls", action="store_true", help="clear screen before start")
ap.add_argument("--rh", type=int, metavar="TRIES", default=0, help="resolve server hostname before upload (good for buggy networks, but TLS certs will break)")
ap = app.add_argument_group("folder sync")
ap.add_argument("--dl", action="store_true", help="delete local files after uploading")
@@ -1096,7 +1126,7 @@ source file/folder selection uses rsync syntax, meaning that:
ar = app.parse_args()
finally:
if EXE and not sys.argv[1:]:
print("*** hit enter to exit ***")
eprint("*** hit enter to exit ***")
try:
input()
except:
@@ -1129,8 +1159,18 @@ source file/folder selection uses rsync syntax, meaning that:
with open(fn, "rb") as f:
ar.a = f.read().decode("utf-8").strip()
for n in range(ar.rh):
try:
ar.url = undns(ar.url)
break
except KeyboardInterrupt:
raise
except:
if n > ar.rh - 2:
raise
if ar.cls:
print("\x1b\x5b\x48\x1b\x5b\x32\x4a\x1b\x5b\x33\x4a", end="")
eprint("\x1b\x5b\x48\x1b\x5b\x32\x4a\x1b\x5b\x33\x4a", end="")
ctl = Ctl(ar)

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.6.13"
pkgver="1.6.15"
pkgrel=1
pkgdesc="Portable file sharing hub"
arch=("any")
@@ -26,7 +26,7 @@ source=("${url}/releases/download/v${pkgver}/${pkgname}-sfx.py"
"https://raw.githubusercontent.com/9001/${pkgname}/v${pkgver}/LICENSE"
)
backup=("etc/${pkgname}.d/init" )
sha256sums=("96b47616c6e671eb2f4a563671726824eaea201b38d138543c8ddd0ae71797bc"
sha256sums=("efe3fe8710ef8f3b50b1eb0f935f8620e378bdee48f592d77a1c5cf3ed7ce244"
"b8565eba5e64dedba1cf6c7aac7e31c5a731ed7153d6810288a28f00a36c28b2"
"f65c207e0670f9d78ad2e399bda18d5502ff30d2ac79e0e7fc48e7fbdc39afdc"
"c4f396b083c9ec02ad50b52412c84d2a82be7f079b2d016e1c9fad22d68285ff"

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.6.13/copyparty-sfx.py",
"version": "1.6.13",
"hash": "sha256-lrR2FsbmcesvSlY2cXJoJOrqIBs40ThUPI3dCucXl7w="
"url": "https://github.com/9001/copyparty/releases/download/v1.6.15/copyparty-sfx.py",
"version": "1.6.15",
"hash": "sha256-7+P+hxDvjztQsesPk1+GION4ve5I9ZLXehxc8+184kQ="
}

View File

@@ -619,9 +619,9 @@ def get_sects():
\033[32macid\033[0m = extremely safe but slow; the old default. Should never lose any data no matter what
\033[32mswal\033[0m = 2.4x faster uploads yet 99.9%% as safe -- theoretical chance of losing metadata for the ~200 most recently uploaded files if there's a power-loss or your OS crashes
\033[32mswal\033[0m = 2.4x faster uploads yet 99.9% as safe -- theoretical chance of losing metadata for the ~200 most recently uploaded files if there's a power-loss or your OS crashes
\033[32mwal\033[0m = another 21x faster on HDDs yet 90%% as safe; same pitfall as \033[33mswal\033[0m except more likely
\033[32mwal\033[0m = another 21x faster on HDDs yet 90% as safe; same pitfall as \033[33mswal\033[0m except more likely
\033[32myolo\033[0m = another 1.5x faster, and removes the occasional sudden upload-pause while the disk syncs, but now you're at risk of losing the entire database in a powerloss / OS-crash
@@ -710,6 +710,8 @@ def add_network(ap):
ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
else:
ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0, help="debug: socket write delay in seconds")
ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0, help="debug: response delay in seconds")
@@ -771,6 +773,7 @@ def add_ftp(ap):
ap2.add_argument("--ftp", metavar="PORT", type=int, help="enable FTP server on PORT, for example \033[32m3921")
ap2.add_argument("--ftps", metavar="PORT", type=int, help="enable FTPS server on PORT, for example \033[32m3990")
ap2.add_argument("--ftpv", action="store_true", help="verbose")
ap2.add_argument("--ftp4", action="store_true", help="only listen on IPv4")
ap2.add_argument("--ftp-wt", metavar="SEC", type=int, default=7, help="grace period for resuming interrupted uploads (any client can write to any file last-modified more recently than SEC seconds ago)")
ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, help="the NAT address to use for passive connections")
ap2.add_argument("--ftp-pr", metavar="P-P", type=u, help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
@@ -781,6 +784,7 @@ def add_webdav(ap):
ap2.add_argument("--daw", action="store_true", help="enable full write support, even if client may not be webdav. \033[1;31mWARNING:\033[0m This has side-effects -- PUT-operations will now \033[1;31mOVERWRITE\033[0m existing files, rather than inventing new filenames to avoid loss of data. You might want to instead set this as a volflag where needed. By not setting this flag, uploaded files can get written to a filename which the client does not expect (which might be okay, depending on client)")
ap2.add_argument("--dav-inf", action="store_true", help="allow depth:infinite requests (recursive file listing); extremely server-heavy but required for spec compliance -- luckily few clients rely on this")
ap2.add_argument("--dav-mac", action="store_true", help="disable apple-garbage filter -- allow macos to create junk files (._* and .DS_Store, .Spotlight-*, .fseventsd, .Trashes, .AppleDouble, __MACOS)")
ap2.add_argument("--dav-rt", action="store_true", help="show symlink-destination's lastmodified instead of the link itself; always enabled for recursive listings (volflag=davrt)")
def add_smb(ap):
@@ -834,6 +838,8 @@ def add_safety(ap, fk_salt):
ap2.add_argument("-ss", action="store_true", help="further increase safety: Prevent js-injection, accidental move/delete, broken symlinks, webdav, 404 on 403, ban on excessive 404s.\n └─Alias of\033[32m -s --unpost=0 --no-del --no-mv --hardlink --vague-403 --ban-404=50,60,1440 -nih")
ap2.add_argument("-sss", action="store_true", help="further increase safety: Enable logging to disk, scan for dangerous symlinks.\n └─Alias of\033[32m -ss --no-dav --no-logues --no-readme -lo=cpp-%%Y-%%m%%d-%%H%%M%%S.txt.xz --ls=**,*,ln,p,r")
ap2.add_argument("--ls", metavar="U[,V[,F]]", type=u, help="do a sanity/safety check of all volumes on startup; arguments \033[33mUSER\033[0m,\033[33mVOL\033[0m,\033[33mFLAGS\033[0m; example [\033[32m**,*,ln,p,r\033[0m]")
ap2.add_argument("--xvol", action="store_true", help="never follow symlinks leaving the volume root, unless the link is into another volume where the user has similar access (volflag=xvol)")
ap2.add_argument("--xdev", action="store_true", help="stay within the filesystem of the volume root; do not descend into other devices (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
ap2.add_argument("--salt", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files -- this one DOES matter")
ap2.add_argument("--no-dot-mv", action="store_true", help="disallow moving dotfiles; makes it impossible to move folders containing dotfiles")
@@ -927,8 +933,6 @@ def add_db_general(ap, hcores):
ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice (volflag=noforget)")
ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see --help-dbd (volflag=dbd)")
ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
ap2.add_argument("--xdev", action="store_true", help="do not descend into other filesystems (symlink or bind-mount to another HDD, ...) (volflag=xdev)")
ap2.add_argument("--xvol", action="store_true", help="skip symlinks leaving the volume root (volflag=xvol)")
ap2.add_argument("--hash-mt", metavar="CORES", type=int, default=hcores, help="num cpu cores to use for file hashing; set 0 or 1 for single-core hashing")
ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval, 0=off (volflag=scan)")
ap2.add_argument("--db-act", metavar="SEC", type=float, default=10, help="defer any scheduled volume reindexing until SEC seconds after last db write (uploads, renames, ...)")

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 6, 14)
CODENAME = "cors k"
BUILD_DT = (2023, 4, 24)
VERSION = (1, 7, 0)
CODENAME = "unlinked"
BUILD_DT = (2023, 4, 29)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -67,9 +67,9 @@ class AXS(object):
self.upget: set[str] = set(upget or [])
def __repr__(self) -> str:
return "AXS({})".format(
return "AXS(%s)" % (
", ".join(
"{}={!r}".format(k, self.__dict__[k])
"%s=%r" % (k, self.__dict__[k])
for k in "uread uwrite umove udel uget upget".split()
)
)
@@ -285,6 +285,8 @@ class VFS(object):
self.vpath = vpath # absolute path in the virtual filesystem
self.axs = axs
self.flags = flags # config options
self.root = self
self.dev = 0 # st_dev
self.nodes: dict[str, VFS] = {} # child nodes
self.histtab: dict[str, str] = {} # all realpath->histpath
self.dbv: Optional[VFS] = None # closest full/non-jump parent
@@ -297,26 +299,42 @@ class VFS(object):
self.apget: dict[str, list[str]] = {}
if realpath:
rp = realpath + ("" if realpath.endswith(os.sep) else os.sep)
vp = vpath + ("/" if vpath else "")
self.histpath = os.path.join(realpath, ".hist") # db / thumbcache
self.all_vols = {vpath: self} # flattened recursive
self.all_aps = [(rp, self)]
self.all_vps = [(vp, self)]
else:
self.histpath = ""
self.all_vols = {}
self.all_aps = []
self.all_vps = []
def __repr__(self) -> str:
return "VFS({})".format(
return "VFS(%s)" % (
", ".join(
"{}={!r}".format(k, self.__dict__[k])
"%s=%r" % (k, self.__dict__[k])
for k in "realpath vpath axs flags".split()
)
)
def get_all_vols(self, outdict: dict[str, "VFS"]) -> None:
def get_all_vols(
self,
vols: dict[str, "VFS"],
aps: list[tuple[str, "VFS"]],
vps: list[tuple[str, "VFS"]],
) -> None:
if self.realpath:
outdict[self.vpath] = self
vols[self.vpath] = self
rp = self.realpath
rp += "" if rp.endswith(os.sep) else os.sep
vp = self.vpath + ("/" if self.vpath else "")
aps.append((rp, self))
vps.append((vp, self))
for v in self.nodes.values():
v.get_all_vols(outdict)
v.get_all_vols(vols, aps, vps)
def add(self, src: str, dst: str) -> "VFS":
"""get existing, or add new path to the vfs"""
@@ -390,7 +408,11 @@ class VFS(object):
self, vpath: str, uname: str
) -> tuple[bool, bool, bool, bool, bool, bool]:
"""can Read,Write,Move,Delete,Get,Upget"""
vn, _ = self._find(undot(vpath))
if vpath:
vn, _ = self._find(undot(vpath))
else:
vn = self
c = vn.axs
return (
uname in c.uread or "*" in c.uread,
@@ -545,9 +567,20 @@ class VFS(object):
self.log("vfs.walk", t.format(seen[-1], fsroot, self.vpath, rem), 3)
return
if "xdev" in self.flags or "xvol" in self.flags:
rm1 = []
for le in vfs_ls:
ap = absreal(os.path.join(fsroot, le[0]))
vn2 = self.chk_ap(ap)
if not vn2 or not vn2.get("", uname, True, False):
rm1.append(le)
_ = [vfs_ls.remove(x) for x in rm1] # type: ignore
seen = seen[:] + [fsroot]
rfiles = [x for x in vfs_ls if not stat.S_ISDIR(x[1].st_mode)]
rdirs = [x for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)]
# if lstat: ignore folder symlinks since copyparty will never make those
# (and we definitely don't want to descend into them)
rfiles.sort()
rdirs.sort()
@@ -641,6 +674,44 @@ class VFS(object):
for d in [{"vp": v, "ap": a, "st": n} for v, a, n in ret2]:
yield d
def chk_ap(self, ap: str, st: Optional[os.stat_result] = None) -> Optional["VFS"]:
aps = ap + os.sep
if "xdev" in self.flags and not ANYWIN:
if not st:
ap2 = ap.replace("\\", "/") if ANYWIN else ap
while ap2:
try:
st = bos.stat(ap2)
break
except:
if "/" not in ap2:
raise
ap2 = ap2.rsplit("/", 1)[0]
assert st
vdev = self.dev
if not vdev:
vdev = self.dev = bos.stat(self.realpath).st_dev
if vdev != st.st_dev:
if self.log:
t = "xdev: {}[{}] => {}[{}]"
self.log("vfs", t.format(vdev, self.realpath, st.st_dev, ap), 3)
return None
if "xvol" in self.flags:
for vap, vn in self.root.all_aps:
if aps.startswith(vap):
return vn
if self.log:
self.log("vfs", "xvol: [{}]".format(ap), 3)
return None
return self
if WINDOWS:
re_vol = re.compile(r"^([a-zA-Z]:[\\/][^:]*|[^:]*):([^:]*):(.*)$")
@@ -1067,7 +1138,13 @@ class AuthSrv(object):
assert vfs
vfs.all_vols = {}
vfs.get_all_vols(vfs.all_vols)
vfs.all_aps = []
vfs.all_vps = []
vfs.get_all_vols(vfs.all_vols, vfs.all_aps, vfs.all_vps)
for vol in vfs.all_vols.values():
vol.all_aps.sort(key=lambda x: len(x[0]), reverse=True)
vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
vol.root = vfs
for perm in "read write move del get pget".split():
axs_key = "u" + perm

View File

@@ -13,6 +13,7 @@ def vf_bmap() -> dict[str, str]:
"no_dedup": "copydupes",
"no_dupe": "nodupe",
"no_forget": "noforget",
"dav_rt": "davrt",
}
for k in (
"dotsrch",
@@ -106,7 +107,7 @@ flagcats = {
"dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
"xlink": "cross-volume dupe detection / linking",
"xdev": "do not descend into other filesystems",
"xvol": "skip symlinks leaving the volume root",
"xvol": "do not follow symlinks leaving the volume root",
"dotsrch": "show dotfiles in search results",
"nodotsrch": "hide dotfiles in search results (default)",
},
@@ -142,7 +143,8 @@ flagcats = {
"lg_sbf": "list of *logue-sandbox safeguards to disable",
},
"others": {
"fk=8": 'generates per-file accesskeys,\nwhich will then be required at the "g" permission'
"fk=8": 'generates per-file accesskeys,\nwhich will then be required at the "g" permission',
"davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",
},
}

View File

@@ -2,6 +2,7 @@
from __future__ import print_function, unicode_literals
import argparse
import errno
import logging
import os
import stat
@@ -46,6 +47,12 @@ if True: # pylint: disable=using-constant-test
from typing import Any, Optional
class FSE(FilesystemError):
def __init__(self, msg: str, severity: int = 0) -> None:
super(FilesystemError, self).__init__(msg)
self.severity = severity
class FtpAuth(DummyAuthorizer):
def __init__(self, hub: "SvcHub") -> None:
super(FtpAuth, self).__init__()
@@ -113,7 +120,8 @@ class FtpFs(AbstractedFS):
def __init__(
self, root: str, cmd_channel: Any
) -> None: # pylint: disable=super-init-not-called
self.h = self.cmd_channel = cmd_channel # type: FTPHandler
self.h = cmd_channel # type: FTPHandler
self.cmd_channel = cmd_channel # type: FTPHandler
self.hub: "SvcHub" = cmd_channel.hub
self.args = cmd_channel.args
self.uname = cmd_channel.uname
@@ -127,10 +135,6 @@ class FtpFs(AbstractedFS):
self.listdirinfo = self.listdir
self.chdir(".")
def die(self, msg):
self.h.die(msg)
raise Exception()
def v2a(
self,
vpath: str,
@@ -140,21 +144,34 @@ class FtpFs(AbstractedFS):
d: bool = False,
) -> tuple[str, VFS, str]:
try:
vpath = vpath.replace("\\", "/").lstrip("/")
vpath = vpath.replace("\\", "/").strip("/")
rd, fn = os.path.split(vpath)
if ANYWIN and relchk(rd):
logging.warning("malicious vpath: %s", vpath)
self.die("Unsupported characters in filepath")
t = "Unsupported characters in [{}]"
raise FSE(t.format(vpath), 1)
fn = sanitize_fn(fn or "", "", [".prologue.html", ".epilogue.html"])
vpath = vjoin(rd, fn)
vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
if not vfs.realpath:
self.die("No filesystem mounted at this path")
t = "No filesystem mounted at [{}]"
raise FSE(t.format(vpath))
if "xdev" in vfs.flags or "xvol" in vfs.flags:
ap = vfs.canonical(rem)
avfs = vfs.chk_ap(ap)
t = "Permission denied in [{}]"
if not avfs:
raise FSE(t.format(vpath), 1)
cr, cw, cm, cd, _, _ = avfs.can_access("", self.h.uname)
if r and not cr or w and not cw or m and not cm or d and not cd:
raise FSE(t.format(vpath), 1)
return os.path.join(vfs.realpath, rem), vfs, rem
except Pebkac as ex:
self.die(str(ex))
raise FSE(str(ex))
def rv2a(
self,
@@ -177,7 +194,7 @@ class FtpFs(AbstractedFS):
def validpath(self, path: str) -> bool:
if "/.hist/" in path:
if "/up2k." in path or path.endswith("/dir.txt"):
self.die("Access to this file is forbidden")
raise FSE("Access to this file is forbidden", 1)
return True
@@ -194,7 +211,7 @@ class FtpFs(AbstractedFS):
td = 0
if td < -1 or td > self.args.ftp_wt:
self.die("Cannot open existing file for writing")
raise FSE("Cannot open existing file for writing")
self.validpath(ap)
return open(fsenc(ap), mode)
@@ -203,9 +220,17 @@ class FtpFs(AbstractedFS):
nwd = join(self.cwd, path)
vfs, rem = self.hub.asrv.vfs.get(nwd, self.uname, False, False)
ap = vfs.canonical(rem)
if not bos.path.isdir(ap):
try:
st = bos.stat(ap)
if not stat.S_ISDIR(st.st_mode):
raise Exception()
except:
# returning 550 is library-default and suitable
self.die("Failed to change directory")
raise FSE("No such file or directory")
avfs = vfs.chk_ap(ap, st)
if not avfs:
raise FSE("Permission denied", 1)
self.cwd = nwd
(
@@ -215,16 +240,18 @@ class FtpFs(AbstractedFS):
self.can_delete,
self.can_get,
self.can_upget,
) = self.hub.asrv.vfs.can_access(self.cwd.lstrip("/"), self.h.uname)
) = avfs.can_access("", self.h.uname)
def mkdir(self, path: str) -> None:
ap = self.rv2a(path, w=True)[0]
bos.makedirs(ap) # filezilla expects this
def listdir(self, path: str) -> list[str]:
vpath = join(self.cwd, path).lstrip("/")
vpath = join(self.cwd, path)
try:
vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, True, False)
ap, vfs, rem = self.v2a(vpath, True, False)
if not bos.path.isdir(ap):
raise FSE("No such file or directory", 1)
fsroot, vfs_ls1, vfs_virt = vfs.ls(
rem,
@@ -240,8 +267,12 @@ class FtpFs(AbstractedFS):
vfs_ls.sort()
return vfs_ls
except:
if vpath:
except Exception as ex:
# panic on malicious names
if getattr(ex, "severity", 0):
raise
if vpath.strip("/"):
# display write-only folders as empty
return []
@@ -251,31 +282,35 @@ class FtpFs(AbstractedFS):
def rmdir(self, path: str) -> None:
ap = self.rv2a(path, d=True)[0]
bos.rmdir(ap)
try:
bos.rmdir(ap)
except OSError as e:
if e.errno != errno.ENOENT:
raise
def remove(self, path: str) -> None:
if self.args.no_del:
self.die("The delete feature is disabled in server config")
raise FSE("The delete feature is disabled in server config")
vp = join(self.cwd, path).lstrip("/")
try:
self.hub.up2k.handle_rm(self.uname, self.h.cli_ip, [vp], [])
self.hub.up2k.handle_rm(self.uname, self.h.cli_ip, [vp], [], False)
except Exception as ex:
self.die(str(ex))
raise FSE(str(ex))
def rename(self, src: str, dst: str) -> None:
if not self.can_move:
self.die("Not allowed for user " + self.h.uname)
raise FSE("Not allowed for user " + self.h.uname)
if self.args.no_mv:
self.die("The rename/move feature is disabled in server config")
raise FSE("The rename/move feature is disabled in server config")
svp = join(self.cwd, src).lstrip("/")
dvp = join(self.cwd, dst).lstrip("/")
try:
self.hub.up2k.handle_mv(self.uname, svp, dvp)
except Exception as ex:
self.die(str(ex))
raise FSE(str(ex))
def chmod(self, path: str, mode: str) -> None:
pass
@@ -284,7 +319,10 @@ class FtpFs(AbstractedFS):
try:
ap = self.rv2a(path, r=True)[0]
return bos.stat(ap)
except:
except FSE as ex:
if ex.severity:
raise
ap = self.rv2a(path)[0]
st = bos.stat(ap)
if not stat.S_ISDIR(st.st_mode):
@@ -304,7 +342,10 @@ class FtpFs(AbstractedFS):
try:
st = self.stat(path)
return stat.S_ISREG(st.st_mode)
except:
except Exception as ex:
if getattr(ex, "severity", 0):
raise
return False # expected for mojibake in ftp_SIZE()
def islink(self, path: str) -> bool:
@@ -315,7 +356,10 @@ class FtpFs(AbstractedFS):
try:
st = self.stat(path)
return stat.S_ISDIR(st.st_mode)
except:
except Exception as ex:
if getattr(ex, "severity", 0):
raise
return True
def getsize(self, path: str) -> int:
@@ -365,14 +409,10 @@ class FtpHandler(FTPHandler):
# reduce non-debug logging
self.log_cmds_list = [x for x in self.log_cmds_list if x not in ("CWD", "XCWD")]
def die(self, msg):
self.respond("550 {}".format(msg))
raise FilesystemError(msg)
def ftp_STOR(self, file: str, mode: str = "w") -> Any:
# Optional[str]
vp = join(self.fs.cwd, file).lstrip("/")
ap, vfs, rem = self.fs.v2a(vp)
ap, vfs, rem = self.fs.v2a(vp, w=True)
self.vfs_map[ap] = vp
xbu = vfs.flags.get("xbu")
if xbu and not runhook(
@@ -388,7 +428,7 @@ class FtpHandler(FTPHandler):
0,
"",
):
self.die("Upload blocked by xbu server config")
raise FSE("Upload blocked by xbu server config")
# print("ftp_STOR: {} {} => {}".format(vp, mode, ap))
ret = FTPHandler.ftp_STOR(self, file, mode)
@@ -488,6 +528,9 @@ class Ftpd(object):
if "::" in ips:
ips.append("0.0.0.0")
if self.args.ftp4:
ips = [x for x in ips if ":" not in x]
ioloop = IOLoop()
for ip in ips:
for h, lp in hs:

View File

@@ -135,6 +135,7 @@ class HttpCli(object):
self.ouparam: dict[str, str] = {}
self.uparam: dict[str, str] = {}
self.cookies: dict[str, str] = {}
self.avn: Optional[VFS] = None
self.vpath = " "
self.uname = " "
self.pw = " "
@@ -219,7 +220,7 @@ class HttpCli(object):
try:
self.s.settimeout(2)
headerlines = read_header(self.sr)
headerlines = read_header(self.sr, self.args.s_thead, self.args.s_thead)
self.in_hdr_recv = False
if not headerlines:
return False
@@ -266,7 +267,7 @@ class HttpCli(object):
)
self.host = self.headers.get("host") or ""
if not self.host:
zs = "{}:{}".format(*list(self.s.getsockname()[:2]))
zs = "%s:%s" % self.s.getsockname()[:2]
self.host = zs[7:] if zs.startswith("::ffff:") else zs
n = self.args.rproxy
@@ -403,10 +404,21 @@ class HttpCli(object):
self.get_pwd_cookie(self.pw)
if self.is_rclone:
# dots: always include dotfiles if permitted
# lt: probably more important showing the correct timestamps of any dupes it just uploaded rather than the lastmod time of any non-copyparty-managed symlinks
# b: basic-browser if it tries to parse the html listing
uparam["dots"] = ""
uparam["lt"] = ""
uparam["b"] = ""
cookies["b"] = ""
vn, rem = self.asrv.vfs.get(self.vpath, self.uname, False, False)
if "xdev" in vn.flags or "xvol" in vn.flags:
ap = vn.canonical(rem)
avn = vn.chk_ap(ap)
else:
avn = vn
(
self.can_read,
self.can_write,
@@ -414,7 +426,12 @@ class HttpCli(object):
self.can_delete,
self.can_get,
self.can_upget,
) = self.asrv.vfs.can_access(self.vpath, self.uname)
) = (
avn.can_access("", self.uname) if avn else [False] * 6
)
self.avn = avn
self.s.settimeout(self.args.s_tbody or None)
try:
cors_k = self._cors()
@@ -530,7 +547,7 @@ class HttpCli(object):
mime: Optional[str] = None,
headers: Optional[dict[str, str]] = None,
) -> None:
response = ["{} {} {}".format(self.http_ver, status, HTTPCODE[status])]
response = ["%s %s %s" % (self.http_ver, status, HTTPCODE[status])]
if length is not None:
response.append("Content-Length: " + unicode(length))
@@ -554,11 +571,10 @@ class HttpCli(object):
self.out_headers["Content-Type"] = mime
for k, zs in list(self.out_headers.items()) + self.out_headerlist:
response.append("{}: {}".format(k, zs))
response.append("%s: %s" % (k, zs))
try:
# best practice to separate headers and body into different packets
self.s.settimeout(None)
self.s.sendall("\r\n".join(response).encode("utf-8") + b"\r\n\r\n")
except:
raise Pebkac(400, "client d/c while replying headers")
@@ -621,7 +637,7 @@ class HttpCli(object):
if not kv:
return ""
r = ["{}={}".format(k, quotep(zs)) if zs else k for k, zs in kv.items()]
r = ["%s=%s" % (k, quotep(zs)) if zs else k for k, zs in kv.items()]
return "?" + "&amp;".join(r)
def redirect(
@@ -864,12 +880,13 @@ class HttpCli(object):
props = set(props_lst)
vn, rem = self.asrv.vfs.get(self.vpath, self.uname, True, False, err=401)
tap = vn.canonical(rem)
depth = self.headers.get("depth", "infinity").lower()
try:
topdir = {"vp": "", "st": bos.stat(vn.canonical(rem))}
topdir = {"vp": "", "st": bos.stat(tap)}
except OSError as ex:
if ex.errno != errno.ENOENT:
if ex.errno not in (errno.ENOENT, errno.ENOTDIR):
raise
raise Pebkac(404)
@@ -883,6 +900,9 @@ class HttpCli(object):
self.reply(zb, 403, "application/xml; charset=utf-8")
return True
# this will return symlink-target timestamps
# because lstat=true would not recurse into subfolders
# and this is a rare case where we actually want that
fgen = vn.zipgen(
rem,
rem,
@@ -896,7 +916,11 @@ class HttpCli(object):
elif depth == "1":
_, vfs_ls, vfs_virt = vn.ls(
rem, self.uname, not self.args.no_scandir, [[True, False]]
rem,
self.uname,
not self.args.no_scandir,
[[True, False]],
lstat="davrt" not in vn.flags,
)
if not self.args.ed:
names = set(exclude_dotfiles([x[0] for x in vfs_ls]))
@@ -930,14 +954,23 @@ class HttpCli(object):
for x in fgen:
rp = vjoin(vtop, x["vp"])
st: os.stat_result = x["st"]
mtime = st.st_mtime
if stat.S_ISLNK(st.st_mode):
try:
st = bos.stat(os.path.join(tap, x["vp"]))
except:
continue
isdir = stat.S_ISDIR(st.st_mode)
t = "<D:response><D:href>/{}{}</D:href><D:propstat><D:prop>"
ret += t.format(quotep(rp), "/" if isdir and rp else "")
ret += "<D:response><D:href>/%s%s</D:href><D:propstat><D:prop>" % (
quotep(rp),
"/" if isdir and rp else "",
)
pvs: dict[str, str] = {
"displayname": html_escape(rp.split("/")[-1]),
"getlastmodified": formatdate(st.st_mtime, usegmt=True),
"getlastmodified": formatdate(mtime, usegmt=True),
"resourcetype": '<D:collection xmlns:D="DAV:"/>' if isdir else "",
"supportedlock": '<D:lockentry xmlns:D="DAV:"><D:lockscope><D:exclusive/></D:lockscope><D:locktype><D:write/></D:locktype></D:lockentry>',
}
@@ -949,13 +982,13 @@ class HttpCli(object):
if k not in props:
continue
elif v:
ret += "<D:{0}>{1}</D:{0}>".format(k, v)
ret += "<D:%s>%s</D:%s>" % (k, v, k)
else:
ret += "<D:{}/>".format(k)
ret += "<D:%s/>" % (k,)
ret += "</D:prop><D:status>HTTP/1.1 200 OK</D:status></D:propstat>"
missing = ["<D:{}/>".format(x) for x in props if x not in pvs]
missing = ["<D:%s/>" % (x,) for x in props if x not in pvs]
if missing and clen:
t = "<D:propstat><D:prop>{}</D:prop><D:status>HTTP/1.1 404 Not Found</D:status></D:propstat>"
ret += t.format("".join(missing))
@@ -1130,18 +1163,9 @@ class HttpCli(object):
dst = self.headers["destination"]
dst = re.sub("^https?://[^/]+", "", dst).lstrip()
dst = unquotep(dst)
if not self._mv(self.vpath, dst):
if not self._mv(self.vpath, dst.lstrip("/")):
return False
# up2k only cares about files and removes all empty folders;
# clients naturally expect empty folders to survive a rename
vn, rem = self.asrv.vfs.get(dst, self.uname, False, False)
dabs = vn.canonical(rem)
try:
bos.makedirs(dabs)
except:
pass
return True
def _applesan(self) -> bool:
@@ -1206,7 +1230,6 @@ class HttpCli(object):
if self.headers.get("expect", "").lower() == "100-continue":
try:
self.s.settimeout(None)
self.s.sendall(b"HTTP/1.1 100 Continue\r\n\r\n")
except:
raise Pebkac(400, "client d/c before 100 continue")
@@ -1218,7 +1241,6 @@ class HttpCli(object):
if self.headers.get("expect", "").lower() == "100-continue":
try:
self.s.settimeout(None)
self.s.sendall(b"HTTP/1.1 100 Continue\r\n\r\n")
except:
raise Pebkac(400, "client d/c before 100 continue")
@@ -1623,7 +1645,7 @@ class HttpCli(object):
spd1 = get_spd(nbytes, self.t0)
spd2 = get_spd(self.conn.nbyte, self.conn.t0)
return "{} {} n{}".format(spd1, spd2, self.conn.nreq)
return "%s %s n%s" % (spd1, spd2, self.conn.nreq)
def handle_post_multipart(self) -> bool:
self.parser = MultipartParser(self.log, self.sr, self.headers)
@@ -3034,8 +3056,8 @@ class HttpCli(object):
ret = self.gen_tree(top, dst)
if self.is_vproxied:
parents = self.args.R.split("/")
for parent in parents[::-1]:
ret = {"k{}".format(parent): ret, "a": []}
for parent in reversed(parents):
ret = {"k%s" % (parent,): ret, "a": []}
zs = json.dumps(ret)
self.reply(zs.encode("utf-8"), mime="application/json")
@@ -3173,7 +3195,9 @@ class HttpCli(object):
nlim = int(self.uparam.get("lim") or 0)
lim = [nlim, nlim] if nlim else []
x = self.conn.hsrv.broker.ask("up2k.handle_rm", self.uname, self.ip, req, lim)
x = self.conn.hsrv.broker.ask(
"up2k.handle_rm", self.uname, self.ip, req, lim, False
)
self.loud_reply(x.get())
return True
@@ -3190,7 +3214,7 @@ class HttpCli(object):
# x-www-form-urlencoded (url query part) uses
# either + or %20 for 0x20 so handle both
dst = unquotep(dst.replace("+", " "))
return self._mv(self.vpath, dst)
return self._mv(self.vpath, dst.lstrip("/"))
def _mv(self, vsrc: str, vdst: str) -> bool:
if not self.can_move:
@@ -3514,7 +3538,11 @@ class HttpCli(object):
return self.tx_zip(k, v, self.vpath, vn, rem, [], self.args.ed)
fsroot, vfs_ls, vfs_virt = vn.ls(
rem, self.uname, not self.args.no_scandir, [[True, False], [False, True]]
rem,
self.uname,
not self.args.no_scandir,
[[True, False], [False, True]],
lstat="lt" in self.uparam,
)
stats = {k: v for k, v in vfs_ls}
ls_names = [x[0] for x in vfs_ls]
@@ -3558,7 +3586,8 @@ class HttpCli(object):
fspath = fsroot + "/" + fn
try:
inf = stats.get(fn) or bos.stat(fspath)
linf = stats.get(fn) or bos.lstat(fspath)
inf = bos.stat(fspath) if stat.S_ISLNK(linf.st_mode) else linf
except:
self.log("broken symlink: {}".format(repr(fspath)))
continue
@@ -3569,19 +3598,26 @@ class HttpCli(object):
if self.args.no_zip:
margin = "DIR"
else:
margin = '<a href="{}?zip" rel="nofollow">zip</a>'.format(
quotep(href)
)
margin = '<a href="%s?zip" rel="nofollow">zip</a>' % (quotep(href),)
elif fn in hist:
margin = '<a href="{}.hist/{}">#{}</a>'.format(
base, html_escape(hist[fn][2], quot=True, crlf=True), hist[fn][0]
margin = '<a href="%s.hist/%s">#%s</a>' % (
base,
html_escape(hist[fn][2], quot=True, crlf=True),
hist[fn][0],
)
else:
margin = "-"
sz = inf.st_size
zd = datetime.utcfromtimestamp(inf.st_mtime)
dt = zd.strftime("%Y-%m-%d %H:%M:%S")
zd = datetime.utcfromtimestamp(linf.st_mtime)
dt = "%04d-%02d-%02d %02d:%02d:%02d" % (
zd.year,
zd.month,
zd.day,
zd.hour,
zd.minute,
zd.second,
)
try:
ext = "---" if is_dir else fn.rsplit(".", 1)[1]
@@ -3591,7 +3627,7 @@ class HttpCli(object):
ext = "%"
if add_fk:
href = "{}?k={}".format(
href = "%s?k=%s" % (
quotep(href),
self.gen_fk(
self.args.fk_salt, fspath, sz, 0 if ANYWIN else inf.st_ino
@@ -3607,7 +3643,7 @@ class HttpCli(object):
"sz": sz,
"ext": ext,
"dt": dt,
"ts": int(inf.st_mtime),
"ts": int(linf.st_mtime),
}
if is_dir:
dirs.append(item)

View File

@@ -261,7 +261,7 @@ class SMB(object):
yeet("blocked delete (no-del-acc): " + vpath)
vpath = vpath.replace("\\", "/").lstrip("/")
self.hub.up2k.handle_rm(LEELOO_DALLAS, "1.7.6.2", [vpath], [])
self.hub.up2k.handle_rm(LEELOO_DALLAS, "1.7.6.2", [vpath], [], False)
def _utime(self, vpath: str, times: tuple[float, float]) -> None:
if not self.args.smbw:

View File

@@ -647,8 +647,14 @@ class SvcHub(object):
return
with self.log_mutex:
ts = datetime.utcnow().strftime("%Y-%m%d-%H%M%S.%f")[:-3]
self.logf.write("@{} [{}\033[0m] {}\n".format(ts, src, msg))
zd = datetime.utcnow()
ts = "%04d-%04d-%06d.%03d" % (
zd.year,
zd.month * 100 + zd.day,
(zd.hour * 100 + zd.minute) * 100 + zd.second,
zd.microsecond // 1000,
)
self.logf.write("@%s [%s\033[0m] %s\n" % (ts, src, msg))
now = time.time()
if now >= self.next_day:
@@ -678,23 +684,29 @@ class SvcHub(object):
print("\033[36m{}\033[0m\n".format(dt.strftime("%Y-%m-%d")), end="")
self._set_next_day()
fmt = "\033[36m{} \033[33m{:21} \033[0m{}\n"
fmt = "\033[36m%s \033[33m%-21s \033[0m%s\n"
if not VT100:
fmt = "{} {:21} {}\n"
fmt = "%s %-21s %s\n"
if "\033" in msg:
msg = ansi_re.sub("", msg)
if "\033" in src:
src = ansi_re.sub("", src)
elif c:
if isinstance(c, int):
msg = "\033[3{}m{}\033[0m".format(c, msg)
msg = "\033[3%sm%s\033[0m" % (c, msg)
elif "\033" not in c:
msg = "\033[{}m{}\033[0m".format(c, msg)
msg = "\033[%sm%s\033[0m" % (c, msg)
else:
msg = "{}{}\033[0m".format(c, msg)
msg = "%s%s\033[0m" % (c, msg)
ts = datetime.utcfromtimestamp(now).strftime("%H:%M:%S.%f")[:-3]
msg = fmt.format(ts, src, msg)
zd = datetime.utcfromtimestamp(now)
ts = "%02d:%02d:%02d.%03d" % (
zd.hour,
zd.minute,
zd.second,
zd.microsecond // 1000,
)
msg = fmt % (ts, src, msg)
try:
print(msg, end="")
except UnicodeEncodeError:

View File

@@ -274,6 +274,10 @@ class ThumbSrv(object):
tdir, tfn = os.path.split(tpath)
ttpath = os.path.join(tdir, "w", tfn)
try:
bos.unlink(ttpath)
except:
pass
for fun in funs:
try:
@@ -570,11 +574,15 @@ class ThumbSrv(object):
want_caf = tpath.endswith(".caf")
tmp_opus = tpath
if want_caf:
tmp_opus = tpath.rsplit(".", 1)[0] + ".opus"
tmp_opus = tpath + ".opus"
try:
bos.unlink(tmp_opus)
except:
pass
caf_src = abspath if src_opus else tmp_opus
if not want_caf or (not src_opus and not bos.path.isfile(tmp_opus)):
if not want_caf or not src_opus:
# fmt: off
cmd = [
b"ffmpeg",
@@ -633,6 +641,12 @@ class ThumbSrv(object):
# fmt: on
self._run_ff(cmd)
if tmp_opus != tpath:
try:
bos.unlink(tmp_opus)
except:
pass
def poke(self, tdir: str) -> None:
if not self.poke_cd.poke(tdir):
return

View File

@@ -380,11 +380,11 @@ class Up2k(object):
if rd.startswith("//") or fn.startswith("//"):
rd, fn = s3dec(rd, fn)
fvp = "{}/{}".format(rd, fn).strip("/")
fvp = ("%s/%s" % (rd, fn)).strip("/")
if vp:
fvp = "{}/{}".format(vp, fvp)
fvp = "%s/%s" % (vp, fvp)
self._handle_rm(LEELOO_DALLAS, "", fvp, [])
self._handle_rm(LEELOO_DALLAS, "", fvp, [], True)
nrm += 1
if nrm:
@@ -2897,7 +2897,9 @@ class Up2k(object):
except:
pass
def handle_rm(self, uname: str, ip: str, vpaths: list[str], lim: list[int]) -> str:
def handle_rm(
self, uname: str, ip: str, vpaths: list[str], lim: list[int], rm_up: bool
) -> str:
n_files = 0
ok = {}
ng = {}
@@ -2906,7 +2908,7 @@ class Up2k(object):
self.log("hit delete limit of {} files".format(lim[1]), 3)
break
a, b, c = self._handle_rm(uname, ip, vp, lim)
a, b, c = self._handle_rm(uname, ip, vp, lim, rm_up)
n_files += a
for k in b:
ok[k] = 1
@@ -2920,7 +2922,7 @@ class Up2k(object):
return "deleted {} files (and {}/{} folders)".format(n_files, iok, iok + ing)
def _handle_rm(
self, uname: str, ip: str, vpath: str, lim: list[int]
self, uname: str, ip: str, vpath: str, lim: list[int], rm_up: bool
) -> tuple[int, list[str], list[str]]:
self.db_act = time.time()
try:
@@ -3027,16 +3029,22 @@ class Up2k(object):
if xad:
runhook(self.log, xad, abspath, vpath, "", uname, 0, 0, ip, 0, "")
ok: list[str] = []
ng: list[str] = []
if is_dir:
ok, ng = rmdirs(self.log_func, scandir, True, atop, 1)
else:
ok = ng = []
ok2, ng2 = rmdirs_up(os.path.dirname(atop), ptop)
if rm_up:
ok2, ng2 = rmdirs_up(os.path.dirname(atop), ptop)
else:
ok2 = ng2 = []
return n_files, ok + ok2, ng + ng2
def handle_mv(self, uname: str, svp: str, dvp: str) -> str:
if svp == dvp or dvp.startswith(svp + "/"):
raise Pebkac(400, "mv: cannot move parent into subfolder")
svn, srem = self.asrv.vfs.get(svp, uname, True, False, True)
svn, srem = svn.get_dbv(srem)
sabs = svn.canonical(srem, False)
@@ -3090,8 +3098,21 @@ class Up2k(object):
curs.clear()
rmdirs(self.log_func, scandir, True, sabs, 1)
rmdirs_up(os.path.dirname(sabs), svn.realpath)
rm_ok, rm_ng = rmdirs(self.log_func, scandir, True, sabs, 1)
for zsl in (rm_ok, rm_ng):
for ap in reversed(zsl):
if not ap.startswith(sabs):
raise Pebkac(500, "mv_d: bug at {}, top {}".format(ap, sabs))
rem = ap[len(sabs) :].replace(os.sep, "/").lstrip("/")
vp = vjoin(dvp, rem)
try:
dvn, drem = self.asrv.vfs.get(vp, uname, False, True)
bos.mkdir(dvn.canonical(drem))
except:
pass
return "k"
def _mv_file(

View File

@@ -537,7 +537,7 @@ class _Unrecv(object):
self.log = log
self.buf: bytes = b""
def recv(self, nbytes: int) -> bytes:
def recv(self, nbytes: int, spins: int = 1) -> bytes:
if self.buf:
ret = self.buf[:nbytes]
self.buf = self.buf[nbytes:]
@@ -548,6 +548,10 @@ class _Unrecv(object):
ret = self.s.recv(nbytes)
break
except socket.timeout:
spins -= 1
if spins <= 0:
ret = b""
break
continue
except:
ret = b""
@@ -590,7 +594,7 @@ class _LUnrecv(object):
self.log = log
self.buf = b""
def recv(self, nbytes: int) -> bytes:
def recv(self, nbytes: int, spins: int) -> bytes:
if self.buf:
ret = self.buf[:nbytes]
self.buf = self.buf[nbytes:]
@@ -609,7 +613,7 @@ class _LUnrecv(object):
def recv_ex(self, nbytes: int, raise_on_trunc: bool = True) -> bytes:
"""read an exact number of bytes"""
try:
ret = self.recv(nbytes)
ret = self.recv(nbytes, 1)
err = False
except:
ret = b""
@@ -617,7 +621,7 @@ class _LUnrecv(object):
while not err and len(ret) < nbytes:
try:
ret += self.recv(nbytes - len(ret))
ret += self.recv(nbytes - len(ret), 1)
except OSError:
err = True
@@ -1292,7 +1296,7 @@ class MultipartParser(object):
rfc1341/rfc1521/rfc2047/rfc2231/rfc2388/rfc6266/the-real-world
(only the fallback non-js uploader relies on these filenames)
"""
for ln in read_header(self.sr):
for ln in read_header(self.sr, 2, 2592000):
self.log(ln)
m = self.re_ctype.match(ln)
@@ -1492,15 +1496,15 @@ def get_boundary(headers: dict[str, str]) -> str:
return m.group(2)
def read_header(sr: Unrecv) -> list[str]:
def read_header(sr: Unrecv, t_idle: int, t_tot: int) -> list[str]:
t0 = time.time()
ret = b""
while True:
if time.time() - t0 > 120:
if time.time() - t0 >= t_tot:
return []
try:
ret += sr.recv(1024)
ret += sr.recv(1024, t_idle // 2)
except:
if not ret:
return []
@@ -1549,7 +1553,7 @@ def rand_name(fdir: str, fn: str, rnd: int) -> str:
def gen_filekey(salt: str, fspath: str, fsize: int, inode: int) -> str:
return base64.urlsafe_b64encode(
hashlib.sha512(
"{} {} {} {}".format(salt, fspath, fsize, inode).encode("utf-8", "replace")
("%s %s %s %s" % (salt, fspath, fsize, inode)).encode("utf-8", "replace")
).digest()
).decode("ascii")
@@ -1658,7 +1662,7 @@ def uncyg(path: str) -> str:
if len(path) > 2 and path[2] != "/":
return path
return "{}:\\{}".format(path[1], path[3:])
return "%s:\\%s" % (path[1], path[3:])
def undot(path: str) -> str:
@@ -1701,7 +1705,7 @@ def sanitize_fn(fn: str, ok: str, bad: list[str]) -> str:
bad = ["con", "prn", "aux", "nul"]
for n in range(1, 10):
bad += "com{0} lpt{0}".format(n).split(" ")
bad += ("com%s lpt%s" % (n, n)).split(" ")
if fn.lower().split(".")[0] in bad:
fn = "_" + fn
@@ -2266,7 +2270,7 @@ def rmdirs(
dirs = [os.path.join(top, x) for x in dirs]
ok = []
ng = []
for d in dirs[::-1]:
for d in reversed(dirs):
a, b = rmdirs(logger, scandir, lstat, d, depth + 1)
ok += a
ng += b

View File

@@ -1159,10 +1159,10 @@ html.y #widget.open {
background: #fff;
background: var(--bg-u3);
}
#wfm, #wzip, #wnp {
#wfs, #wfm, #wzip, #wnp {
display: none;
}
#wzip, #wnp {
#wfs, #wzip, #wnp {
margin-right: .2em;
padding-right: .2em;
border: 1px solid var(--bg-u5);
@@ -1174,6 +1174,7 @@ html.y #widget.open {
padding-left: .2em;
border-left-width: .1em;
}
#wfs.act,
#wfm.act {
display: inline-block;
}
@@ -1197,6 +1198,13 @@ html.y #widget.open {
position: relative;
display: inline-block;
}
#wfs {
font-size: .36em;
text-align: right;
line-height: 1.3em;
padding: 0 .3em 0 0;
border-width: 0 .25em 0 0;
}
#wfm span,
#wnp span {
font-size: .6em;

View File

@@ -261,6 +261,7 @@ var Ls = {
"mm_e403": "Could not play audio; error 403: Access denied.\n\nTry pressing F5 to reload, maybe you got logged out",
"mm_e5xx": "Could not play audio; server error ",
"mm_nof": "not finding any more audio files nearby",
"mm_pwrsv": "<p>it looks like playback is being interrupted by your phone's power-saving settings!</p>" + '<p>please go to <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">the app settings of your browser</a> and then <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">allow unrestricted battery usage</a> to fix it.</p><p>(probably a good idea to use a separate browser dedicated for just music streaming...)</p>',
"mm_hnf": "that song no longer exists",
"im_hnf": "that image no longer exists",
@@ -721,6 +722,7 @@ var Ls = {
"mm_e403": "Avspilling feilet: Tilgang nektet.\n\nKanskje du ble logget ut?\nPrøv å trykk F5 for å laste siden på nytt.",
"mm_e5xx": "Avspilling feilet: ",
"mm_nof": "finner ikke flere sanger i nærheten",
"mm_pwrsv": "<p>det ser ut som musikken ble avbrutt av telefonen sine strømsparings-innstillinger!</p>" + '<p>ta en tur innom <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262121-2ffc51ae-7821-4310-a322-c3b7a507890c.png">app-innstillingene til nettleseren din</a> og så <a target="_blank" href="https://user-images.githubusercontent.com/241032/235262123-c328cca9-3930-4948-bd18-3949b9fd3fcf.png">tillat ubegrenset batteriforbruk</a></p><p>(sikkert smart å ha en egen nettleser kun for musikkspilling...)</p>',
"mm_hnf": "sangen finnes ikke lenger",
"im_hnf": "bildet finnes ikke lenger",
@@ -952,6 +954,7 @@ ebi('ops').innerHTML = (
// media player
ebi('widget').innerHTML = (
'<div id="wtoggle">' +
'<span id="wfs"></span>' +
'<span id="wfm"><a' +
' href="#" id="fren" tt="' + L.wt_ren + '">✎<span>name</span></a><a' +
' href="#" id="fdel" tt="' + L.wt_del + '">⌫<span>del.</span></a><a' +
@@ -1482,7 +1485,8 @@ var mpl = (function () {
ebi('np_title').textContent = np.title || '';
ebi('np_dur').textContent = np['.dur'] || '';
ebi('np_url').textContent = get_vpath() + np.file.split('?')[0];
ebi('np_img').setAttribute('src', cover); // dont give last.fm the pwd
if (!MOBILE)
ebi('np_img').setAttribute('src', cover || ''); // dont give last.fm the pwd
navigator.mediaSession.metadata = new MediaMetadata(tags);
navigator.mediaSession.setActionHandler('play', mplay);
@@ -1499,6 +1503,7 @@ var mpl = (function () {
if (!r.os_ctl)
return;
// dead code; left for debug
navigator.mediaSession.metadata = null;
navigator.mediaSession.playbackState = "paused";
@@ -1538,12 +1543,14 @@ var re_au_native = can_ogg ? /\.(aac|flac|m4a|mp3|ogg|opus|wav)$/i :
// extract songs + add play column
var mpo = { "au": null, "au2": null, "acs": null };
var t_fchg = 0;
function MPlayer() {
var r = this;
r.id = Date.now();
r.au = null;
r.au = null;
r.au2 = null;
r.au = mpo.au;
r.au2 = mpo.au2;
r.acs = mpo.acs;
r.tracks = {};
r.order = [];
r.cd_pause = 0;
@@ -2159,17 +2166,30 @@ function song_skip(n) {
else
play(mp.order[n == -1 ? mp.order.length - 1 : 0]);
}
function next_song_sig(e) {
t_fchg = document.hasFocus() ? 0 : Date.now();
return next_song_cmn(e);
}
function next_song(e) {
t_fchg = 0;
return next_song_cmn(e);
}
function next_song_cmn(e) {
ev(e);
if (mp.order.length) {
mpl.traversals = 0;
return song_skip(1);
}
if (mpl.traversals++ < 5) {
treectl.ls_cb = next_song;
if (MOBILE && t_fchg && Date.now() - t_fchg > 30 * 1000)
modal.alert(L.mm_pwrsv);
t_fchg = document.hasFocus() ? 0 : Date.now();
treectl.ls_cb = next_song_cmn;
return tree_neigh(1);
}
toast.inf(10, L.mm_nof);
t_fchg = 0;
}
function prev_song(e) {
ev(e);
@@ -2285,6 +2305,10 @@ var mpui = (function () {
return;
}
var pos = mp.au.currentTime;
if (!isNum(pos))
pos = 0;
// indicate playback state in ui
widget.paused(mp.au.paused);
@@ -2307,10 +2331,18 @@ var mpui = (function () {
pbar.drawbuf();
}
if (pos > 0.3 && t_fchg) {
// cannot check document.hasFocus to avoid false positives;
// it continues on power-on, doesn't need to be in-browser
if (MOBILE && Date.now() - t_fchg > 30 * 1000)
modal.alert(L.mm_pwrsv);
t_fchg = 0;
}
// preload next song
if (mpl.preload && preloaded != mp.au.rsrc) {
var pos = mp.au.currentTime,
len = mp.au.duration,
var len = mp.au.duration,
rem = pos > 1 ? len - pos : 999,
full = null;
@@ -2483,7 +2515,7 @@ var afilt = (function () {
if (mp.acs)
mp.acs.disconnect();
mp.acs = null;
mp.acs = mpo.acs = null;
};
r.apply = function () {
@@ -2703,6 +2735,7 @@ function play(tid, is_ev, seek) {
tn = 0;
}
else if (mpl.pb_mode == 'next') {
t_fchg = document.hasFocus() ? 0 : Date.now();
treectl.ls_cb = next_song;
return tree_neigh(1);
}
@@ -2722,7 +2755,9 @@ function play(tid, is_ev, seek) {
if (mp.au) {
mp.au.pause();
clmod(ebi('a' + mp.au.tid), 'act');
var el = ebi('a' + mp.au.tid);
if (el)
clmod(el, 'act');
}
else {
mp.au = new Audio();
@@ -2730,7 +2765,7 @@ function play(tid, is_ev, seek) {
mp.au.onerror = evau_error;
mp.au.onprogress = pbar.drawpos;
mp.au.onplaying = mpui.progress_updater;
mp.au.onended = next_song;
mp.au.onended = next_song_sig;
widget.open();
}
@@ -2747,7 +2782,7 @@ function play(tid, is_ev, seek) {
mp.au.onerror = evau_error;
mp.au.onprogress = pbar.drawpos;
mp.au.onplaying = mpui.progress_updater;
mp.au.onended = next_song;
mp.au.onended = next_song_sig;
t = mp.au.currentTime;
if (isNum(t) && t > 0.1)
mp.au.currentTime = 0;
@@ -2807,7 +2842,7 @@ function play(tid, is_ev, seek) {
toast.err(0, esc(L.mm_playerr + basenames(ex)));
}
clmod(ebi(oid), 'act');
setTimeout(next_song, 5000);
setTimeout(next_song_sig, 5000);
}
@@ -2895,7 +2930,7 @@ function autoplay_blocked(seek) {
modal.confirm('<h6>' + L.mm_hashplay + '</h6>\n«' + esc(fn) + '»', function () {
// chrome 91 may permanently taint on a failed play()
// depending on win10 settings or something? idk
mp.au = null;
mp.au = mpo.au = null;
play(tid, true, seek);
mp.fade_in();
@@ -3234,7 +3269,9 @@ var fileman = (function () {
if (r.clip === null)
r.clip = jread('fman_clip', []).slice(1);
var nsel = msel.getsel().length;
var sel = msel.getsel(),
nsel = sel.length;
clmod(bren, 'en', nsel);
clmod(bdel, 'en', nsel);
clmod(bcut, 'en', nsel);
@@ -3246,9 +3283,51 @@ var fileman = (function () {
clmod(bpst, 'hide', !(have_mv && has(perms, 'write')));
clmod(ebi('wfm'), 'act', QS('#wfm a.en:not(.hide)'));
var wfs = ebi('wfs'), h = '';
try {
wfs.innerHTML = h = r.fsi(sel);
}
catch (ex) { }
clmod(wfs, 'act', h);
bpst.setAttribute('tt', L.ft_paste.format(r.clip.length));
};
r.fsi = function (sel) {
if (!sel.length)
return '';
var lf = treectl.lsc.files,
nf = 0,
sz = 0,
dur = 0,
ntab = new Set();
for (var a = 0; a < sel.length; a++)
ntab.add(sel[a].vp.split('/').pop());
for (var a = 0; a < lf.length; a++) {
if (!ntab.has(lf[a].href.split('?')[0]))
continue;
var f = lf[a];
nf++;
sz += f.sz;
if (f.tags && f.tags['.dur'])
dur += f.tags['.dur']
}
if (!nf)
return '';
var ret = '{0}<br />{1}<small>F</small>'.format(humansize(sz), nf);
if (dur)
ret += ' ' + s2ms(dur);
return ret;
};
r.rename = function (e) {
ev(e);
if (clgot(bren, 'hide'))
@@ -7277,21 +7356,25 @@ ebi('files').onclick = ebi('docul').onclick = function (e) {
function reload_mp() {
if (mp && mp.au) {
if (afilt)
afilt.stop();
mp.au.pause();
mp.au = null;
mpo.au = mp.au;
mpo.au2 = mp.au2;
mpo.acs = mp.acs;
mpl.unbuffer();
}
mpl.stop();
var plays = QSA('tr>td:first-child>a.play');
for (var a = plays.length - 1; a >= 0; a--)
plays[a].parentNode.innerHTML = '-';
mp = new MPlayer();
if (afilt)
afilt.acst = {};
if (mp.au && mp.au.tid) {
var el = QS('a#a' + mp.au.tid);
if (el)
clmod(el, 'act', 1);
el = el && el.closest('tr');
if (el)
clmod(el, 'play', 1);
}
setTimeout(pbar.onresize, 1);
}

View File

@@ -1,3 +1,51 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-0426-2300 `v1.6.15` unexpected boost
## new features
* 30% faster folder listings due to [the very last thing](https://github.com/9001/copyparty/commit/55c74ad164633a0a64dceb51f7f534da0422cbb5) i'd ever expect to be a bottleneck, [thx perf](https://docs.python.org/3.12/howto/perf_profiling.html)
* option to see the lastmod timestamps of symlinks instead of the target files
* makes the turbo mode of [u2cli, the commandline uploader and folder-sync tool](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) more turbo since copyparty dedupes uploads by symlinking to an existing copy and the symlink is stamped with the deduped file's lastmod
* **webdav:** enabled by default (because rclone will want this), can be disabled with arg `--dav-rt` or volflag `davrt`
* **http:** disabled by default, can be enabled per-request with urlparam `lt`
* [u2cli](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py): option `--rh` to resolve server hostname only once at start of upload
* fantastic for buggy networks, but it'll break TLS
## bugfixes
* new arg `--s-tbody` specifies the network timeout before a dead connection gets dropped (default 3min)
* before there was no timeout at all, which could hang uploads or possibly consume all server resources
* ...but this is only relevant if your copyparty is directly exposed to the internet with no reverse proxy
* with nginx/caddy/etc you can disable the timeout with `--s-tbody 0` for a 3% performance boost (*wow!*)
* iPhone audio transcoder could turn bad and stop transcoding
* ~~maybe android phones no longer pause playback at the end of an album~~
* nope, that was due to [android's powersaver](https://github.com/9001/copyparty#fix-unreliable-playback-on-android), oh well
* ***bonus unintended feature:*** navigate into other folders while a song is plaing
* [installing from the source tarball](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#build-from-release-tarball) should be ok now
* good base for making distro packages probably
## other changes
* since the network timeout fix is relevant for the single usecase that [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.6.15/copyparty-winpe64.exe) covers, there is now a new version of that
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-0424-0609 `v1.6.14` unsettable flags
## new features
* unset a volflag (override a global option) by negating it (setting volflag `-flagname`)
* new argument `--cert` to specify TLS certificate location
* defaults to `~/.config/copyparty/cert.pem` like before
## bugfixes
* in zip/tar downloads, always use the parent-folder name as the archive root
* more reliable ftp authentication when providing password as username
* connect-page: fix rclone ftps example
## other changes
* stop suggesting `--http-only` and `--https-only` for performance since the difference is negligible
* mention how some antivirus (avast, avg, mcafee) thinks that pillow's webp encoder is a virus, affecting `copyparty.exe`
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2023-0420-2141 `v1.6.12` as seen on nixos

View File

@@ -17,6 +17,7 @@
* [building](#building)
* [dev env setup](#dev-env-setup)
* [just the sfx](#just-the-sfx)
* [build from release tarball](#build-from-release-tarball) - uses the included prebuilt webdeps
* [complete release](#complete-release)
* [todo](#todo) - roughly sorted by priority
* [discarded ideas](#discarded-ideas)
@@ -110,6 +111,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?ls&dots` | list files/folders at URL as JSON, including dotfiles |
| GET | `?ls=t` | list files/folders at URL as plaintext |
| GET | `?ls=v` | list files/folders at URL, terminal-formatted |
| GET | `?lt` | in listings, use symlink timestamps rather than targets |
| GET | `?b` | list files/folders at URL as simplified HTML |
| GET | `?tree=.` | list one level of subdirectories inside URL |
| GET | `?tree` | list one level of subdirectories for each level until URL |
@@ -255,10 +257,31 @@ then build the sfx using any of the following examples:
```
## build from release tarball
uses the included prebuilt webdeps
if you downloaded a [release](https://github.com/9001/copyparty/releases) source tarball from github (for example [copyparty-1.6.15.tar.gz](https://github.com/9001/copyparty/releases/download/v1.6.15/copyparty-1.6.15.tar.gz) so not the autogenerated one) you can build it like so,
```bash
python3 setup.py install --user
```
or if you're packaging it for a linux distro (nice), maybe something like
```bash
bash scripts/run-tests.sh python3 # optional
python3 setup.py build
python3 setup.py install --skip-build --prefix=/usr --root=$HOME/pe/copyparty
```
## complete release
also builds the sfx so skip the sfx section above
does everything completely from scratch, straight from your local repo
in the `scripts` folder:
* run `make -C deps-docker` to build all dependencies

View File

@@ -194,6 +194,9 @@ sqlite3 .hist/up2k.db 'select * from mt where k="fgsfds" or k="t:mtp"' | tee /de
for ((f=420;f<1200;f++)); do sz=$(ffmpeg -y -f lavfi -i sine=frequency=$f:duration=2 -vf volume=0.1 -ac 1 -ar 44100 -f s16le /dev/shm/a.wav 2>/dev/null; base64 -w0 </dev/shm/a.wav | gzip -c | wc -c); printf '%d %d\n' $f $sz; done | tee /dev/stderr | sort -nrk2,2
ffmpeg -y -f lavfi -i sine=frequency=1050:duration=2 -vf volume=0.1 -ac 1 -ar 44100 /dev/shm/a.wav
# better sine
sox -DnV -r8000 -b8 -c1 /dev/shm/a.wav synth 1.1 sin 400 vol 0.02
# play icon calibration pics
for w in 150 170 190 210 230 250; do for h in 130 150 170 190 210; do /c/Program\ Files/ImageMagick-7.0.11-Q16-HDRI/magick.exe convert -size ${w}x${h} xc:brown -fill orange -draw "circle $((w/2)),$((h/2)) $((w/2)),$((h/3))" $w-$h.png; done; done

View File

@@ -64,6 +64,8 @@ git archive hovudstraum | tar -xC "$rls_dir"
echo ">>> export untracked deps"
tar -c copyparty/web/deps | tar -xC "$rls_dir"
scripts/genlic.sh "$rls_dir/copyparty/res/COPYING.txt"
cd "$rls_dir"
find -type d -exec chmod 755 '{}' \+
find -type f -exec chmod 644 '{}' \+
@@ -93,7 +95,7 @@ rm \
.gitattributes \
.gitignore
mv LICENSE LICENSE.txt
cp -pv LICENSE LICENSE.txt
# the regular cleanup memes
find -name '*.pyc' -delete

View File

@@ -1,7 +1,7 @@
#!/bin/bash
set -e
parallel=2
parallel=1
[ -e make-sfx.sh ] || cd scripts
[ -e make-sfx.sh ] && [ -e deps-docker ] || {

View File

@@ -98,7 +98,7 @@ class Cfg(Namespace):
def __init__(self, a=None, v=None, c=None):
ka = {}
ex = "daw dav_inf dav_mac dotsrch e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_thumb no_vthumb no_zip nrand nw rand vc xdev xlink xvol"
ex = "daw dav_inf dav_mac dav_rt dotsrch e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp ed emp force_js getmod hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_thumb no_vthumb no_zip nrand nw rand vc xdev xlink xvol"
ka.update(**{k: False for k in ex.split()})
ex = "dotpart no_rescan no_sendfile no_voldump plain_ip"
@@ -107,6 +107,9 @@ class Cfg(Namespace):
ex = "css_browser hist js_browser no_forget no_hash no_idx"
ka.update(**{k: None for k in ex.split()})
ex = "s_thead s_tbody"
ka.update(**{k: 9 for k in ex.split()})
ex = "df loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp theme themes turbo"
ka.update(**{k: 0 for k in ex.split()})