Compare commits

...

391 Commits

Author SHA1 Message Date
ed
a90dde94e1 v1.16.2 2024-11-23 23:36:15 +00:00
ed
7dfbfc7227 fix v1.16.0 webdav upload regression; closes #119 2024-11-23 23:32:56 +00:00
ed
b10843d051 cosmetic eta improvements:
* u2c: strip hh:mm:ss past 30 days
* u2js: fix "infini.ty" in elapsed-times
2024-11-23 19:58:25 +00:00
ed
520ac8f4dc fix opening md files from gridview 2024-11-23 17:55:05 +00:00
ed
537a6e50e9 javascript... 2024-11-22 23:10:23 +00:00
ed
2d0cbdf1a8 video-player: support mov files 2024-11-22 22:47:42 +00:00
ed
5afb562aa3 avoid layout-shift for qr-codes 2024-11-22 22:44:44 +00:00
ed
db069c3d4a fix shares qr-code on chrome 2024-11-22 22:28:00 +00:00
ed
fae40c7e2f black 2024-11-22 22:26:34 +00:00
ed
0c43b592dc pave the way for more ux volflags
makes directory listings a tiny bit faster, about 7% or so
2024-11-22 22:24:56 +00:00
ed
2ab8924e2d tests/debug: plug some resource leaks 2024-11-22 22:21:43 +00:00
akp
0e31cfa784 Allow multiple CIDR ranges when using lan shorthands
Signed-off-by: akp <abi@tdpain.net>
2024-11-22 19:51:56 +00:00
ed
8f7ffcf350 add nsort option/volflag 2024-11-19 18:39:40 +00:00
ed
9c8507a0fd fix downloads-eta layout jank 2024-11-17 19:39:44 +00:00
ed
e9b2cab088 update pkgs to 1.16.1 2024-11-15 22:40:41 +00:00
ed
d3ccacccb1 v1.16.1 2024-11-15 22:18:11 +00:00
ed
df386c8fbc ux: fix paste msg + cleanup css 2024-11-15 22:11:51 +00:00
ed
4d15dd6e17 cbz thumbnails 2024-11-15 21:33:37 +00:00
ed
56a0499636 fix gallery links when msel enabled 2024-11-15 20:04:13 +00:00
ed
10fc4768e8 fix dl from jumpvols with -j0 2024-11-15 19:29:44 +00:00
ed
2b63d7d10d detect invalid config (prevent db loss) 2024-11-15 08:04:58 +00:00
ed
1f177528c1 fix advanced options for password-hashing
and allow raising scrypt ram usage past OpenSSL's default 32 MiB
2024-11-15 00:42:08 +00:00
ed
fc3bbb70a3 update pkgs to 1.16.0 2024-11-10 20:00:38 +00:00
ed
ce3cab0295 v1.16.0 2024-11-10 19:32:37 +00:00
ed
c784e5285e u2c: adaptive connection:keepalive expiration 2024-11-10 17:43:40 +00:00
ed
2bf9055cae detect free RAM on startup for sane defaults
* if free ram on startup is less than 2 GiB,
   use smaller chunks for parallel file hashing

* if --th-max-ram is lower than 0.25 (256 MiB),
   print a warning that thumbnails will not work

* make thumbnail cleaner immediately do a sweep on startup,
   forgetting any failed conversions so they can be retried
   in case the memory limit was increased since last run
2024-11-10 15:43:19 +00:00
ed
8aba5aed4f list active downloads in controlpanel 2024-11-10 02:12:18 +00:00
ed
0ce7cf5e10 update comparison / versus.md 2024-11-09 14:44:03 +00:00
ed
96edcbccd7 https://ocv.me/stuff/goed-gedaan.jpg 2024-11-08 22:11:33 +00:00
ed
4603afb6de don't consume ctrl-shift-c (devtools inspector) 2024-11-08 21:51:54 +00:00
ed
56317b00af filecopy: ui for resolving name conflicts 2024-11-08 02:12:28 +00:00
ed
cacec9c1f3 support copying files/folders; closes #115
behaves according to the target volume's deduplication config;
will create symlinks / hardlinks instead if dedup is enabled
2024-11-07 21:41:53 +00:00
ed
44ee07f0b2 IdP: async reload; closes #114
whenever a new idp user is registered, up2k will continuously
reload in the background until all users have been processed

just like before, this blocks up2k uploads from each user
until said user makes it into a reload, but as of now,
reloads will batch and execute without interrupting read-access

needs further testing before next release,
probably some rough edges to sand down
2024-11-04 22:31:48 +00:00
ed
6a8d5e1731 ui: batch-rename: remember last regex + format 2024-11-02 18:06:39 +00:00
ed
d9962f65b3 ui: folder loading indicator stole focus
show a spinning halfcircle around the +/- instead of
moving the focus to the selected folder in the sidebar,
since that could mess with keyboard scrolling
2024-11-02 17:58:30 +00:00
ed
119e88d87b bubble OS-filesystem errors to client
send a 500 or 404 if a folder is inaccessible or does not exist

previously it would return an empty directory listing instead
2024-11-02 17:38:17 +00:00
ed
71d9e010d9 ui: make hotkey-help less eager to show itself
would appear when typing `?` into textboxes
2024-10-30 19:40:48 +00:00
ed
5718caa957 ui: url-options to set grid/thumbs on/off 2024-10-30 19:24:00 +00:00
ed
efd8a32ed6 ui: show switch-to-https on 403s too 2024-10-28 03:38:15 +00:00
ed
b22d700e16 update pkgs to 1.15.10 2024-10-27 09:27:38 +00:00
ed
ccdacea0c4 v1.15.10 2024-10-27 07:51:11 +00:00
ed
4bdcbc1cb5 shares: allow upload, unpost
* files can be uploaded into writeable shares

* add "write-only" button to the create-share ui

* unpost is possible while viewing the relevant share
2024-10-26 21:36:07 +00:00
ed
833c6cf2ec partyfuse: bump dircache size
dircache size should exceed max dir depth, because the OS
may periodically listdir all parents of current folder
2024-10-26 18:25:21 +00:00
ed
dd6dbdd90a http 304: client-option to force-disable cache
an extremely brutish workaround for issues such as #110 where
browsers receive an HTTP 304 and misinterpret as HTTP 200

option `--no304=1` adds the button `no304` to the controlpanel
which can be enabled to force-disable caching in that browser

the button is default-disabled; by specifying `--no304=2`
instead of `--no304=1` the button becomes default-enabled

can also always be enabled by accessing `/?setck=no304=y`
2024-10-26 17:56:54 +00:00
ed
63013cc565 http 304: k304 obsoleted for ie11 by Vary
the Vary header killed caching in all versions of internet explorer
so there's no point conditionally enabling k304 for trident anymore
2024-10-25 22:32:58 +00:00
ed
912402364a http 304: strip Content-Length and Content-Type
these response headers are usually not included in 304 replies,
and their presence are suspected to confuse some clients (#110)

also strip `out_headerlist` (primarily cookie assignments)
2024-10-25 22:24:40 +00:00
ed
159f51b12b http 304: if-range, backdating
add support for the `If-Range` header which is generally used to
prevent resuming a partial download after the source file on the
server has been modified, by returning HTTP 200 instead of a 206

also simplifies `If-Modified-Since` and `If-Range` handling;
previously this was a spec-compliant lexical comparison,
now it's a basic string-comparison instead. The server will now
reply 200 also when the server mtime is older than the client's.
This is technically not according to spec, but should be safer,
as it allows backdating timestamps without purging client cache
2024-10-25 22:05:59 +00:00
ed
7678a91b0e debug: --ohead (log response headers) 2024-10-25 20:00:19 +00:00
ed
b13899c63d make --u2sz more intuitive
previously, it only accepted the 3-tuple `min,default,max`

if given a single integer (or any other unexpected value),
the up2k js would enter an infinite loop, eat all the ram
and crash the browser (nice)

fix this by accepting a single integer (for example 96)
and translating it to `1,96,96`
2024-10-22 21:37:51 +00:00
ed
3a0d882c5e fix NetMap -j0 compat
would crash on startup if `-j0` was
combined with `--ipa` or `--ipu`
2024-10-22 20:53:19 +00:00
ed
cb81f0ad6d readme: add nintendo 3ds to supported browsers 2024-10-21 00:06:13 +00:00
ed
518bacf628 add pingvin-share to comparison 2024-10-20 01:18:07 +00:00
ed
ca63b03e55 update pkgs to 1.15.9 2024-10-18 23:54:46 +00:00
ed
cecef88d6b v1.15.9 2024-10-18 23:42:20 +00:00
ed
7ffd805a03 add RSS feed output; closes #109 2024-10-18 23:24:12 +00:00
ed
a7e2a0c981 up2k: fix chinese-specific js crash; closes #108
the client-side ETA, included as metadata in POSTs,
would crash the js with the initial "Starting..." text
2024-10-18 19:04:22 +00:00
ed
2a570bb4ca fix --df for webdav; closes #107
PUT uploads, as used by webdav, would stat the absolute
path of the file to be created, which would throw ENOENT

strip components until the path is an existing directory

and also try to enforce disk space / volume size limits
even when the incoming file is of unknown size
2024-10-18 18:14:35 +00:00
ed
5ca8f0706d up2k.js: detect broken webworkers;
the first time a file is to be hashed after a website refresh,
a set of webworkers are launched for efficient parallelization

in the unlikely event of a network outage exactly at this point,
the workers will fail to start, and the hashing would never begin

add a ping/pong sequence to smoketest the workers, and
fallback to hashing on the main-thread when necessary
2024-10-18 16:50:15 +00:00
ed
a9b4436cdc up2k: improve upload retry/timeout
* `js:` make handshake retries more aggressive
* `u2c:` reduce chunks timeout + ^
* `main:` reduce tcp timeout to 128sec (js is 42s)
* `httpcli:` less confusing log messages
2024-10-18 16:24:31 +00:00
ed
5f91999512 update pkgs to 1.15.8 2024-10-16 22:22:29 +00:00
ed
9f000beeaf v1.15.8 2024-10-16 21:53:23 +00:00
ed
ff0a71f212 gallery: play m4v videos 2024-10-16 21:36:11 +00:00
ed
22dfc6ec24 ui-toast: hide countdown if infinite 2024-10-16 21:32:47 +00:00
ed
48147c079e subchunks: fix eta, cfg-ui 2024-10-16 21:17:00 +00:00
ed
d715479ef6 add chickenbit to force hashwasm 2024-10-16 20:23:02 +00:00
ed
fc8298c468 up2k: avoid cloudflare upload size-limit
previously, the biggest file that could be uploaded through
cloudflare was 383 GiB, due to max num chunks being 4096

`--u2sz`, which takes three ints (min-size, target, max-size)
can now be used to enforce a max chunksize; chunks larger
than max-size gets split into smaller subchunks / chunklets

subchunks cannot be stitched/joined, and subchunks of the
same chunk must be uploaded sequentially, one at a time

if a subchunk fails due to bitflips or connection-loss,
then the entire chunk must (and will) be reuploaded
2024-10-16 19:29:08 +00:00
ed
e94ca5dc91 up2k: improve logging 2024-10-16 15:41:19 +00:00
ed
114b71b751 up2k: fix filesystem toctou
previously and currently, as an upload completes, its "done" flag
is not set until all the data has been flushed to disk

however, the list of missing chunks becomes empty before the flush,
and that list was incorrectly used to determine completion state
in some dedup-related logic

as a result, duplicate uploads could initially fail, and would
succeed after the client automatically retried a handful of times
2024-10-16 15:32:58 +00:00
ed
b2770a2087 u2c: support more crazy filenames
newlines, invalid utf8, and worst of all... %20 (whitespace)

due to up2k protocol limitations,
filenames are normalized when they hit the server,
but folders get to keep their intended jank
2024-10-15 23:01:07 +00:00
ed
cba1878bb2 u2c: don't get stuck at fifos and such 2024-10-15 22:53:55 +00:00
ed
a2e037d6af u2c: fix chunksize calculation
files which were exactly 128 GiB large would fail
(you can't make this shit up)
2024-10-15 22:39:48 +00:00
ed
65a2b6a223 u2c: fix excessive FDs
it would open separate FDs for all chunks to be uploaded...

open and close files as they are needed during upload instead
2024-10-15 22:30:15 +00:00
ed
9ed799e803 update pkgs to 1.15.7 2024-10-13 23:07:31 +00:00
ed
c1c0ecca13 v1.15.7 2024-10-13 22:44:57 +00:00
ed
ee62836383 bitflip logging 2024-10-13 22:37:35 +00:00
ed
705f598b1a up2k non-e2d fixes:
* respect noforget when loading snaps
* ...but actually forget deleted files otherwise
* insert empty need/hash as necessary
2024-10-13 22:10:27 +00:00
ed
414de88925 u2c v2.2 2024-10-13 22:07:41 +00:00
ed
53ffd245dd u2c: fix progress indicator for resumed uploads 2024-10-13 22:07:07 +00:00
ed
cf1b756206 u2c: option to list chunk hashes 2024-10-13 22:06:02 +00:00
ed
22b58e31ef unpost: authed users can see anon on same ip 2024-10-13 22:00:15 +00:00
ed
b7f9bf5a28 cidr-based autologin 2024-10-13 21:56:26 +00:00
ed
aba680b6c2 update pkgs to 1.15.6 2024-10-11 23:16:24 +00:00
ed
fabada95f6 v1.15.6 2024-10-11 22:56:10 +00:00
ed
9ccd8bb3ea support viewing dotfile docs; closes #104 2024-10-11 22:06:43 +00:00
ed
1d68acf8f0 add preadme.md; closes #105 2024-10-11 21:52:44 +00:00
ed
1e7697b551 misc cleanup;
* more typos
* python 3.13 deprecations
2024-10-11 20:46:40 +00:00
ed
4a4ec88d00 up2k: fix hs after bitflips / net-glitch
chunk stitching could cause handshakes to initiate
a new upload rather than resume an ongoing one
2024-10-11 19:48:44 +00:00
ed
6adc778d62 fix a buttload of typos 2024-10-11 18:58:14 +00:00
ed
6b7ebdb7e9 upgrade old snaps to dwrk + fix ptop
ptop would be wrong if a volume was moved on-disk since last run
2024-10-09 06:05:55 +00:00
ed
3d7facd774 add option to entirely disable dedup
global-option `--no-clone` / volflag `noclone` entirely disables
serverside deduplication; clients will then fully upload dupe files

can be useful when `--safe-dedup=1` is not an option due to other
software tampering with the on-disk files, and your filesystem has
prohibitively slow or expensive reads
2024-10-08 21:27:19 +00:00
ed
eaee1f2cab update pkgs to 1.15.5 2024-10-05 18:20:20 +00:00
ed
ff012221ae v1.15.5 2024-10-05 18:03:04 +00:00
ed
c398553748 pkgres: fix multiprocessing 2024-10-05 17:32:08 +00:00
ed
3ccbcf6185 update pkgs to 1.15.4 2024-10-04 23:56:45 +00:00
ed
f0abc0ef59 v1.15.4 2024-10-04 23:19:28 +00:00
ed
a99fa3375d the impresources.files traversible is not threadsafe 2024-10-04 22:37:29 +00:00
ed
22c7e09b3f small fixes;
* make-sfx: delete failed deps downloads
* tlcheck: detect untranslated strings
2024-10-04 20:56:16 +00:00
ed
0dfe1d5b35 toast countdown bar 2024-10-04 19:29:54 +00:00
ed
a99a3bc6d7 audio-player: fix compact-mode rendering glitch on narrow screens 2024-10-04 18:15:18 +00:00
ed
9804f25de3 add option for natural sorting; thx @oshiteku 2024-10-04 00:30:04 +00:00
ed
ae98200660 og: support filekeys 2024-10-03 23:52:11 +00:00
ed
e45420646f share folders as qr-codes 2024-10-03 23:14:06 +00:00
ed
21be82ef8b fix #101 (show logues even if dotfiles are hidden) 2024-10-03 22:19:32 +02:00
ed
001afe00cb i18n: time plurals 2024-10-03 07:38:33 +00:00
ed
19a5985f29 allow uploading logues; closes #100 2024-10-02 23:16:59 +00:00
ed
2715ee6c61 fix confusing toast on F2 with nothing selected (#100) 2024-10-02 23:11:29 +00:00
ed
dc157fa28f webdav: support explicit <allprop/> (WinSCP) 2024-10-02 22:28:23 +00:00
ed
1ff14b4e05 optimizations, failsafes, formatting 2024-10-02 21:59:53 +00:00
ed
480ac254ab webdav: show toplevel volumes when root is unmapped
previously, only real folders could be listed by a webdav client;
a server which does not have any filesystem paths mapped to `/`
would cause clients to panic when trying to list the server root

now, assuming volumes `/foo` and `/bar/qux` exist, when accessing `/`
the user will see `/foo` but not `/bar` due to limitations in `walk`,
and `qux` will only appear when viewing `/bar`

a future rework of the recursion logic should further improve this
2024-10-02 21:12:58 +00:00
ed
4b95db81aa thx keth 2024-10-01 22:37:50 +00:00
ed
c81e898435 partyfuse: also support mounting nginx, iis
these additional parsers are not included in the sfx-embedded
copy of partyfuse.py; grab it from github when necessary
2024-10-01 22:37:07 +00:00
ed
f1646b96ca dist: strip some pointless code 2024-10-01 18:35:36 +00:00
ed
44f2b63e43 partyfuse: embed fuse.py into sfx 2024-10-01 18:27:42 +00:00
ed
847a2bdc85 partyfuse: bump datacache chunksize
previous approach:
* cache 64K on first read
* cache 1M on subsequent intersecting reads

new approach:
* cache 64K on first read
* cache 1M on the next intersecting read
* cache 8M on subsequent intersecting reads
* cache 4M on standalone reads at offsets >1M

improves performance by 50% on windows
and should help on high-latency connections
2024-10-01 17:15:35 +00:00
ed
03f0f99469 partyfuse: fix extremely slow dircache lookups
the cache was a list of files instead of a dict... dude

also adds a max-num dircache limit
in addition to the expiration time
2024-10-01 17:07:28 +00:00
ed
3900e66158 partyfuse: modernize html parser (just in case) 2024-10-01 17:00:17 +00:00
ed
3dff6cda40 partyfuse: normalize naming in parsers 2024-10-01 16:55:00 +00:00
ed
73d05095b5 partyfuse: misc correctness;
* support more unix envs with granular fuse config
* generated URLs were OK but technically incorrect
2024-10-01 16:49:39 +00:00
ed
fcdc1728eb #102: make UI translation easier in docker 2024-10-01 00:04:07 +00:00
ed
8b942ea237 partyfuse: cleanup logging and exceptions
windows runs 50% faster with recentlog on infos too...
2024-09-29 23:19:33 +00:00
ed
88a1c5ca5d optimize non-e2d ram usage down to 10% or so
drop chunk-hashes in the up2k snap, plus other insignificant attribs
to reduce both the snapfile size and the ram usage by about 90%

reduces startup/shutdown time by a lot since there's less to serdes
(does not affect -e2d which was already optimal)

other changes:

* improve incoming-eta accuracy when the initial handshake
   was made a long time before the upload actually started

* move the list of incoming files in the controlpanel to the top
2024-09-27 21:11:10 +00:00
ed
047176b297 py2 fix 2024-09-27 21:06:01 +00:00
ed
dc4d0d8e71 smb: upto 2x faster; but still very buggy:
* do not absreal paths unless necessary
* do not determine username if no users configured
* impacket 0.12 fixed the foldersize limit, but now
   you get extremely poor performance in large folders
   so the previous workaround is still default-enabled
2024-09-27 17:09:48 +00:00
ed
b9c5c7bbde u2c: early exclude + fix py2 perf
* don't traverse into excluded folders
* 3.5x faster on python2.7
2024-09-23 17:20:04 +00:00
ed
9daeed923f u2c: remove all deps to become 3x faster on small files
reduces performance on python 2.7, but that is ok

also fixes `unknown encoding: idna` due to cpython race
2024-09-22 18:07:36 +00:00
ed
66b260cea9 pkgres: fix tiny leak in template loader 2024-09-20 22:25:36 +00:00
ed
58cf01c2ad fix linter warnings 2024-09-20 22:24:39 +00:00
ed
d866841c19 pkgres:
* pyz: yeet the resource tar which is now pointless thanks to pkgres
* cache impresource stuff because pyz lookups are Extremely slow
* prefer tx_file when possible for slightly better performance
* use hardcoded list of expected resources instead of dynamic
   discovery at runtime; much simpler and probably safer
* fix some forgotten resources (copying.txt, insecure.pem)
* fix loading jinja templates on windows
2024-09-19 22:04:49 +00:00
Shiz
a462a644fb Python 3.7 package resources support (#98)
add support for reading webdeps and jinja-templates using either
importlib_resources or pkg_resources, which removes the need for
extracting these to a temporary folder on the filesystem

* util: add helper functions to abstract embedded resource access
* http*: serve embedded resources through resource abstraction
* main: check webdeps through resource abstraction
* httpconn: remove unused method `respath(name)`
* use __package__ to find package resources
* util: use importlib_resources backport if available
* pass E.pkg as module object for importlib_resources compatibility
* util: add pkg_resources compatibility to resource abstraction
2024-09-19 09:00:34 +00:00
ed
678675a9a6 fix prometheus metrics; broke in 609c5921 2024-09-16 21:04:58 +00:00
ed
de9069ef1d update pkgs to 1.15.3 2024-09-16 01:20:16 +00:00
ed
c0c0a1a83a v1.15.3 2024-09-16 01:07:50 +00:00
ed
1d004b6dbd update pkgs to 1.15.2 2024-09-16 00:44:48 +00:00
ed
b90e1200d7 v1.15.2 2024-09-16 00:20:20 +00:00
ed
4493a0a804 misc mojibake filename support 2024-09-16 00:12:49 +00:00
ed
58835b2b42 ux bugfixes:
* show media tags in shares
* html hydrator assumed a folder named `foo.txt` was a doc
* due to sessions, use `pwd` as password placeholder on services
2024-09-15 23:37:24 +00:00
ed
427597b603 show total directory size in listings
sizes are computed during `-e2ds` indexing, and new uploads
are counted, but a rescan is necessary after a move or delete
2024-09-15 23:01:18 +00:00
ed
7d64879ba8 more optimizations,
* 5% less cpu load from clients fetching thumbnails
* and slight improvement to up2k stuff
2024-09-15 17:46:43 +00:00
ed
bb715704b7 ren_open was too fancy 2024-09-15 14:39:35 +00:00
ed
d67e9cc507 sqlite and misc optimizations:
* exponentially slow upload handshakes caused by lack of rd+fn
   sqlite index; became apparent after a volume hit 200k files
* listing big folders 5% faster due to `_quotep3b`
* optimize `unquote`, 20% faster but only used rarely
* reindex on startup 150x faster in some rare cases
   (same filename in MANY folders)

the database is now around 10% larger (likely worst-case)
2024-09-15 13:18:43 +00:00
ed
2927bbb2d6 strip dev-only asserts at build stage 2024-09-14 22:17:35 +00:00
ed
0527b59180 cosmetic: only print hostname warning once 2024-09-14 20:37:56 +00:00
ed
a5ce1032d3 tlnote + nginx unix-socket example 2024-09-12 21:42:33 +00:00
ed
1c2acdc985 add flameshot client example 2024-09-11 20:56:38 +00:00
ed
4e75534ef8 optimize BrokerThr, 7x faster:
reduce the overhead of function-calls from the client thread
to the svchub singletons (up2k, thumbs, metrics) down to 14%

and optimize up2k chunk-receiver to spend 5x less time bookkeeping
which restores up2k performance to before introducing incoming-ETA
2024-09-11 20:37:10 +00:00
ultwcz
7a573cafd1 fix: translation: Check the newly added Chinese translation (#97) 2024-09-11 19:03:53 +00:00
ed
844194ee29 incoming-ETA: improve accuracy 2024-09-11 06:56:12 +00:00
ed
609c5921d4 list incoming files + ETA in controlpanel 2024-09-10 21:24:05 +00:00
ed
c79eaa089a update pkgs to 1.15.1 2024-09-09 23:55:37 +00:00
ed
e9d962f273 v1.15.1 2024-09-09 23:43:43 +00:00
ed
b5405174ec add login sessions 2024-09-09 23:39:20 +00:00
ed
6eee601521 fix u2c --ow (overwrite/replace)
the u2c flag to overwrite files on the server became no-op in v1.13.8
2024-09-09 19:40:38 +00:00
ed
2fac2bee7c update pkgs to 1.15.0 2024-09-08 20:02:25 +00:00
ed
c140eeee6b v1.15.0 2024-09-08 19:25:46 +00:00
ed
c5988a04f9 up2k.js: bump handshake timeout for safededup 2024-09-08 18:06:37 +00:00
ed
a2e0f98693 disable upload deduplication by default;
dedup is still encouraged and fully supported, but
being default-enabled has caused too many surprises

enabling `--dedup` restores the previous default behavior

also renames `--never-symlink` to `--hardlink-only`
2024-09-08 17:09:14 +00:00
ed
1111153f06 test dedup relinking 2024-09-08 12:55:27 +00:00
ed
e5a836cb7d og: fix links to textfiles 2024-09-08 12:12:34 +00:00
ed
b0de84cbc5 db-verify: support newlines in filenames + flag 404s 2024-09-08 00:44:22 +00:00
ed
cbb718e10d css fixes:
* improve hotdog-stand theme
* fix up2k tabs glow (went poof in a syntax error)
2024-09-07 19:29:40 +00:00
ed
b5ad9369fe confine xlink behavior behind its volflag
symlinks between volumes will only be created if xlink is
enabled, so such symlinks should be ignored if xlink is
disabled, as they might originate from other software

this prevents accidental rewriting of non-dedup symlinks
2024-09-07 19:17:32 +00:00
ed
4401de0413 fix mv with --no-dedup in volumes with dupes;
if --no-dedup was enabled in a volume which already contained
symlinked duplicate files, renaming/moving folders could fail

this is due to folder contents being moved one file at a time
(which is how symlink breakage is prevented) except the links
are moved assuming the final directory layout, meaning they
may be intermittently broken during the movie

with no-dedup, the symlinks are converted into full files as
each symlink is encountered, but a temporarily broken symlink
would crash the procedure

fix this by giving `_symlink` a new parameter `fsrc`
which is a known valid inode for data copying purposes
2024-09-07 00:47:12 +00:00
ed
6e671c5245 verify on-disk contents before dedup;
previously, the assumption was made that the database and filesystem
would not desync, and that an upload could safely be substituted with
a symlink to an existing copy on-disk, assuming said copy still
existed on-disk at all

this is fine if copyparty is the only software that makes changes to
the filesystem, but that is a shitty assumption to make in hindsight

add `--safe-dedup` which takes a "safety level", and by default (50)
it will no longer blindly expect that the filesystem has not been
altered through other means; the file contents will now be hashed
and compared to the database

deduplication can be much slower as a result, but definitely worth it
as this avoids some potentially very unpleasant surprises

the previous behavior can be restored with `--safe-dedup 1`
2024-09-06 19:08:14 +00:00
ed
08848be784 u2c: add hashgen mode + fix shutdown lag 2024-09-06 00:31:25 +00:00
ed
b599fbae97 use local timezone in log messages; closes #96
timezone can be changed with `export TZ=Europe/Oslo` before launch

using naive timestamps like this appears to be safe as of 3.13-rc1,
no deprecation warnings, just a tiny bit slower than assuming UTC
2024-09-05 19:31:33 +00:00
ed
a8dabc99f6 add more translations 2024-09-04 23:46:32 +00:00
ed
f1130db131 fix confusing message when uploading dupes
due to deduplication, it is intentionally impossible to
upload several identical copies of a file in parallel

by default, the up2k client will upload files sorted by
size, which usually leads to dupes being grouped together,
and it will try to do just that

this is by design, as it improves performance on average,
but it also shows the confusing (but technically-correct)
message "resume the partial upload into the original path"

fix this with a more appropriate message

note that this approach was selected in favor of pausing
handshakes while the initial copy finishes uploading,
because that could severely reduce upload performance
by preventing optimal use of multiple connections
2024-09-04 22:03:26 +00:00
ed
735ec35546 update pkgs to 1.14.4 2024-09-02 01:21:07 +00:00
ed
5a009a2a64 v1.14.4 2024-09-02 01:08:41 +00:00
ed
d9e9526247 fix js typo (could panic on network glitches) 2024-09-02 00:58:15 +00:00
ed
5a8c3b8be0 optimize test_httpcli.py too, from 1.64 to 1.51s 2024-08-31 22:03:06 +00:00
ed
1c9c17fb9b optimize test_dedup.py
* 7.71s originally
* 4.51s with fstab reuse
* 4.34s without db_wd
* 4.02s with no pp start
* 3.73s with Cfg reuse
2024-08-31 21:54:47 +00:00
ed
7f82449179 changelog: cleanup historic entries 2024-08-31 20:39:37 +00:00
ed
e455ec994e logo tweaks (kerning, footer-slant) 2024-08-31 20:37:58 +00:00
ed
c111027420 update pkgs to 1.14.3 2024-08-30 23:29:47 +00:00
ed
abcdf479e6 v1.14.3 2024-08-30 23:11:22 +00:00
ed
ad2371f810 shares: add revival and expiration extension 2024-08-30 22:25:50 +00:00
ed
c4e2b0f95f doc-viewer: always wordwrap code 2024-08-30 22:13:10 +00:00
ed
3da62ec234 fix dedup bug as of v1.13.8:
* v1.13.8 broke collision resolving for non-identical files;
   the correct filename was reserved but not symlinked to
   the original file, leaving a zerobyte file instead.
   See v1.14.3 github release notes for remediation info

* add sanchecks for early detection of index/fs desync;
   saves performance and gives less confusing logs
2024-08-30 22:06:25 +00:00
ed
01233991f3 tftp: support unmapped root 2024-08-30 16:08:50 +00:00
ed
ee35974273 readme hacking 2024-08-29 22:17:13 +00:00
ed
7037e7365e add logo 2024-08-29 22:00:08 +00:00
ed
03b13e8a1c sfx-customizer:
* better translation stripping
* add support in bruteforcer
* add examples

and fix login-banner usage example
2024-08-28 05:53:26 +00:00
ed
cdd2da0208 update pkgs to 1.14.2 2024-08-23 23:43:46 +00:00
ed
cec0e0cf02 v1.14.2 2024-08-23 23:07:18 +00:00
ed
8122ddedfe share multiple files (#84);
if files (one or more) are selected for sharing, then
a virtual folder is created to hold the selected files

if a single file is selected for sharing, then
the returned URL will point directly to that file

and fix some shares-related bugs:
* password coalescing
* log-spam on reload
2024-08-23 22:55:31 +00:00
ultwcz
55a77c5e89 Chinese translation fixes (#95)
* fix: translation: changing from `" "` to `' '` for some strings;
	using `./scripts/tlcheck.sh eng chi copyparty/web/browser.js`

* fix: translation: Check the newly added Chinese translation
2024-08-23 08:14:24 +00:00
ed
461f31582d add IDs for ricing (#93) + fix a11y bleed 2024-08-22 20:14:08 +00:00
ed
f356faa278 u2c: support multiple exclusion patterns 2024-08-22 20:03:25 +00:00
ed
9f034d9c4c fix confusing logmsg for zerobyte files 2024-08-22 19:54:10 +00:00
ed
ba52590ae4 translation tweaks 2024-08-22 19:52:20 +00:00
ultwcz
92edea1de5 add translation: Chinese (#94) 2024-08-22 17:19:16 +00:00
ed
7ff46966da fix some issues with shares mentioned in #84;
* crash when root volume is unmapped
* rephrase login-page for shares
* add chrome support (lol)
* fix confusing helptext
* improve ux
  * placeholders in share creator
  * button to disable expiration in share creator
  * human-readable timestamps in share listing
2024-08-19 21:38:47 +00:00
ed
fca70b3508 update pkgs to 1.14.1 2024-08-19 00:24:52 +00:00
ed
70009cd984 v1.14.1 2024-08-19 00:14:44 +00:00
ed
8d8b88c4fd update pkgs to 1.14.0 2024-08-18 23:36:57 +00:00
ed
c4b0cccefd v1.14.0 2024-08-18 23:11:36 +00:00
ed
7c2beba555 add file/folder sharing; closes #84 2024-08-18 22:49:13 +00:00
ed
7d8d94388b invert volume scrollwheel
<daniiooo> also iirc some time ago we were talking about the scroll for volume ed
<daniiooo> and how its reversed
<ed> is it reversed though? most people said it worked the way they expected
<daniiooo> fuck maybe i agreed back then too
<daniiooo> its the opposite in both aimp and mpv though
<ed> is it w
<tatsu> its a feature
<Devices> it's to keep you on your toes
<Devices> consciously use copyparty
<ed> i can invert it no problem
<ed> would be a nice surprise for anyone who's used it
<Flaminator> Scroll down turns the audio down right?
<daniiooo> ye it makes it louder in cpp
<Devices> why would scrolling down make something louder
<Vin> yeah that's odd
<Vin> scrolling up should make it louder
<Flaminator> It's what it does for me in winamp, mpc-hc and foobar2000.
<daniiooo> so now the question is who itc agreed to whats currently in cpp
<daniiooo> haha
<ed> idk but i'm inverting it
<ed> let's invert it every 6 months
2024-08-17 20:36:59 +00:00
ed
0b46b1a614 fix some vproxy issues (#93):
* navpane would always feed the vproxy paths into the tree
   instead of only when necessary (the initial load)

* mkdir would return `X-New-Dir` without the `rp-loc` prefix
  * chpw and some other redirects also sent raw vpaths

Reported-by: @iridial
2024-08-17 18:17:40 +00:00
ed
5153db6bff ux: login margin; theme2: yellow buttons
the red buttons from protonmail's monokai theme look better,
but they're confusing because intuitively red means off
2024-08-17 15:55:55 +00:00
ed
b0af4b3712 hook/reloc: dupe in one vol doesn't mean dupe in another 2024-08-16 21:08:22 +00:00
ed
c8f4aeaefa hook/reloc: fix up2k jank
* wark landed in the wrong registry when moved to another volume
   (harmless; upload would succeed on the next handshake)

* dedup did not apply correctly when moved into another volume,
   since all the checks were done based on the previous vol;
   fix this by recursing the whole thing

also update the reloc example after some real-world experience

Reported-by: @daniiooo
2024-08-15 19:26:06 +00:00
ed
00da74400c password-changer fixes:
* fix `--chpw-no` which did nothing
* print list of users with unchanged passwords by default
* more granular verbosity levels
2024-08-15 17:30:01 +00:00
ed
83fb569d61 make passwords user-changeable; closes #92 2024-08-14 20:09:57 +00:00
ed
5a62cb4869 fix custom fonts in sandboxed docs;
`@import` must be at the very start of a `<style>` tag

Reported-by: @thaddeuskkr (thx!)
2024-08-14 15:30:04 +00:00
ed
687df2fabd unix-socket fixes:
* support x-forwarded-for
* option to specify socket permissions and group
* in containers, avoid collision during restart
* add --help-bind with examples
2024-08-14 04:47:10 +00:00
ed
cdd0794d6e update pkgs to 1.13.8 2024-08-13 00:20:04 +00:00
ed
dcc988135e v1.13.8 2024-08-13 00:08:23 +00:00
ed
3db117d85f list status of optional dependencies 2024-08-12 22:48:53 +00:00
ed
ee9aad82dd support listening on unix sockets 2024-08-12 21:58:02 +00:00
ed
2d6eb63fce scripts/uncomment: python 3.12 support;
`tokenize.FSTRING_MIDDLE` was introduced, changing the
representation of `f"x{{y"` from `STRING(f"x{{y")` to:

* `FSTRING_START('f"')`
* `FSTRING_MIDDLE('x{')`
* `FSTRING_MIDDLE('y')`
* `FSTRING_END('"')`

each literal `{` (encoded as `{{` in the input) now appears as a
single `{` as the final character of its `FSTRING_MIDDLE`, with
additional consecutive `FSTRING_MIDDLE` tokens if necessary

regular interpolating `{` are encoded as separate `OP` tokens

the fact that the literal `{` is encoded as a single `{` instead
of `{{` breaks the assumption that the string-value of each token
maps directly to the original code

fix this by replacing `{` with `{{` and `}` with `}}` in
`FSTRING_MIDDLE` tokens, and not adding whitespace after
`FSTRING_MIDDLE` tokens
2024-08-12 19:55:17 +00:00
ed
ca001c8504 update deps (pyftpdlib, win10-python) 2024-08-12 18:51:52 +00:00
ed
4e581c59da fix s390x w/a, up2k name-randomizer 2024-08-12 17:45:19 +00:00
ed
dbd42bc6bf add option to load custom js on all pages 2024-08-11 23:51:17 +00:00
ed
c862ec1b64 up2k.js: optimal pipelining 2024-08-11 21:15:44 +00:00
ed
f709140571 hook/reloc: helptext mentioned jank that doesn't exist anymore 2024-08-11 15:07:21 +00:00
ed
ef1c4b7a20 this guy didn't make it in 2024-08-11 14:55:51 +00:00
ed
6c94a63f1c add hook side-effects; closes #86
hooks can now interrupt or redirect actions, and initiate
related actions, by printing json on stdout with commands

mainly to mitigate limitations such as sharex/sharex#3992

xbr/xau can redirect uploads to other destinations with `reloc`
and most hooks can initiate indexing or deletion of additional
files by giving a list of vpaths in json-keys `idx` or `del`

there are limitations;
* xbu/xau effects don't apply to ftp, tftp, smb
* xau will intentionally fail if a reloc destination exists
* xau effects do not apply to up2k

also provides more details for hooks:
* xbu/xau: basic-uploader vpath with filename
* xbr/xar: add client ip
2024-08-11 14:52:32 +00:00
ed
20669c73d3 rm dead code (gridview conditional dl/play)
and maybe fix negative eta when a chunk gets eaten by the network
2024-08-09 21:57:42 +00:00
ed
0da719f4c2 up2k: shrink request headers
v1.13.5 made some proxies angry with its massive chunklists

when stitching chunks, only list the first chunk hash in full,
and include a truncated hash for the consecutive chunks

should be enough for logfiles to make sense
and to smoketest that clients are behaving
2024-08-08 18:24:18 +00:00
ed
373194c38a better up2k stitching on fat32 servers:
* the batches don't need to be window-aligned
* improve js backoff (in case of more funnies)
2024-08-05 19:52:50 +00:00
ed
3d245431fc linter fixes 2024-08-05 18:48:16 +00:00
ed
250c8c56f0 fix deadlock on IBM mainframes (s390x) 2024-08-02 23:05:44 +00:00
ed
e136231c8e docker: add portainer howto 2024-08-02 23:01:32 +00:00
ed
98ffaadf52 docker: use less RAM at runtime
compile to bytecode so cpython doesn't have to keep it in memory

ram usage reduced by:
* min: 5.4 MiB (32.6 to 27.2)
* ac/im: 5.2 MiB (39.0 to 33.8)
* dj/iv: 10.6 MiB (67.3 to 56.7)

startup time reduced from:
* min: 1.3s to 0.6s
* ac/im: 1.6s to 0.9s
* dj/iv: 2.0s to 1.1s

image size increased by 4 MiB (min), 6 MiB (ac/im/iv), 9 MiB (dj)

ram usage measured on idle with:
while true; do ps aux | grep -E 'R[S]S|no[-]crt'; read -n1; echo; done

startup time measured with:
time podman run --rm -it localhost/copyparty-min-amd64 --exit=idx
2024-08-02 22:11:23 +00:00
ed
ebb1981803 py2: reduce ram usage 2024-08-01 20:01:42 +00:00
ed
72361c99e1 add import chickenbits 2024-08-01 18:29:25 +00:00
ed
d5c9c8ebbd make it 5% faster 2024-07-31 17:51:53 +00:00
ed
746229846d add test for zip-download 2024-07-30 22:44:29 +00:00
ed
ffd7cd3ca8 update pkgs to 1.13.6 2024-07-29 20:56:00 +00:00
ed
b3cecabca3 v1.13.6 2024-07-29 20:28:51 +00:00
ed
662541c64c audio-player: show status while loading 2024-07-29 20:14:39 +00:00
ed
225bd80ea8 up2k.js: fix overshoot in chunk stitcher 2024-07-29 19:19:22 +00:00
ed
85e54980cc up2k.js: set timeouts for uploads
in the event that an upload chunk gets stuck, the js would
never stop waiting for a response, requiring a page reload

improves reliability when running behind a reverse-proxy
which is configured to never timeout requests (can make
sense when combined with other services on the same box)
2024-07-29 19:17:03 +00:00
ed
a19a0fa9f3 fix modal wordwrap in firefox;
with overflow:auto, firefox picks the div-width before estimating
the height, causing it to undershoot by the scrollbar width
and then messing up the text alignment

fix: conditionally set overflow-y:scroll using js
2024-07-29 18:04:35 +00:00
ed
9bb6e0dc62 misc ux:
* wait until page (au) has loaded to register hotkeys
* hotkey `m` would grow sidebar if tree was minimized
* more exact warning about num.parallel uploads
* keep more console logs in memory
* message phrasing
2024-07-29 17:59:34 +00:00
ed
15ddcf53e7 add bsod theme 2024-07-26 22:09:59 +00:00
ed
6b54972ec0 update comparison vs similar software:
* general changes:
  * upload speed comparisons considering v1.13.5

* hfs2:
  * dead project with unfixed vulnerabilities

* hfs3:
  * has replaced hfs2
  * uploads are now resumable
  * add new functionality:
    * write-only folders
    * unmap subfolders
    * move and delete files
    * folder-rproxy
    * themes
    * basic audio player, image viewer

* filebrowser:
  * uploads are now parallelized, resumable, segmented
    * but single large files are not accelerated
  * can listen on unix sockets
  * folder-rproxy is supported
  * more cpu efficient than copyparty
2024-07-26 19:46:03 +00:00
ed
0219eada23 cleanup: strip trailing whitespace 2024-07-26 19:33:56 +00:00
ed
8916bce306 u2c fixes:
* `--sz` was num.chunks, not the intended MiB
* crash on exit with `-z` and no modified files
* summary upload elapsed-time could exceed wallclock
2024-07-26 19:28:47 +00:00
ed
99edba4fd9 change xm examples to reject users without write-access; #68 2024-07-25 19:23:08 +00:00
ed
64de3e01e8 update pkgs to 1.13.5 2024-07-22 23:48:24 +00:00
ed
8222ccc40b v1.13.5 2024-07-22 23:23:53 +00:00
ed
dc449bf8b0 fix grid toolbar undocking after viewing a pic/vid 2024-07-22 23:09:25 +00:00
ed
ef0ecf878b recommend rclone over davfs2; closes #90 2024-07-22 22:46:24 +00:00
ed
53f1e3c91d ui option to play video as audio
audio extraction happens serverside to opus or mp3
depending on browser support

remuxing (extracting audio without transcoding)
is currently not supported, and is not planned
2024-07-22 22:30:21 +00:00
ed
eeef80919f css-fix for firefox52 (centos6) 2024-07-22 20:59:05 +00:00
ed
987bce2182 u2c fixes:
* don't stitch across deduplicated blocks
* print speed/time for hash/upload
* more compact json in handshakes
2024-07-22 20:55:32 +00:00
ed
b511d686f0 up2k fixes:
* progress donuts should include inflight bytes
* changes to stitch-size in settings didn't apply until next refresh
* serverlog was too verbose; truncate chunk hashes
* mention absolute cloudflare limit in readme
2024-07-22 19:06:01 +00:00
ed
132a83501e add chunk stitching; twice as fast long-distance uploads:
rather than sending each file chunk as a separate HTTP request,
sibling chunks will now be fused together into larger HTTP POSTs
which results in unreasonably huge speed boosts on some routes
( `2.6x` from Norway to US-East,  `1.6x` from US-West to Finland )

the `x-up2k-hash` request header now takes a comma-separated list
of chunk hashes, which must all be sibling chunks, resulting in
one large consecutive range of file data as the post body

a new global-option `--u2sz`, default `1,64,96`, sets the target
request size as 64 MiB, allowing the settings ui to specify any
value between 1 and 96 MiB, which is cloudflare's max value

this does not cause any issues for resumable uploads; thanks to the
streaming HTTP POST parser, each chunk will be verified and written
to disk as they arrive, meaning only the untransmitted chunks will
have to be resent in the event of a connection drop -- of course
assuming there are no misconfigured WAFs or caching-proxies

the previous up2k approach of uploading each chunk in a separate HTTP
POST was inefficient in many real-world scenarios, mainly due to TCP
window-scaling behaving erratically in some IXPs / along some routes

a particular link from Norway to Virginia,US is unusably slow for
the first 4 MiB, only reaching optimal speeds after 100 MiB, and
then immediately resets the scale when the request has been sent;
connection reuse does not help in this case

on this route, the basic-uploader was somehow faster than up2k
with 6 parallel uploads; only time i've seen this
2024-07-21 23:35:37 +00:00
ed
e565ad5f55 better errors through broker 2024-07-21 20:36:50 +00:00
ed
f955d2bd58 dangit 2024-07-20 22:28:40 +00:00
ed
5953399090 add helptext exporters (html, txt) 2024-07-17 23:06:01 +00:00
ed
d26a944d95 hooks: add cache-warmer 2024-07-17 21:00:59 +00:00
ed
50dac15568 update pkgs to 1.13.4 2024-07-16 05:48:45 +00:00
ed
ac1e11e4ce v1.13.4 2024-07-16 04:57:26 +00:00
ed
d749683d48 hooks: add permission filtering, argv-prepend;
hooks can be restricted to users with certain permissions, for example
`--xm aw,notify-send` will only `notify-send` if user has write-access

the user's list of permissions are now also included in the json
that is passed to the hook if enabled; `--xm aw,j,notify-send`

will now also stop parsing flags when encountering a blank value,
allowing to specify any initial arguments to the command:
`--xm aw,j,,notify-send,hey` would run `notify-send` with `hey`
as its first argument, and the json would be the 2nd argument,
similarly `--xm ,notify-send,hey` when no flags specified

this is somewhat explained in `--help-hooks`, but
additional related features are planned in the near future
and will all be better documented when the dust settles
2024-07-16 04:45:02 +00:00
ed
84e8e1ddfb ftpd: only mention vols that user can access
if an ftp client tried to list the toplevel folder on a server
where nothing is mounted toplevel, it would syntheisze a
directory listing which included all volumes, even those
which the user would not be able to access

so basically not a problem, just very confusing
2024-07-15 21:24:26 +00:00
ed
6e58514b84 update deps:
* win10:
  * python 3.12
  * pillow 10.4
  * pyinstaller 6.9
* win: upx 4.2.4
* web: dompurify 3.1.6
2024-07-15 21:16:19 +00:00
ed
803e156509 hooks: improve torrent downloader 2024-07-14 17:57:36 +00:00
ed
c06aa683eb allow audio-DL regardless of current folder 2024-07-13 17:10:24 +02:00
ed
6644ceef49 mention davfs2 workaround, closes #91 2024-07-13 16:55:27 +02:00
ed
bd3b3863ae hooks: add bittorrent downloader 2024-07-13 01:37:17 +02:00
ed
ffd4f9c8b9 hooks: describe examples better 2024-07-13 01:32:26 +02:00
ed
760ff2db72 other linter nitpicks (not actually bugs) 2024-07-13 01:18:14 +02:00
ed
f37187a041 fix bugs detected by pyright but not pylance:
* race-the-beam broke in v1.13.3 (i'm good at this)

* wrong logger type in certgen
2024-07-13 01:09:19 +02:00
ed
1cdb170290 order-significant --th-covers;
the first matching filename as listed in the
`--th-covers` global-option will always be selected
2024-07-13 00:54:38 +02:00
ed
d5de3f2fe0 improve --cgen (configfile generator) 2024-07-12 22:57:57 +02:00
ed
d76673e62d use correct mtime for folder thumbs;
mtime the file that was used to produce the folder thumbnail
(rather than the folder itself) since the folder-thumb is
always resolved to the file's thumb in the on-disk cache
2024-07-11 23:12:51 +02:00
ed
c549f367c1 reduce timeout of unbounded socket reads;
if a request body is expected, but request has no content-length,
set the timeout to 1/20 of `--s-tbody`, so 9 seconds by default,
or 3 seconds if it's 60 as recommended in helptext

this gives less confusing behavior if a client accidentally does
something invalid, replying with an error response before the
previous timeout of 186 seconds

also raise the slowloris flag, in case a client bugs out and
keeps making such requests
2024-07-10 11:14:42 +02:00
ed
927c3bce96 support descript.ion; makes listings 2% slower 2024-07-06 17:02:33 +02:00
ed
d75a2c77da og: fix viewing readmes 2024-07-06 16:55:15 +02:00
ed
e6c55d7ff9 systemd service: fix install notes, closes #88
the linked issue mentions that creating the `th` folder inside `.hist`
failed when RestrictSUIDSGID=true was enabled; this was on raspbian11
inside an ext4 chmod 777 owned by another user, so I have no idea why
that option would make any difference... but might as well mention it
2024-06-27 17:35:23 +02:00
ed
4c2cb26991 readme: add mimetype mapping examples; closes #89 2024-06-27 15:24:15 +02:00
ed
dfe7f1d9af point out that HTTP/2 tends to be slower than HTTP/1.1
re discord, someone with a fairly standard setup (cpp behind nginx)
found that switching from HTTP/2 to HTTP/1.1 made it 5x faster
2024-06-27 15:19:21 +02:00
ed
666297f6fb remove excessive warning on ancient machines;
sqlite<3.9 combined with python<3.6> would always warn
that `-e2t` is not supported, even when not requested
2024-06-27 14:55:12 +02:00
ed
55a011b9c1 fix jank when trying to play a corrupt audio file
if a song fails to play for some reason (network loss,
corrupt file), a timer plays the next track after 5s

the timer was not cancelled if the user
started another track in the meantime
2024-06-23 01:59:02 +02:00
ed
27aff12a1e fix helptext, closes #87 2024-06-19 10:42:41 +02:00
ed
9a87ee2fe4 add gsel option; closes #85
global-option `--gsel`, volflag `gsel` default-enables the
client setting to select files by ctrl-clicking them in the grid
2024-06-18 22:47:17 +02:00
ed
0a9f4c6074 ftpd: allow implicit overwrite if user has delete perms
the spec doesn't say what you're supposed to do if the target filename of an upload is already taken, but this seems to be the most common behavior on other ftp servers, and is required by wondows 2000 (otherwise it'll freak out and issue a delete and then not actually upload it, nice)

new option `--ftp-no-ow` restores old default behavior of rejecting upload if target filename exists
2024-06-18 12:07:45 +02:00
ed
7219331057 bugfixes;
* `--og` went 500 if thumbnails were disabled / not available
* strip_hints wasn't very helpful explaining why it crashed
2024-06-18 12:01:48 +02:00
ed
2fd12a839c more windows2000 support 2024-06-18 12:01:21 +02:00
ed
8c73e0cbc2 support windows 2000 and XP 2024-06-17 00:09:52 +02:00
ed
52e06226a2 make thumbnails compatible with dirkeys/filekeys
was intentionally skipped to avoid complexity but enough people have
asked why it doesn't work that it's time to do something about it

turns out it wasn't that bad
2024-06-16 21:35:43 +02:00
ed
452592519d tftp:
* upgrade to partftpy 0.4.0
  * workarounds for buggy clients/servers
  * improved ipv6 support, especially on macos
  * improved robustness on unreliable networks

* make `--tftp4` separate from `--ftp4`
2024-06-16 21:20:09 +02:00
ed
c9281f8912 option to return media-links for uploads 2024-06-07 12:56:02 +00:00
ed
36d6d29a0c set audio volume by scrollwheel 2024-06-07 12:23:55 +00:00
ed
db6059e100 music preloader fixes:
* stop scanning after 5 folders
* don't walk into errorpages (such as unmapped root)

and improve errortoast in case of network issues
2024-06-07 11:38:40 +00:00
ed
aab57cb24b update pkgs to 1.13.3 2024-06-01 23:51:14 +00:00
ed
f00b939402 v1.13.3 2024-06-01 23:24:35 +00:00
ed
bef9617638 u2c.exe: explain that https is disabled 2024-06-01 22:26:47 +00:00
ed
692175f5b0 md-editor autoindent was duplicating hr markers
only keep characters `>+-*` if there's less than three of them,
and discard entire prefix if there's more

markdown spec only cares about exactly-one or three-or-more, but
let's keep pairs in case anyone use that as unconventional markup
2024-06-01 20:56:15 +00:00
ed
5ad65450c4 more intuitive df option/volflag, closes #83 2024-06-01 01:15:34 +00:00
ed
60c96f990a ux: hide video ui + floor seekbar text
* hide lightbox buttons when a video is playing

* move audio seekbar text to the bottom, so it
   hides less of the waveform and minute-markers
2024-06-01 00:35:44 +00:00
ed
07b2bf1104 better support for 700+ connections
when there was more than ~700 active connections,
* sendfile (non-https downloads) could fail
* mdns and ssdp could fail to reinitialize on network changes

...because `select` can't handle FDs higher than 512 on windows
(1024 on linux/macos), so prefer `poll` where possible (linux/macos)

but apple keeps breaking and unbreaking `poll` in macos,
so use `--no-poll` if necessary to force `select` instead
2024-05-31 23:31:32 +00:00
ed
ac1bc232a9 black 2024-05-31 08:57:33 +00:00
ed
5919607ad0 sanitize fs-paths in archive error summary
also gets rid of a dumb debug print i forgot
2024-05-30 23:55:37 +00:00
ed
07ea629ca5 keep most tags during audio transcode
metadata is no longer discarded when transcoding to opus or mp3;
this was a good idea back when the transcodes were only used by
the webplayer, but now that folders can be batch-downloaded with
on-the-fly transcoding, it makes sense to keep most of the tags

individual tags are discarded if its value exceeds 1023 letters

this should mainly affect the following:
* traktor beatmaps, size usually somewhere around 100 KiB
* non-standard cover-art embeddings, size around 250 KiB
* XMP (project data from adobe premiere), around 48 KiB
2024-05-30 23:46:56 +00:00
ed
b629d18df6 print helpful warning if unix env is inhospitable
thx kipu you're the best
2024-05-11 18:34:41 +00:00
ed
566cbb6507 update pkgs to 1.13.2 2024-05-10 15:04:33 +00:00
ed
400d700845 v1.13.2 2024-05-10 14:31:50 +00:00
ed
82ce6862ee option to use pngquant for smaller waveform PNGs 2024-05-10 13:06:02 +00:00
ed
38e4fdfe03 batch-convert audio waveforms with ?tar&p 2024-05-10 12:55:35 +00:00
ed
c04662798d play compressed s3xmodit chiptunes
adds support for playing gz, xz, and zip-compressed tracker files

using the de-facto naming convention for compressed modules;

* mod: mdz, mdgz, mdxz
* s3m: s3z, s3gz, s3xz
* xm: xmz, xmgz, xmxz
* it: itz, itgz, itxz
2024-05-10 12:45:17 +00:00
ed
19d156ff4e option to add custom UI translations 2024-05-09 23:09:45 +00:00
ed
87c60a1ec9 ensure OS signals hit main-thread as intended;
use sigmasks to block SIGINT, SIGTERM, SIGUSR1 from all other threads

also initiate shutdown by calling sighandler directly,
in case this misses anything and that is still unreliable
(discovered by `--exit=idx` being noop once in a blue moon)
2024-05-09 22:28:16 +00:00
ed
2c92dab165 fix small annoyances,
* mute exception on early shutdown
* sfx: give the utime thread a name
2024-05-09 14:17:53 +00:00
ed
5c1e23907d og: append full original filename as url suffix 2024-05-09 13:18:15 +00:00
ed
925c7f0a57 in gridview, assume .ts files are video, not typescript 2024-05-08 22:20:29 +00:00
ed
feed08deb2 doc: export --help to html and link it 2024-05-08 22:01:58 +00:00
ed
560d7b6672 option to add or change mimetype mappings 2024-05-08 21:12:14 +00:00
ed
565daee98b fix mimetype detection for uppercase file extensions 2024-05-08 20:08:11 +00:00
ed
e396c5c2b5 only drop index caches if necessary;
prevents having to rebuild covers due to unrelated changes
2024-05-08 20:03:51 +00:00
ed
1ee2cdd089 update pkgs to 1.13.1 2024-05-06 01:11:01 +00:00
ed
beacedab50 v1.13.1 2024-05-06 00:29:15 +00:00
ed
25139a4358 qr-code: better fallback ip when no default-route 2024-05-05 23:36:05 +00:00
ed
f8491970fd remember url-hash during login from 403 2024-05-05 22:37:41 +00:00
ed
da091aec85 "volume" is too overloaded, make it --au-vol instead 2024-05-05 21:27:07 +00:00
ed
e9eb5affcd and option to set default audio/video volume 2024-05-05 19:10:29 +00:00
ed
c1918bc36c expand tcolor early to avoid listing in volume props 2024-05-05 18:52:02 +00:00
ed
fdda567f50 ux: add "this folder is empty" banner 2024-05-05 18:44:36 +00:00
ed
603d0ed72b misc: messages, docs, ie4 / win311 support
* docker: improve config-not-found warning message
* readme: mention markdown variable expansion
* basic-browser: use zip=crc to support ie4 / win-3.11
2024-05-05 17:32:50 +00:00
ed
b15a4ef79f failed attempt at making images load on android-discord 2024-05-05 14:16:22 +00:00
ed
48a6789d36 use --og-title as fallback if template gives blank result 2024-05-05 11:25:52 +00:00
ed
36f2c446af opengraph stuff:
* template-based title formatting
* picture embeds are no longer ant-sized
* `--og-color` sets accent color; default #333
* `--og-s-title` forces default title, ignoring e2t
* add a music indicator to song titles because discord doesn't
2024-05-03 00:11:40 +00:00
ed
69517e4624 add general-purpose query-string parcelling;
currently only being used to workaround discord discarding
query strings in opengraph tags, but i'm sure there will be
plenty more wonderful usecases for this atrocity
2024-05-02 22:49:27 +00:00
ed
ea270ab9f2 add og / opengraph / discord embeds 2024-05-01 23:40:56 +00:00
ed
b6cf2d3089 --html-head can take a filepath and/or jinja2 2024-05-01 20:24:18 +00:00
ed
e8db3dd37f fix tests on windows 2024-04-25 22:25:38 +00:00
ed
27485a4cb1 add pyz builder 2024-04-24 23:45:01 +00:00
ed
253a414443 better ctrl-v upload ux 2024-04-24 23:49:34 +02:00
ed
f6e693f0f5 reevaluate support for sparse files periodically
if a given filesystem were to disappear (e.g. removable storage)
followed by another filesystem appearing at the same location,
this would not get noticed by up2k in a timely manner

fix this by discarding the mtab cache after `--mtab-age` seconds and
rebuild it from scratch, unless the previous values are definitely
correct (as indicated by identical output from `/bin/mount`)

probably reduces windows performance by an acceptable amount
2024-04-24 21:18:26 +00:00
ed
c5f7cfc355 upload files/images with CTRL-V (from explorer etc.) 2024-04-23 19:46:54 +00:00
ed
bc2c1e427a config-reset forgot the dots cookie 2024-04-23 19:39:43 +00:00
ed
95d9e693c6 d2d should disable search/unpost even if db exists 2024-04-22 18:55:13 +00:00
ed
70a3cf36d1 pipe: only flush FDs when necessary
should give higher performance on servers with slow storage
2024-04-21 23:53:04 +00:00
ed
aa45fccf11 update pkgs to 1.13.0 2024-04-20 22:48:16 +00:00
ed
42d00050c1 v1.13.0 2024-04-20 22:32:50 +00:00
ed
4bb0e6e75a pipe: windows: make it safe with aggressive flushing 2024-04-20 22:15:08 +00:00
ed
2f7f9de3f5 pipe: optimize (1 GiB/s @ ryzen5-4500U) 2024-04-20 20:13:31 +00:00
ed
f31ac90932 less confusing help-text for --re-dhash 2024-04-20 16:42:56 +00:00
ed
439cb7f85b u2c: add --ow (previously part of --dr) 2024-04-20 16:36:10 +00:00
ed
af193ee834 keep up2k state integrity on abort 2024-04-20 16:13:32 +00:00
ed
c06126cc9d pipe: add volflag to disable 2024-04-19 23:54:23 +00:00
ed
897ffbbbd0 pipe: add to docs 2024-04-19 00:02:28 +00:00
ed
8244d3b4fc pipe: add tapering to keep tcp alive 2024-04-18 23:10:37 +00:00
ed
74266af6d1 pipe: warn when trying to download a .PARTIAL
and fix file sorting indicators on firefox
2024-04-18 23:10:11 +00:00
ed
8c552f1ad1 windows: fix upload-abort 2024-04-18 23:08:05 +00:00
ed
bf5850785f add opt-out from storing uploader IPs 2024-04-18 17:16:00 +00:00
ed
feecb3e0b8 up2k: fix put-hasher dying + a harmless race
* hasher thread could die if a client would rapidly
   upload and delete files (so very unlikely)

* two unprotected calls to register_vpath which was
   almost-definitely safe because the volumes
   already existed in the registry
2024-04-18 16:43:38 +00:00
ed
08d8c82167 PoC: ongoing uploads can be downloaded in lockstep 2024-04-18 00:10:54 +00:00
ed
5239e7ac0c separate registry mutex for faster access
also fix a harmless toctou in handle_json where clients
could get stuck hanging for a bit longer than necessary
2024-04-18 00:07:56 +00:00
ed
9937c2e755 add ArozOS to comparison 2024-04-16 21:00:47 +00:00
ed
f1e947f37d rehost deps from a flaky server 2024-04-12 21:49:01 +00:00
ed
a70a49b9c9 update pkgs to 1.12.2 2024-04-12 21:25:21 +00:00
ed
fe700dcf1a v1.12.2 2024-04-12 21:10:02 +00:00
ed
c8e3ed3aae retry failed renames on windows
theoretical issue which nobody has ran into yet,
probably because nobody uses this on windows
2024-04-12 20:38:30 +00:00
ed
b8733653a3 fix audio transcoding with filekeys 2024-04-11 21:54:15 +00:00
ed
b772a4f8bb fix wordwrap of buttons on ios 2024-04-11 21:31:40 +00:00
ed
9e5253ef87 ie11: restore load-bearing thing 2024-04-11 20:53:15 +00:00
ed
7b94e4edf3 configurable basic-auth preference;
adds options `--bauth-last` to lower the preference for
taking the basic-auth password in case of conflict,
and `--no-bauth` to entirely disable basic-authentication

if a client is providing multiple passwords, for example when
"logged in" with one password (the `cppwd` cookie) and switching
to another account by also sending a PW header/url-param, then
the default evaluation order to determine which password to use is:

url-param `pw`, header `pw`, basic-auth header, cookie (cppwd/cppws)

so if a client supplies a basic-auth header, it will ignore the cookie
and use the basic-auth password instead, which usually makes sense

but this can become a problem if you have other webservers running
on the same domain which also support basic-authentication

--bauth-last is a good choice for cooperating with such services, as
--no-bauth currently breaks support for the android app...
2024-04-11 20:15:49 +00:00
ed
da26ec36ca add password placeholder on login page
was easy to assume you were supposed to put a username there
2024-04-11 19:31:02 +00:00
ed
443acf2f8b update nuitka notes 2024-04-10 22:04:43 +00:00
ed
6c90e3893d update pkgs to 1.12.1 2024-04-09 23:53:43 +00:00
ed
ea002ee71d v1.12.1 2024-04-09 23:34:31 +00:00
ed
ab18893cd2 update deps 2024-04-09 23:25:54 +00:00
ed
844d16b9e5 bbox: scrollwheel for prev/next pic
inspired by d385305f5e
2024-04-09 20:39:07 +00:00
ed
989cc613ef fix tree-rendering when history-popping into bbox
plus misc similar technically-incorrect addq usages;
most of these don't matter in practice since they'll
never get a url with a hash, but makes the intent clear

and make sure hashes never get passed around
like they're part of a dirkey, harmless as it is
2024-04-09 19:54:15 +00:00
ed
4f0cad5468 fix bbox destructor, closes #81 for real 2024-04-09 19:10:55 +00:00
ed
f89de6b35d preloading too aggressive, chill a bit 2024-04-09 18:44:23 +00:00
ed
e0bcb88ee7 update pkgs to 1.12.0 2024-04-06 20:56:52 +00:00
ed
a0022805d1 v1.12.0 (closes #64) 2024-04-06 20:11:49 +00:00
ed
853adb5d04 update deps 2024-04-06 19:51:38 +00:00
ed
7744226b5c apply audio equalizer to videos too 2024-04-06 18:44:08 +00:00
ed
d94b5b3fc9 fau doesn't work on iphones; compensate by preloading much earlier 2024-04-06 18:43:45 +00:00
ed
e6ba065bc2 improve cachebusters 2024-04-06 00:27:06 +00:00
ed
59a53ba9ac on phones, fix playback halting if next song didn't buffer in time 2024-04-06 00:25:28 +00:00
ed
b88cc7b5ce turns out it doesn't need to be audible... 2024-04-05 23:06:26 +00:00
ed
5ab54763c6 remove pyoxidizer (unmaintained)
partially reverts e430b2567a

the remaining stuff might be useful for other cpython alternatives
2024-04-05 17:51:26 +00:00
ed
59f815ff8c deps: add busy.mp3 2024-04-04 09:27:01 +00:00
ed
9c42cbec6f maybe fix #81 2024-04-03 00:28:15 +00:00
ed
f471b05aa4 todo: fix playback stopping on phones if slow preload 2024-04-02 23:20:58 +00:00
ed
34c32e3e89 golf:
util.js ensures `WebAssembly`, `Notification`, and `FormData`
are always declared, setting them false when not available
2024-04-02 20:25:06 +00:00
ed
a080759a03 add transcoding to mp3
because CU's car stereo can't do opus...

incidentally adds support for playing any audio format in ie11
2024-03-29 16:36:56 +00:00
ed
0ae12868e5 dirkeys: add volflag dky (skip keycheck) 2024-03-27 21:03:58 +00:00
ed
ef52e2c06c dirkeys: fix 403 in dks volumes 2024-03-27 20:34:34 +00:00
ed
32c912bb16 fix a bunch of dirkey stuff:
* breadcrumb navigation
* tree generation in `recvls`
* dirkeys in initial tree
2024-03-27 16:05:05 +00:00
ed
20870fda79 Merge branch 'dirkeys' into hovudstraum 2024-03-25 10:34:08 +00:00
ed
bdfe2c1a5f mention unproductive optimizations 2024-03-24 22:07:23 +00:00
ed
10bc2d9205 unsuccessful attempt at dirkeys (#64) 2023-12-17 22:30:22 +00:00
145 changed files with 16472 additions and 3582 deletions

2
.gitignore vendored
View File

@@ -12,6 +12,7 @@ copyparty.egg-info/
/dist/
/py2/
/sfx*
/pyz/
/unt/
/log/
@@ -29,6 +30,7 @@ copyparty/res/COPYING.txt
copyparty/web/deps/
srv/
scripts/docker/i/
scripts/deps-docker/uncomment.py
contrib/package/arch/pkg/
contrib/package/arch/src/

24
.vscode/settings.json vendored
View File

@@ -22,6 +22,9 @@
"terminal.ansiBrightCyan": "#9cf0ed",
"terminal.ansiBrightWhite": "#ffffff",
},
"python.terminal.activateEnvironment": false,
"python.analysis.enablePytestSupport": false,
"python.analysis.typeCheckingMode": "standard",
"python.testing.pytestEnabled": false,
"python.testing.unittestEnabled": true,
"python.testing.unittestArgs": [
@@ -31,23 +34,8 @@
"-p",
"test_*.py"
],
"python.linting.pylintEnabled": true,
"python.linting.flake8Enabled": true,
"python.linting.banditEnabled": true,
"python.linting.mypyEnabled": true,
"python.linting.flake8Args": [
"--max-line-length=120",
"--ignore=E722,F405,E203,W503,W293,E402,E501,E128,E226",
],
"python.linting.banditArgs": [
"--ignore=B104,B110,B112"
],
// python3 -m isort --py=27 --profile=black copyparty/
"python.formatting.provider": "none",
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter"
},
"editor.formatOnSave": true,
// python3 -m isort --py=27 --profile=black ~/dev/copyparty/{copyparty,tests}/*.py && python3 -m black -t py27 ~/dev/copyparty/{copyparty,tests,bin}/*.py $(find ~/dev/copyparty/copyparty/stolen -iname '*.py')
"editor.formatOnSave": false,
"[html]": {
"editor.formatOnSave": false,
"editor.autoIndent": "keep",
@@ -58,6 +46,4 @@
"files.associations": {
"*.makefile": "makefile"
},
"python.linting.enabled": true,
"python.pythonPath": "/usr/bin/python3"
}

494
README.md
View File

@@ -1,4 +1,6 @@
# 💾🎉 copyparty
<img src="https://github.com/9001/copyparty/raw/hovudstraum/docs/logo.svg" width="250" align="right"/>
### 💾🎉 copyparty
turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser
@@ -10,13 +12,16 @@ turn almost any device into a file server with resumable uploads/downloads using
📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer)
🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)
## readme toc
* top
* [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
* [at home](#at-home) - make it accessible over the internet
* [on servers](#on-servers) - you may also want these, especially on servers
* [features](#features)
* [features](#features) - also see [comparison to similar software](./docs/versus.md)
* [testimonials](#testimonials) - small collection of user feedback
* [motivations](#motivations) - project goals / philosophy
* [notes](#notes) - general notes
@@ -37,12 +42,17 @@ turn almost any device into a file server with resumable uploads/downloads using
* [file-search](#file-search) - dropping files into the browser also lets you see if they exist on the server
* [unpost](#unpost) - undo/delete accidental uploads
* [self-destruct](#self-destruct) - uploads can be given a lifetime
* [race the beam](#race-the-beam) - download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm))
* [incoming files](#incoming-files) - the control-panel shows the ETA for all incoming files
* [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
* [shares](#shares) - share a file or folder by creating a temporary link
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [rss feeds](#rss-feeds) - monitor a folder with your RSS reader
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
* [markdown viewer](#markdown-viewer) - and there are *two* editors
* [markdown vars](#markdown-vars) - dynamic docs with serverside variable expansion
* [other tricks](#other-tricks)
* [searching](#searching) - search by size, date, path/name, mp3-tags, ...
* [server config](#server-config) - using arguments or config files, or a mix of both
@@ -56,7 +66,9 @@ turn almost any device into a file server with resumable uploads/downloads using
* [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
* [smb server](#smb-server) - unsafe, slow, not recommended for wan
* [browser ux](#browser-ux) - tweaking the ui
* [file indexing](#file-indexing) - enables dedup and music search ++
* [opengraph](#opengraph) - discord and social-media embeds
* [file deduplication](#file-deduplication) - enable symlink-based upload deduplication
* [file indexing](#file-indexing) - enable music search, upload-undo, and better dedup
* [exclude-patterns](#exclude-patterns) - to save some time
* [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
* [periodic rescan](#periodic-rescan) - filesystem monitoring
@@ -69,14 +81,20 @@ turn almost any device into a file server with resumable uploads/downloads using
* [event hooks](#event-hooks) - trigger a program on uploads, renames etc ([examples](./bin/hooks/))
* [upload events](#upload-events) - the older, more powerful approach ([examples](./bin/mtag/))
* [handlers](#handlers) - redefine behavior with plugins ([examples](./bin/handlers/))
* [ip auth](#ip-auth) - autologin based on IP range (CIDR)
* [identity providers](#identity-providers) - replace copyparty passwords with oauth and such
* [user-changeable passwords](#user-changeable-passwords) - if permitted, users can change their own passwords
* [using the cloud as storage](#using-the-cloud-as-storage) - connecting to an aws s3 bucket and similar
* [hiding from google](#hiding-from-google) - tell search engines you dont wanna be indexed
* [hiding from google](#hiding-from-google) - tell search engines you don't wanna be indexed
* [themes](#themes)
* [complete examples](#complete-examples)
* [listen on port 80 and 443](#listen-on-port-80-and-443) - become a *real* webserver
* [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
* [real-ip](#real-ip) - teaching copyparty how to see client IPs
* [prometheus](#prometheus) - metrics/stats can be enabled
* [other extremely specific features](#other-extremely-specific-features) - you'll never find a use for these
* [custom mimetypes](#custom-mimetypes) - change the association of a file extension
* [feature chickenbits](#feature-chickenbits) - buggy feature? rip it out
* [packages](#packages) - the party might be closer than you think
* [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
* [fedora package](#fedora-package) - does not exist yet
@@ -94,17 +112,20 @@ turn almost any device into a file server with resumable uploads/downloads using
* [gotchas](#gotchas) - behavior that might be unexpected
* [cors](#cors) - cross-site request config
* [filekeys](#filekeys) - prevent filename bruteforcing
* [dirkeys](#dirkeys) - share specific folders in a volume
* [password hashing](#password-hashing) - you can hash passwords
* [https](#https) - both HTTP and HTTPS are accepted
* [recovering from crashes](#recovering-from-crashes)
* [client crashes](#client-crashes)
* [frefox wsod](#frefox-wsod) - firefox 87 can crash during uploads
* [firefox wsod](#firefox-wsod) - firefox 87 can crash during uploads
* [HTTP API](#HTTP-API) - see [devnotes](./docs/devnotes.md#http-api)
* [dependencies](#dependencies) - mandatory deps
* [optional dependencies](#optional-dependencies) - install these to enable bonus features
* [dependency chickenbits](#dependency-chickenbits) - prevent loading an optional dependency
* [optional gpl stuff](#optional-gpl-stuff)
* [sfx](#sfx) - the self-contained "binary"
* [sfx](#sfx) - the self-contained "binary" (recommended!)
* [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
* [zipapp](#zipapp) - another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)
* [install on android](#install-on-android)
* [reporting bugs](#reporting-bugs) - ideas for context to include, and where to submit them
* [devnotes](#devnotes) - for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
@@ -114,10 +135,11 @@ turn almost any device into a file server with resumable uploads/downloads using
just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
* or install through pypi: `python3 -m pip install --user -U copyparty`
* or install through [pypi](https://pypi.org/project/copyparty/): `python3 -m pip install --user -U copyparty`
* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
* or install [on arch](#arch-package) [on NixOS](#nixos-module) [through nix](#nix-package)
* or if you are on android, [install copyparty in termux](#install-on-android)
* or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
* docker has all deps built-in, so skip this step:
@@ -125,7 +147,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
* **Alpine:** `apk add py3-pillow ffmpeg`
* **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
* **MacOS:** `port install py-Pillow ffmpeg`
* **MacOS** (alternative): `brew install pillow ffmpeg`
@@ -146,6 +168,17 @@ some recommended options:
* see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
### at home
make it accessible over the internet by starting a [cloudflare quicktunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/) like so:
first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) and then start the tunnel with `cloudflared tunnel --url http://127.0.0.1:3923`
as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you
since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
### on servers
you may also want these, especially on servers:
@@ -169,8 +202,10 @@ firewall-cmd --reload
## features
also see [comparison to similar software](./docs/versus.md)
* backend stuff
* ☑ IPv6
* ☑ IPv6 + unix-sockets
* ☑ [multiprocessing](#performance) (actual multithreading)
* ☑ volumes (mountpoints)
* ☑ [accounts](#accounts-and-volumes)
@@ -185,12 +220,13 @@ firewall-cmd --reload
* upload
* ☑ basic: plain multipart, ie6 support
* ☑ [up2k](#uploading): js, resumable, multithreaded
* unaffected by cloudflare's max-upload-size (100 MiB)
* **no filesize limit!** even on Cloudflare
* ☑ stash: simple PUT filedropper
* ☑ filename randomizer
* ☑ write-only folders
* ☑ [unpost](#unpost): undo/delete accidental uploads
* ☑ [self-destruct](#self-destruct) (specified server-side or client-side)
* ☑ [race the beam](#race-the-beam) (almost like peer-to-peer)
* ☑ symlink/discard duplicates (content-matching)
* download
* ☑ single files in browser
@@ -199,7 +235,8 @@ firewall-cmd --reload
* browser
* ☑ [navpane](#navpane) (directory tree sidebar)
* ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
* ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus transcoding)
* ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus/mp3 transcoding)
* ☑ play video files as audio (converted on server)
* ☑ image gallery with webm player
* ☑ textfile browser with syntax hilighting
* ☑ [thumbnails](#thumbnails)
@@ -207,6 +244,7 @@ firewall-cmd --reload
* ☑ ...of videos using FFmpeg
* ☑ ...of audio (spectrograms) using FFmpeg
* ☑ cache eviction (max-age; maybe max-size eventually)
* ☑ multilingual UI (english, norwegian, chinese, [add your own](./docs/rice/#translations)))
* ☑ SPA (browse while uploading)
* server indexing
* ☑ [locate files by contents](#file-search)
@@ -215,9 +253,11 @@ firewall-cmd --reload
* client support
* ☑ [folder sync](#folder-sync)
* ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
* ☑ [opengraph](#opengraph) (discord embeds)
* markdown
* ☑ [viewer](#markdown-viewer)
* ☑ editor (sure why not)
* ☑ [variables](#markdown-vars)
PS: something missing? post any crazy ideas you've got as a [feature request](https://github.com/9001/copyparty/issues/new?assignees=9001&labels=enhancement&template=feature_request.md) or [discussion](https://github.com/9001/copyparty/discussions/new?category=ideas) 🤙
@@ -387,8 +427,8 @@ configuring accounts/volumes with arguments:
`-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
permissions:
* `r` (read): browse folder contents, download files, download as zip/tar
* `w` (write): upload files, move files *into* this folder
* `r` (read): browse folder contents, download files, download as zip/tar, see filekeys/dirkeys
* `w` (write): upload files, move/copy files *into* this folder
* `m` (move): move files/folders *from* this folder
* `d` (delete): delete files/folders
* `.` (dots): user can ask to show dotfiles in directory listings
@@ -468,7 +508,8 @@ the browser has the following hotkeys (always qwerty)
* `ESC` close various things
* `ctrl-K` delete selected files/folders
* `ctrl-X` cut selected files/folders
* `ctrl-V` paste
* `ctrl-C` copy selected files/folders to clipboard
* `ctrl-V` paste (move/copy)
* `Y` download selected files
* `F2` [rename](#batch-rename) selected file/folder
* when a file/folder is selected (in not-grid-view):
@@ -537,6 +578,7 @@ click the `🌲` or pressing the `B` hotkey to toggle between breadcrumbs path (
press `g` or `` to toggle grid-view instead of the file listing and `t` toggles icons / thumbnails
* can be made default globally with `--grid` or per-volume with volflag `grid`
* enable by adding `?imgs` to a link, or disable with `?imgs=0`
![copyparty-thumbs-fs8](https://user-images.githubusercontent.com/241032/129636211-abd20fa2-a953-4366-9423-1c88ebb96ba9.png)
@@ -544,16 +586,15 @@ it does static images with Pillow / pyvips / FFmpeg, and uses FFmpeg for video f
* pyvips is 3x faster than Pillow, Pillow is 3x faster than FFmpeg
* disable thumbnails for specific volumes with volflag `dthumb` for all, or `dvthumb` / `dathumb` / `dithumb` for video/audio/images only
audio files are covnerted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
audio files are converted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
* the order is significant, so if both `cover.png` and `folder.jpg` exist in a folder, it will pick the first matching `--th-covers` entry (`folder.jpg`)
* and, if you enable [file indexing](#file-indexing), it will also try those names as dotfiles (`.folder.jpg` and so), and then fallback on the first picture in the folder (if it has any pictures at all)
in the grid/thumbnail view, if the audio player panel is open, songs will start playing when clicked
* indicated by the audio files having the ▶ icon instead of 💾
enabling `multiselect` lets you click files to select them, and then shift-click another file for range-select
* `multiselect` is mostly intended for phones/tablets, but the `sel` option in the `[⚙️] settings` tab is better suited for desktop use, allowing selection by CTRL-clicking and range-selection with SHIFT-click, all without affecting regular clicking
* the `sel` option can be made default globally with `--gsel` or per-volume with volflag `gsel`
## zip downloads
@@ -587,17 +628,23 @@ you can also zip a selection of files or folders by clicking them in the browser
![copyparty-zipsel-fs8](https://user-images.githubusercontent.com/241032/129635374-e5136e01-470a-49b1-a762-848e8a4c9cdc.png)
cool trick: download a folder by appending url-params `?tar&opus` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus before they're added to the archive
cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
* super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images (`&p` for audio waveforms)
* can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
## uploading
drag files/folders into the web-browser to upload (or use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy))
drag files/folders into the web-browser to upload
this initiates an upload using `up2k`; there are two uploaders available:
dragdrop is the recommended way, but you may also:
* select some files (not folders) in your file explorer and press CTRL-V inside the browser window
* use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy)
* upload using [curl or sharex](#client-examples)
when uploading files through dragdrop or CTRL-V, this initiates an upload using `up2k`; there are two browser-based uploaders available:
* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
* `[🚀] up2k`, the good / fancy one
@@ -610,13 +657,14 @@ up2k has several advantages:
* uploads resume if you reboot your browser or pc, just upload the same files again
* server detects any corruption; the client reuploads affected chunks
* the client doesn't upload anything that already exists on the server
* no filesize limit, even when a proxy limits the request size (for example Cloudflare)
* much higher speeds than ftp/scp/tarpipe on some internet connections (mainly american ones) thanks to parallel connections
* the last-modified timestamp of the file is preserved
> it is perfectly safe to restart / upgrade copyparty while someone is uploading to it!
> all known up2k clients will resume just fine 💪
see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
see [up2k](./docs/devnotes.md#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
@@ -624,7 +672,7 @@ see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a
**protip:** if you enable `favicon` in the `[⚙️] settings` tab (by typing something into the textbox), the icon in the browser tab will indicate upload progress -- also, the `[🔔]` and/or `[🔊]` switches enable visible and/or audible notifications on upload completion
the up2k UI is the epitome of polished inutitive experiences:
the up2k UI is the epitome of polished intuitive experiences:
* "parallel uploads" specifies how many chunks to upload at the same time
* `[🏃]` analysis of other files should continue while one is uploading
* `[🥔]` shows a simpler UI for faster uploads from slow devices
@@ -645,6 +693,8 @@ note that since up2k has to read each file twice, `[🎈] bup` can *theoreticall
if you are resuming a massive upload and want to skip hashing the files which already finished, you can enable `turbo` in the `[⚙️] config` tab, but please read the tooltip on that button
if the server is behind a proxy which imposes a request-size limit, you can configure up2k to sneak below the limit with server-option `--u2sz` (the default is 96 MiB to support Cloudflare)
### file-search
@@ -673,20 +723,34 @@ you can unpost even if you don't have regular move/delete access, however only f
### self-destruct
uploads can be given a lifetime, afer which they expire / self-destruct
uploads can be given a lifetime, after which they expire / self-destruct
the feature must be enabled per-volume with the `lifetime` [upload rule](#upload-rules) which sets the upper limit for how long a file gets to stay on the server
clients can specify a shorter expiration time using the [up2k ui](#uploading) -- the relevant options become visible upon navigating into a folder with `lifetimes` enabled -- or by using the `life` [upload modifier](#write)
clients can specify a shorter expiration time using the [up2k ui](#uploading) -- the relevant options become visible upon navigating into a folder with `lifetimes` enabled -- or by using the `life` [upload modifier](./docs/devnotes.md#write)
specifying a custom expiration time client-side will affect the timespan in which unposts are permitted, so keep an eye on the estimates in the up2k ui
### race the beam
download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)) -- it's almost like peer-to-peer
requires the file to be uploaded using up2k (which is the default drag-and-drop uploader), alternatively the command-line program
### incoming files
the control-panel shows the ETA for all incoming files , but only for files being uploaded into volumes where you have read-access
![copyparty-cpanel-upload-eta-or8](https://github.com/user-attachments/assets/fd275ffa-698c-4fca-a307-4d2181269a6a)
## file manager
cut/paste, rename, and delete files/folders (if you have permission)
file selection: click somewhere on the line (not the link itsef), then:
file selection: click somewhere on the line (not the link itself), then:
* `space` to toggle
* `up/down` to move
* `shift-up/down` to move-and-select
@@ -694,10 +758,48 @@ file selection: click somewhere on the line (not the link itsef), then:
* shift-click another line for range-select
* cut: select some files and `ctrl-x`
* copy: select some files and `ctrl-c`
* paste: `ctrl-v` in another folder
* rename: `F2`
you can move files across browser tabs (cut in one tab, paste in another)
you can copy/move files across browser tabs (cut/copy in one tab, paste in another)
## shares
share a file or folder by creating a temporary link
when enabled in the server settings (`--shr`), click the bottom-right `share` button to share the folder you're currently in, or alternatively:
* select a folder first to share that folder instead
* select one or more files to share only those files
this feature was made with [identity providers](#identity-providers) in mind -- configure your reverseproxy to skip the IdP's access-control for a given URL prefix and use that to safely share specific files/folders sans the usual auth checks
when creating a share, the creator can choose any of the following options:
* password-protection
* expire after a certain time; `0` or blank means infinite
* allow visitors to upload (if the user who creates the share has write-access)
semi-intentional limitations:
* cleanup of expired shares only works when global option `e2d` is set, and/or at least one volume on the server has volflag `e2d`
* only folders from the same volume are shared; if you are sharing a folder which contains other volumes, then the contents of those volumes will not be available
* related to [IdP volumes being forgotten on shutdown](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#idp-volumes-are-forgotten-on-shutdown), any shares pointing into a user's IdP volume will be unavailable until that user makes their first request after a restart
* no option to "delete after first access" because tricky
* when linking something to discord (for example) it'll get accessed by their scraper and that would count as a hit
* browsers wouldn't be able to resume a broken download unless the requester's IP gets allowlisted for X minutes (ref. tricky)
specify `--shr /foobar` to enable this feature; a toplevel virtual folder named `foobar` is then created, and that's where all the shares will be served from
* you can name it whatever, `foobar` is just an example
* if you're using config files, put `shr: /foobar` inside the `[global]` section instead
users can delete their own shares in the controlpanel, and a list of privileged users (`--shr-adm`) are allowed to see and/or delet any share on the server
after a share has expired, it remains visible in the controlpanel for `--shr-rt` minutes (default is 1 day), and the owner can revive it by extending the expiration time there
**security note:** using this feature does not mean that you can skip the [accounts and volumes](#accounts-and-volumes) section -- you still need to restrict access to volumes that you do not intend to share with unauthenticated users! it is not sufficient to use rules in the reverseproxy to restrict access to just the `/share` folder.
## batch rename
@@ -747,6 +849,30 @@ or a mix of both:
the metadata keys you can use in the format field are the ones in the file-browser table header (whatever is collected with `-mte` and `-mtp`)
## rss feeds
monitor a folder with your RSS reader , optionally recursive
must be enabled per-volume with volflag `rss` or globally with `--rss`
the feed includes itunes metadata for use with podcast readers such as [AntennaPod](https://antennapod.org/)
a feed example: https://cd.ocv.me/a/d2/d22/?rss&fext=mp3
url parameters:
* `pw=hunter2` for password auth
* `recursive` to also include subfolders
* `title=foo` changes the feed title (default: folder name)
* `fext=mp3,opus` only include mp3 and opus files (default: all)
* `nf=30` only show the first 30 results (default: 250)
* `sort=m` sort by mtime (file last-modified), newest first (default)
* `u` = upload-time; NOTE: non-uploaded files have upload-time `0`
* `n` = filename
* `a` = filesize
* uppercase = reverse-sort; `M` = oldest file first
## media player
plays almost every audio format there is (if the server has FFmpeg installed for on-demand transcoding)
@@ -757,6 +883,7 @@ some hilights:
* OS integration; control playback from your phone's lockscreen ([windows](https://user-images.githubusercontent.com/241032/233213022-298a98ba-721a-4cf1-a3d4-f62634bc53d5.png) // [iOS](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) // [android](https://user-images.githubusercontent.com/241032/233212311-a7368590-08c7-4f9f-a1af-48ccf3f36fad.png))
* shows the audio waveform in the seekbar
* not perfectly gapless but can get really close (see settings + eq below); good enough to enjoy gapless albums as intended
* videos can be played as audio, without wasting bandwidth on the video
click the `play` link next to an audio file, or copy the link target to [share it](https://a.ocv.me/pub/demo/music/Ubiktune%20-%20SOUNDSHOCK%202%20-%20FM%20FUNK%20TERRROR!!/#af-1fbfba61&t=18) (optionally with a timestamp to start playing from, like that example does)
@@ -778,9 +905,9 @@ open the `[🎺]` media-player-settings tab to configure it,
* `[loop]` keeps looping the folder
* `[next]` plays into the next folder
* "transcode":
* `[flac]` converts `flac` and `wav` files into opus
* `[aac]` converts `aac` and `m4a` files into opus
* `[oth]` converts all other known formats into opus
* `[flac]` converts `flac` and `wav` files into opus (if supported by browser) or mp3
* `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
* `[oth]` converts all other known formats into opus (if supported by browser) or mp3
* `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
* "tint" reduces the contrast of the playback bar
@@ -817,6 +944,13 @@ other notes,
* the document preview has a max-width which is the same as an A4 paper when printed
### markdown vars
dynamic docs with serverside variable expansion to replace stuff like `{{self.ip}}` with the client's IP, or `{{srv.htime}}` with the current time on the server
see [./srv/expand/](./srv/expand/) for usage and examples
## other tricks
* you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
@@ -831,8 +965,12 @@ other notes,
* files named `.prologue.html` / `.epilogue.html` will be rendered before/after directory listings unless `--no-logues`
* files named `descript.ion` / `DESCRIPT.ION` are parsed and displayed in the file listing, or as the epilogue if nonstandard
* files named `README.md` / `readme.md` will be rendered after directory listings unless `--no-readme` (but `.epilogue.html` takes precedence)
* and `PREADME.md` / `preadme.md` is shown above directory listings unless `--no-readme` or `.prologue.html`
* `README.md` and `*logue.html` can contain placeholder values which are replaced server-side before embedding into directory listings; see `--help-exp`
@@ -865,6 +1003,8 @@ using arguments or config files, or a mix of both:
**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with `--help` to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in `--help-flags` can be used in volumes as volflags.
* if running in docker/podman, try this: `docker run --rm -it copyparty/ac --help`
* or see this (probably outdated): https://ocv.me/copyparty/helptext.html
* or if you prefer plaintext, https://ocv.me/copyparty/helptext.txt
## zeroconf
@@ -882,7 +1022,11 @@ uses [multicast dns](https://en.wikipedia.org/wiki/Multicast_DNS) to give copypa
all enabled services ([webdav](#webdav-server), [ftp](#ftp-server), [smb](#smb-server)) will appear in mDNS-aware file managers (KDE, gnome, macOS, ...)
the domain will be http://partybox.local if the machine's hostname is `partybox` unless `--name` specifies soemthing else
the domain will be `partybox.local` if the machine's hostname is `partybox` unless `--name` specifies something else
and the web-UI will be available at http://partybox.local:3923/
* if you want to get rid of the `:3923` so you can use http://partybox.local/ instead then see [listen on port 80 and 443](#listen-on-port-80-and-443)
### ssdp
@@ -908,7 +1052,7 @@ print a qr-code [(screenshot)](https://user-images.githubusercontent.com/241032/
* `--qrz 1` forces 1x zoom instead of autoscaling to fit the terminal size
* 1x may render incorrectly on some terminals/fonts, but 2x should always work
it uses the server hostname if [mdns](#mdns) is enbled, otherwise it'll use your external ip (default route) unless `--qri` specifies a specific ip-prefix or domain
it uses the server hostname if [mdns](#mdns) is enabled, otherwise it'll use your external ip (default route) unless `--qri` specifies a specific ip-prefix or domain
## ftp server
@@ -933,7 +1077,7 @@ some recommended FTP / FTPS clients; `wark` = example password:
## webdav server
with read-write support, supports winXP and later, macos, nautilus/gvfs
with read-write support, supports winXP and later, macos, nautilus/gvfs ... a great way to [access copyparty straight from the file explorer in your OS](#mount-as-drive)
click the [connect](http://127.0.0.1:3923/?hc) button in the control-panel to see connection instructions for windows, linux, macos
@@ -1010,12 +1154,12 @@ some **BIG WARNINGS** specific to SMB/CIFS, in decreasing importance:
* [shadowing](#shadowing) probably works as expected but no guarantees
and some minor issues,
* clients only see the first ~400 files in big folders; [impacket#1433](https://github.com/SecureAuthCorp/impacket/issues/1433)
* clients only see the first ~400 files in big folders;
* this was originally due to [impacket#1433](https://github.com/SecureAuthCorp/impacket/issues/1433) which was fixed in impacket-0.12, so you can disable the workaround with `--smb-nwa-1` but then you get unacceptably poor performance instead
* hot-reload of server config (`/?reload=cfg`) does not include the `[global]` section (commandline args)
* listens on the first IPv4 `-i` interface only (default = :: = 0.0.0.0 = all)
* login doesn't work on winxp, but anonymous access is ok -- remove all accounts from copyparty config for that to work
* win10 onwards does not allow connecting anonymously / without accounts
* on windows, creating a new file through rightclick --> new --> textfile throws an error due to impacket limitations -- hit OK and F5 to get your file
* python3 only
* slow (the builtin webdav support in windows is 5x faster, and rclone-webdav is 30x faster)
@@ -1037,14 +1181,65 @@ authenticate with one of the following:
tweaking the ui
* set default sort order globally with `--sort` or per-volume with the `sort` volflag; specify one or more comma-separated columns to sort by, and prefix the column name with `-` for reverse sort
* the column names you can use are visible as tooltips when hovering over the column headers in the directory listing, for example `href ext sz ts tags/.up_at tags/Cirle tags/.tn tags/Artist tags/Title`
* to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Cirle,tags/.tn,tags/Artist,tags/Title,href`
* the column names you can use are visible as tooltips when hovering over the column headers in the directory listing, for example `href ext sz ts tags/.up_at tags/Circle tags/.tn tags/Artist tags/Title`
* to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Circle,tags/.tn,tags/Artist,tags/Title,href`
* to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`
see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag, or to add your own translation
## opengraph
discord and social-media embeds
can be enabled globally with `--og` or per-volume with volflag `og`
note that this disables hotlinking because the opengraph spec demands it; to sneak past this intentional limitation, you can enable opengraph selectively by user-agent, for example `--og-ua '(Discord|Twitter|Slack)bot'` (or volflag `og_ua`)
you can also hotlink files regardless by appending `?raw` to the url
if you want to entirely replace the copyparty response with your own jinja2 template, give the template filepath to `--og-tpl` or volflag `og_tpl` (all members of `HttpCli` are available through the `this` object)
## file deduplication
enable symlink-based upload deduplication globally with `--dedup` or per-volume with volflag `dedup`
by default, when someone tries to upload a file that already exists on the server, the upload will be politely declined, and the server will copy the existing file over to where the upload would have gone
if you enable deduplication with `--dedup` then it'll create a symlink instead of a full copy, thus reducing disk space usage
* on the contrary, if your server is hooked up to s3-glacier or similar storage where reading is expensive, and you cannot use `--safe-dedup=1` because you have other software tampering with your files, so you want to entirely disable detection of duplicate data instead, then you can specify `--no-clone` globally or `noclone` as a volflag
**warning:** when enabling dedup, you should also:
* enable indexing with `-e2dsa` or volflag `e2dsa` (see [file indexing](#file-indexing) section below); strongly recommended
* ...and/or `--hardlink-only` to use hardlink-based deduplication instead of symlinks; see explanation below
it will not be safe to rename/delete files if you only enable dedup and none of the above; if you enable indexing then it is not *necessary* to also do hardlinks (but you may still want to)
by default, deduplication is done based on symlinks (symbolic links); these are tiny files which are pointers to the nearest full copy of the file
you can choose to use hardlinks instead of softlinks, globally with `--hardlink-only` or volflag `hardlinkonly`;
advantages of using hardlinks:
* hardlinks are more compatible with other software; they behave entirely like regular files
* you can safely move and rename files using other file managers
* symlinks need to be managed by copyparty to ensure the destinations remain correct
advantages of using symlinks (default):
* each symlink can have its own last-modified timestamp, but a single timestamp is shared by all hardlinks
* symlinks make it more obvious to other software that the file is not a regular file, so this can be less dangerous
* hardlinks look like regular files, so other software may assume they are safe to edit without affecting the other copies
**warning:** if you edit the contents of a deduplicated file, then you will also edit all other copies of that file! This is especially surprising with hardlinks, because they look like regular files, but that same file exists in multiple locations
global-option `--xlink` / volflag `xlink` additionally enables deduplication across volumes, but this is probably buggy and not recommended
## file indexing
enables dedup and music search ++
enable music search, upload-undo, and better dedup
file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volflags, or a mix of both.
@@ -1055,10 +1250,9 @@ through arguments:
* `-e2t` enables metadata indexing on upload
* `-e2ts` also scans for tags in all files that don't have tags yet
* `-e2tsr` also deletes all existing tags, doing a full reindex
* `-e2v` verfies file integrity at startup, comparing hashes from the db
* `-e2v` verifies file integrity at startup, comparing hashes from the db
* `-e2vu` patches the database with the new hashes from the filesystem
* `-e2vp` panics and kills copyparty instead
* `--xlink` enables deduplication across volumes
the same arguments can be set as volflags, in addition to `d2d`, `d2ds`, `d2t`, `d2ts`, `d2v` for disabling:
* `-v ~/music::r:c,e2ds,e2tsr` does a full reindex of everything on startup
@@ -1071,7 +1265,6 @@ note:
* upload-times can be displayed in the file listing by enabling the `.up_at` metadata key, either globally with `-e2d -mte +.up_at` or per-volume with volflags `e2d,mte=+.up_at` (will have a ~17% performance impact on directory listings)
* `e2tsr` is probably always overkill, since `e2ds`/`e2dsa` would pick up any file modifications and `e2ts` would then reindex those, unless there is a new copyparty version with new parsers and the release note says otherwise
* the rescan button in the admin panel has no effect unless the volume has `-e2ds` or higher
* deduplication is possible on windows if you run copyparty as administrator (not saying you should!)
### exclude-patterns
@@ -1240,6 +1433,8 @@ you can set hooks before and/or after an event happens, and currently you can ho
there's a bunch of flags and stuff, see `--help-hooks`
if you want to write your own hooks, see [devnotes](./docs/devnotes.md#event-hooks)
### upload events
@@ -1268,11 +1463,27 @@ redefine behavior with plugins ([examples](./bin/handlers/))
replace 404 and 403 errors with something completely different (that's it for now)
## ip auth
autologin based on IP range (CIDR) , using the global-option `--ipu`
for example, if everyone with an IP that starts with `192.168.123` should automatically log in as the user `spartacus`, then you can either specify `--ipu=192.168.123.0/24=spartacus` as a commandline option, or put this in a config file:
```yaml
[global]
ipu: 192.168.123.0/24=spartacus
```
repeat the option to map additional subnets
**be careful with this one!** if you have a reverseproxy, then you definitely want to make sure you have [real-ip](#real-ip) configured correctly, and it's probably a good idea to nullmap the reverseproxy's IP just in case; so if your reverseproxy is sending requests from `172.24.27.9` then that would be `--ipu=172.24.27.9/32=`
## identity providers
replace copyparty passwords with oauth and such
you can disable the built-in password-based login sysem, and instead replace it with a separate piece of software (an identity provider) which will then handle authenticating / authorizing of users; this makes it possible to login with passkeys / fido2 / webauthn / yubikey / ldap / active directory / oauth / many other single-sign-on contraptions
you can disable the built-in password-based login system, and instead replace it with a separate piece of software (an identity provider) which will then handle authenticating / authorizing of users; this makes it possible to login with passkeys / fido2 / webauthn / yubikey / ldap / active directory / oauth / many other single-sign-on contraptions
a popular choice is [Authelia](https://www.authelia.com/) (config-file based), another one is [authentik](https://goauthentik.io/) (GUI-based, more complex)
@@ -1280,6 +1491,29 @@ there is a [docker-compose example](./docs/examples/docker/idp-authelia-traefik)
a more complete example of the copyparty configuration options [look like this](./docs/examples/docker/idp/copyparty.conf)
but if you just want to let users change their own passwords, then you probably want [user-changeable passwords](#user-changeable-passwords) instead
## user-changeable passwords
if permitted, users can change their own passwords in the control-panel
* not compatible with [identity providers](#identity-providers)
* must be enabled with `--chpw` because account-sharing is a popular usecase
* if you want to enable the feature but deny password-changing for a specific list of accounts, you can do that with `--chpw-no name1,name2,name3,...`
* to perform a password reset, edit the server config and give the user another password there, then do a [config reload](#server-config) or server restart
* the custom passwords are kept in a textfile at filesystem-path `--chpw-db`, by default `chpw.json` in the copyparty config folder
* if you run multiple copyparty instances with different users you *almost definitely* want to specify separate DBs for each instance
* if [password hashing](#password-hashing) is enabled, the passwords in the db are also hashed
* ...which means that all user-defined passwords will be forgotten if you change password-hashing settings
## using the cloud as storage
@@ -1296,7 +1530,7 @@ you may improve performance by specifying larger values for `--iobuf` / `--s-rd-
## hiding from google
tell search engines you dont wanna be indexed, either using the good old [robots.txt](https://www.robotstxt.org/robotstxt.html) or through copyparty settings:
tell search engines you don't wanna be indexed, either using the good old [robots.txt](https://www.robotstxt.org/robotstxt.html) or through copyparty settings:
* `--no-robots` adds HTTP (`X-Robots-Tag`) and HTML (`<meta>`) headers with `noindex, nofollow` globally
* volflag `[...]:c,norobots` does the same thing for that single volume
@@ -1371,6 +1605,33 @@ if you want to change the fonts, see [./docs/rice/](./docs/rice/)
`-lo log/cpp-%Y-%m%d-%H%M%S.txt.xz`
## listen on port 80 and 443
become a *real* webserver which people can access by just going to your IP or domain without specifying a port
**if you're on windows,** then you just need to add the commandline argument `-p 80,443` and you're done! nice
**if you're on macos,** sorry, I don't know
**if you're on Linux,** you have the following 4 options:
* **option 1:** set up a [reverse-proxy](#reverse-proxy) -- this one makes a lot of sense if you're running on a proper headless server, because that way you get real HTTPS too
* **option 2:** NAT to port 3923 -- this is cumbersome since you'll need to do it every time you reboot, and the exact command may depend on your linux distribution:
```bash
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 3923
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 3923
```
* **option 3:** disable the [security policy](https://www.w3.org/Daemon/User/Installation/PrivilegedPorts.html) which prevents the use of 80 and 443; this is *probably* fine:
```
setcap CAP_NET_BIND_SERVICE=+eip $(realpath $(which python))
python copyparty-sfx.py -p 80,443
```
* **option 4:** run copyparty as root (please don't)
## reverse-proxy
running copyparty next to other websites hosted on an existing webserver such as nginx, caddy, or apache
@@ -1380,8 +1641,13 @@ you can either:
* or do location-based proxying, using `--rp-loc=/stuff` to tell copyparty where it is mounted -- has a slight performance cost and higher chance of bugs
* if copyparty says `incorrect --rp-loc or webserver config; expected vpath starting with [...]` it's likely because the webserver is stripping away the proxy location from the request URLs -- see the `ProxyPass` in the apache example below
some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which could be a nice speed boost
* **warning:** nginx-QUIC is still experimental and can make uploads much slower, so HTTP/2 is recommended for now
when running behind a reverse-proxy (this includes services like cloudflare), it is important to configure real-ip correctly, as many features rely on knowing the client's IP. Look out for red and yellow log messages which explain how to do this. But basically, set `--xff-hdr` to the name of the http header to read the IP from (usually `x-forwarded-for`, but cloudflare uses `cf-connecting-ip`), and then `--xff-src` to the IP of the reverse-proxy so copyparty will trust the xff-hdr. Note that `--rp-loc` in particular will not work at all unless you do this
some reverse proxies (such as [Caddy](https://caddyserver.com/)) can automatically obtain a valid https/tls certificate for you, and some support HTTP/2 and QUIC which *could* be a nice speed boost, depending on a lot of factors
* **warning:** nginx-QUIC (HTTP/3) is still experimental and can make uploads much slower, so HTTP/1.1 is recommended for now
* depending on server/client, HTTP/1.1 can also be 5x faster than HTTP/2
for improved security (and a 10% performance boost) consider listening on a unix-socket with `-i unix:770:www:/tmp/party.sock` (permission `770` means only members of group `www` can access it)
example webserver configs:
@@ -1421,6 +1687,7 @@ scrape_configs:
currently the following metrics are available,
* `cpp_uptime_seconds` time since last copyparty restart
* `cpp_boot_unixtime_seconds` same but as an absolute timestamp
* `cpp_active_dl` number of active downloads
* `cpp_http_conns` number of open http(s) connections
* `cpp_http_reqs` number of http(s) requests handled
* `cpp_sus_reqs` number of 403/422/malicious requests
@@ -1461,6 +1728,45 @@ the following options are available to disable some of the metrics:
note: the following metrics are counted incorrectly if multiprocessing is enabled with `-j`: `cpp_http_conns`, `cpp_http_reqs`, `cpp_sus_reqs`, `cpp_active_bans`, `cpp_total_bans`
## other extremely specific features
you'll never find a use for these:
### custom mimetypes
change the association of a file extension
using commandline args, you can do something like `--mime gif=image/jif` and `--mime ts=text/x.typescript` (can be specified multiple times)
in a config-file, this is the same as:
```yaml
[global]
mime: gif=image/jif
mime: ts=text/x.typescript
```
run copyparty with `--mimes` to list all the default mappings
### feature chickenbits
buggy feature? rip it out by setting any of the following environment variables to disable its associated bell or whistle,
| env-var | what it does |
| -------------------- | ------------ |
| `PRTY_NO_IFADDR` | disable ip/nic discovery by poking into your OS with ctypes |
| `PRTY_NO_IPV6` | disable some ipv6 support (should not be necessary since windows 2000) |
| `PRTY_NO_LZMA` | disable streaming xz compression of incoming uploads |
| `PRTY_NO_MP` | disable all use of the python `multiprocessing` module (actual multithreading, cpu-count for parsers/thumbnailers) |
| `PRTY_NO_SQLITE` | disable all database-related functionality (file indexing, metadata indexing, most file deduplication logic) |
| `PRTY_NO_TLS` | disable native HTTPS support; if you still want to accept HTTPS connections then TLS must now be terminated by a reverse-proxy |
| `PRTY_NO_TPOKE` | disable systemd-tmpfilesd avoider |
example: `PRTY_NO_IFADDR=1 python3 copyparty-sfx.py`
# packages
the party might be closer than you think
@@ -1631,6 +1937,9 @@ quick summary of more eccentric web-browsers trying to view a directory index:
| **ie4** and **netscape** 4.0 | can browse, upload with `?b=u`, auth with `&pw=wark` |
| **ncsa mosaic** 2.7 | does not get a pass, [pic1](https://user-images.githubusercontent.com/241032/174189227-ae816026-cf6f-4be5-a26e-1b3b072c1b2f.png) - [pic2](https://user-images.githubusercontent.com/241032/174189225-5651c059-5152-46e9-ac26-7e98e497901b.png) |
| **SerenityOS** (7e98457) | hits a page fault, works with `?b=u`, file upload not-impl |
| **nintendo 3ds** | can browse, upload, view thumbnails (thx bnjmn) |
<p align="center"><img src="https://github.com/user-attachments/assets/88deab3d-6cad-4017-8841-2f041472b853" /></p>
# client examples
@@ -1664,16 +1973,18 @@ interact with copyparty using non-browser clients
* FUSE: mount a copyparty server as a local filesystem
* cross-platform python client available in [./bin/](bin/)
* able to mount nginx and iis directory listings too, not just copyparty
* can be downloaded from copyparty: controlpanel -> connect -> [partyfuse.py](http://127.0.0.1:3923/.cpr/a/partyfuse.py)
* [rclone](https://rclone.org/) as client can give ~5x performance, see [./docs/rclone.md](docs/rclone.md)
* sharex (screenshot utility): see [./contrib/sharex.sxcu](contrib/#sharexsxcu)
* and for screenshots on linux, see [./contrib/flameshot.sh](./contrib/flameshot.sh)
* contextlet (web browser integration); see [contrib contextlet](contrib/#send-to-cppcontextletjson)
* [igloo irc](https://iglooirc.com/): Method: `post` Host: `https://you.com/up/?want=url&pw=hunter2` Multipart: `yes` File parameter: `f`
copyparty returns a truncated sha512sum of your PUT/POST as base64; you can generate the same checksum locally to verify uplaods:
copyparty returns a truncated sha512sum of your PUT/POST as base64; you can generate the same checksum locally to verify uploads:
b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
b512 <movie.mkv
@@ -1702,14 +2013,16 @@ alternatively, some alternatives roughly sorted by speed (unreproducible benchma
* [rclone-webdav](./docs/rclone.md) (25s), read/WRITE (rclone v1.63 or later)
* [rclone-http](./docs/rclone.md) (26s), read-only
* [partyfuse.py](./bin/#partyfusepy) (35s), read-only
* [partyfuse.py](./bin/#partyfusepy) (26s), read-only
* [rclone-ftp](./docs/rclone.md) (47s), read/WRITE
* davfs2 (103s), read/WRITE, *very fast* on small files
* davfs2 (103s), read/WRITE
* [win10-webdav](#webdav-server) (138s), read/WRITE
* [win10-smb2](#smb-server) (387s), read/WRITE
most clients will fail to mount the root of a copyparty server unless there is a root volume (so you get the admin-panel instead of a browser when accessing it) -- in that case, mount a specific volume instead
if you have volumes that are accessible without a password, then some webdav clients (such as davfs2) require the global-option `--dav-auth` to access any password-protected areas
# android app
@@ -1738,9 +2051,13 @@ defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
below are some tweaks roughly ordered by usefulness:
* disabling HTTP/2 and HTTP/3 can make uploads 5x faster, depending on server/client software
* `-q` disables logging and can help a bunch, even when combined with `-lo` to redirect logs to file
* `--hist` pointing to a fast location (ssd) will make directory listings and searches faster when `-e2d` or `-e2t` is set
* and also makes thumbnails load faster, regardless of e2d/e2t
* `--dedup` enables deduplication and thus avoids writing to the HDD if someone uploads a dupe
* `--safe-dedup 1` makes deduplication much faster during upload by skipping verification of file contents; safe if there is no other software editing/moving the files in the volumes
* `--no-dirsz` shows the size of folder inodes instead of the total size of the contents, giving about 30% faster folder listings
* `--no-hash .` when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable
* if your volumes are on a network-disk such as NFS / SMB / s3, specifying larger values for `--iobuf` and/or `--s-rd-sz` and/or `--s-wr-sz` may help; try setting all of them to `524288` or `1048576` or `4194304`
* `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
@@ -1767,7 +2084,7 @@ when uploading files,
* up to 30% faster uploads if you hide the upload status list by switching away from the `[🚀]` up2k ui-tab (or closing it)
* optionally you can switch to the lightweight potato ui by clicking the `[🥔]`
* switching to another browser-tab also works, the favicon will update every 10 seconds in that case
* unlikely to be a problem, but can happen when uploding many small files, or your internet is too fast, or PC too slow
* unlikely to be a problem, but can happen when uploading many small files, or your internet is too fast, or PC too slow
# security
@@ -1780,6 +2097,7 @@ some notes on hardening
* cors doesn't work right otherwise
* if you allow anonymous uploads or otherwise don't trust the contents of a volume, you can prevent XSS with volflag `nohtml`
* this returns html documents as plaintext, and also disables markdown rendering
* when running behind a reverse-proxy, listen on a unix-socket for tighter access control (and more performance); see [reverse-proxy](#reverse-proxy) or `--help-bind`
safety profiles:
@@ -1794,7 +2112,7 @@ safety profiles:
* `--hardlink` creates hardlinks instead of symlinks when deduplicating uploads, which is less maintenance
* however note if you edit one file it will also affect the other copies
* `--vague-403` returns a "404 not found" instead of "401 unauthorized" which is a common enterprise meme
* `--nih` removes the server hostname from directory listings
* `-nih` removes the server hostname from directory listings
* option `-sss` is a shortcut for the above plus:
* `--no-dav` disables webdav support
@@ -1814,7 +2132,7 @@ other misc notes:
behavior that might be unexpected
* users without read-access to a folder can still see the `.prologue.html` / `.epilogue.html` / `README.md` contents, for the purpose of showing a description on how to use the uploader for example
* users without read-access to a folder can still see the `.prologue.html` / `.epilogue.html` / `PREADME.md` / `README.md` contents, for the purpose of showing a description on how to use the uploader for example
* users can submit `<script>`s which autorun (in a sandbox) for other visitors in a few ways;
* uploading a `README.md` -- avoid with `--no-readme`
* renaming `some.html` to `.epilogue.html` -- avoid with either `--no-logues` or `--no-dot-ren`
@@ -1839,12 +2157,31 @@ cors can be configured with `--acao` and `--acam`, or the protections entirely d
prevent filename bruteforcing
volflag `c,fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
volflag `fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
by default, filekeys are generated based on salt (`--fk-salt`) + filesystem-path + file-size + inode (if not windows); add volflag `fka` to generate slightly weaker filekeys which will not be invalidated if the file is edited (only salt + path)
permissions `wG` (write + upget) lets users upload files and receive their own filekeys, still without being able to see other uploads
### dirkeys
share specific folders in a volume without giving away full read-access to the rest -- the visitor only needs the `g` (get) permission to view the link
volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, granting read-access to that folder; by default only that folder itself, no subfolders
volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
* `dk` + `dky` gives the same behavior as if all users with `g` access have full read-access, but subfolders are hidden files (as if their names start with a dot), so `dky` is an alternative to renaming all the folders for that purpose, maybe just for some users
volflag `dks` lets people enter subfolders as well, and also enables download-as-zip/tar
if you enable dirkeys, it is probably a good idea to enable filekeys too, otherwise it will be impossible to hotlink files from a folder which was accessed using a dirkey
dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and have a few limitations:
* the key does not change if the contents of the folder is modified
* if you need a new dirkey, either change the salt or rename the folder
* linking to a textfile (so it opens in the textfile viewer) is not possible if recipient doesn't have read-access
## password hashing
@@ -1861,7 +2198,7 @@ the default configs take about 0.4 sec and 256 MiB RAM to process a new password
both HTTP and HTTPS are accepted by default, but letting a [reverse proxy](#reverse-proxy) handle the https/tls/ssl would be better (probably more secure by default)
copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well
copyparty doesn't speak HTTP/2 or QUIC, so using a reverse proxy would solve that as well -- but note that HTTP/1 is usually faster than both HTTP/2 and HTTP/3
if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, copyparty will automatically create a CA and server-cert on startup
* the certs are written to `--crt-dir` for distribution, see `--help` for the other `--crt` options
@@ -1873,13 +2210,13 @@ if [cfssl](https://github.com/cloudflare/cfssl/releases/latest) is installed, co
## client crashes
### frefox wsod
### firefox wsod
firefox 87 can crash during uploads -- the entire browser goes, including all other browser tabs, everything turns white
however you can hit `F12` in the up2k tab and use the devtools to see how far you got in the uploads:
* get a complete list of all uploads, organized by statuts (ok / no-good / busy / queued):
* get a complete list of all uploads, organized by status (ok / no-good / busy / queued):
`var tabs = { ok:[], ng:[], bz:[], q:[] }; for (var a of up2k.ui.tab) tabs[a.in].push(a); tabs`
* list of filenames which failed:
@@ -1922,11 +2259,42 @@ enable [thumbnails](#thumbnails) of...
* **JPEG XL pictures:** `pyvips` or `ffmpeg`
enable [smb](#smb-server) support (**not** recommended):
* `impacket==0.11.0`
* `impacket==0.12.0`
`pyvips` gives higher quality thumbnails than `Pillow` and is 320% faster, using 270% more ram: `sudo apt install libvips42 && python3 -m pip install --user -U pyvips`
### dependency chickenbits
prevent loading an optional dependency , for example if:
* you have an incompatible version installed and it causes problems
* you just don't want copyparty to use it, maybe to save ram
set any of the following environment variables to disable its associated optional feature,
| env-var | what it does |
| -------------------- | ------------ |
| `PRTY_NO_ARGON2` | disable argon2-cffi password hashing |
| `PRTY_NO_CFSSL` | never attempt to generate self-signed certificates using [cfssl](https://github.com/cloudflare/cfssl) |
| `PRTY_NO_FFMPEG` | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips |
| `PRTY_NO_FFPROBE` | **audio transcoding** goes byebye, **thumbnailing** must be handled by Pillow/libvips, **metadata-scanning** must be handled by mutagen |
| `PRTY_NO_MUTAGEN` | do not use [mutagen](https://pypi.org/project/mutagen/) for reading metadata from media files; will fallback to ffprobe |
| `PRTY_NO_PIL` | disable all [Pillow](https://pypi.org/project/pillow/)-based thumbnail support; will fallback to libvips or ffmpeg |
| `PRTY_NO_PILF` | disable Pillow `ImageFont` text rendering, used for folder thumbnails |
| `PRTY_NO_PIL_AVIF` | disable 3rd-party Pillow plugin for [AVIF support](https://pypi.org/project/pillow-avif-plugin/) |
| `PRTY_NO_PIL_HEIF` | disable 3rd-party Pillow plugin for [HEIF support](https://pypi.org/project/pyheif-pillow-opener/) |
| `PRTY_NO_PIL_WEBP` | disable use of native webp support in Pillow |
| `PRTY_NO_PSUTIL` | do not use [psutil](https://pypi.org/project/psutil/) for reaping stuck hooks and plugins on Windows |
| `PRTY_NO_VIPS` | disable all [libvips](https://pypi.org/project/pyvips/)-based thumbnail support; will fallback to Pillow or ffmpeg |
example: `PRTY_NO_PIL=1 python3 copyparty-sfx.py`
* `PRTY_NO_PIL` saves ram
* `PRTY_NO_VIPS` saves ram and startup time
* python2.7 on windows: `PRTY_NO_FFMPEG` + `PRTY_NO_FFPROBE` saves startup time
## optional gpl stuff
some bundled tools have copyleft dependencies, see [./bin/#mtag](bin/#mtag)
@@ -1936,7 +2304,7 @@ these are standalone programs and will never be imported / evaluated by copypart
# sfx
the self-contained "binary" [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
the self-contained "binary" (recommended!) [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
@@ -1963,6 +2331,16 @@ meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/d
then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) every once ina while if you can afford the size
## zipapp
another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) has less features, is slow, requires python 3.7 or newer, worse compression, and more importantly is unable to benefit from more recent versions of jinja2 and such (which makes it less secure)... lots of drawbacks with this one really -- but it does not unpack any temporary files to disk, so it *may* just work if the regular sfx fails to start because the computer is messed up in certain funky ways, so it's worth a shot if all else fails
run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails
it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
# install on android
install [Termux](https://termux.com/) + its companion app `Termux:API` (see [ocv.me/termux](https://ocv.me/termux/)) and then copy-paste this into Termux (long-tap) all at once:

View File

@@ -15,22 +15,18 @@ produces a chronological list of all uploads by collecting info from up2k databa
# [`partyfuse.py`](partyfuse.py)
* mount a copyparty server as a local filesystem (read-only)
* **supports Windows!** -- expect `194 MiB/s` sequential read
* **supports Linux** -- expect `117 MiB/s` sequential read
* **supports Linux** -- expect `600 MiB/s` sequential read
* **supports macos** -- expect `85 MiB/s` sequential read
filecache is default-on for windows and macos;
* macos readsize is 64kB, so speed ~32 MiB/s without the cache
* windows readsize varies by software; explorer=1M, pv=32k
note that copyparty should run with `-ed` to enable dotfiles (hidden otherwise)
also consider using [../docs/rclone.md](../docs/rclone.md) instead for 5x performance
and consider using [../docs/rclone.md](../docs/rclone.md) instead; usually a bit faster, especially on windows
## to run this on windows:
* install [winfsp](https://github.com/billziss-gh/winfsp/releases/latest) and [python 3](https://www.python.org/downloads/)
* [x] add python 3.x to PATH (it asks during install)
* `python -m pip install --user fusepy`
* `python -m pip install --user fusepy` (or grab a copy of `fuse.py` from the `connect` page on your copyparty, and keep it in the same folder)
* `python ./partyfuse.py n: http://192.168.1.69:3923/`
10% faster in [msys2](https://www.msys2.org/), 700% faster if debug prints are enabled:

View File

@@ -2,7 +2,7 @@ standalone programs which are executed by copyparty when an event happens (uploa
these programs either take zero arguments, or a filepath (the affected file), or a json message with filepath + additional info
run copyparty with `--help-hooks` for usage details / hook type explanations (xbu/xau/xiu/xbr/xar/xbd/xad)
run copyparty with `--help-hooks` for usage details / hook type explanations (xm/xbu/xau/xiu/xbc/xac/xbr/xar/xbd/xad/xban)
> **note:** in addition to event hooks (the stuff described here), copyparty has another api to run your programs/scripts while providing way more information such as audio tags / video codecs / etc and optionally daisychaining data between scripts in a processing pipeline; if that's what you want then see [mtp plugins](../mtag/) instead
@@ -13,6 +13,7 @@ run copyparty with `--help-hooks` for usage details / hook type explanations (xb
* [image-noexif.py](image-noexif.py) removes image exif by overwriting / directly editing the uploaded file
* [discord-announce.py](discord-announce.py) announces new uploads on discord using webhooks ([example](https://user-images.githubusercontent.com/241032/215304439-1c1cb3c8-ec6f-4c17-9f27-81f969b1811a.png))
* [reject-mimetype.py](reject-mimetype.py) rejects uploads unless the mimetype is acceptable
* [into-the-cache-it-goes.py](into-the-cache-it-goes.py) avoids bugs in caching proxies by immediately downloading each file that is uploaded
# upload batches
@@ -23,7 +24,10 @@ these are `--xiu` hooks; unlike `xbu` and `xau` (which get executed on every sin
# before upload
* [reject-extension.py](reject-extension.py) rejects uploads if they match a list of file extensions
* [reloc-by-ext.py](reloc-by-ext.py) redirects an upload to another destination based on the file extension
# on message
* [wget.py](wget.py) lets you download files by POSTing URLs to copyparty
* [qbittorrent-magnet.py](qbittorrent-magnet.py) starts downloading a torrent if you post a magnet url
* [msg-log.py](msg-log.py) is a guestbook; logs messages to a doc in the same folder

View File

@@ -12,19 +12,28 @@ announces a new upload on discord
example usage as global config:
--xau f,t5,j,bin/hooks/discord-announce.py
parameters explained,
xau = execute after upload
f = fork; don't delay other hooks while this is running
t5 = timeout if it's still running after 5 sec
j = this hook needs upload information as json (not just the filename)
example usage as a volflag (per-volume config):
-v srv/inc:inc:r:rw,ed:c,xau=f,t5,j,bin/hooks/discord-announce.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/inc as volume /inc,
readable by everyone, read-write for user 'ed',
running this plugin on all uploads with the params listed below)
running this plugin on all uploads with the params explained above)
parameters explained,
xbu = execute after upload
f = fork; don't wait for it to finish
t5 = timeout if it's still running after 5 sec
j = provide upload information as json; not just the filename
example usage as a volflag in a copyparty config file:
[/inc]
srv/inc
accs:
r: *
rw: ed
flags:
xau: f,t5,j,bin/hooks/discord-announce.py
replace "xau" with "xbu" to announce Before upload starts instead of After completion

View File

@@ -0,0 +1,140 @@
#!/usr/bin/env python3
import sys
import json
import shutil
import platform
import subprocess as sp
from urllib.parse import quote
_ = r"""
try to avoid race conditions in caching proxies
(primarily cloudflare, but probably others too)
by means of the most obvious solution possible:
just as each file has finished uploading, use
the server's external URL to download the file
so that it ends up in the cache, warm and snug
this intentionally delays the upload response
as it waits for the file to finish downloading
before copyparty is allowed to return the URL
NOTE: you must edit this script before use,
replacing https://example.com with your URL
NOTE: if the files are only accessible with a
password and/or filekey, you must also add
a cromulent password in the PASSWORD field
NOTE: needs either wget, curl, or "requests":
python3 -m pip install --user -U requests
example usage as global config:
--xau j,t10,bin/hooks/into-the-cache-it-goes.py
parameters explained,
xau = execute after upload
j = this hook needs upload information as json (not just the filename)
t10 = abort download and continue if it takes longer than 10sec
example usage as a volflag (per-volume config):
-v srv/inc:inc:r:rw,ed:c,xau=j,t10,bin/hooks/into-the-cache-it-goes.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/inc as volume /inc,
readable by everyone, read-write for user 'ed',
running this plugin on all uploads with params explained above)
example usage as a volflag in a copyparty config file:
[/inc]
srv/inc
accs:
r: *
rw: ed
flags:
xau: j,t10,bin/hooks/into-the-cache-it-goes.py
"""
# replace this with your site's external URL
# (including the :portnumber if necessary)
SITE_URL = "https://example.com"
# if downloading is protected by passwords or filekeys,
# specify a valid password between the quotes below:
PASSWORD = ""
# if file is larger than this, skip download
MAX_MEGABYTES = 8
# =============== END OF CONFIG ===============
WINDOWS = platform.system() == "Windows"
def main():
fun = download_with_python
if shutil.which("curl"):
fun = download_with_curl
elif shutil.which("wget"):
fun = download_with_wget
inf = json.loads(sys.argv[1])
if inf["sz"] > 1024 * 1024 * MAX_MEGABYTES:
print("[into-the-cache] file is too large; will not download")
return
file_url = "/"
if inf["vp"]:
file_url += inf["vp"] + "/"
file_url += inf["ap"].replace("\\", "/").split("/")[-1]
file_url = SITE_URL.rstrip("/") + quote(file_url, safe=b"/")
print("[into-the-cache] %s(%s)" % (fun.__name__, file_url))
fun(file_url, PASSWORD.strip())
print("[into-the-cache] Download OK")
def download_with_curl(url, pw):
cmd = ["curl"]
if pw:
cmd += ["-HPW:%s" % (pw,)]
nah = sp.DEVNULL
sp.check_call(cmd + [url], stdout=nah, stderr=nah)
def download_with_wget(url, pw):
cmd = ["wget", "-O"]
cmd += ["nul" if WINDOWS else "/dev/null"]
if pw:
cmd += ["--header=PW:%s" % (pw,)]
nah = sp.DEVNULL
sp.check_call(cmd + [url], stdout=nah, stderr=nah)
def download_with_python(url, pw):
import requests
headers = {}
if pw:
headers["PW"] = pw
with requests.get(url, headers=headers, stream=True) as r:
r.raise_for_status()
for _ in r.iter_content(chunk_size=1024 * 256):
pass
if __name__ == "__main__":
main()

View File

@@ -14,19 +14,32 @@ except:
from datetime import datetime
"""
_ = r"""
use copyparty as a dumb messaging server / guestbook thing;
accepts guestbook entries from 📟 (message-to-server-log) in the web-ui
initially contributed by @clach04 in https://github.com/9001/copyparty/issues/35 (thanks!)
Sample usage:
example usage as global config:
python copyparty-sfx.py --xm j,bin/hooks/msg-log.py
Where:
parameters explained,
xm = execute on message (📟)
j = this hook needs message information as json (not just the message-text)
xm = execute on message-to-server-log
j = provide message information as json; not just the text - this script REQUIRES json
t10 = timeout and kill download after 10 secs
example usage as a volflag (per-volume config):
python copyparty-sfx.py -v srv/log:log:r:c,xm=j,bin/hooks/msg-log.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/log as volume /log, readable by everyone,
running this plugin on all messages with the params explained above)
example usage as a volflag in a copyparty config file:
[/log]
srv/log
accs:
r: *
flags:
xm: j,bin/hooks/msg-log.py
"""

128
bin/hooks/qbittorrent-magnet.py Executable file
View File

@@ -0,0 +1,128 @@
#!/usr/bin/env python3
# coding: utf-8
import os
import sys
import json
import shutil
import subprocess as sp
_ = r"""
start downloading a torrent by POSTing a magnet URL to copyparty,
for example using 📟 (message-to-server-log) in the web-ui
by default it will download the torrent to the folder you were in
when you pasted the magnet into the message-to-server-log field
you can optionally specify another location by adding a whitespace
after the magnet URL followed by the name of the subfolder to DL into,
or for example "anime/airing" would download to /srv/media/anime/airing
because the keyword "anime" is in the DESTS config below
needs python3
example usage as global config (not a good idea):
python copyparty-sfx.py --xm aw,f,j,t60,bin/hooks/qbittorrent-magnet.py
parameters explained,
xm = execute on message (📟)
aw = only users with write-access can use this
f = fork; don't delay other hooks while this is running
j = provide message information as json (not just the text)
t60 = abort if qbittorrent has to think about it for more than 1 min
example usage as a volflag (per-volume config, much better):
-v srv/qb:qb:A,ed:c,xm=aw,f,j,t60,bin/hooks/qbittorrent-magnet.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/qb as volume /qb with Admin for user 'ed',
running this plugin on all messages with the params explained above)
example usage as a volflag in a copyparty config file:
[/qb]
srv/qb
accs:
A: ed
flags:
xm: aw,f,j,t60,bin/hooks/qbittorrent-magnet.py
the volflag examples only kicks in if you send the torrent magnet
while you're in the /qb folder (or any folder below there)
"""
# list of usernames to allow
ALLOWLIST = [ "ed", "morpheus" ]
# list of destination aliases to translate into full filesystem
# paths; takes effect if the first folder component in the
# custom download location matches anything in this dict
DESTS = {
"iso": "/srv/pub/linux-isos",
"anime": "/srv/media/anime",
}
def main():
inf = json.loads(sys.argv[1])
url = inf["txt"]
if not url.lower().startswith("magnet:?"):
# not a magnet, abort
return
if inf["user"] not in ALLOWLIST:
print("🧲 denied for user", inf["user"])
return
# might as well run the command inside the filesystem folder
# which matches the URL that the magnet message was sent to
os.chdir(inf["ap"])
# is there is a custom download location in the url?
dst = ""
if " " in url:
url, dst = url.split(" ", 1)
# is the location in the predefined list of locations?
parts = dst.replace("\\", "/").split("/")
if parts[0] in DESTS:
dst = os.path.join(DESTS[parts[0]], *(parts[1:]))
else:
# nope, so download to the current folder instead;
# comment the dst line below to instead use the default
# download location from your qbittorrent settings
dst = inf["ap"]
pass
# archlinux has a -nox suffix for qbittorrent if headless
# so check if we should be using that
if shutil.which("qbittorrent-nox"):
torrent_bin = "qbittorrent-nox"
else:
torrent_bin = "qbittorrent"
# the command to add a new torrent, adjust if necessary
cmd = [torrent_bin, url]
if dst:
cmd += ["--save-path=%s" % (dst,)]
# if copyparty and qbittorrent are running as different users
# you may have to do something like the following
# (assuming qbittorrent* is nopasswd-allowed in sudoers):
#
# cmd = ["sudo", "-u", "qbitter"] + cmd
print("🧲", cmd)
try:
sp.check_call(cmd)
except:
print("🧲 FAILED TO ADD", url)
if __name__ == "__main__":
main()

127
bin/hooks/reloc-by-ext.py Normal file
View File

@@ -0,0 +1,127 @@
#!/usr/bin/env python3
import json
import os
import re
import sys
_ = r"""
relocate/redirect incoming uploads according to file extension or name
example usage as global config:
--xbu j,c1,bin/hooks/reloc-by-ext.py
parameters explained,
xbu = execute before upload
j = this hook needs upload information as json (not just the filename)
c1 = this hook returns json on stdout, so tell copyparty to read that
example usage as a volflag (per-volume config):
-v srv/inc:inc:r:rw,ed:c,xbu=j,c1,bin/hooks/reloc-by-ext.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/inc as volume /inc,
readable by everyone, read-write for user 'ed',
running this plugin on all uploads with the params explained above)
example usage as a volflag in a copyparty config file:
[/inc]
srv/inc
accs:
r: *
rw: ed
flags:
xbu: j,c1,bin/hooks/reloc-by-ext.py
note: this could also work as an xau hook (after-upload), but
because it doesn't need to read the file contents its better
as xbu (before-upload) since that's safer / less buggy,
and only xbu works with up2k (dragdrop into browser)
"""
PICS = "avif bmp gif heic heif jpeg jpg jxl png psd qoi tga tif tiff webp"
VIDS = "3gp asf avi flv mkv mov mp4 mpeg mpeg2 mpegts mpg mpg2 nut ogm ogv rm ts vob webm wmv"
MUSIC = "aac aif aiff alac amr ape dfpwm flac m4a mp3 ogg opus ra tak tta wav wma wv"
def main():
inf = json.loads(sys.argv[1])
vdir, fn = os.path.split(inf["vp"])
try:
fn, ext = fn.rsplit(".", 1)
except:
# no file extension; pretend it's "bin"
ext = "bin"
ext = ext.lower()
# this function must end by printing the action to perform;
# that's handled by the print(json.dumps(... at the bottom
#
# the action can contain the following keys:
# "vp" is the folder URL to move the upload to,
# "ap" is the filesystem-path to move it to (but "vp" is safer),
# "fn" overrides the final filename to use
##
## some example actions to take; pick one by
## selecting it inside the print at the end:
##
# create a subfolder named after the filetype and move it into there
into_subfolder = {"vp": ext}
# move it into a toplevel folder named after the filetype
into_toplevel = {"vp": "/" + ext}
# move it into a filetype-named folder next to the target folder
into_sibling = {"vp": "../" + ext}
# move images into "/just/pics", vids into "/just/vids",
# music into "/just/tunes", and anything else as-is
if ext in PICS.split():
by_category = {"vp": "/just/pics"}
elif ext in VIDS.split():
by_category = {"vp": "/just/vids"}
elif ext in MUSIC.split():
by_category = {"vp": "/just/tunes"}
else:
by_category = {} # no action
# now choose the default effect to apply; can be any of these:
# into_subfolder into_toplevel into_sibling by_category
effect = {"vp": "/junk"}
##
## but we can keep going, adding more speicifc rules
## which can take precedence, replacing the fallback
## effect we just specified:
##
fn = fn.lower() # lowercase filename to make this easier
if "screenshot" in fn:
effect = {"vp": "/ss"}
if "mpv_" in fn:
effect = {"vp": "/anishots"}
elif "debian" in fn or "biebian" in fn:
effect = {"vp": "/linux-ISOs"}
elif re.search(r"ep(isode |\.)?[0-9]", fn):
effect = {"vp": "/podcasts"}
# regex lets you grab a part of the matching
# text and use that in the upload path:
m = re.search(r"\b(op|ed)([^a-z]|$)", fn)
if m:
# the regex matched; use "anime-op" or "anime-ed"
effect = {"vp": "/anime-" + m[1]}
# aaand DO IT
print(json.dumps({"reloc": effect}))
if __name__ == "__main__":
main()

View File

@@ -9,25 +9,38 @@ import subprocess as sp
_ = r"""
use copyparty as a file downloader by POSTing URLs as
application/x-www-form-urlencoded (for example using the
message/pager function on the website)
📟 message-to-server-log in the web-ui)
example usage as global config:
--xm f,j,t3600,bin/hooks/wget.py
example usage as a volflag (per-volume config):
-v srv/inc:inc:r:rw,ed:c,xm=f,j,t3600,bin/hooks/wget.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/inc as volume /inc,
readable by everyone, read-write for user 'ed',
running this plugin on all messages with the params listed below)
--xm aw,f,j,t3600,bin/hooks/wget.py
parameters explained,
xm = execute on message-to-server-log
f = fork so it doesn't block uploads
j = provide message information as json; not just the text
aw = only users with write-access can use this
f = fork; don't delay other hooks while this is running
j = provide message information as json (not just the text)
c3 = mute all output
t3600 = timeout and kill download after 1 hour
t3600 = timeout and abort download after 1 hour
example usage as a volflag (per-volume config):
-v srv/inc:inc:r:rw,ed:c,xm=aw,f,j,t3600,bin/hooks/wget.py
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(share filesystem-path srv/inc as volume /inc,
readable by everyone, read-write for user 'ed',
running this plugin on all messages with the params explained above)
example usage as a volflag in a copyparty config file:
[/inc]
srv/inc
accs:
r: *
rw: ed
flags:
xm: aw,f,j,t3600,bin/hooks/wget.py
the volflag examples only kicks in if you send the message
while you're in the /inc folder (or any folder below there)
"""

View File

@@ -231,7 +231,7 @@ install_vamp() {
cd "$td"
echo '#include <vamp-sdk/Plugin.h>' | g++ -x c++ -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
printf '\033[33mcould not find the vamp-sdk, building from source\033[0m\n'
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/2691/vamp-plugin-sdk-2.10.0.tar.gz)
(dl_files yolo https://ocv.me/mirror/vamp-plugin-sdk-2.10.0.tar.gz)
sha512sum -c <(
echo "153b7f2fa01b77c65ad393ca0689742d66421017fd5931d216caa0fcf6909355fff74706fabbc062a3a04588a619c9b515a1dae00f21a57afd97902a355c48ed -"
) <vamp-plugin-sdk-2.10.0.tar.gz
@@ -247,7 +247,7 @@ install_vamp() {
cd "$td"
have_beatroot || {
printf '\033[33mcould not find the vamp beatroot plugin, building from source\033[0m\n'
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/885/beatroot-vamp-v1.0.tar.gz)
(dl_files yolo https://ocv.me/mirror/beatroot-vamp-v1.0.tar.gz)
sha512sum -c <(
echo "1f444d1d58ccf565c0adfe99f1a1aa62789e19f5071e46857e2adfbc9d453037bc1c4dcb039b02c16240e9b97f444aaff3afb625c86aa2470233e711f55b6874 -"
) <beatroot-vamp-v1.0.tar.gz

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -19,6 +19,9 @@
* the `act:bput` thing is optional since copyparty v1.9.29
* using an older sharex version, maybe sharex v12.1.1 for example? dw fam i got your back 👉😎👉 [`sharex12.sxcu`](sharex12.sxcu)
### [`flameshot.sh`](flameshot.sh)
* takes a screenshot with [flameshot](https://flameshot.org/) on Linux, uploads it, and writes the URL to clipboard
### [`send-to-cpp.contextlet.json`](send-to-cpp.contextlet.json)
* browser integration, kind of? custom rightclick actions and stuff
* rightclick a pic and send it to copyparty straight from your browser

14
contrib/flameshot.sh Executable file
View File

@@ -0,0 +1,14 @@
#!/bin/bash
set -e
# take a screenshot with flameshot and send it to copyparty;
# the image url will be placed on your clipboard
password=wark
url=https://a.ocv.me/up/
filename=$(date +%Y-%m%d-%H%M%S).png
flameshot gui -s -r |
curl -T- $url$filename?pw=$password |
tail -n 1 |
xsel -ib

View File

@@ -1,14 +1,10 @@
# when running copyparty behind a reverse proxy,
# the following arguments are recommended:
#
# -i 127.0.0.1 only accept connections from nginx
#
# -nc must match or exceed the webserver's max number of concurrent clients;
# copyparty default is 1024 if OS permits it (see "max clients:" on startup),
# look for "max clients:" when starting copyparty, as nginx should
# not accept more consecutive clients than what copyparty is able to;
# nginx default is 512 (worker_processes 1, worker_connections 512)
#
# you may also consider adding -j0 for CPU-intensive configurations
# (5'000 requests per second, or 20gbps upload/download in parallel)
# rarely, in some extreme usecases, it can be good to add -j0
# (40'000 requests per second, or 20gbps upload/download in parallel)
# but this is usually counterproductive and slightly buggy
#
# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
#
@@ -20,10 +16,33 @@
#
# and then enable it below by uncomenting the cloudflare-only.conf line
upstream cpp {
upstream cpp_tcp {
# alternative 1: connect to copyparty using tcp;
# cpp_uds is slightly faster and more secure, but
# cpp_tcp is easier to setup and "just works"
# ...you should however restrict copyparty to only
# accept connections from nginx by adding these args:
# -i 127.0.0.1
server 127.0.0.1:3923 fail_timeout=1s;
keepalive 1;
}
upstream cpp_uds {
# alternative 2: unix-socket, aka. "unix domain socket";
# 5-10% faster, and better isolation from other software,
# but there must be at least one unix-group which both
# nginx and copyparty is a member of; if that group is
# "www" then run copyparty with the following args:
# -i unix:770:www:/tmp/party.sock
server unix:/tmp/party.sock fail_timeout=1s;
keepalive 1;
}
server {
listen 443 ssl;
listen [::]:443 ssl;
@@ -34,7 +53,8 @@ server {
#include /etc/nginx/cloudflare-only.conf;
location / {
proxy_pass http://cpp;
# recommendation: replace cpp_tcp with cpp_uds below
proxy_pass http://cpp_tcp;
proxy_redirect off;
# disable buffering (next 4 lines)
proxy_http_version 1.1;
@@ -52,6 +72,7 @@ server {
}
}
# default client_max_body_size (1M) blocks uploads larger than 256 MiB
client_max_body_size 1024M;
client_header_timeout 610m;

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.11.2"
pkgver="1.16.1"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" )
sha256sums=("0b37641746d698681691ea9e7070096404afc64a42d3d4e96cc4e036074fded9")
sha256sums=("48506881f7920ad9d528763833a8cc3d1b6df39402bbe1cb90c3ff58c865dfc6")
build() {
cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.11.2/copyparty-sfx.py",
"version": "1.11.2",
"hash": "sha256-3nIHLM4xJ9RQH3ExSGvBckHuS40IdzyREAtMfpJmfug="
"url": "https://github.com/9001/copyparty/releases/download/v1.16.1/copyparty-sfx.py",
"version": "1.16.1",
"hash": "sha256-vlxAuVtd/o11CIC6E6K6UDUdtYDzQ6u7kG3Qc3eqJ+U="
}

View File

@@ -20,6 +20,13 @@ point `--js-browser` to one of these by URL:
## example any-js
point `--js-browser` and/or `--js-other` to one of these by URL:
* [`banner.js`](banner.js) shows a very enterprise [legal-banner](https://github.com/user-attachments/assets/8ae8e087-b209-449c-b08d-74e040f0284b)
## example browser-css
point `--css-browser` to one of these by URL:

93
contrib/plugins/banner.js Normal file
View File

@@ -0,0 +1,93 @@
(function() {
// usage: copy this to '.banner.js' in your webroot,
// and run copyparty with the following arguments:
// --js-browser /.banner.js --js-other /.banner.js
// had to pick the most chuuni one as the default
var bannertext = '' +
'<h3>You are accessing a U.S. Government (USG) Information System (IS) that is provided for USG-authorized use only.</h3>' +
'<p>By using this IS (which includes any device attached to this IS), you consent to the following conditions:</p>' +
'<ul>' +
'<li>The USG routinely intercepts and monitors communications on this IS for purposes including, but not limited to, penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM), law enforcement (LE), and counterintelligence (CI) investigations.</li>' +
'<li>At any time, the USG may inspect and seize data stored on this IS.</li>' +
'<li>Communications using, or data stored on, this IS are not private, are subject to routine monitoring, interception, and search, and may be disclosed or used for any USG-authorized purpose.</li>' +
'<li>This IS includes security measures (e.g., authentication and access controls) to protect USG interests -- not for your personal benefit or privacy.</li>' +
'<li>Notwithstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching or monitoring of the content of privileged communications, or work product, related to personal representation or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work product are private and confidential. See User Agreement for details.</li>' +
'</ul>';
// fancy div to insert into pages
function bannerdiv(border) {
var ret = mknod('div', null, bannertext);
if (border)
ret.setAttribute("style", "border:1em solid var(--fg); border-width:.3em 0; margin:3em 0");
return ret;
}
// keep all of these false and then selectively enable them in the if-blocks below
var show_msgbox = false,
login_top = false,
top = false,
bottom = false,
top_bordered = false,
bottom_bordered = false;
if (QS("h1#cc") && QS("a#k")) {
// this is the controlpanel
// (you probably want to keep just one of these enabled)
show_msgbox = true;
login_top = true;
bottom = true;
}
else if (ebi("swin") && ebi("smac")) {
// this is the connect-page, same deal here
show_msgbox = true;
top_bordered = true;
bottom_bordered = true;
}
else if (ebi("op_cfg") || ebi("div#mw") ) {
// we're running in the main filebrowser (op_cfg) or markdown-viewer/editor (div#mw),
// fragile pages which break if you do something too fancy
show_msgbox = true;
}
// shows a fullscreen messagebox; works on all pages
if (show_msgbox) {
var now = Math.floor(Date.now() / 1000),
last_shown = sread("bannerts") || 0;
// 60 * 60 * 17 = 17 hour cooldown
if (now - last_shown > 60 * 60 * 17) {
swrite("bannerts", now);
modal.confirm(bannertext, null, function () {
location = 'https://this-page-intentionally-left-blank.org/';
});
}
}
// show a message on the page footer; only works on the connect-page
if (top || top_bordered) {
var dst = ebi('wrap');
dst.insertBefore(bannerdiv(top_bordered), dst.firstChild);
}
// show a message on the page footer; only works on the controlpanel and connect-page
if (bottom || bottom_bordered) {
ebi('wrap').appendChild(bannerdiv(bottom_bordered));
}
// show a message on the top of the page; only works on the controlpanel
if (login_top) {
var dst = QS('h1');
dst.parentNode.insertBefore(bannerdiv(false), dst);
}
})();

View File

@@ -4,7 +4,7 @@
#
# installation:
# wget https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py -O /usr/local/bin/copyparty-sfx.py
# useradd -r -s /sbin/nologin -d /var/lib/copyparty copyparty
# useradd -r -s /sbin/nologin -m -d /var/lib/copyparty copyparty
# firewall-cmd --permanent --add-port=3923/tcp # --zone=libvirt
# firewall-cmd --reload
# cp -pv copyparty.service /etc/systemd/system/
@@ -12,11 +12,18 @@
# restorecon -vr /etc/systemd/system/copyparty.service # on fedora/rhel
# systemctl daemon-reload && systemctl enable --now copyparty
#
# every time you edit this file, you must "systemctl daemon-reload"
# for the changes to take effect and then "systemctl restart copyparty"
#
# if it fails to start, first check this: systemctl status copyparty
# then try starting it while viewing logs:
# journalctl -fan 100
# tail -Fn 100 /var/log/copyparty/$(date +%Y-%m%d.log)
#
# if you run into any issues, for example thumbnails not working,
# try removing the "some quick hardening" section and then please
# let me know if that actually helped so we can look into it
#
# you may want to:
# - change "User=copyparty" and "/var/lib/copyparty/" to another user
# - edit /etc/copyparty.conf to configure copyparty

116
contrib/themes/bsod.css Normal file
View File

@@ -0,0 +1,116 @@
/* copy bsod.* into a folder named ".themes" in your webroot and then
--themes=10 --theme=9 --css-browser=/.themes/bsod.css
*/
html.ey {
--w2: #3d7bbc;
--w3: #5fcbec;
--fg: #fff;
--fg-max: #fff;
--fg-weak: var(--w3);
--bg: #2067b2;
--bg-d3: var(--bg);
--bg-d2: var(--w2);
--bg-d1: var(--fg-weak);
--bg-u2: var(--bg);
--bg-u3: var(--bg);
--bg-u5: var(--w2);
--tab-alt: var(--fg-weak);
--row-alt: var(--w2);
--scroll: var(--w3);
--a: #fff;
--a-b: #fff;
--a-hil: #fff;
--a-h-bg: var(--fg-weak);
--a-dark: var(--a);
--a-gray: var(--fg-weak);
--btn-fg: var(--a);
--btn-bg: var(--w2);
--btn-h-fg: var(--w2);
--btn-1-fg: var(--bg);
--btn-1-bg: var(--a);
--txt-sh: a;
--txt-bg: var(--w2);
--u2-b1-bg: var(--w2);
--u2-b2-bg: var(--w2);
--u2-txt-bg: var(--w2);
--u2-tab-bg: a;
--u2-tab-1-bg: var(--w2);
--sort-1: var(--a);
--sort-1: var(--fg-weak);
--tree-bg: var(--bg);
--g-b1: a;
--g-b2: a;
--g-f-bg: var(--w2);
--f-sh1: 0.1;
--f-sh2: 0.02;
--f-sh3: 0.1;
--f-h-b1: a;
--srv-1: var(--a);
--srv-3: var(--a);
--mp-sh: a;
}
html.ey {
background: url('bsod.png') top 5em right 4.5em no-repeat fixed var(--bg);
}
html.ey body#b {
background: var(--bg); /*sandbox*/
}
html.ey #ops {
margin: 1.7em 1.5em 0 1.5em;
border-radius: .3em;
border-width: 1px 0;
}
html.ey #ops a {
text-shadow: 1px 1px 0 rgba(0,0,0,0.5);
}
html.ey .opbox {
margin: 1.5em 0 0 0;
}
html.ey #tree {
box-shadow: none;
}
html.ey #tt {
border-color: var(--w2);
background: var(--w2);
}
html.ey .mdo a {
background: none;
text-decoration: underline;
}
html.ey .mdo pre,
html.ey .mdo code {
color: #fff;
background: var(--w2);
border: none;
}
html.ey .mdo h1,
html.ey .mdo h2 {
background: none;
border-color: var(--w2);
}
html.ey .mdo ul ul,
html.ey .mdo ul ol,
html.ey .mdo ol ul,
html.ey .mdo ol ol {
border-color: var(--w2);
}
html.ey .mdo p>em,
html.ey .mdo li>em,
html.ey .mdo td>em {
color: #fd0;
}

BIN
contrib/themes/bsod.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

View File

@@ -16,9 +16,10 @@ except:
TYPE_CHECKING = False
if True:
from typing import Any, Callable
from typing import Any, Callable, Optional
PY2 = sys.version_info < (3,)
PY36 = sys.version_info > (3, 6)
if not PY2:
unicode: Callable[[Any], str] = str
else:
@@ -50,13 +51,67 @@ try:
except:
CORES = (os.cpu_count() if hasattr(os, "cpu_count") else 0) or 2
# all embedded resources to be retrievable over http
zs = """
web/a/partyfuse.py
web/a/u2c.py
web/a/webdav-cfg.bat
web/baguettebox.js
web/browser.css
web/browser.html
web/browser.js
web/browser2.html
web/cf.html
web/copyparty.gif
web/dd/2.png
web/dd/3.png
web/dd/4.png
web/dd/5.png
web/deps/busy.mp3
web/deps/easymde.css
web/deps/easymde.js
web/deps/marked.js
web/deps/fuse.py
web/deps/mini-fa.css
web/deps/mini-fa.woff
web/deps/prism.css
web/deps/prism.js
web/deps/prismd.css
web/deps/scp.woff2
web/deps/sha512.ac.js
web/deps/sha512.hw.js
web/iiam.gif
web/md.css
web/md.html
web/md.js
web/md2.css
web/md2.js
web/mde.css
web/mde.html
web/mde.js
web/msg.css
web/msg.html
web/shares.css
web/shares.html
web/shares.js
web/splash.css
web/splash.html
web/splash.js
web/svcs.html
web/svcs.js
web/ui.css
web/up2k.js
web/util.js
web/w.hash.js
"""
RES = set(zs.strip().split("\n"))
class EnvParams(object):
def __init__(self) -> None:
self.t0 = time.time()
self.mod = ""
self.cfg = ""
self.ox = getattr(sys, "oxidized", None)
E = EnvParams()

458
copyparty/__main__.py Executable file → Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 11, 2)
CODENAME = "You Can (Not) Proceed"
BUILD_DT = (2024, 3, 23)
VERSION = (1, 16, 2)
CODENAME = "COPYparty"
BUILD_DT = (2024, 11, 23)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

File diff suppressed because it is too large Load Diff

View File

@@ -9,14 +9,14 @@ import queue
from .__init__ import CORES, TYPE_CHECKING
from .broker_mpw import MpWorker
from .broker_util import ExceptionalQueue, try_exec
from .broker_util import ExceptionalQueue, NotExQueue, try_exec
from .util import Daemon, mp
if TYPE_CHECKING:
from .svchub import SvcHub
if True: # pylint: disable=using-constant-test
from typing import Any
from typing import Any, Union
class MProcess(mp.Process):
@@ -43,6 +43,9 @@ class BrokerMp(object):
self.procs = []
self.mutex = threading.Lock()
self.retpend: dict[int, Any] = {}
self.retpend_mutex = threading.Lock()
self.num_workers = self.args.j or CORES
self.log("broker", "booting {} subprocesses".format(self.num_workers))
for n in range(1, self.num_workers + 1):
@@ -54,14 +57,13 @@ class BrokerMp(object):
self.procs.append(proc)
proc.start()
Daemon(self.periodic, "mp-periodic")
def shutdown(self) -> None:
self.log("broker", "shutting down")
for n, proc in enumerate(self.procs):
thr = threading.Thread(
target=proc.q_pend.put((0, "shutdown", [])),
name="mp-shutdown-{}-{}".format(n, len(self.procs)),
)
thr.start()
name = "mp-shut-%d-%d" % (n, len(self.procs))
Daemon(proc.q_pend.put, name, ((0, "shutdown", []),))
with self.mutex:
procs = self.procs
@@ -79,6 +81,10 @@ class BrokerMp(object):
for _, proc in enumerate(self.procs):
proc.q_pend.put((0, "reload", []))
def reload_sessions(self) -> None:
for _, proc in enumerate(self.procs):
proc.q_pend.put((0, "reload_sessions", []))
def collector(self, proc: MProcess) -> None:
"""receive message from hub in other process"""
while True:
@@ -89,8 +95,10 @@ class BrokerMp(object):
self.log(*args)
elif dest == "retq":
# response from previous ipc call
raise Exception("invalid broker_mp usage")
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
retq.put(args[0])
else:
# new ipc invoking managed service in hub
@@ -107,8 +115,7 @@ class BrokerMp(object):
if retq_id:
proc.q_pend.put((retq_id, "retq", rv))
def ask(self, dest: str, *args: Any) -> ExceptionalQueue:
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
# new non-ipc invoking managed service in hub
obj = self.hub
for node in dest.split("."):
@@ -120,17 +127,30 @@ class BrokerMp(object):
retq.put(rv)
return retq
def wask(self, dest: str, *args: Any) -> list[Union[ExceptionalQueue, NotExQueue]]:
# call from hub to workers
ret = []
for p in self.procs:
retq = ExceptionalQueue(1)
retq_id = id(retq)
with self.retpend_mutex:
self.retpend[retq_id] = retq
p.q_pend.put((retq_id, dest, list(args)))
ret.append(retq)
return ret
def say(self, dest: str, *args: Any) -> None:
"""
send message to non-hub component in other process,
returns a Queue object which eventually contains the response if want_retval
(not-impl here since nothing uses it yet)
"""
if dest == "listen":
if dest == "httpsrv.listen":
for p in self.procs:
p.q_pend.put((0, dest, [args[0], len(self.procs)]))
elif dest == "set_netdevs":
elif dest == "httpsrv.set_netdevs":
for p in self.procs:
p.q_pend.put((0, dest, list(args)))
@@ -139,3 +159,19 @@ class BrokerMp(object):
else:
raise Exception("what is " + str(dest))
def periodic(self) -> None:
while True:
time.sleep(1)
tdli = {}
tdls = {}
qs = self.wask("httpsrv.read_dls")
for q in qs:
qr = q.get()
dli, dls = qr
tdli.update(dli)
tdls.update(dls)
tdl = (tdli, tdls)
for p in self.procs:
p.q_pend.put((0, "httpsrv.write_dls", tdl))

View File

@@ -11,7 +11,7 @@ import queue
from .__init__ import ANYWIN
from .authsrv import AuthSrv
from .broker_util import BrokerCli, ExceptionalQueue
from .broker_util import BrokerCli, ExceptionalQueue, NotExQueue
from .httpsrv import HttpSrv
from .util import FAKE_MP, Daemon, HMaccas
@@ -82,35 +82,40 @@ class MpWorker(BrokerCli):
while True:
retq_id, dest, args = self.q_pend.get()
# self.logw("work: [{}]".format(d[0]))
if dest == "retq":
# response from previous ipc call
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
retq.put(args)
continue
if dest == "shutdown":
self.httpsrv.shutdown()
self.logw("ok bye")
sys.exit(0)
return
elif dest == "reload":
if dest == "reload":
self.logw("mpw.asrv reloading")
self.asrv.reload()
self.logw("mpw.asrv reloaded")
continue
elif dest == "listen":
self.httpsrv.listen(args[0], args[1])
if dest == "reload_sessions":
with self.asrv.mutex:
self.asrv.load_sessions()
continue
elif dest == "set_netdevs":
self.httpsrv.set_netdevs(args[0])
obj = self
for node in dest.split("."):
obj = getattr(obj, node)
elif dest == "retq":
# response from previous ipc call
with self.retpend_mutex:
retq = self.retpend.pop(retq_id)
rv = obj(*args) # type: ignore
if retq_id:
self.say("retq", rv, retq_id=retq_id)
retq.put(args)
else:
raise Exception("what is " + str(dest))
def ask(self, dest: str, *args: Any) -> ExceptionalQueue:
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
retq = ExceptionalQueue(1)
retq_id = id(retq)
with self.retpend_mutex:
@@ -119,5 +124,5 @@ class MpWorker(BrokerCli):
self.q_yield.put((retq_id, dest, list(args)))
return retq
def say(self, dest: str, *args: Any) -> None:
self.q_yield.put((0, dest, list(args)))
def say(self, dest: str, *args: Any, retq_id=0) -> None:
self.q_yield.put((retq_id, dest, list(args)))

View File

@@ -5,7 +5,7 @@ import os
import threading
from .__init__ import TYPE_CHECKING
from .broker_util import BrokerCli, ExceptionalQueue, try_exec
from .broker_util import BrokerCli, ExceptionalQueue, NotExQueue
from .httpsrv import HttpSrv
from .util import HMaccas
@@ -13,7 +13,7 @@ if TYPE_CHECKING:
from .svchub import SvcHub
if True: # pylint: disable=using-constant-test
from typing import Any
from typing import Any, Union
class BrokerThr(BrokerCli):
@@ -34,6 +34,7 @@ class BrokerThr(BrokerCli):
self.iphash = HMaccas(os.path.join(self.args.E.cfg, "iphash"), 8)
self.httpsrv = HttpSrv(self, None)
self.reload = self.noop
self.reload_sessions = self.noop
def shutdown(self) -> None:
# self.log("broker", "shutting down")
@@ -42,26 +43,21 @@ class BrokerThr(BrokerCli):
def noop(self) -> None:
pass
def ask(self, dest: str, *args: Any) -> ExceptionalQueue:
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
# new ipc invoking managed service in hub
obj = self.hub
for node in dest.split("."):
obj = getattr(obj, node)
rv = try_exec(True, obj, *args)
# pretend we're broker_mp
retq = ExceptionalQueue(1)
retq.put(rv)
return retq
return NotExQueue(obj(*args)) # type: ignore
def say(self, dest: str, *args: Any) -> None:
if dest == "listen":
if dest == "httpsrv.listen":
self.httpsrv.listen(args[0], 1)
return
if dest == "set_netdevs":
if dest == "httpsrv.set_netdevs":
self.httpsrv.set_netdevs(args[0])
return
@@ -70,4 +66,4 @@ class BrokerThr(BrokerCli):
for node in dest.split("."):
obj = getattr(obj, node)
try_exec(False, obj, *args)
obj(*args) # type: ignore

View File

@@ -28,11 +28,23 @@ class ExceptionalQueue(Queue, object):
if rv[1] == "pebkac":
raise Pebkac(*rv[2:])
else:
raise Exception(rv[2])
raise rv[2]
return rv
class NotExQueue(object):
"""
BrokerThr uses this instead of ExceptionalQueue; 7x faster
"""
def __init__(self, rv: Any) -> None:
self.rv = rv
def get(self) -> Any:
return self.rv
class BrokerCli(object):
"""
helps mypy understand httpsrv.broker but still fails a few levels deeper,
@@ -48,7 +60,7 @@ class BrokerCli(object):
def __init__(self) -> None:
pass
def ask(self, dest: str, *args: Any) -> ExceptionalQueue:
def ask(self, dest: str, *args: Any) -> Union[ExceptionalQueue, NotExQueue]:
return ExceptionalQueue(1)
def say(self, dest: str, *args: Any) -> None:
@@ -65,8 +77,8 @@ def try_exec(want_retval: Union[bool, int], func: Any, *args: list[Any]) -> Any:
return ["exception", "pebkac", ex.code, str(ex)]
except:
except Exception as ex:
if not want_retval:
raise
return ["exception", "stack", traceback.format_exc()]
return ["exception", "stack", ex]

View File

@@ -6,12 +6,19 @@ import os
import shutil
import time
from .util import Netdev, runcmd
from .__init__ import ANYWIN
from .util import Netdev, load_resource, runcmd, wrename, wunlink
HAVE_CFSSL = True
HAVE_CFSSL = not os.environ.get("PRTY_NO_CFSSL")
if True: # pylint: disable=using-constant-test
from .util import RootLogger
from .util import NamedLogger, RootLogger
if ANYWIN:
VF = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
else:
VF = {"mv_re_t": 0, "rm_re_t": 0}
def ensure_cert(log: "RootLogger", args) -> None:
@@ -22,13 +29,15 @@ def ensure_cert(log: "RootLogger", args) -> None:
i feel awful about this and so should they
"""
cert_insec = os.path.join(args.E.mod, "res/insecure.pem")
with load_resource(args.E, "res/insecure.pem") as f:
cert_insec = f.read()
cert_appdata = os.path.join(args.E.cfg, "cert.pem")
if not os.path.isfile(args.cert):
if cert_appdata != args.cert:
raise Exception("certificate file does not exist: " + args.cert)
shutil.copy(cert_insec, args.cert)
with open(args.cert, "wb") as f:
f.write(cert_insec)
with open(args.cert, "rb") as f:
buf = f.read()
@@ -43,7 +52,9 @@ def ensure_cert(log: "RootLogger", args) -> None:
raise Exception(m + "private key must appear before server certificate")
try:
if filecmp.cmp(args.cert, cert_insec):
with open(args.cert, "rb") as f:
active_cert = f.read()
if active_cert == cert_insec:
t = "using default TLS certificate; https will be insecure:\033[36m {}"
log("cert", t.format(args.cert), 3)
except:
@@ -76,6 +87,8 @@ def _read_crt(args, fn):
def _gen_ca(log: "RootLogger", args):
nlog: "NamedLogger" = lambda msg, c=0: log("cert-gen-ca", msg, c)
expiry = _read_crt(args, "ca.pem")[0]
if time.time() + args.crt_cdays * 60 * 60 * 24 * 0.1 < expiry:
return
@@ -105,13 +118,19 @@ def _gen_ca(log: "RootLogger", args):
raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
bname = os.path.join(args.crt_dir, "ca")
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
try:
wunlink(nlog, bname + ".key", VF)
except:
pass
wrename(nlog, bname + "-key.pem", bname + ".key", VF)
wunlink(nlog, bname + ".csr", VF)
log("cert", "new ca OK", 2)
def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
nlog: "NamedLogger" = lambda msg, c=0: log("cert-gen-srv", msg, c)
names = args.crt_ns.split(",") if args.crt_ns else []
if not args.crt_exact:
for n in names[:]:
@@ -136,14 +155,22 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
raise Exception("no useable cert found")
expired = time.time() + args.crt_sdays * 60 * 60 * 24 * 0.5 > expiry
cert_insec = os.path.join(args.E.mod, "res/insecure.pem")
if expired:
raise Exception("old server-cert has expired")
for n in names:
if n not in inf["sans"]:
raise Exception("does not have {}".format(n))
if expired:
raise Exception("old server-cert has expired")
if not filecmp.cmp(args.cert, cert_insec):
with load_resource(args.E, "res/insecure.pem") as f:
cert_insec = f.read()
with open(args.cert, "rb") as f:
active_cert = f.read()
if active_cert and active_cert != cert_insec:
return
except Exception as ex:
log("cert", "will create new server-cert; {}".format(ex))
@@ -185,11 +212,11 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
bname = os.path.join(args.crt_dir, "srv")
try:
os.unlink(bname + ".key")
wunlink(nlog, bname + ".key", VF)
except:
pass
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
wrename(nlog, bname + "-key.pem", bname + ".key", VF)
wunlink(nlog, bname + ".csr", VF)
with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:
ca = f.read()

View File

@@ -2,7 +2,7 @@
from __future__ import print_function, unicode_literals
# awk -F\" '/add_argument\("-[^-]/{print(substr($2,2))}' copyparty/__main__.py | sort | tr '\n' ' '
zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nw p q s ss sss v z zv"
zs = "a c e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vp e2vu ed emp i j lo mcr mte mth mtm mtp nb nc nid nih nth nw p q s ss sss v z zv"
onedash = set(zs.split())
@@ -12,16 +12,19 @@ def vf_bmap() -> dict[str, str]:
"dav_auth": "davauth",
"dav_rt": "davrt",
"ed": "dots",
"never_symlink": "neversymlink",
"no_dedup": "copydupes",
"hardlink_only": "hardlinkonly",
"no_clone": "noclone",
"no_dirsz": "nodirsz",
"no_dupe": "nodupe",
"no_forget": "noforget",
"no_pipe": "nopipe",
"no_robots": "norobots",
"no_thumb": "dthumb",
"no_vthumb": "dvthumb",
"no_athumb": "dathumb",
}
for k in (
"dedup",
"dotsrch",
"e2d",
"e2ds",
@@ -34,11 +37,17 @@ def vf_bmap() -> dict[str, str]:
"e2vp",
"exp",
"grid",
"gsel",
"hardlink",
"magic",
"no_sb_md",
"no_sb_lg",
"nsort",
"og",
"og_no_head",
"og_s_title",
"rand",
"rss",
"xdev",
"xlink",
"xvol",
@@ -53,6 +62,7 @@ def vf_vmap() -> dict[str, str]:
"no_hash": "nohash",
"no_idx": "noidx",
"re_maxage": "scan",
"safe_dedup": "safededup",
"th_convt": "convt",
"th_size": "thsize",
"th_crop": "crop",
@@ -60,11 +70,23 @@ def vf_vmap() -> dict[str, str]:
}
for k in (
"dbd",
"html_head",
"lg_sbf",
"md_sbf",
"nrand",
"og_desc",
"og_site",
"og_th",
"og_title",
"og_title_a",
"og_title_v",
"og_title_i",
"og_tpl",
"og_ua",
"mv_retry",
"rm_retry",
"sort",
"tcolor",
"unlist",
"u2abort",
"u2ts",
@@ -79,14 +101,15 @@ def vf_cmap() -> dict[str, str]:
for k in (
"exp_lg",
"exp_md",
"html_head",
"mte",
"mth",
"mtp",
"xac",
"xad",
"xar",
"xau",
"xban",
"xbc",
"xbd",
"xbr",
"xbu",
@@ -113,10 +136,12 @@ permdescs = {
flagcats = {
"uploads, general": {
"nodupe": "rejects existing files (instead of symlinking them)",
"hardlink": "does dedup with hardlinks instead of symlinks",
"neversymlink": "disables symlink fallback; full copy instead",
"copydupes": "disables dedup, always saves full copies of dupes",
"dedup": "enable symlink-based file deduplication",
"hardlink": "enable hardlink-based file deduplication,\nwith fallback on symlinks when that is impossible",
"hardlinkonly": "dedup with hardlink only, never symlink;\nmake a full copy if hardlink is impossible",
"safededup": "verify on-disk data before using it for dedup",
"noclone": "take dupe data from clients, even if available on HDD",
"nodupe": "rejects existing files (instead of linking/cloning them)",
"sparse": "force use of sparse files, mainly for s3-backed storage",
"daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
"nosub": "forces all uploads into the top folder of the vfs",
@@ -129,6 +154,7 @@ flagcats = {
"maxb=1g,300": "max 1 GiB over 5min (suffixes: b, k, m, g, t)",
"vmaxb=1g": "total volume size max 1 GiB (suffixes: b, k, m, g, t)",
"vmaxn=4k": "max 4096 files in volume (suffixes: b, k, m, g, t)",
"medialinks": "return medialinks for non-up2k uploads (not hotlinks)",
"rand": "force randomized filenames, 9 chars long by default",
"nrand=N": "randomized filenames are N chars long",
"u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
@@ -142,7 +168,7 @@ flagcats = {
"lifetime=3600": "uploads are deleted after 1 hour",
},
"database, general": {
"e2d": "enable database; makes files searchable + enables upload dedup",
"e2d": "enable database; makes files searchable + enables upload-undo",
"e2ds": "scan writable folders for new files on startup; also sets -e2d",
"e2dsa": "scans all folders for new files on startup; also sets -e2d",
"e2t": "enable multimedia indexing; makes it possible to search for tags",
@@ -160,7 +186,7 @@ flagcats = {
"noforget": "don't forget files when deleted from disk",
"fat32": "avoid excessive reindexing on android sdcardfs",
"dbd=[acid|swal|wal|yolo]": "database speed-durability tradeoff",
"xlink": "cross-volume dupe detection / linking",
"xlink": "cross-volume dupe detection / linking (dangerous)",
"xdev": "do not descend into other filesystems",
"xvol": "do not follow symlinks leaving the volume root",
"dotsrch": "show dotfiles in search results",
@@ -175,6 +201,7 @@ flagcats = {
"dvthumb": "disables video thumbnails",
"dathumb": "disables audio thumbnails (spectrograms)",
"dithumb": "disables image thumbnails",
"pngquant": "compress audio waveforms 33% better",
"thsize": "thumbnail res; WxH",
"crop": "center-cropping (y/n/fy/fn)",
"th3x": "3x resolution (y/n/fy/fn)",
@@ -188,6 +215,8 @@ flagcats = {
"xbu=CMD": "execute CMD before a file upload starts",
"xau=CMD": "execute CMD after a file upload finishes",
"xiu=CMD": "execute CMD after all uploads finish and volume is idle",
"xbc=CMD": "execute CMD before a file copy",
"xac=CMD": "execute CMD after a file copy",
"xbr=CMD": "execute CMD before a file rename/move",
"xar=CMD": "execute CMD after a file rename/move",
"xbd=CMD": "execute CMD before a file delete",
@@ -197,9 +226,10 @@ flagcats = {
},
"client and ux": {
"grid": "show grid/thumbnails by default",
"gsel": "select files in grid by ctrl-click",
"sort": "default sort order",
"unlist": "dont list files matching REGEX",
"html_head=TXT": "includes TXT in the <head>",
"html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
"robots": "allows indexing by search engines (default)",
"norobots": "kindly asks search engines to leave",
"no_sb_md": "disable js sandbox for markdown files",
@@ -214,6 +244,7 @@ flagcats = {
"dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
"fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
"fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
"mv_retry": "ms-windows: timeout for renaming busy files",
"rm_retry": "ms-windows: timeout for deleting busy files",
"davauth": "ask webdav clients to login for all folders",
"davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",

View File

@@ -1,6 +1,7 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import re
import time
@@ -8,29 +9,35 @@ import time
from .__init__ import ANYWIN, MACOS
from .authsrv import AXS, VFS
from .bos import bos
from .util import chkcmd, min_ex
from .util import chkcmd, min_ex, undot
if True: # pylint: disable=using-constant-test
from typing import Optional, Union
from .util import RootLogger
from .util import RootLogger, undot
class Fstab(object):
def __init__(self, log: "RootLogger"):
def __init__(self, log: "RootLogger", args: argparse.Namespace):
self.log_func = log
self.warned = False
self.trusted = False
self.tab: Optional[VFS] = None
self.oldtab: Optional[VFS] = None
self.srctab = "a"
self.cache: dict[str, str] = {}
self.age = 0.0
self.maxage = args.mtab_age
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("fstab", msg, c)
def get(self, path: str) -> str:
if len(self.cache) > 9000:
self.age = time.time()
now = time.time()
if now - self.age > self.maxage or len(self.cache) > 9000:
self.age = now
self.oldtab = self.tab or self.oldtab
self.tab = None
self.cache = {}
@@ -45,7 +52,7 @@ class Fstab(object):
self.log(msg.format(path, fs, min_ex()), 3)
return fs
path = path.lstrip("/")
path = undot(path)
try:
return self.cache[path]
except:
@@ -75,7 +82,7 @@ class Fstab(object):
self.trusted = False
def build_tab(self) -> None:
self.log("building tab")
self.log("inspecting mtab for changes")
sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
if MACOS:
@@ -84,6 +91,7 @@ class Fstab(object):
ptn = re.compile(sptn)
so, _ = chkcmd(["mount"])
tab1: list[tuple[str, str]] = []
atab = []
for ln in so.split("\n"):
m = ptn.match(ln)
if not m:
@@ -91,6 +99,15 @@ class Fstab(object):
zs1, zs2 = m.groups()
tab1.append((str(zs1), str(zs2)))
atab.append(ln)
# keep empirically-correct values if mounttab unchanged
srctab = "\n".join(sorted(atab))
if srctab == self.srctab:
self.tab = self.oldtab
return
self.log("mtab has changed; reevaluating support for sparse files")
tab1.sort(key=lambda x: (len(x[0]), x[0]))
path1, fs1 = tab1[0]
@@ -99,14 +116,15 @@ class Fstab(object):
tab.add(fs, path.lstrip("/"))
self.tab = tab
self.srctab = srctab
def relabel(self, path: str, nval: str) -> None:
assert self.tab
assert self.tab # !rm
self.cache = {}
if ANYWIN:
path = self._winpath(path)
path = path.lstrip("/")
path = undot(path)
ptn = re.compile(r"^[^\\/]*")
vn, rem = self.tab._find(path)
if not self.trusted:
@@ -133,10 +151,12 @@ class Fstab(object):
self.trusted = True
except:
# prisonparty or other restrictive environment
self.log("failed to build tab:\n{}".format(min_ex()), 3)
if not self.warned:
self.warned = True
self.log("failed to build tab:\n{}".format(min_ex()), 3)
self.build_fallback()
assert self.tab
assert self.tab # !rm
ret = self.tab._find(path)[0]
if self.trusted or path == ret.vpath:
return ret.realpath.split("/")[0]
@@ -147,6 +167,6 @@ class Fstab(object):
if not self.tab:
self.build_fallback()
assert self.tab
assert self.tab # !rm
ret = self.tab._find(path)[0]
return ret.realpath

View File

@@ -19,6 +19,7 @@ from .__init__ import PY2, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .util import (
VF_CAREFUL,
Daemon,
ODict,
Pebkac,
@@ -30,6 +31,7 @@ from .util import (
runhook,
sanitize_fn,
vjoin,
wunlink,
)
if TYPE_CHECKING:
@@ -37,7 +39,10 @@ if TYPE_CHECKING:
if True: # pylint: disable=using-constant-test
import typing
from typing import Any, Optional
from typing import Any, Optional, Union
if PY2:
range = xrange # type: ignore
class FSE(FilesystemError):
@@ -71,6 +76,7 @@ class FtpAuth(DummyAuthorizer):
else:
raise AuthenticationFailed("banned")
args = self.hub.args
asrv = self.hub.asrv
uname = "*"
if username != "anonymous":
@@ -81,6 +87,9 @@ class FtpAuth(DummyAuthorizer):
uname = zs
break
if args.ipu and uname == "*":
uname = args.ipu_iu[args.ipu_nm.map(ip)]
if not uname or not (asrv.vfs.aread.get(uname) or asrv.vfs.awrite.get(uname)):
g = self.hub.gpwd
if g.lim:
@@ -139,6 +148,9 @@ class FtpFs(AbstractedFS):
self.listdirinfo = self.listdir
self.chdir(".")
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.hub.log("ftpd", msg, c)
def v2a(
self,
vpath: str,
@@ -155,7 +167,7 @@ class FtpFs(AbstractedFS):
t = "Unsupported characters in [{}]"
raise FSE(t.format(vpath), 1)
fn = sanitize_fn(fn or "", "", [".prologue.html", ".epilogue.html"])
fn = sanitize_fn(fn or "", "")
vpath = vjoin(rd, fn)
vfs, rem = self.hub.asrv.vfs.get(vpath, self.uname, r, w, m, d)
if not vfs.realpath:
@@ -207,17 +219,37 @@ class FtpFs(AbstractedFS):
w = "w" in mode or "a" in mode or "+" in mode
ap = self.rv2a(filename, r, w)[0]
self.validpath(ap)
if w:
try:
st = bos.stat(ap)
td = time.time() - st.st_mtime
need_unlink = True
except:
need_unlink = False
td = 0
if td < -1 or td > self.args.ftp_wt:
raise FSE("Cannot open existing file for writing")
if w and need_unlink:
if td >= -1 and td <= self.args.ftp_wt:
# within permitted timeframe; unlink and accept
do_it = True
elif self.args.no_del or self.args.ftp_no_ow:
# file too old, or overwrite not allowed; reject
do_it = False
else:
# allow overwrite if user has delete permission
# (avoids win2000 freaking out and deleting the server copy without uploading its own)
try:
self.rv2a(filename, False, True, False, True)
do_it = True
except:
do_it = False
if not do_it:
raise FSE("File already exists")
wunlink(self.log, ap, VF_CAREFUL)
self.validpath(ap)
return open(fsenc(ap), mode, self.args.iobuf)
def chdir(self, path: str) -> None:
@@ -264,6 +296,7 @@ class FtpFs(AbstractedFS):
self.uname,
not self.args.no_scandir,
[[True, False], [False, True]],
throw=True,
)
vfs_ls = [x[0] for x in vfs_ls1]
vfs_ls.extend(vfs_virt.keys())
@@ -282,9 +315,20 @@ class FtpFs(AbstractedFS):
# display write-only folders as empty
return []
# return list of volumes
r = {x.split("/")[0]: 1 for x in self.hub.asrv.vfs.all_vols.keys()}
return list(sorted(list(r.keys())))
# return list of accessible volumes
ret = []
for vn in self.hub.asrv.vfs.all_vols.values():
if "/" in vn.vpath or not vn.vpath:
continue # only include toplevel-mounted vols
try:
self.hub.asrv.vfs.get(vn.vpath, self.uname, True, False)
ret.append(vn.vpath)
except:
pass
ret.sort()
return ret
def rmdir(self, path: str) -> None:
ap = self.rv2a(path, d=True)[0]
@@ -314,7 +358,7 @@ class FtpFs(AbstractedFS):
svp = join(self.cwd, src).lstrip("/")
dvp = join(self.cwd, dst).lstrip("/")
try:
self.hub.up2k.handle_mv(self.uname, svp, dvp)
self.hub.up2k.handle_mv(self.uname, self.h.cli_ip, svp, dvp)
except Exception as ex:
raise FSE(str(ex))
@@ -432,15 +476,19 @@ class FtpHandler(FTPHandler):
xbu = vfs.flags.get("xbu")
if xbu and not runhook(
None,
None,
self.hub.up2k,
"xbu.ftpd",
xbu,
ap,
vfs.canonical(rem),
vp,
"",
self.uname,
self.hub.asrv.vfs.get_perms(vp, self.uname),
0,
0,
self.cli_ip,
0,
time.time(),
"",
):
raise FSE("Upload blocked by xbu server config")
@@ -543,9 +591,15 @@ class Ftpd(object):
if "::" in ips:
ips.append("0.0.0.0")
ips = [x for x in ips if "unix:" not in x]
if self.args.ftp4:
ips = [x for x in ips if ":" not in x]
if not ips:
lgr.fatal("cannot start ftp-server; no compatible IPs in -i")
return
ips = list(ODict.fromkeys(ips)) # dedup
ioloop = IOLoop()

File diff suppressed because it is too large Load Diff

View File

@@ -9,6 +9,9 @@ import threading # typechk
import time
try:
if os.environ.get("PRTY_NO_TLS"):
raise Exception()
HAVE_SSL = True
import ssl
except:
@@ -55,6 +58,9 @@ class HttpConn(object):
self.E: EnvParams = self.args.E
self.asrv: AuthSrv = hsrv.asrv # mypy404
self.u2fh: Util.FHC = hsrv.u2fh # mypy404
self.pipes: Util.CachedDict = hsrv.pipes # mypy404
self.ipu_iu: Optional[dict[str, str]] = hsrv.ipu_iu
self.ipu_nm: Optional[NetMap] = hsrv.ipu_nm
self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
self.xff_nm: Optional[NetMap] = hsrv.xff_nm
self.xff_lan: NetMap = hsrv.xff_lan # type: ignore
@@ -99,9 +105,6 @@ class HttpConn(object):
self.log_src = ("%s \033[%dm%d" % (ip, color, self.addr[1])).ljust(26)
return self.log_src
def respath(self, res_name: str) -> str:
return os.path.join(self.E.mod, "web", res_name)
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func(self.log_src, msg, c)
@@ -161,6 +164,7 @@ class HttpConn(object):
self.log_src = self.log_src.replace("[36m", "[35m")
try:
assert ssl # type: ignore # !rm
ctx = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
ctx.load_cert_chain(self.args.cert)
if self.args.ssl_ver:
@@ -186,7 +190,7 @@ class HttpConn(object):
if self.args.ssl_dbg and hasattr(self.s, "shared_ciphers"):
ciphers = self.s.shared_ciphers()
assert ciphers
assert ciphers # !rm
overlap = [str(y[::-1]) for y in ciphers]
self.log("TLS cipher overlap:" + "\n".join(overlap))
for k, v in [

View File

@@ -1,7 +1,7 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import base64
import hashlib
import math
import os
import re
@@ -12,7 +12,7 @@ import time
import queue
from .__init__ import ANYWIN, CORES, EXE, MACOS, TYPE_CHECKING, EnvParams
from .__init__ import ANYWIN, CORES, EXE, MACOS, PY2, TYPE_CHECKING, EnvParams, unicode
try:
MNFE = ModuleNotFoundError
@@ -61,28 +61,44 @@ from .u2idx import U2idx
from .util import (
E_SCK,
FHC,
CachedDict,
Daemon,
Garda,
Magician,
Netdev,
NetMap,
absreal,
build_netmap,
has_resource,
ipnorm,
load_ipu,
load_resource,
min_ex,
shut_socket,
spack,
start_log_thrs,
start_stackmon,
ub64enc,
)
if TYPE_CHECKING:
from .authsrv import VFS
from .broker_util import BrokerCli
from .ssdp import SSDPr
if True: # pylint: disable=using-constant-test
from typing import Any, Optional
if PY2:
range = xrange # type: ignore
if not hasattr(socket, "AF_UNIX"):
setattr(socket, "AF_UNIX", -9001)
def load_jinja2_resource(E: EnvParams, name: str):
with load_resource(E, "web/" + name, "r") as f:
return f.read()
class HttpSrv(object):
"""
@@ -115,6 +131,12 @@ class HttpSrv(object):
self.bans: dict[str, int] = {}
self.aclose: dict[str, int] = {}
dli: dict[str, tuple[float, int, "VFS", str, str]] = {} # info
dls: dict[str, tuple[float, int]] = {} # state
self.dli = self.tdli = dli
self.dls = self.tdls = dls
self.iiam = '<img src="%s.cpr/iiam.gif?cache=i" />' % (self.args.SRS,)
self.bound: set[tuple[str, int]] = set()
self.name = "hsrv" + nsuf
self.mutex = threading.Lock()
@@ -130,6 +152,8 @@ class HttpSrv(object):
self.t_periodic: Optional[threading.Thread] = None
self.u2fh = FHC()
self.u2sc: dict[str, tuple[int, "hashlib._Hash"]] = {}
self.pipes = CachedDict(0.2)
self.metrics = Metrics(self)
self.nreq = 0
self.nsus = 0
@@ -144,22 +168,35 @@ class HttpSrv(object):
self.u2idx_free: dict[str, U2idx] = {}
self.u2idx_n = 0
assert jinja2 # type: ignore # !rm
env = jinja2.Environment()
env.loader = jinja2.FileSystemLoader(os.path.join(self.E.mod, "web"))
jn = ["splash", "svcs", "browser", "browser2", "msg", "md", "mde", "cf"]
env.loader = jinja2.FunctionLoader(lambda f: load_jinja2_resource(self.E, f))
jn = [
"splash",
"shares",
"svcs",
"browser",
"browser2",
"msg",
"md",
"mde",
"cf",
]
self.j2 = {x: env.get_template(x + ".html") for x in jn}
zs = os.path.join(self.E.mod, "web", "deps", "prism.js.gz")
self.prism = os.path.exists(zs)
self.prism = has_resource(self.E, "web/deps/prism.js.gz")
if self.args.ipu:
self.ipu_iu, self.ipu_nm = load_ipu(self.log, self.args.ipu)
else:
self.ipu_iu = self.ipu_nm = None
self.ipa_nm = build_netmap(self.args.ipa)
self.xff_nm = build_netmap(self.args.xff_src)
self.xff_lan = build_netmap("lan")
self.statics: set[str] = set()
self._build_statics()
self.ptn_cc = re.compile(r"[\x00-\x1f]")
self.ptn_hsafe = re.compile(r"[\x00-\x1f<>\"'&]")
self.uparam_cc_ok = set("doc move tree".split())
self.mallow = "GET HEAD POST PUT DELETE OPTIONS".split()
if not self.args.no_dav:
@@ -175,6 +212,9 @@ class HttpSrv(object):
self.start_threads(4)
if nid:
self.tdli = {}
self.tdls = {}
if self.args.stackmon:
start_stackmon(self.args.stackmon, nid)
@@ -191,14 +231,6 @@ class HttpSrv(object):
except:
pass
def _build_statics(self) -> None:
for dp, _, df in os.walk(os.path.join(self.E.mod, "web")):
for fn in df:
ap = absreal(os.path.join(dp, fn))
self.statics.add(ap)
if ap.endswith(".gz"):
self.statics.add(ap[:-3])
def set_netdevs(self, netdevs: dict[str, Netdev]) -> None:
ips = set()
for ip, _ in self.bound:
@@ -219,7 +251,7 @@ class HttpSrv(object):
if self.args.log_htp:
self.log(self.name, "workers -= {} = {}".format(n, self.tp_nthr), 6)
assert self.tp_q
assert self.tp_q # !rm
for _ in range(n):
self.tp_q.put(None)
@@ -238,15 +270,24 @@ class HttpSrv(object):
return
def listen(self, sck: socket.socket, nlisteners: int) -> None:
tcp = sck.family != socket.AF_UNIX
if self.args.j != 1:
# lost in the pickle; redefine
if not ANYWIN or self.args.reuseaddr:
sck.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sck.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
if tcp:
sck.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
sck.settimeout(None) # < does not inherit, ^ opts above do
ip, port = sck.getsockname()[:2]
if tcp:
ip, port = sck.getsockname()[:2]
else:
ip = re.sub(r"\.[0-9]+$", "", sck.getsockname().split("/")[-1])
port = 0
self.srvs.append(sck)
self.bound.add((ip, port))
self.nclimax = math.ceil(self.args.nc * 1.0 / nlisteners)
@@ -258,16 +299,22 @@ class HttpSrv(object):
def thr_listen(self, srv_sck: socket.socket) -> None:
"""listens on a shared tcp server"""
ip, port = srv_sck.getsockname()[:2]
fno = srv_sck.fileno()
hip = "[{}]".format(ip) if ":" in ip else ip
msg = "subscribed @ {}:{} f{} p{}".format(hip, port, fno, os.getpid())
if srv_sck.family == socket.AF_UNIX:
ip = re.sub(r"\.[0-9]+$", "", srv_sck.getsockname())
msg = "subscribed @ %s f%d p%d" % (ip, fno, os.getpid())
ip = ip.split("/")[-1]
port = 0
tcp = False
else:
tcp = True
ip, port = srv_sck.getsockname()[:2]
hip = "[%s]" % (ip,) if ":" in ip else ip
msg = "subscribed @ %s:%d f%d p%d" % (hip, port, fno, os.getpid())
self.log(self.name, msg)
def fun() -> None:
self.broker.say("cb_httpsrv_up")
threading.Thread(target=fun, name="sig-hsrv-up1").start()
Daemon(self.broker.say, "sig-hsrv-up1", ("cb_httpsrv_up",))
while not self.stopping:
if self.args.log_conn:
@@ -336,11 +383,13 @@ class HttpSrv(object):
try:
sck, saddr = srv_sck.accept()
cip, cport = saddr[:2]
if cip.startswith("::ffff:"):
cip = cip[7:]
addr = (cip, cport)
if tcp:
cip = unicode(saddr[0])
if cip.startswith("::ffff:"):
cip = cip[7:]
addr = (cip, saddr[1])
else:
addr = ("127.8.3.7", sck.fileno())
except (OSError, socket.error) as ex:
if self.stopping:
break
@@ -396,7 +445,7 @@ class HttpSrv(object):
)
def thr_poolw(self) -> None:
assert self.tp_q
assert self.tp_q # !rm
while True:
task = self.tp_q.get()
if not task:
@@ -508,8 +557,8 @@ class HttpSrv(object):
except:
pass
v = base64.urlsafe_b64encode(spack(b">xxL", int(v)))
self.cb_v = v.decode("ascii")[-4:]
# spack gives 4 lsb, take 3 lsb, get 4 ch
self.cb_v = ub64enc(spack(b">L", int(v))[1:]).decode("ascii")
self.cb_ts = time.time()
return self.cb_v
@@ -540,3 +589,32 @@ class HttpSrv(object):
ident += "a"
self.u2idx_free[ident] = u2idx
def read_dls(
self,
) -> tuple[
dict[str, tuple[float, int, str, str, str]], dict[str, tuple[float, int]]
]:
"""
mp-broker asking for local dl-info + dl-state;
reduce overhead by sending just the vfs vpath
"""
dli = {k: (a, b, c.vpath, d, e) for k, (a, b, c, d, e) in self.dli.items()}
return (dli, self.dls)
def write_dls(
self,
sdli: dict[str, tuple[float, int, str, str, str]],
dls: dict[str, tuple[float, int]],
) -> None:
"""
mp-broker pushing total dl-info + dl-state;
swap out the vfs vpath with the vfs node
"""
dli: dict[str, tuple[float, int, "VFS", str, str]] = {}
for k, (a, b, c, d, e) in sdli.items():
vn = self.asrv.vfs.all_nodes[c]
dli[k] = (a, b, vn, d, e)
self.tdli = dli
self.tdls = dls

View File

@@ -74,7 +74,7 @@ class Ico(object):
try:
_, _, tw, th = pb.textbbox((0, 0), ext)
except:
tw, th = pb.textsize(ext)
tw, th = pb.textsize(ext) # type: ignore
tw += len(ext)
cw = tw // len(ext)

View File

@@ -292,6 +292,22 @@ class MDNS(MCast):
def run2(self) -> None:
last_hop = time.time()
ihop = self.args.mc_hop
try:
if self.args.no_poll:
raise Exception()
fd2sck = {}
srvpoll = select.poll()
for sck in self.srv:
fd = sck.fileno()
fd2sck[fd] = sck
srvpoll.register(fd, select.POLLIN)
except Exception as ex:
srvpoll = None
if not self.args.no_poll:
t = "WARNING: failed to poll(), will use select() instead: %r"
self.log(t % (ex,), 3)
while self.running:
timeout = (
0.02 + random.random() * 0.07
@@ -300,8 +316,13 @@ class MDNS(MCast):
if self.unsolicited
else (last_hop + ihop if ihop else 180)
)
rdy = select.select(self.srv, [], [], timeout)
rx: list[socket.socket] = rdy[0] # type: ignore
if srvpoll:
pr = srvpoll.poll(timeout * 1000)
rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
else:
rdy = select.select(self.srv, [], [], timeout)
rx: list[socket.socket] = rdy[0] # type: ignore
self.rx4.cln()
self.rx6.cln()
buf = b""
@@ -340,7 +361,7 @@ class MDNS(MCast):
except:
pass
self.srv = {}
self.srv.clear()
def eat(self, buf: bytes, addr: tuple[str, int], sck: socket.socket) -> None:
cip = addr[0]

View File

@@ -72,6 +72,9 @@ class Metrics(object):
v = "{:.3f}".format(self.hsrv.t0)
addug("cpp_boot_unixtime", "seconds", v, t)
t = "number of active downloads"
addg("cpp_active_dl", str(len(self.hsrv.tdls)), t)
t = "number of open http(s) client connections"
addg("cpp_http_conns", str(self.hsrv.ncli), t)
@@ -88,7 +91,7 @@ class Metrics(object):
addg("cpp_total_bans", str(self.hsrv.nban), t)
if not args.nos_vst:
x = self.hsrv.broker.ask("up2k.get_state")
x = self.hsrv.broker.ask("up2k.get_state", True, "")
vs = json.loads(x.get())
nvidle = 0
@@ -128,7 +131,7 @@ class Metrics(object):
addbh("cpp_disk_size_bytes", "total HDD size of volume")
addbh("cpp_disk_free_bytes", "free HDD space in volume")
for vpath, vol in allvols:
free, total = get_df(vol.realpath)
free, total, _ = get_df(vol.realpath, False)
if free is None or total is None:
continue
@@ -179,7 +182,7 @@ class Metrics(object):
tnbytes = 0
tnfiles = 0
for vpath, vol in allvols:
cur = idx.get_cur(vol.realpath)
cur = idx.get_cur(vol)
if not cur:
continue

View File

@@ -4,15 +4,19 @@ from __future__ import print_function, unicode_literals
import argparse
import json
import os
import re
import shutil
import subprocess as sp
import sys
import tempfile
from .__init__ import ANYWIN, EXE, PY2, WINDOWS, E, unicode
from .authsrv import VFS
from .bos import bos
from .util import (
FFMPEG_URL,
REKOBO_LKEY,
VF_CAREFUL,
fsenc,
min_ex,
pybin,
@@ -20,12 +24,24 @@ from .util import (
runcmd,
sfsenc,
uncyg,
wunlink,
)
if True: # pylint: disable=using-constant-test
from typing import Any, Union
from typing import Any, Optional, Union
from .util import RootLogger
from .util import NamedLogger, RootLogger
try:
if os.environ.get("PRTY_NO_MUTAGEN"):
raise Exception()
from mutagen import version # noqa: F401
HAVE_MUTAGEN = True
except:
HAVE_MUTAGEN = False
def have_ff(scmd: str) -> bool:
@@ -44,8 +60,11 @@ def have_ff(scmd: str) -> bool:
return bool(shutil.which(scmd))
HAVE_FFMPEG = have_ff("ffmpeg")
HAVE_FFPROBE = have_ff("ffprobe")
HAVE_FFMPEG = not os.environ.get("PRTY_NO_FFMPEG") and have_ff("ffmpeg")
HAVE_FFPROBE = not os.environ.get("PRTY_NO_FFPROBE") and have_ff("ffprobe")
CBZ_PICS = set("png jpg jpeg gif bmp tga tif tiff webp avif".split())
CBZ_01 = re.compile(r"(^|[^0-9v])0+[01]\b")
class MParser(object):
@@ -107,6 +126,81 @@ class MParser(object):
raise Exception()
def au_unpk(
log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
) -> str:
ret = ""
maxsz = 1024 * 1024 * 64
try:
ext = abspath.split(".")[-1].lower()
au, pk = fmt_map[ext].split(".")
fd, ret = tempfile.mkstemp("." + au)
if pk == "gz":
import gzip
fi = gzip.GzipFile(abspath, mode="rb")
elif pk == "xz":
import lzma
fi = lzma.open(abspath, "rb")
elif pk == "zip":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
zil = zf.infolist()
zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
if not zil:
raise Exception("no audio inside zip")
fi = zf.open(zil[0])
elif pk == "cbz":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
znil = [(x.filename.lower(), x) for x in zf.infolist()]
nf = len(znil)
znil = [x for x in znil if x[0].split(".")[-1] in CBZ_PICS]
znil = [x for x in znil if "cover" in x[0]] or znil
znil = [x for x in znil if CBZ_01.search(x[0])] or znil
t = "cbz: %d files, %d hits" % (nf, len(znil))
if znil:
t += ", using " + znil[0][1].filename
log(t)
if not znil:
raise Exception("no images inside cbz")
fi = zf.open(znil[0][1])
else:
raise Exception("unknown compression %s" % (pk,))
fsz = 0
with os.fdopen(fd, "wb") as fo:
while True:
buf = fi.read(32768)
if not buf:
break
fsz += len(buf)
if fsz > maxsz:
raise Exception("zipbomb defused")
fo.write(buf)
return ret
except Exception as ex:
if ret:
t = "failed to decompress audio file [%s]: %r"
log(t % (abspath, ex))
wunlink(log, ret, vn.flags if vn else VF_CAREFUL)
return abspath
def ffprobe(
abspath: str, timeout: int = 60
) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]]]:
@@ -281,16 +375,14 @@ class MTag(object):
or_ffprobe = " or FFprobe"
if self.backend == "mutagen":
self.get = self.get_mutagen
try:
from mutagen import version # noqa: F401
except:
self._get = self.get_mutagen
if not HAVE_MUTAGEN:
self.log("could not load Mutagen, trying FFprobe instead", c=3)
self.backend = "ffprobe"
if self.backend == "ffprobe":
self.usable = self.can_ffprobe
self.get = self.get_ffprobe
self._get = self.get_ffprobe
self.prefer_mt = True
if not HAVE_FFPROBE:
@@ -410,7 +502,7 @@ class MTag(object):
sv = str(zv).split("/")[0].strip().lstrip("0")
ret[sk] = sv or 0
# normalize key notation to rkeobo
# normalize key notation to rekobo
okey = ret.get("key")
if okey:
key = str(okey).replace(" ", "").replace("maj", "").replace("min", "m")
@@ -460,6 +552,17 @@ class MTag(object):
return r1
def get(self, abspath: str) -> dict[str, Union[str, float]]:
ext = abspath.split(".")[-1].lower()
if ext not in self.args.au_unpk:
return self._get(abspath)
ap = au_unpk(self.log, self.args.au_unpk, abspath)
ret = self._get(ap)
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret
def get_mutagen(self, abspath: str) -> dict[str, Union[str, float]]:
ret: dict[str, tuple[int, Any]] = {}
@@ -513,7 +616,7 @@ class MTag(object):
continue
if k == ".aq":
v /= 1000
v /= 1000 # type: ignore
if k == "ac" and v.startswith("mp4a.40."):
v = "aac"
@@ -551,13 +654,18 @@ class MTag(object):
pypath = str(os.pathsep.join(zsl))
env["PYTHONPATH"] = pypath
except:
if not E.ox and not EXE:
raise
raise # might be expected outside cpython
ext = abspath.split(".")[-1].lower()
if ext in self.args.au_unpk:
ap = au_unpk(self.log, self.args.au_unpk, abspath)
else:
ap = abspath
ret: dict[str, Any] = {}
for tagname, parser in sorted(parsers.items(), key=lambda x: (x[1].pri, x[0])):
try:
cmd = [parser.bin, abspath]
cmd = [parser.bin, ap]
if parser.bin.endswith(".py"):
cmd = [pybin] + cmd
@@ -594,4 +702,7 @@ class MTag(object):
t = "mtag error: tagname {}, parser {}, file {} => {}"
self.log(t.format(tagname, parser.bin, abspath, min_ex()))
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret

View File

@@ -206,6 +206,7 @@ class MCast(object):
except:
t = "announce failed on {} [{}]:\n{}"
self.log(t.format(netdev, ip, min_ex()), 3)
sck.close()
if self.args.zm_msub:
for s1 in self.srv.values():

View File

@@ -4,27 +4,33 @@ from __future__ import print_function, unicode_literals
import argparse
import base64
import hashlib
import os
import sys
import threading
from .__init__ import unicode
try:
if os.environ.get("PRTY_NO_ARGON2"):
raise Exception()
HAVE_ARGON2 = True
from argon2 import __version__ as argon2ver
except:
HAVE_ARGON2 = False
class PWHash(object):
def __init__(self, args: argparse.Namespace):
self.args = args
try:
alg, ac = args.ah_alg.split(",")
except:
alg = args.ah_alg
ac = {}
zsl = args.ah_alg.split(",")
alg = zsl[0]
if alg == "none":
alg = ""
self.alg = alg
self.ac = ac
self.ac = zsl[1:]
if not alg:
self.on = False
self.hash = unicode
@@ -80,17 +86,23 @@ class PWHash(object):
its = 2
blksz = 8
para = 4
ramcap = 0 # openssl 1.1 = 32 MiB
try:
cost = 2 << int(self.ac[0])
its = int(self.ac[1])
blksz = int(self.ac[2])
para = int(self.ac[3])
ramcap = int(self.ac[4]) * 1024 * 1024
except:
pass
cfg = {"salt": self.salt, "n": cost, "r": blksz, "p": para, "dklen": 24}
if ramcap:
cfg["maxmem"] = ramcap
ret = plain.encode("utf-8")
for _ in range(its):
ret = hashlib.scrypt(ret, salt=self.salt, n=cost, r=blksz, p=para, dklen=24)
ret = hashlib.scrypt(ret, **cfg)
return "+" + base64.urlsafe_b64encode(ret).decode("utf-8")

View File

@@ -12,7 +12,7 @@ from types import SimpleNamespace
from .__init__ import ANYWIN, EXE, TYPE_CHECKING
from .authsrv import LEELOO_DALLAS, VFS
from .bos import bos
from .util import Daemon, min_ex, pybin, runhook
from .util import Daemon, absreal, min_ex, pybin, runhook, vjoin
if True: # pylint: disable=using-constant-test
from typing import Any, Union
@@ -127,7 +127,7 @@ class SMB(object):
self.log("smb", msg, c)
def start(self) -> None:
Daemon(self.srv.start)
Daemon(self.srv.start, "smbd")
def _auth_cb(self, *a, **ka):
debug("auth-result: %s %s", a, ka)
@@ -151,6 +151,8 @@ class SMB(object):
def _uname(self) -> str:
if self.noacc:
return LEELOO_DALLAS
if not self.asrv.acct:
return "*"
try:
# you found it! my single worst bit of code so far
@@ -187,7 +189,9 @@ class SMB(object):
debug('%s("%s", %s) %s @%s\033[K\033[0m', caller, vpath, str(a), perms, uname)
vfs, rem = self.asrv.vfs.get(vpath, uname, *perms)
return vfs, vfs.canonical(rem)
if not vfs.realpath:
raise Exception("unmapped vfs")
return vfs, vjoin(vfs.realpath, rem)
def _listdir(self, vpath: str, *a: Any, **ka: Any) -> list[str]:
vpath = vpath.replace("\\", "/").lstrip("/")
@@ -195,6 +199,8 @@ class SMB(object):
uname = self._uname()
# debug('listdir("%s", %s) @%s\033[K\033[0m', vpath, str(a), uname)
vfs, rem = self.asrv.vfs.get(vpath, uname, False, False)
if not vfs.realpath:
raise Exception("unmapped vfs")
_, vfs_ls, vfs_virt = vfs.ls(
rem, uname, not self.args.no_scandir, [[False, False]]
)
@@ -209,7 +215,7 @@ class SMB(object):
sz = 112 * 2 # ['.', '..']
for n, fn in enumerate(ls):
if sz >= 64000:
t = "listing only %d of %d files (%d byte) in /%s; see impacket#1433"
t = "listing only %d of %d files (%d byte) in /%s for performance; see --smb-nwa-1"
warning(t, n, len(ls), sz, vpath)
break
@@ -238,9 +244,24 @@ class SMB(object):
t = "blocked write (no-write-acc %s): /%s @%s"
yeet(t % (vfs.axs.uwrite, vpath, uname))
ap = absreal(ap)
xbu = vfs.flags.get("xbu")
if xbu and not runhook(
self.nlog, xbu, ap, vpath, "", "", 0, 0, "1.7.6.2", 0, ""
self.nlog,
None,
self.hub.up2k,
"xbu.smb",
xbu,
ap,
vpath,
"",
"",
"",
0,
0,
"1.7.6.2",
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
@@ -297,7 +318,7 @@ class SMB(object):
t = "blocked rename (no-move-acc %s): /%s @%s"
yeet(t % (vfs1.axs.umove, vp1, uname))
self.hub.up2k.handle_mv(uname, vp1, vp2)
self.hub.up2k.handle_mv(uname, "1.7.6.2", vp1, vp2)
try:
bos.makedirs(ap2)
except:

View File

@@ -5,11 +5,11 @@ import errno
import re
import select
import socket
from email.utils import formatdate
import time
from .__init__ import TYPE_CHECKING
from .multicast import MC_Sck, MCast
from .util import CachedSet, html_escape, min_ex
from .util import CachedSet, formatdate, html_escape, min_ex
if TYPE_CHECKING:
from .broker_util import BrokerCli
@@ -84,7 +84,7 @@ class SSDPr(object):
name = self.args.doctitle
zs = zs.strip().format(c(ubase), c(url), c(name), c(self.args.zsid))
hc.reply(zs.encode("utf-8", "replace"))
return False # close connectino
return False # close connection
class SSDPd(MCast):
@@ -141,9 +141,29 @@ class SSDPd(MCast):
self.log("stopped", 2)
def run2(self) -> None:
try:
if self.args.no_poll:
raise Exception()
fd2sck = {}
srvpoll = select.poll()
for sck in self.srv:
fd = sck.fileno()
fd2sck[fd] = sck
srvpoll.register(fd, select.POLLIN)
except Exception as ex:
srvpoll = None
if not self.args.no_poll:
t = "WARNING: failed to poll(), will use select() instead: %r"
self.log(t % (ex,), 3)
while self.running:
rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
rx: list[socket.socket] = rdy[0] # type: ignore
if srvpoll:
pr = srvpoll.poll((self.args.z_chk or 180) * 1000)
rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
else:
rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
rx: list[socket.socket] = rdy[0] # type: ignore
self.rxc.cln()
buf = b""
addr = ("0", 0)
@@ -168,7 +188,7 @@ class SSDPd(MCast):
except:
pass
self.srv = {}
self.srv.clear()
def eat(self, buf: bytes, addr: tuple[str, int]) -> None:
cip = addr[0]
@@ -209,7 +229,7 @@ CONFIGID.UPNP.ORG: 1
"""
v4 = srv.ip.replace("::ffff:", "")
zs = zs.format(formatdate(usegmt=True), v4, srv.hport, self.args.zsid)
zs = zs.format(formatdate(), v4, srv.hport, self.args.zsid)
zb = zs[1:].replace("\n", "\r\n").encode("utf-8", "replace")
srv.sck.sendto(zb, addr[:2])

View File

@@ -1,13 +1,13 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import re
import stat
import tarfile
from queue import Queue
from .authsrv import AuthSrv
from .bos import bos
from .sutil import StreamArc, errdesc
from .util import Daemon, fsenc, min_ex
@@ -45,12 +45,12 @@ class StreamTar(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
cmp: str = "",
**kwargs: Any
):
super(StreamTar, self).__init__(log, args, fgen)
super(StreamTar, self).__init__(log, asrv, fgen)
self.ci = 0
self.co = 0
@@ -148,7 +148,7 @@ class StreamTar(StreamArc):
errors.append((f["vp"], ex))
if errors:
self.errf, txt = errdesc(errors)
self.errf, txt = errdesc(self.asrv.vfs, errors)
self.log("\n".join(([repr(self.errf)] + txt[1:])))
self.ser(self.errf)

View File

@@ -12,6 +12,12 @@ from .label import DNSBuffer, DNSLabel
from .ranges import IP4, IP6, H, I, check_bytes
try:
range = xrange
except:
pass
class DNSError(Exception):
pass

View File

@@ -11,7 +11,21 @@ import os
from ._shared import IP, Adapter
if os.name == "nt":
def nope(include_unconfigured=False):
return []
try:
S390X = os.uname().machine == "s390x"
except:
S390X = False
if os.environ.get("PRTY_NO_IFADDR") or S390X:
# s390x deadlocks at libc.getifaddrs
get_adapters = nope
elif os.name == "nt":
from ._win32 import get_adapters
elif os.name == "posix":
from ._posix import get_adapters

View File

@@ -17,6 +17,7 @@ if not PY2:
U: Callable[[str], str] = str
else:
U = unicode # noqa: F821 # pylint: disable=undefined-variable,self-assigning-variable
range = xrange # noqa: F821 # pylint: disable=undefined-variable,self-assigning-variable
class Adapter(object):

View File

@@ -16,6 +16,11 @@ if True: # pylint: disable=using-constant-test
from typing import Callable, List, Optional, Tuple, Union
try:
range = xrange
except:
pass
def num_char_count_bits(ver: int) -> int:
return 16 if (ver + 7) // 17 else 8
@@ -589,3 +594,20 @@ def _get_bit(x: int, i: int) -> bool:
class DataTooLongError(ValueError):
pass
def qr2svg(qr: QrCode, border: int) -> str:
parts: list[str] = []
for y in range(qr.size):
sy = border + y
for x in range(qr.size):
if qr.modules[y][x]:
parts.append("M%d,%dh1v1h-1z" % (border + x, sy))
t = """\
<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 {0} {0}" stroke="none">
<rect width="100%" height="100%" fill="#F7F7F7"/>
<path d="{1}" fill="#111111"/>
</svg>
"""
return t.format(qr.size + border * 2, " ".join(parts))

View File

@@ -1,15 +1,15 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import tempfile
from datetime import datetime
from .__init__ import CORES
from .authsrv import VFS, AuthSrv
from .bos import bos
from .th_cli import ThumbCli
from .util import UTC, vjoin
from .util import UTC, vjoin, vol_san
if True: # pylint: disable=using-constant-test
from typing import Any, Generator, Optional
@@ -21,12 +21,13 @@ class StreamArc(object):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
**kwargs: Any
):
self.log = log
self.args = args
self.asrv = asrv
self.args = asrv.args
self.fgen = fgen
self.stopped = False
@@ -81,7 +82,9 @@ def enthumb(
) -> dict[str, Any]:
rem = f["vp"]
ext = rem.rsplit(".", 1)[-1].lower()
if fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|"):
if (fmt == "mp3" and ext == "mp3") or (
fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|")
):
raise Exception()
vp = vjoin(vtop, rem.split("/", 1)[1])
@@ -101,15 +104,20 @@ def enthumb(
return f
def errdesc(errors: list[tuple[str, str]]) -> tuple[dict[str, Any], list[str]]:
def errdesc(
vfs: VFS, errors: list[tuple[str, str]]
) -> tuple[dict[str, Any], list[str]]:
report = ["copyparty failed to add the following files to the archive:", ""]
for fn, err in errors:
report.extend([" file: {}".format(fn), "error: {}".format(err), ""])
btxt = "\r\n".join(report).encode("utf-8", "replace")
btxt = vol_san(list(vfs.all_vols.values()), btxt)
with tempfile.NamedTemporaryFile(prefix="copyparty-", delete=False) as tf:
tf_path = tf.name
tf.write("\r\n".join(report).encode("utf-8", "replace"))
tf.write(btxt)
dt = datetime.now(UTC).strftime("%Y-%m%d-%H%M%S")

View File

@@ -2,8 +2,6 @@
from __future__ import print_function, unicode_literals
import argparse
import base64
import calendar
import errno
import gzip
import logging
@@ -16,7 +14,7 @@ import string
import sys
import threading
import time
from datetime import datetime, timedelta
from datetime import datetime
# from inspect import currentframe
# print(currentframe().f_lineno)
@@ -28,18 +26,30 @@ if True: # pylint: disable=using-constant-test
import typing
from typing import Any, Optional, Union
from .__init__ import ANYWIN, EXE, MACOS, TYPE_CHECKING, E, EnvParams, unicode
from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, E, EnvParams, unicode
from .authsrv import BAD_CFG, AuthSrv
from .cert import ensure_cert
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, HAVE_MUTAGEN
from .pwhash import HAVE_ARGON2
from .tcpsrv import TcpSrv
from .th_srv import HAVE_PIL, HAVE_VIPS, HAVE_WEBP, ThumbSrv
from .th_srv import (
HAVE_AVIF,
HAVE_FFMPEG,
HAVE_FFPROBE,
HAVE_HEIF,
HAVE_PIL,
HAVE_VIPS,
HAVE_WEBP,
ThumbSrv,
)
from .up2k import Up2k
from .util import (
DEF_EXP,
DEF_MTE,
DEF_MTH,
FFMPEG_URL,
HAVE_PSUTIL,
HAVE_SQLITE3,
UTC,
VERSIONS,
Daemon,
@@ -50,12 +60,14 @@ from .util import (
alltrace,
ansi_re,
build_netmap,
load_ipu,
min_ex,
mp,
odfusion,
pybin,
start_log_thrs,
start_stackmon,
ub64enc,
)
if TYPE_CHECKING:
@@ -65,6 +77,9 @@ if TYPE_CHECKING:
except:
pass
if PY2:
range = xrange # type: ignore
class SvcHub(object):
"""
@@ -89,20 +104,23 @@ class SvcHub(object):
self.argv = argv
self.E: EnvParams = args.E
self.no_ansi = args.no_ansi
self.tz = UTC if args.log_utc else None
self.logf: Optional[typing.TextIO] = None
self.logf_base_fn = ""
self.is_dut = False # running in unittest; always False
self.stop_req = False
self.stopping = False
self.stopped = False
self.reload_req = False
self.reloading = 0
self.reload_mutex = threading.Lock()
self.stop_cond = threading.Condition()
self.nsigs = 3
self.retcode = 0
self.httpsrv_up = 0
self.log_mutex = threading.Lock()
self.next_day = 0
self.cday = 0
self.cmon = 0
self.tstack = 0.0
self.iphash = HMaccas(os.path.join(self.E.cfg, "iphash"), 8)
@@ -193,6 +211,38 @@ class SvcHub(object):
t = "WARNING: --s-rd-sz (%d) is larger than --iobuf (%d); this may lead to reduced performance"
self.log("root", t % (args.s_rd_sz, args.iobuf), 3)
zs = ""
if args.th_ram_max < 0.22:
zs = "generate thumbnails"
elif args.th_ram_max < 1:
zs = "generate audio waveforms or spectrograms"
if zs:
t = "WARNING: --th-ram-max is very small (%.2f GiB); will not be able to %s"
self.log("root", t % (args.th_ram_max, zs), 3)
if args.chpw and args.idp_h_usr:
t = "ERROR: user-changeable passwords is incompatible with IdP/identity-providers; you must disable either --chpw or --idp-h-usr"
self.log("root", t, 1)
raise Exception(t)
noch = set()
for zs in args.chpw_no or []:
zsl = [x.strip() for x in zs.split(",")]
noch.update([x for x in zsl if x])
args.chpw_no = noch
if args.ipu:
iu, nm = load_ipu(self.log, args.ipu, True)
setattr(args, "ipu_iu", iu)
setattr(args, "ipu_nm", nm)
if not self.args.no_ses:
self.setup_session_db()
args.shr1 = ""
if args.shr:
self.setup_share_db()
bri = "zy"[args.theme % 2 :][:1]
ch = "abcdefghijklmnopqrstuvwx"[int(args.theme / 2)]
args.theme = "{0}{1} {0} {1}".format(ch, bri)
@@ -232,6 +282,8 @@ class SvcHub(object):
self.up2k = Up2k(self)
self._feature_test()
decs = {k: 1 for k in self.args.th_dec.split(",")}
if not HAVE_VIPS:
decs.pop("vips", None)
@@ -240,6 +292,10 @@ class SvcHub(object):
if not HAVE_FFMPEG or not HAVE_FFPROBE:
decs.pop("ff", None)
# compressed formats; "s3z=s3m.zip, s3gz=s3m.gz, ..."
zlss = [x.strip().lower().split("=", 1) for x in args.au_unpk.split(",")]
args.au_unpk = {x[0]: x[1] for x in zlss}
self.args.th_dec = list(decs.keys())
self.thumbsrv = None
want_ff = False
@@ -276,6 +332,13 @@ class SvcHub(object):
if want_ff and ANYWIN:
self.log("thumb", "download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
if not args.no_acode:
if not re.match("^(0|[qv][0-9]|[0-9]{2,3}k)$", args.q_mp3.lower()):
t = "invalid mp3 transcoding quality [%s] specified; only supports [0] to disable, a CBR value such as [192k], or a CQ/CRF value such as [v2]"
raise Exception(t % (args.q_mp3,))
else:
args.au_unpk = {}
args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
zms = ""
@@ -288,13 +351,14 @@ class SvcHub(object):
from .ftpd import Ftpd
self.ftpd: Optional[Ftpd] = None
Daemon(self.start_ftpd, "start_ftpd")
zms += "f" if args.ftp else "F"
if args.tftp:
from .tftpd import Tftpd
self.tftpd: Optional[Tftpd] = None
if args.ftp or args.ftps or args.tftp:
Daemon(self.start_ftpd, "start_tftpd")
if args.smb:
@@ -324,6 +388,160 @@ class SvcHub(object):
self.broker = Broker(self)
# create netmaps early to avoid firewall gaps,
# but the mutex blocks multiprocessing startup
for zs in "ipu_iu ftp_ipa_nm tftp_ipa_nm".split():
try:
getattr(args, zs).mutex = threading.Lock()
except:
pass
def setup_session_db(self) -> None:
if not HAVE_SQLITE3:
self.args.no_ses = True
t = "WARNING: sqlite3 not available; disabling sessions, will use plaintext passwords in cookies"
self.log("root", t, 3)
return
import sqlite3
create = True
db_path = self.args.ses_db
self.log("root", "opening sessions-db %s" % (db_path,))
for n in range(2):
try:
db = sqlite3.connect(db_path)
cur = db.cursor()
try:
cur.execute("select count(*) from us").fetchone()
create = False
break
except:
pass
except Exception as ex:
if n:
raise
t = "sessions-db corrupt; deleting and recreating: %r"
self.log("root", t % (ex,), 3)
try:
cur.close() # type: ignore
except:
pass
try:
db.close() # type: ignore
except:
pass
os.unlink(db_path)
sch = [
r"create table kv (k text, v int)",
r"create table us (un text, si text, t0 int)",
# username, session-id, creation-time
r"create index us_un on us(un)",
r"create index us_si on us(si)",
r"create index us_t0 on us(t0)",
r"insert into kv values ('sver', 1)",
]
assert db # type: ignore # !rm
assert cur # type: ignore # !rm
if create:
for cmd in sch:
cur.execute(cmd)
self.log("root", "created new sessions-db")
db.commit()
cur.close()
db.close()
def setup_share_db(self) -> None:
al = self.args
if not HAVE_SQLITE3:
self.log("root", "sqlite3 not available; disabling --shr", 1)
al.shr = ""
return
import sqlite3
al.shr = al.shr.strip("/")
if "/" in al.shr or not al.shr:
t = "config error: --shr must be the name of a virtual toplevel directory to put shares inside"
self.log("root", t, 1)
raise Exception(t)
al.shr = "/%s/" % (al.shr,)
al.shr1 = al.shr[1:]
create = True
modified = False
db_path = self.args.shr_db
self.log("root", "opening shares-db %s" % (db_path,))
for n in range(2):
try:
db = sqlite3.connect(db_path)
cur = db.cursor()
try:
cur.execute("select count(*) from sh").fetchone()
create = False
break
except:
pass
except Exception as ex:
if n:
raise
t = "shares-db corrupt; deleting and recreating: %r"
self.log("root", t % (ex,), 3)
try:
cur.close() # type: ignore
except:
pass
try:
db.close() # type: ignore
except:
pass
os.unlink(db_path)
sch1 = [
r"create table kv (k text, v int)",
r"create table sh (k text, pw text, vp text, pr text, st int, un text, t0 int, t1 int)",
# sharekey, password, src, perms, numFiles, owner, created, expires
]
sch2 = [
r"create table sf (k text, vp text)",
r"create index sf_k on sf(k)",
r"create index sh_k on sh(k)",
r"create index sh_t1 on sh(t1)",
]
assert db # type: ignore # !rm
assert cur # type: ignore # !rm
if create:
dver = 2
modified = True
for cmd in sch1 + sch2:
cur.execute(cmd)
self.log("root", "created new shares-db")
else:
(dver,) = cur.execute("select v from kv where k = 'sver'").fetchall()[0]
if dver == 1:
modified = True
for cmd in sch2:
cur.execute(cmd)
cur.execute("update sh set st = 0")
self.log("root", "shares-db schema upgrade ok")
if modified:
for cmd in [
r"delete from kv where k = 'sver'",
r"insert into kv values ('sver', %d)" % (2,),
]:
cur.execute(cmd)
db.commit()
cur.close()
db.close()
def start_ftpd(self) -> None:
time.sleep(30)
@@ -383,7 +601,7 @@ class SvcHub(object):
self.sigterm()
def sigterm(self) -> None:
os.kill(os.getpid(), signal.SIGTERM)
self.signal_handler(signal.SIGTERM, None)
def cb_httpsrv_up(self) -> None:
self.httpsrv_up += 1
@@ -408,6 +626,58 @@ class SvcHub(object):
Daemon(self.sd_notify, "sd-notify")
def _feature_test(self) -> None:
fok = []
fng = []
t_ff = "transcode audio, create spectrograms, video thumbnails"
to_check = [
(HAVE_SQLITE3, "sqlite", "sessions and file/media indexing"),
(HAVE_PIL, "pillow", "image thumbnails (plenty fast)"),
(HAVE_VIPS, "vips", "image thumbnails (faster, eats more ram)"),
(HAVE_WEBP, "pillow-webp", "create thumbnails as webp files"),
(HAVE_FFMPEG, "ffmpeg", t_ff + ", good-but-slow image thumbnails"),
(HAVE_FFPROBE, "ffprobe", t_ff + ", read audio/media tags"),
(HAVE_MUTAGEN, "mutagen", "read audio tags (ffprobe is better but slower)"),
(HAVE_ARGON2, "argon2", "secure password hashing (advanced users only)"),
(HAVE_HEIF, "pillow-heif", "read .heif images with pillow (rarely useful)"),
(HAVE_AVIF, "pillow-avif", "read .avif images with pillow (rarely useful)"),
]
if ANYWIN:
to_check += [
(HAVE_PSUTIL, "psutil", "improved plugin cleanup (rarely useful)")
]
verbose = self.args.deps
if verbose:
self.log("dependencies", "")
for have, feat, what in to_check:
lst = fok if have else fng
lst.append((feat, what))
if verbose:
zi = 2 if have else 5
sgot = "found" if have else "missing"
t = "%7s: %s \033[36m(%s)"
self.log("dependencies", t % (sgot, feat, what), zi)
if verbose:
self.log("dependencies", "")
return
sok = ", ".join(x[0] for x in fok)
sng = ", ".join(x[0] for x in fng)
t = ""
if sok:
t += "OK: \033[32m" + sok
if sng:
if t:
t += ", "
t += "\033[0mNG: \033[35m" + sng
t += "\033[0m, see --deps"
self.log("dependencies", t, 6)
def _check_env(self) -> None:
try:
files = os.listdir(E.cfg)
@@ -419,6 +689,12 @@ class SvcHub(object):
t = "WARNING: found config files in [%s]: %s\n config files are not expected here, and will NOT be loaded (unless your setup is intentionally hella funky)"
self.log("root", t % (E.cfg, ", ".join(hits)), 3)
if self.args.no_bauth:
t = "WARNING: --no-bauth disables support for the Android app; you may want to use --bauth-last instead"
self.log("root", t, 3)
if self.args.bauth_last:
self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
def _process_config(self) -> bool:
al = self.args
@@ -461,8 +737,10 @@ class SvcHub(object):
zsl = al.th_covers.split(",")
zsl = [x.strip() for x in zsl]
zsl = [x for x in zsl if x]
al.th_covers = set(zsl)
al.th_coversd = set(zsl + ["." + x for x in zsl])
al.th_covers = zsl
al.th_coversd = zsl + ["." + x for x in zsl]
al.th_covers_set = set(al.th_covers)
al.th_coversd_set = set(al.th_coversd)
for k in "c".split(" "):
vl = getattr(al, k)
@@ -502,8 +780,8 @@ class SvcHub(object):
al.idp_h_grp = al.idp_h_grp.lower()
al.idp_h_key = al.idp_h_key.lower()
al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa)
al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa)
al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa, True)
al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa, True)
mte = ODict.fromkeys(DEF_MTE.split(","), True)
al.mte = odfusion(mte, al.mte)
@@ -515,7 +793,7 @@ class SvcHub(object):
al.exp_md = odfusion(exp, al.exp_md.replace(" ", ","))
al.exp_lg = odfusion(exp, al.exp_lg.replace(" ", ","))
for k in ["no_hash", "no_idx"]:
for k in ["no_hash", "no_idx", "og_ua"]:
ptn = getattr(self.args, k)
if ptn:
setattr(self.args, k, re.compile(ptn))
@@ -539,6 +817,35 @@ class SvcHub(object):
except:
raise Exception("invalid --rm-retry [%s]" % (self.args.rm_retry,))
try:
zf1, zf2 = self.args.mv_retry.split("/")
self.args.mv_re_t = float(zf1)
self.args.mv_re_r = float(zf2)
except:
raise Exception("invalid --mv-retry [%s]" % (self.args.mv_retry,))
al.tcolor = al.tcolor.lstrip("#")
if len(al.tcolor) == 3: # fc5 => ffcc55
al.tcolor = "".join([x * 2 for x in al.tcolor])
zs = al.u2sz
zsl = zs.split(",")
if len(zsl) not in (1, 3):
t = "invalid --u2sz; must be either one number, or a comma-separated list of three numbers (min,default,max)"
raise Exception(t)
if len(zsl) < 3:
zsl = ["1", zs, zs]
zi2 = 1
for zs in zsl:
zi = int(zs)
# arbitrary constraint (anything above 2 GiB is probably unintended)
if zi < 1 or zi > 2047:
raise Exception("invalid --u2sz; minimum is 1, max is 2047")
if zi < zi2:
raise Exception("invalid --u2sz; values must be equal or ascending")
zi2 = zi
al.u2sz = ",".join(zsl)
return True
def _ipa2re(self, txt) -> Optional[re.Pattern]:
@@ -589,7 +896,7 @@ class SvcHub(object):
self.args.nc = min(self.args.nc, soft // 2)
def _logname(self) -> str:
dt = datetime.now(UTC)
dt = datetime.now(self.tz)
fn = str(self.args.lo)
for fs in "YmdHMS":
fs = "%" + fs
@@ -706,38 +1013,23 @@ class SvcHub(object):
except:
self.log("root", "ssdp startup failed;\n" + min_ex(), 3)
def reload(self) -> str:
with self.up2k.mutex:
if self.reloading:
return "cannot reload; already in progress"
self.reloading = 1
Daemon(self._reload, "reloading")
return "reload initiated"
def _reload(self, rescan_all_vols: bool = True) -> None:
with self.up2k.mutex:
if self.reloading != 1:
return
self.reloading = 2
def reload(self, rescan_all_vols: bool, up2k: bool) -> str:
t = "config has been reloaded"
with self.reload_mutex:
self.log("root", "reloading config")
self.asrv.reload()
self.up2k.reload(rescan_all_vols)
self.asrv.reload(9 if up2k else 4)
if up2k:
self.up2k.reload(rescan_all_vols)
t += "; volumes are now reinitializing"
else:
self.log("root", "reload done")
self.broker.reload()
self.reloading = 0
return t
def _reload_blocking(self, rescan_all_vols: bool = True) -> None:
while True:
with self.up2k.mutex:
if self.reloading < 2:
self.reloading = 1
break
time.sleep(0.05)
# try to handle multiple pending IdP reloads at once:
time.sleep(0.2)
self._reload(rescan_all_vols=rescan_all_vols)
def _reload_sessions(self) -> None:
with self.asrv.mutex:
self.asrv.load_sessions(True)
self.broker.reload_sessions()
def stop_thr(self) -> None:
while not self.stop_req:
@@ -746,7 +1038,7 @@ class SvcHub(object):
if self.reload_req:
self.reload_req = False
self.reload()
self.reload(True, True)
self.shutdown()
@@ -859,12 +1151,12 @@ class SvcHub(object):
return
with self.log_mutex:
zd = datetime.now(UTC)
dt = datetime.now(self.tz)
ts = self.log_dfmt % (
zd.year,
zd.month * 100 + zd.day,
(zd.hour * 100 + zd.minute) * 100 + zd.second,
zd.microsecond // self.log_div,
dt.year,
dt.month * 100 + dt.day,
(dt.hour * 100 + dt.minute) * 100 + dt.second,
dt.microsecond // self.log_div,
)
if c and not self.args.no_ansi:
@@ -885,41 +1177,26 @@ class SvcHub(object):
if not self.args.no_logflush:
self.logf.flush()
now = time.time()
if int(now) >= self.next_day:
self._set_next_day()
if dt.day != self.cday or dt.month != self.cmon:
self._set_next_day(dt)
def _set_next_day(self) -> None:
if self.next_day and self.logf and self.logf_base_fn != self._logname():
def _set_next_day(self, dt: datetime) -> None:
if self.cday and self.logf and self.logf_base_fn != self._logname():
self.logf.close()
self._setup_logfile("")
dt = datetime.now(UTC)
# unix timestamp of next 00:00:00 (leap-seconds safe)
day_now = dt.day
while dt.day == day_now:
dt += timedelta(hours=12)
dt = dt.replace(hour=0, minute=0, second=0)
try:
tt = dt.utctimetuple()
except:
# still makes me hella uncomfortable
tt = dt.timetuple()
self.next_day = calendar.timegm(tt)
self.cday = dt.day
self.cmon = dt.month
def _log_enabled(self, src: str, msg: str, c: Union[int, str] = 0) -> None:
"""handles logging from all components"""
with self.log_mutex:
now = time.time()
if int(now) >= self.next_day:
dt = datetime.fromtimestamp(now, UTC)
dt = datetime.now(self.tz)
if dt.day != self.cday or dt.month != self.cmon:
zs = "{}\n" if self.no_ansi else "\033[36m{}\033[0m\n"
zs = zs.format(dt.strftime("%Y-%m-%d"))
print(zs, end="")
self._set_next_day()
self._set_next_day(dt)
if self.logf:
self.logf.write(zs)
@@ -938,12 +1215,11 @@ class SvcHub(object):
else:
msg = "%s%s\033[0m" % (c, msg)
zd = datetime.fromtimestamp(now, UTC)
ts = self.log_efmt % (
zd.hour,
zd.minute,
zd.second,
zd.microsecond // self.log_div,
dt.hour,
dt.minute,
dt.second,
dt.microsecond // self.log_div,
)
msg = fmt % (ts, src, msg)
try:
@@ -1041,5 +1317,5 @@ class SvcHub(object):
zs = "{}\n{}".format(VERSIONS, alltrace())
zb = zs.encode("utf-8", "replace")
zb = gzip.compress(zb)
zs = base64.b64encode(zb).decode("ascii")
zs = ub64enc(zb).decode("ascii")
self.log("stacks", zs)

View File

@@ -1,12 +1,12 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import calendar
import stat
import time
import zlib
from .authsrv import AuthSrv
from .bos import bos
from .sutil import StreamArc, errdesc
from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile
@@ -37,9 +37,7 @@ def dostime2unix(buf: bytes) -> int:
def unixtime2dos(ts: int) -> bytes:
tt = time.gmtime(ts + 1)
dy, dm, dd, th, tm, ts = list(tt)[:6]
dy, dm, dd, th, tm, ts, _, _, _ = time.gmtime(ts + 1)
bd = ((dy - 1980) << 9) + (dm << 5) + dd
bt = (th << 11) + (tm << 5) + ts // 2
try:
@@ -102,12 +100,12 @@ def gen_hdr(
# spec says to put zeros when !crc if bit3 (streaming)
# however infozip does actual sz and it even works on winxp
# (same reasning for z64 extradata later)
# (same reasoning for z64 extradata later)
vsz = 0xFFFFFFFF if z64 else sz
ret += spack(b"<LL", vsz, vsz)
# windows support (the "?" replace below too)
fn = sanitize_fn(fn, "/", [])
fn = sanitize_fn(fn, "/")
bfn = fn.encode("utf-8" if utf8 else "cp437", "replace").replace(b"?", b"_")
# add ntfs (0x24) and/or unix (0x10) extrafields for utc, add z64 if requested
@@ -219,13 +217,13 @@ class StreamZip(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
utf8: bool = False,
pre_crc: bool = False,
**kwargs: Any
) -> None:
super(StreamZip, self).__init__(log, args, fgen)
super(StreamZip, self).__init__(log, asrv, fgen)
self.utf8 = utf8
self.pre_crc = pre_crc
@@ -302,7 +300,7 @@ class StreamZip(StreamArc):
mbuf = b""
if errors:
errf, txt = errdesc(errors)
errf, txt = errdesc(self.asrv.vfs, errors)
self.log("\n".join(([repr(errf)] + txt[1:])))
for x in self.ser(errf):
yield x

View File

@@ -15,19 +15,25 @@ from .util import (
E_ADDR_IN_USE,
E_ADDR_NOT_AVAIL,
E_UNREACH,
HAVE_IPV6,
IP6ALL,
VF_CAREFUL,
Netdev,
atomic_move,
min_ex,
sunpack,
termsize,
)
if True:
from typing import Generator
from typing import Generator, Union
if TYPE_CHECKING:
from .svchub import SvcHub
if not hasattr(socket, "AF_UNIX"):
setattr(socket, "AF_UNIX", -9001)
if not hasattr(socket, "IPPROTO_IPV6"):
setattr(socket, "IPPROTO_IPV6", 41)
@@ -89,7 +95,7 @@ class TcpSrv(object):
continue
# binding 0.0.0.0 after :: fails on dualstack
# but is necessary on non-dualstakc
# but is necessary on non-dualstack
if successful_binds:
continue
@@ -111,8 +117,10 @@ class TcpSrv(object):
eps = {
"127.0.0.1": Netdev("127.0.0.1", 0, "", "local only"),
"::1": Netdev("::1", 0, "", "local only"),
}
if HAVE_IPV6:
eps["::1"] = Netdev("::1", 0, "", "local only")
nonlocals = [x for x in self.args.i if x not in [k.split("/")[0] for k in eps]]
if nonlocals:
try:
@@ -214,14 +222,41 @@ class TcpSrv(object):
if self.args.qr or self.args.qrs:
self.qr = self._qr(qr1, qr2)
def nlog(self, msg: str, c: Union[int, str] = 0) -> None:
self.log("tcpsrv", msg, c)
def _listen(self, ip: str, port: int) -> None:
ipv = socket.AF_INET6 if ":" in ip else socket.AF_INET
uds_perm = uds_gid = -1
if "unix:" in ip:
tcp = False
ipv = socket.AF_UNIX
uds = ip.split(":")
ip = uds[-1]
if len(uds) > 2:
uds_perm = int(uds[1], 8)
if len(uds) > 3:
try:
uds_gid = int(uds[2])
except:
import grp
uds_gid = grp.getgrnam(uds[2]).gr_gid
elif ":" in ip:
tcp = True
ipv = socket.AF_INET6
else:
tcp = True
ipv = socket.AF_INET
srv = socket.socket(ipv, socket.SOCK_STREAM)
if not ANYWIN or self.args.reuseaddr:
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
if tcp:
srv.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
srv.settimeout(None) # < does not inherit, ^ opts above do
try:
@@ -233,8 +268,25 @@ class TcpSrv(object):
srv.setsockopt(socket.SOL_IP, socket.IP_FREEBIND, 1)
try:
srv.bind((ip, port))
sport = srv.getsockname()[1]
if tcp:
srv.bind((ip, port))
else:
if ANYWIN or self.args.rm_sck:
if os.path.exists(ip):
os.unlink(ip)
srv.bind(ip)
else:
tf = "%s.%d" % (ip, os.getpid())
if os.path.exists(tf):
os.unlink(tf)
srv.bind(tf)
if uds_gid != -1:
os.chown(tf, -1, uds_gid)
if uds_perm != -1:
os.chmod(tf, uds_perm)
atomic_move(self.nlog, tf, ip, VF_CAREFUL)
sport = srv.getsockname()[1] if tcp else port
if port != sport:
# linux 6.0.16 lets you bind a port which is in use
# except it just gives you a random port instead
@@ -246,12 +298,23 @@ class TcpSrv(object):
except:
pass
e = ""
if ex.errno in E_ADDR_IN_USE:
e = "\033[1;31mport {} is busy on interface {}\033[0m".format(port, ip)
if not tcp:
e = "\033[1;31munix-socket {} is busy\033[0m".format(ip)
elif ex.errno in E_ADDR_NOT_AVAIL:
e = "\033[1;31minterface {} does not exist\033[0m".format(ip)
else:
if not e:
if not tcp:
t = "\n\n\n NOTE: this crash may be due to a unix-socket bug; try --rm-sck\n"
self.log("tcpsrv", t, 2)
raise
if not tcp and not self.args.rm_sck:
e += "; maybe this is a bug? try --rm-sck"
raise Exception(e)
def run(self) -> None:
@@ -259,7 +322,14 @@ class TcpSrv(object):
bound: list[tuple[str, int]] = []
srvs: list[socket.socket] = []
for srv in self.srv:
ip, port = srv.getsockname()[:2]
if srv.family == socket.AF_UNIX:
tcp = False
ip = re.sub(r"\.[0-9]+$", "", srv.getsockname())
port = 0
else:
tcp = True
ip, port = srv.getsockname()[:2]
if ip == IP6ALL:
ip = "::" # jython
@@ -291,13 +361,17 @@ class TcpSrv(object):
bound.append((ip, port))
srvs.append(srv)
fno = srv.fileno()
hip = "[{}]".format(ip) if ":" in ip else ip
msg = "listening @ {}:{} f{} p{}".format(hip, port, fno, os.getpid())
if tcp:
hip = "[{}]".format(ip) if ":" in ip else ip
msg = "listening @ {}:{} f{} p{}".format(hip, port, fno, os.getpid())
else:
msg = "listening @ {} f{} p{}".format(ip, fno, os.getpid())
self.log("tcpsrv", msg)
if self.args.q:
print(msg)
self.hub.broker.say("listen", srv)
self.hub.broker.say("httpsrv.listen", srv)
self.srv = srvs
self.bound = bound
@@ -305,7 +379,7 @@ class TcpSrv(object):
self._distribute_netdevs()
def _distribute_netdevs(self):
self.hub.broker.say("set_netdevs", self.netdevs)
self.hub.broker.say("httpsrv.set_netdevs", self.netdevs)
self.hub.start_zeroconf()
gencert(self.log, self.args, self.netdevs)
self.hub.restart_ftpd()
@@ -345,6 +419,8 @@ class TcpSrv(object):
def detect_interfaces(self, listen_ips: list[str]) -> dict[str, Netdev]:
from .stolen.ifaddr import get_adapters
listen_ips = [x for x in listen_ips if "unix:" not in x]
nics = get_adapters(True)
eps: dict[str, Netdev] = {}
for nic in nics:
@@ -463,6 +539,12 @@ class TcpSrv(object):
sys.stderr.flush()
def _qr(self, t1: dict[str, list[int]], t2: dict[str, list[int]]) -> str:
t2c = {zs: zli for zs, zli in t2.items() if zs in ("127.0.0.1", "::1")}
t2b = {zs: zli for zs, zli in t2.items() if ":" in zs and zs not in t2c}
t2 = {zs: zli for zs, zli in t2.items() if zs not in t2b and zs not in t2c}
t2.update(t2b) # first ipv4, then ipv6...
t2.update(t2c) # ...and finally localhost
ip = None
ips = list(t1) + list(t2)
qri = self.args.qri

View File

@@ -33,10 +33,10 @@ from partftpy import (
)
from partftpy.TftpShared import TftpException
from .__init__ import EXE, TYPE_CHECKING
from .__init__ import EXE, PY2, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .util import BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
from .util import UTC, BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
if True: # pylint: disable=using-constant-test
from typing import Any, Union
@@ -44,6 +44,9 @@ if True: # pylint: disable=using-constant-test
if TYPE_CHECKING:
from .svchub import SvcHub
if PY2:
range = xrange # type: ignore
lg = logging.getLogger("tftp")
debug, info, warning, error = (lg.debug, lg.info, lg.warning, lg.error)
@@ -95,7 +98,7 @@ class Tftpd(object):
TftpServer,
]
cbak = []
if not self.args.tftp_no_fast and not EXE:
if not self.args.tftp_no_fast and not EXE and not PY2:
try:
ptn = re.compile(r"(^\s*)log\.debug\(.*\)$")
for C in Cs:
@@ -105,7 +108,7 @@ class Tftpd(object):
cfn = C.__spec__.origin
exec (compile(src2, filename=cfn, mode="exec"), C.__dict__)
except Exception:
t = "failed to optimize tftp code; run with --tftp-noopt if there are issues:\n"
t = "failed to optimize tftp code; run with --tftp-no-fast if there are issues:\n"
self.log("tftp", t + min_ex(), 3)
for n, zd in enumerate(cbak):
Cs[n].__dict__ = zd
@@ -150,11 +153,6 @@ class Tftpd(object):
self._disarm(fos)
ip = next((x for x in self.args.i if ":" not in x), None)
if not ip:
self.log("tftp", "IPv6 not supported for tftp; listening on 0.0.0.0", 3)
ip = "0.0.0.0"
self.port = int(self.args.tftp)
self.srv = []
self.ips = []
@@ -168,9 +166,16 @@ class Tftpd(object):
if "::" in ips:
ips.append("0.0.0.0")
if self.args.ftp4:
ips = [x for x in ips if "unix:" not in x]
if self.args.tftp4:
ips = [x for x in ips if ":" not in x]
if not ips:
t = "cannot start tftp-server; no compatible IPs in -i"
self.nlog(t, 1)
return
ips = list(ODict.fromkeys(ips)) # dedup
for ip in ips:
@@ -246,6 +251,8 @@ class Tftpd(object):
debug('%s("%s", %s) %s\033[K\033[0m', caller, vpath, str(a), perms)
vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
if not vfs.realpath:
raise Exception("unmapped vfs")
return vfs, vfs.canonical(rem)
def _ls(self, vpath: str, raddress: str, rport: int, force=False) -> Any:
@@ -262,12 +269,13 @@ class Tftpd(object):
"*",
not self.args.no_scandir,
[[True, False]],
throw=True,
)
dnames = set([x[0] for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)])
dirs1 = [(v.st_mtime, v.st_size, k + "/") for k, v in vfs_ls if k in dnames]
fils1 = [(v.st_mtime, v.st_size, k) for k, v in vfs_ls if k not in dnames]
real1 = dirs1 + fils1
realt = [(datetime.fromtimestamp(mt), sz, fn) for mt, sz, fn in real1]
realt = [(datetime.fromtimestamp(mt, UTC), sz, fn) for mt, sz, fn in real1]
reals = [
(
"%04d-%02d-%02d %02d:%02d:%02d"
@@ -333,7 +341,21 @@ class Tftpd(object):
xbu = vfs.flags.get("xbu")
if xbu and not runhook(
self.nlog, xbu, ap, vpath, "", "", 0, 0, "8.3.8.7", 0, ""
self.nlog,
None,
self.hub.up2k,
"xbu.tftpd",
xbu,
ap,
vpath,
"",
"",
"",
0,
0,
"8.3.8.7",
time.time(),
"",
):
yeet("blocked by xbu server config: " + vpath)
@@ -341,7 +363,7 @@ class Tftpd(object):
return self._ls(vpath, "", 0, True)
if not a:
a = [self.args.iobuf]
a = (self.args.iobuf,)
return open(ap, mode, *a, **ka)
@@ -382,7 +404,7 @@ class Tftpd(object):
bos.stat(ap)
return True
except:
return False
return vpath == "/"
def _p_isdir(self, vpath: str) -> bool:
try:
@@ -390,7 +412,7 @@ class Tftpd(object):
ret = stat.S_ISDIR(st.st_mode)
return ret
except:
return False
return vpath == "/"
def _hook(self, *a: Any, **ka: Any) -> None:
src = inspect.currentframe().f_back.f_code.co_name

View File

@@ -57,9 +57,10 @@ class ThumbCli(object):
if is_vid and "dvthumb" in dbv.flags:
return None
want_opus = fmt in ("opus", "caf")
want_opus = fmt in ("opus", "caf", "mp3")
is_au = ext in self.fmt_ffa
if is_au:
is_vau = want_opus and ext in self.fmt_ffv
if is_au or is_vau:
if want_opus:
if self.args.no_acode:
return None
@@ -107,6 +108,11 @@ class ThumbCli(object):
fmt = sfmt
elif fmt[:1] == "p" and not is_au and not is_vid:
t = "cannot thumbnail [%s]: png only allowed for waveforms"
self.log(t % (rem), 6)
return None
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import base64
import hashlib
import logging
import os
@@ -12,22 +11,23 @@ import time
from queue import Queue
from .__init__ import ANYWIN, TYPE_CHECKING
from .__init__ import ANYWIN, PY2, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, au_unpk, ffprobe
from .util import BytesIO # type: ignore
from .util import (
FFMPEG_URL,
Cooldown,
Daemon,
Pebkac,
afsenc,
fsenc,
min_ex,
runcmd,
statdir,
ub64enc,
vsplit,
wrename,
wunlink,
)
@@ -37,6 +37,9 @@ if True: # pylint: disable=using-constant-test
if TYPE_CHECKING:
from .svchub import SvcHub
if PY2:
range = xrange # type: ignore
HAVE_PIL = False
HAVE_PILF = False
HAVE_HEIF = False
@@ -44,22 +47,34 @@ HAVE_AVIF = False
HAVE_WEBP = False
try:
if os.environ.get("PRTY_NO_PIL"):
raise Exception()
from PIL import ExifTags, Image, ImageFont, ImageOps
HAVE_PIL = True
try:
if os.environ.get("PRTY_NO_PILF"):
raise Exception()
ImageFont.load_default(size=16)
HAVE_PILF = True
except:
pass
try:
if os.environ.get("PRTY_NO_PIL_WEBP"):
raise Exception()
Image.new("RGB", (2, 2)).save(BytesIO(), format="webp")
HAVE_WEBP = True
except:
pass
try:
if os.environ.get("PRTY_NO_PIL_HEIF"):
raise Exception()
from pyheif_pillow_opener import register_heif_opener
register_heif_opener()
@@ -68,6 +83,9 @@ try:
pass
try:
if os.environ.get("PRTY_NO_PIL_AVIF"):
raise Exception()
import pillow_avif # noqa: F401 # pylint: disable=unused-import
HAVE_AVIF = True
@@ -79,6 +97,9 @@ except:
pass
try:
if os.environ.get("PRTY_NO_VIPS"):
raise Exception()
HAVE_VIPS = True
import pyvips
@@ -87,6 +108,9 @@ except:
HAVE_VIPS = False
th_dir_cache = {}
def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -> str:
# base16 = 16 = 256
# b64-lc = 38 = 1444
@@ -100,16 +124,22 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
if ext in ffa and fmt[:2] in ("wf", "jf"):
fmt = fmt.replace("f", "")
rd += "\n" + fmt
h = hashlib.sha512(afsenc(rd)).digest()
b64 = base64.urlsafe_b64encode(h).decode("ascii")[:24]
rd = ("%s/%s/" % (b64[:2], b64[2:4])).lower() + b64
dcache = th_dir_cache
rd_key = rd + "\n" + fmt
rd = dcache.get(rd_key)
if not rd:
h = hashlib.sha512(afsenc(rd_key)).digest()
b64 = ub64enc(h).decode("ascii")[:24]
rd = ("%s/%s/" % (b64[:2], b64[2:4])).lower() + b64
if len(dcache) > 9001:
dcache.clear()
dcache[rd_key] = rd
# could keep original filenames but this is safer re pathlen
h = hashlib.sha512(afsenc(fn)).digest()
fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
fn = ub64enc(h).decode("ascii")[:24]
if fmt in ("opus", "caf"):
if fmt in ("opus", "caf", "mp3"):
cat = "ac"
else:
fc = fmt[:1]
@@ -133,6 +163,7 @@ class ThumbSrv(object):
self.ram: dict[str, float] = {}
self.memcond = threading.Condition(self.mutex)
self.stopping = False
self.rm_nullthumbs = True # forget failed conversions on startup
self.nthr = max(1, self.args.th_mt)
self.q: Queue[Optional[tuple[str, str, str, VFS]]] = Queue(self.nthr * 4)
@@ -296,25 +327,38 @@ class ThumbSrv(object):
ext = abspath.split(".")[-1].lower()
png_ok = False
funs = []
if ext in self.args.au_unpk:
ap_unpk = au_unpk(self.log, self.args.au_unpk, abspath, vn)
else:
ap_unpk = abspath
if not bos.path.exists(tpath):
want_mp3 = tpath.endswith(".mp3")
want_opus = tpath.endswith(".opus") or tpath.endswith(".caf")
want_png = tpath.endswith(".png")
want_au = want_mp3 or want_opus
for lib in self.args.th_dec:
can_au = lib == "ff" and (
ext in self.fmt_ffa or ext in self.fmt_ffv
)
if lib == "pil" and ext in self.fmt_pil:
funs.append(self.conv_pil)
elif lib == "vips" and ext in self.fmt_vips:
funs.append(self.conv_vips)
elif lib == "ff" and ext in self.fmt_ffi or ext in self.fmt_ffv:
funs.append(self.conv_ffmpeg)
elif lib == "ff" and ext in self.fmt_ffa:
if tpath.endswith(".opus") or tpath.endswith(".caf"):
elif can_au and (want_png or want_au):
if want_opus:
funs.append(self.conv_opus)
elif tpath.endswith(".png"):
elif want_mp3:
funs.append(self.conv_mp3)
elif want_png:
funs.append(self.conv_waves)
png_ok = True
else:
funs.append(self.conv_spec)
if not png_ok and tpath.endswith(".png"):
raise Pebkac(400, "png only allowed for waveforms")
elif lib == "ff" and (ext in self.fmt_ffi or ext in self.fmt_ffv):
funs.append(self.conv_ffmpeg)
elif lib == "ff" and ext in self.fmt_ffa and not want_au:
funs.append(self.conv_spec)
tdir, tfn = os.path.split(tpath)
ttpath = os.path.join(tdir, "w", tfn)
@@ -325,7 +369,10 @@ class ThumbSrv(object):
for fun in funs:
try:
fun(abspath, ttpath, fmt, vn)
if not png_ok and tpath.endswith(".png"):
raise Exception("png only allowed for waveforms")
fun(ap_unpk, ttpath, fmt, vn)
break
except Exception as ex:
msg = "{} could not create thumbnail of {}\n{}"
@@ -343,8 +390,11 @@ class ThumbSrv(object):
except:
pass
if abspath != ap_unpk:
wunlink(self.log, ap_unpk, vn.flags)
try:
bos.rename(ttpath, tpath)
wrename(self.log, ttpath, tpath, vn.flags)
except:
pass
@@ -438,7 +488,7 @@ class ThumbSrv(object):
if c == crops[-1]:
raise
assert img # type: ignore
assert img # type: ignore # !rm
img.write_to_file(tpath, Q=40)
def conv_ffmpeg(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
@@ -581,6 +631,25 @@ class ThumbSrv(object):
cmd += [fsenc(tpath)]
self._run_ff(cmd, vn)
if "pngquant" in vn.flags:
wtpath = tpath + ".png"
cmd = [
b"pngquant",
b"--strip",
b"--nofs",
b"--output",
fsenc(wtpath),
fsenc(tpath),
]
ret = runcmd(cmd, timeout=vn.flags["convt"], nice=True, oom=400)[0]
if ret:
try:
wunlink(self.log, wtpath, vn.flags)
except:
pass
else:
wrename(self.log, wtpath, tpath, vn.flags)
def conv_spec(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
@@ -637,21 +706,60 @@ class ThumbSrv(object):
cmd += [fsenc(tpath)]
self._run_ff(cmd, vn)
def conv_opus(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
if self.args.no_acode:
def conv_mp3(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
quality = self.args.q_mp3.lower()
if self.args.no_acode or not quality:
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in tags:
raise Exception("not audio")
if quality.endswith("k"):
qk = b"-b:a"
qv = quality.encode("ascii")
else:
qk = b"-q:a"
qv = quality[1:].encode("ascii")
# extremely conservative choices for output format
# (always 2ch 44k1) because if a device is old enough
# to not support opus then it's probably also super picky
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
] + self.big_tags(rawtags) + [
b"-map", b"0:a:0",
b"-ar", b"44100",
b"-ac", b"2",
b"-c:a", b"libmp3lame",
qk, qv,
fsenc(tpath)
]
# fmt: on
self._run_ff(cmd, vn, oom=300)
def conv_opus(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
if self.args.no_acode or not self.args.q_opus:
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in tags:
raise Exception("not audio")
try:
dur = ret[".dur"][1]
dur = tags[".dur"][1]
except:
dur = 0
src_opus = abspath.lower().endswith(".opus") or ret["ac"][1] == "opus"
src_opus = abspath.lower().endswith(".opus") or tags["ac"][1] == "opus"
want_caf = tpath.endswith(".caf")
tmp_opus = tpath
if want_caf:
@@ -662,6 +770,7 @@ class ThumbSrv(object):
pass
caf_src = abspath if src_opus else tmp_opus
bq = ("%dk" % (self.args.q_opus,)).encode("ascii")
if not want_caf or not src_opus:
# fmt: off
@@ -671,10 +780,10 @@ class ThumbSrv(object):
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
b"-map_metadata", b"-1",
] + self.big_tags(rawtags) + [
b"-map", b"0:a:0",
b"-c:a", b"libopus",
b"-b:a", b"128k",
b"-b:a", bq,
fsenc(tmp_opus)
]
# fmt: on
@@ -697,7 +806,7 @@ class ThumbSrv(object):
b"-map_metadata", b"-1",
b"-ac", b"2",
b"-c:a", b"libopus",
b"-b:a", b"128k",
b"-b:a", bq,
b"-f", b"caf",
fsenc(tpath)
]
@@ -728,6 +837,16 @@ class ThumbSrv(object):
except:
pass
def big_tags(self, raw_tags: dict[str, list[str]]) -> list[bytes]:
ret = []
for k, vs in raw_tags.items():
for v in vs:
if len(str(v)) >= 1024:
bv = k.encode("utf-8", "replace")
ret += [b"-metadata", bv + b"="]
break
return ret
def poke(self, tdir: str) -> None:
if not self.poke_cd.poke(tdir):
return
@@ -743,7 +862,6 @@ class ThumbSrv(object):
def cleaner(self) -> None:
interval = self.args.th_clean
while True:
time.sleep(interval)
ndirs = 0
for vol, histpath in self.asrv.vfs.histtab.items():
if histpath.startswith(vol):
@@ -757,6 +875,8 @@ class ThumbSrv(object):
self.log("\033[Jcln err in %s: %r" % (histpath, ex), 3)
self.log("\033[Jcln ok; rm {} dirs".format(ndirs))
self.rm_nullthumbs = False
time.sleep(interval)
def clean(self, histpath: str) -> int:
ret = 0
@@ -771,13 +891,15 @@ class ThumbSrv(object):
def _clean(self, cat: str, thumbpath: str) -> int:
# self.log("cln {}".format(thumbpath))
exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf"]
exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf", "mp3"]
maxage = getattr(self.args, cat + "_maxage")
now = time.time()
prev_b64 = None
prev_fp = ""
try:
t1 = statdir(self.log_func, not self.args.no_scandir, False, thumbpath)
t1 = statdir(
self.log_func, not self.args.no_scandir, False, thumbpath, False
)
ents = sorted(list(t1))
except:
return 0
@@ -818,6 +940,10 @@ class ThumbSrv(object):
continue
if self.rm_nullthumbs and not inf.st_size:
bos.unlink(fp)
continue
if b64 == prev_b64:
self.log("rm replaced [{}]".format(fp))
bos.unlink(prev_fp)

View File

@@ -8,7 +8,7 @@ import threading
import time
from operator import itemgetter
from .__init__ import ANYWIN, TYPE_CHECKING, unicode
from .__init__ import ANYWIN, PY2, TYPE_CHECKING, unicode
from .authsrv import LEELOO_DALLAS, VFS
from .bos import bos
from .up2k import up2k_wark_from_hashlist
@@ -38,6 +38,9 @@ if True: # pylint: disable=using-constant-test
if TYPE_CHECKING:
from .httpsrv import HttpSrv
if PY2:
range = xrange # type: ignore
class U2idx(object):
def __init__(self, hsrv: "HttpSrv") -> None:
@@ -50,18 +53,42 @@ class U2idx(object):
self.log("your python does not have sqlite3; searching will be disabled")
return
assert sqlite3 # type: ignore # !rm
self.active_id = ""
self.active_cur: Optional["sqlite3.Cursor"] = None
self.cur: dict[str, "sqlite3.Cursor"] = {}
self.mem_cur = sqlite3.connect(":memory:", check_same_thread=False).cursor()
self.mem_cur.execute(r"create table a (b text)")
self.sh_cur: Optional["sqlite3.Cursor"] = None
self.p_end = 0.0
self.p_dur = 0.0
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("u2idx", msg, c)
def shutdown(self) -> None:
if not HAVE_SQLITE3:
return
for cur in self.cur.values():
db = cur.connection
try:
db.interrupt()
except:
pass
cur.close()
db.close()
for cur in (self.mem_cur, self.sh_cur):
if cur:
db = cur.connection
cur.close()
db.close()
def fsearch(
self, uname: str, vols: list[VFS], body: dict[str, Any]
) -> list[dict[str, Any]]:
@@ -77,18 +104,36 @@ class U2idx(object):
uv: list[Union[str, int]] = [wark[:16], wark]
try:
return self.run_query(uname, vols, uq, uv, False, 99999)[0]
return self.run_query(uname, vols, uq, uv, False, True, 99999)[0]
except:
raise Pebkac(500, min_ex())
def get_cur(self, ptop: str) -> Optional["sqlite3.Cursor"]:
if not HAVE_SQLITE3:
def get_shr(self) -> Optional["sqlite3.Cursor"]:
if self.sh_cur:
return self.sh_cur
if not HAVE_SQLITE3 or not self.args.shr:
return None
cur = self.cur.get(ptop)
assert sqlite3 # type: ignore # !rm
db = sqlite3.connect(self.args.shr_db, timeout=2, check_same_thread=False)
cur = db.cursor()
cur.execute('pragma table_info("sh")').fetchall()
self.sh_cur = cur
return cur
def get_cur(self, vn: VFS) -> Optional["sqlite3.Cursor"]:
cur = self.cur.get(vn.realpath)
if cur:
return cur
if not HAVE_SQLITE3 or "e2d" not in vn.flags:
return None
assert sqlite3 # type: ignore # !rm
ptop = vn.realpath
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
@@ -265,7 +310,7 @@ class U2idx(object):
q += " lower({}) {} ? ) ".format(field, oper)
try:
return self.run_query(uname, vols, q, va, have_mt, lim)
return self.run_query(uname, vols, q, va, have_mt, True, lim)
except Exception as ex:
raise Pebkac(500, repr(ex))
@@ -276,6 +321,7 @@ class U2idx(object):
uq: str,
uv: list[Union[str, int]],
have_mt: bool,
sort: bool,
lim: int,
) -> tuple[list[dict[str, Any]], list[str], bool]:
if self.args.srch_dbg:
@@ -317,7 +363,7 @@ class U2idx(object):
ptop = vol.realpath
flags = vol.flags
cur = self.get_cur(ptop)
cur = self.get_cur(vol)
if not cur:
continue
@@ -422,7 +468,8 @@ class U2idx(object):
done_flag.append(True)
self.active_id = ""
ret.sort(key=itemgetter("rp"))
if sort:
ret.sort(key=itemgetter("rp"))
return ret, list(taglist.keys()), lim < 0 and not clamped
@@ -433,5 +480,5 @@ class U2idx(object):
return
if identifier == self.active_id:
assert self.active_cur
assert self.active_cur # !rm
self.active_cur.connection.interrupt()

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -29,8 +29,10 @@ window.baguetteBox = (function () {
isOverlayVisible = false,
touch = {}, // start-pos
touchFlag = false, // busy
scrollCSS = ['', ''],
scrollTimer = 0,
re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4|m4v|mov)(\?|$)/i,
anims = ['slideIn', 'fadeIn', 'none'],
data = {}, // all galleries
imagesElements = [],
@@ -91,6 +93,30 @@ window.baguetteBox = (function () {
touchendHandler();
};
var overlayWheelHandler = function (e) {
if (!options.noScrollbars || anymod(e))
return;
ev(e);
var x = e.deltaX,
y = e.deltaY,
d = Math.abs(x) > Math.abs(y) ? x : y;
if (e.deltaMode)
d *= 10;
if (Date.now() - scrollTimer < (Math.abs(d) > 20 ? 100 : 300))
return;
scrollTimer = Date.now();
if (d > 0)
showNextImage();
else
showPreviousImage();
};
var trapFocusInsideOverlay = function (e) {
if (overlay.style.display === 'block' && (overlay.contains && !overlay.contains(e.target))) {
e.stopPropagation();
@@ -394,8 +420,7 @@ window.baguetteBox = (function () {
}
function dlpic() {
var url = findfile()[3].href;
url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache';
var url = addq(findfile()[3].href, 'cache');
dl_file(url);
}
@@ -452,6 +477,7 @@ window.baguetteBox = (function () {
bind(document, 'keyup', keyUpHandler);
bind(document, 'fullscreenchange', onFSC);
bind(overlay, 'click', overlayClickHandler);
bind(overlay, 'wheel', overlayWheelHandler);
bind(btnPrev, 'click', showPreviousImage);
bind(btnNext, 'click', showNextImage);
bind(btnClose, 'click', hideOverlay);
@@ -474,6 +500,7 @@ window.baguetteBox = (function () {
unbind(document, 'keyup', keyUpHandler);
unbind(document, 'fullscreenchange', onFSC);
unbind(overlay, 'click', overlayClickHandler);
unbind(overlay, 'wheel', overlayWheelHandler);
unbind(btnPrev, 'click', showPreviousImage);
unbind(btnNext, 'click', showNextImage);
unbind(btnClose, 'click', hideOverlay);
@@ -541,6 +568,12 @@ window.baguetteBox = (function () {
function showOverlay(chosenImageIndex) {
if (options.noScrollbars) {
var a = document.documentElement.style.overflowY,
b = document.body.style.overflowY;
if (a != 'hidden' || b != 'scroll')
scrollCSS = [a, b];
document.documentElement.style.overflowY = 'hidden';
document.body.style.overflowY = 'scroll';
}
@@ -584,15 +617,23 @@ window.baguetteBox = (function () {
isOverlayVisible = true;
}
function hideOverlay(e) {
function hideOverlay(e, dtor) {
ev(e);
playvid(false);
removeFromCache('#files');
if (options.noScrollbars) {
document.documentElement.style.overflowY = 'auto';
document.body.style.overflowY = 'auto';
document.documentElement.style.overflowY = scrollCSS[0];
document.body.style.overflowY = scrollCSS[1];
}
if (overlay.style.display === 'none')
try {
if (document.fullscreenElement)
document.exitFullscreen();
}
catch (ex) { }
isFullscreen = false;
if (dtor || overlay.style.display === 'none')
return;
if (options.duringHide)
@@ -600,11 +641,6 @@ window.baguetteBox = (function () {
sethash('');
unbindEvents();
try {
document.exitFullscreen();
isFullscreen = false;
}
catch (ex) { }
// Fade out and hide the overlay
overlay.className = '';
@@ -682,7 +718,7 @@ window.baguetteBox = (function () {
options.captions.call(currentGallery, imageElement) :
imageElement.getAttribute('data-caption') || imageElement.title;
imageSrc += imageSrc.indexOf('?') < 0 ? '?cache' : '&cache';
imageSrc = addq(imageSrc, 'cache');
if (is_vid && index != currentIndex)
return; // no preload
@@ -711,8 +747,11 @@ window.baguetteBox = (function () {
});
image.setAttribute('src', imageSrc);
if (is_vid) {
image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
image.setAttribute('controls', 'controls');
image.onended = vidEnd;
image.onplay = function () { show_buttons(1); };
image.onpause = function () { show_buttons(); };
}
image.alt = thumbnailElement ? thumbnailElement.alt || '' : '';
if (options.titleTag && imageCaption)
@@ -720,6 +759,9 @@ window.baguetteBox = (function () {
figure.appendChild(image);
if (is_vid && window.afilt)
afilt.apply(undefined, image);
if (options.async && callback)
callback();
}
@@ -955,6 +997,12 @@ window.baguetteBox = (function () {
}
}
function show_buttons(v) {
clmod(ebi('bbox-btns'), 'off', v);
clmod(btnPrev, 'off', v);
clmod(btnNext, 'off', v);
}
function bounceAnimation(direction) {
slider.className = options.animation == 'slideIn' ? 'bounce-from-' + direction : 'eog';
setTimeout(function () {
@@ -1018,9 +1066,7 @@ window.baguetteBox = (function () {
if (fx > 0.7)
return showNextImage();
clmod(ebi('bbox-btns'), 'off', 't');
clmod(btnPrev, 'off', 't');
clmod(btnNext, 'off', 't');
show_buttons('t');
if (Date.now() - ctime <= 500 && !IPHONE)
tglfull();
@@ -1062,6 +1108,7 @@ window.baguetteBox = (function () {
}
function destroyPlugin() {
hideOverlay(undefined, true);
unbindEvents();
clearCachedData();
document.getElementsByTagName('body')[0].removeChild(ebi('bbox-overlay'));

View File

@@ -10,7 +10,6 @@
--fg2-max: #fff;
--fg-weak: #bbb;
--bg-u7: #555;
--bg-u6: #4c4c4c;
--bg-u5: #444;
--bg-u4: #383838;
@@ -28,6 +27,8 @@
--row-alt: #282828;
--scroll: #eb0;
--sel-fg: var(--bg-d1);
--sel-bg: var(--fg);
--a: #fc5;
--a-b: #c90;
@@ -41,8 +42,14 @@
--btn-h-bg: #805;
--btn-1-fg: #400;
--btn-1-bg: var(--a);
--btn-h-bs: var(--btn-bs);
--btn-h-bb: var(--btn-bb);
--btn-1-bs: var(--btn-bs);
--btn-1-bb: var(--btn-bb);
--btn-1h-fg: var(--btn-1-fg);
--btn-1h-bg: #fe8;
--btn-1h-bs: var(--btn-1-bs);
--btn-1h-bb: var(--btn-1-bb);
--chk-fg: var(--tab-alt);
--txt-sh: var(--bg-d2);
--txt-bg: var(--btn-bg);
@@ -57,7 +64,7 @@
--u2-tab-bg: linear-gradient(to bottom, var(--bg), var(--bg-u1));
--u2-tab-b1: rgba(128,128,128,0.8);
--u2-tab-1-fg: #fd7;
--u2-tab-1-bg: linear-gradient(to bottom, var(#353), var(--bg) 80%);
--u2-tab-1-bg: linear-gradient(to bottom, #353, var(--bg) 80%);
--u2-tab-1-b1: #7c5;
--u2-tab-1-b2: #583;
--u2-tab-1-sh: #280;
@@ -181,7 +188,6 @@ html.y {
--srv-1: #555;
--srv-2: #c83;
--srv-3: #c0a;
--srv-3b: rgba(255,68,204,0.6);
--tree-bg: #fff;
@@ -210,22 +216,19 @@ html.y {
html.a {
--op-aa-sh: 0 0 .2em var(--bg-d3) inset;
--u2-o-bg: #603;
--u2-o-b1: #a16;
--u2-o-sh: #a00;
--u2-o-h-bg: var(--u2-o-bg);
--u2-o-h-b1: #fb0;
--u2-o-h-sh: #fb0;
--u2-o-1-bg: #6a1;
--u2-o-1-b1: #efa;
--u2-o-1-sh: #0c0;
--u2-o-1h-bg: var(--u2-o-1-bg);
--btn-bs: 0 0 .2em var(--bg-d3);
}
html.az {
--btn-1-bs: 0 0 .1em var(--fg) inset;
}
html.ay {
--op-aa-sh: 0 .1em .2em #ccc;
--op-aa-bg: var(--bg-max);
}
html.b {
--btn-bs: 0 .05em 0 var(--bg-d3) inset;
--btn-1-bs: 0 .05em 0 var(--btn-1h-bg) inset;
--tree-bg: var(--bg);
--g-bg: var(--bg);
@@ -242,17 +245,13 @@ html.b {
--u2-b1-bg: rgba(128,128,128,0.15);
--u2-b2-bg: var(--u2-b1-bg);
--u2-o-bg: var(--btn-bg);
--u2-o-h-bg: var(--btn-h-bg);
--u2-o-1-bg: var(--a);
--u2-o-1h-bg: var(--a-hil);
--f-sh1: 0.1;
--mp-b-bg: transparent;
}
html.bz {
--fg: #cce;
--fg-weak: #bbd;
--bg-u5: #3b3f58;
--bg-u4: #1e2130;
--bg-u3: #1e2130;
@@ -264,11 +263,14 @@ html.bz {
--row-alt: #181a27;
--a-b: #fb4;
--btn-bg: #202231;
--btn-h-bg: #2d2f45;
--btn-1-bg: #ba2959;
--btn-1-fg: #fff;
--btn-1-bg: #eb6;
--btn-1-fg: #000;
--btn-1h-fg: #000;
--btn-1h-bg: #ff9;
--txt-sh: a;
--u2-tab-b1: var(--bg-u5);
@@ -283,6 +285,7 @@ html.bz {
--f-h-b1: #34384e;
--mp-sh: #11121d;
/*--mp-b-bg: #2c3044;*/
--f-play-bg: var(--btn-1-bg);
}
html.by {
--bg: #f2f2f2;
@@ -303,6 +306,7 @@ html.by {
}
html.c {
font-weight: bold;
--fg: #fff;
--fg-weak: #cef;
--bg-u5: #409;
@@ -323,15 +327,25 @@ html.c {
--chk-fg: #d90;
--op-aa-bg: #f9dd22;
--u2-o-1-bg: #4cf;
--srv-1: #ea0;
--mp-b-bg: transparent;
}
html.cz {
--bgg: var(--bg-u2);
--sel-bg: var(--bg-u5);
--sel-fg: var(--fg);
--btn-bb: .2em solid #709;
--btn-bs: 0 .1em .6em rgba(255,0,185,0.5);
--btn-1-bb: .2em solid #e90;
--btn-1-bs: 0 .1em .8em rgba(255,205,0,0.9);
--srv-3: #fff;
--u2-tab-b1: var(--bg-d3);
--u2-tab-1-bg: a;
}
html.cy {
--fg: #fff;
@@ -343,6 +357,8 @@ html.cy {
--bg-d3: #f77;
--bg-d2: #ff0;
--sel-bg: #f77;
--a: #fff;
--a-hil: #fff;
--a-h-bg: #000;
@@ -356,24 +372,25 @@ html.cy {
--btn-h-fg: #fff;
--btn-1-bg: #ff0;
--btn-1-fg: #000;
--btn-bs: 0 .25em 0 #f00;
--chk-fg: #fd0;
--txt-bg: #000;
--srv-1: #f00;
--srv-3: #fff;
--op-aa-bg: #fff;
--u2-b1-bg: #f00;
--u2-b2-bg: #f00;
--u2-o-bg: #ff0;
--u2-o-1-bg: #f00;
--g-sel-fg: #fff;
--g-sel-bg: #aaa;
--g-fsel-bg: #aaa;
}
html.dz {
--fg: #4d4;
--fg-max: #fff;
--fg2-max: #fff;
--fg-weak: #2a2;
--bg-u7: #020;
--bg-u6: #020;
--bg-u5: #050;
--bg-u4: #020;
@@ -381,11 +398,9 @@ html.dz {
--bg-u2: #020;
--bg-u1: #020;
--bg: #010;
--bgg: var(--bg);
--bg-d1: #000;
--bg-d2: #020;
--bg-d3: #000;
--bg-max: #000;
--tab-alt: #6f6;
--row-alt: #030;
@@ -398,48 +413,21 @@ html.dz {
--a-dark: #afa;
--a-gray: #2a2;
--btn-fg: var(--a);
--btn-bg: rgba(64,128,64,0.15);
--btn-h-fg: var(--a-hil);
--btn-h-bg: #050;
--btn-1-fg: #000;
--btn-1-bg: #4f4;
--btn-1h-fg: var(--btn-1-fg);
--btn-1h-bg: #3f3;
--chk-fg: var(--tab-alt);
--txt-sh: var(--bg-d2);
--txt-bg: var(--btn-bg);
--op-aa-fg: var(--a);
--op-aa-bg: var(--bg-d2);
--op-a-sh: rgba(0,0,0,0.5);
--btn-bs: 0 0 0 .1em #080 inset;
--btn-1-bs: a;
--u2-btn-b1: var(--fg-weak);
--u2-sbtn-b1: var(--fg-weak);
--u2-txt-bg: var(--bg-u5);
--u2-tab-bg: linear-gradient(to bottom, var(--bg), var(--bg-u1));
--u2-tab-b1: var(--fg-weak);
--u2-tab-1-fg: #fff;
--u2-tab-1-bg: linear-gradient(to bottom, var(#353), var(--bg) 80%);
--u2-tab-1-b1: #7c5;
--u2-tab-1-b2: #583;
--u2-tab-1-sh: #280;
--u2-b-fg: #fff;
--u2-tab-1-bg: linear-gradient(to bottom, #151, var(--bg) 80%);
--u2-b1-bg: #3a3;
--u2-b2-bg: #3a3;
--u2-o-bg: var(--btn-bg);
--u2-o-b1: var(--bg-u5);
--u2-o-h-bg: var(--fg-weak);
--u2-o-1-bg: var(--fg-weak);
--u2-o-1-b1: var(--a);
--u2-o-1h-bg: var(--a);
--u2-inf-bg: #07a;
--u2-inf-b1: #0be;
--u2-ok-bg: #380;
--u2-ok-b1: #8e4;
--u2-err-bg: #900;
--u2-err-b1: #d06;
--ud-b1: #888;
--sort-1: #fff;
--sort-2: #3f3;
@@ -451,47 +439,12 @@ html.dz {
--tree-bg: #010;
--g-play-bg: #750;
--g-play-b1: #c90;
--g-play-b2: #da4;
--g-play-sh: #b83;
--g-sel-fg: #fff;
--g-sel-bg: #925;
--g-sel-b1: #c37;
--g-sel-sh: #b36;
--g-fsel-bg: #d39;
--g-fsel-b1: #d48;
--g-fsel-ts: #804;
--g-fg: var(--a-hil);
--g-bg: var(--bg-u2);
--g-b1: var(--bg-u4);
--g-b2: var(--bg-u5);
--g-g1: var(--bg-u2);
--g-g2: var(--bg-u5);
--g-f-bg: var(--bg-u4);
--g-f-b1: var(--bg-u5);
--g-f-fg: var(--a-hil);
--g-sh: rgba(0,0,0,0.3);
--f-sh1: 0.33;
--f-sh2: 0.02;
--f-sh3: 0.2;
--f-h-b1: #3b3;
--f-play-bg: #fc5;
--f-play-fg: #000;
--f-sel-sh: #fc0;
--f-gray: #999;
--fm-off: #f6c;
--mp-sh: var(--bg-d3);
--err-fg: #fff;
--err-bg: #a20;
--err-b1: #f00;
--err-ts: #500;
text-shadow: none;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
@@ -544,10 +497,6 @@ html.dy {
--u2-tab-1-bg: a;
--u2-b1-bg: #000;
--u2-b2-bg: #000;
--u2-o-h-bg: #999;
--u2-o-1h-bg: #999;
--u2-o-bg: #eee;
--u2-o-1-bg: #000;
--ud-b1: a;
@@ -588,11 +537,11 @@ html.dy {
line-height: 1.2em;
}
::selection {
color: var(--bg-d1);
background: var(--fg);
color: var(--sel-fg);
background: var(--sel-bg);
text-shadow: none;
}
html,body,tr,th,td,#files,a {
html,body,tr,th,td,#files,a,#blogout {
color: inherit;
background: none;
font-weight: inherit;
@@ -620,6 +569,7 @@ pre, code, tt, #doc, #doc>code {
overflow: hidden;
width: 0;
height: 0;
color: var(--bg);
}
html .ayjump:focus {
z-index: 80386;
@@ -674,11 +624,15 @@ html.y #path {
#files tbody div a {
color: var(--tab-alt);
}
a, #files tbody div a:last-child {
a, #blogout, #files tbody div a:last-child {
color: var(--a);
padding: .2em;
text-decoration: none;
}
#blogout {
margin: -.2em;
}
#blogout:hover,
a:hover {
color: var(--a-hil);
background: var(--a-h-bg);
@@ -699,12 +653,12 @@ a:hover {
.s0:after,
.s1:after {
content: '⌄';
margin-left: -.1em;
margin-left: -.15em;
}
.s0r:after,
.s1r:after {
content: '⌃';
margin-left: -.1em;
margin-left: -.15em;
}
.s0:after,
.s0r:after {
@@ -715,7 +669,7 @@ a:hover {
color: var(--sort-2);
}
#files thead th:after {
margin-right: -.7em;
margin-right: -.5em;
}
#files tbody tr:hover td,
#files tbody tr:hover td+td {
@@ -744,6 +698,15 @@ html #files.hhpick thead th {
word-wrap: break-word;
overflow: hidden;
}
#files tr.fade a {
color: #999;
color: rgba(255, 255, 255, 0.4);
font-style: italic;
}
html.y #files tr.fade a {
color: #999;
color: rgba(0, 0, 0, 0.4);
}
#files tr:nth-child(2n) td {
background: var(--row-alt);
}
@@ -870,7 +833,7 @@ html.y #path a:hover {
max-width: 52em;
}
.mdo.sb,
#epi.logue.mdo>iframe {
.logue.mdo>iframe {
max-width: 54em;
}
.mdo,
@@ -913,6 +876,9 @@ html.y #path a:hover {
color: var(--srv-3);
border-bottom: 1px solid var(--srv-3b);
}
#flogout {
display: inline;
}
#goh+span {
color: var(--bg-u5);
padding-left: .5em;
@@ -947,6 +913,8 @@ html.y #path a:hover {
#files tbody tr.play a:hover {
color: var(--btn-1h-fg);
background: var(--btn-1h-bg);
box-shadow: var(--btn-1h-bs);
border-bottom: var(--btn-1h-bb);
}
#ggrid {
margin: -.2em -.5em;
@@ -955,6 +923,7 @@ html.y #path a:hover {
overflow: hidden;
display: block;
display: -webkit-box;
line-clamp: var(--grid-ln);
-webkit-line-clamp: var(--grid-ln);
-webkit-box-orient: vertical;
padding-top: .3em;
@@ -1001,9 +970,6 @@ html.y #path a:hover {
color: var(--g-dfg);
}
#ggrid>a.au:before {
content: '💾';
}
html.np_open #ggrid>a.au:before {
content: '▶';
}
#ggrid>a:before {
@@ -1132,6 +1098,7 @@ html.y #widget.open {
width: 100%;
height: 100%;
}
#fshr,
#wtgrid,
#wtico {
position: relative;
@@ -1318,6 +1285,7 @@ html.y #widget.open {
#widget.cmp #wtoggle {
font-size: 1.2em;
}
#widget.cmp #fshr,
#widget.cmp #wtgrid {
display: none;
}
@@ -1331,6 +1299,7 @@ html.y #widget.open {
}
#widget.cmp #barpos,
#widget.cmp #barbuf {
height: 1.6em;
width: calc(100% - 11em);
border-radius: 0;
left: 5em;
@@ -1418,7 +1387,11 @@ input[type="checkbox"]+label {
input[type="radio"]:checked+label,
input[type="checkbox"]:checked+label {
color: #0e0;
color: var(--a);
color: var(--btn-1-bg);
}
input[type="checkbox"]:checked+label {
box-shadow: var(--btn-1-bs);
border-bottom: var(--btn-1-bb);
}
html.dz input {
font-family: 'scp', monospace, monospace;
@@ -1596,10 +1569,13 @@ html {
color: var(--btn-fg);
background: #eee;
background: var(--btn-bg);
box-shadow: var(--btn-bs);
border-bottom: var(--btn-bb);
border-radius: .3em;
padding: .2em .4em;
font-size: 1.2em;
margin: .2em;
display: inline-block;
white-space: pre;
position: relative;
top: -.12em;
@@ -1608,20 +1584,14 @@ html.c .btn,
html.a .btn {
border-radius: .2em;
}
html.cz .btn {
box-shadow: 0 .1em .6em rgba(255,0,185,0.5);
border-bottom: .2em solid #709;
}
html.dz .btn {
font-size: 1em;
box-shadow: 0 0 0 .1em #080 inset;
}
html.dz .tgl.btn.on {
box-shadow: 0 0 0 .1em var(--btn-1-bg) inset;
}
.btn:hover {
color: var(--btn-h-fg);
background: var(--btn-h-bg);
box-shadow: var(--btn-h-bs);
border-bottom: var(--btn-h-bb);
}
.tgl.btn.on {
background: #000;
@@ -1629,14 +1599,14 @@ html.dz .tgl.btn.on {
color: #fff;
color: var(--btn-1-fg);
text-shadow: none;
}
html.cz .tgl.btn.on {
box-shadow: 0 .1em .8em rgba(255,205,0,0.9);
border-bottom: .2em solid #e90;
box-shadow: var(--btn-1-bs);
border-bottom: var(--btn-1-bb);
}
.tgl.btn.on:hover {
background: var(--btn-1h-bg);
color: var(--btn-1h-fg);
background: var(--btn-1h-bg);
box-shadow: var(--btn-1h-bs);
border-bottom: var(--btn-1h-bb);
}
#detree {
padding: .3em .5em;
@@ -1677,6 +1647,18 @@ html.cz .tgl.btn.on {
background: var(--btn-1-bg);
text-shadow: none;
}
#tree ul a.ld::before {
font-weight: bold;
font-family: sans-serif;
display: inline-block;
text-align: center;
width: 1em;
margin: 0 .3em 0 -1.3em;
color: var(--fg-max);
opacity: 0;
content: '◠';
animation: .5s linear infinite forwards spin, ease .25s 1 forwards fadein;
}
#tree ul a.par {
color: var(--fg-max);
}
@@ -1736,6 +1718,7 @@ html.y #tree.nowrap .ntree a+a:hover {
}
#files th span {
position: relative;
white-space: nowrap;
}
#files>thead>tr>th.min,
#files td.min {
@@ -1773,9 +1756,6 @@ html.y #tree.nowrap .ntree a+a:hover {
margin: .7em 0 .7em .5em;
padding-left: .5em;
}
.opwide>div>div>a {
line-height: 2em;
}
.opwide>div>h3 {
color: var(--fg-weak);
margin: 0 .4em;
@@ -1843,6 +1823,7 @@ html.y #tree.nowrap .ntree a+a:hover {
#unpost td:nth-child(4) {
text-align: right;
}
#shui,
#rui {
background: #fff;
background: var(--bg);
@@ -1858,13 +1839,25 @@ html.y #tree.nowrap .ntree a+a:hover {
padding: 1em;
z-index: 765;
}
#shui div+div,
#rui div+div {
margin-top: 1em;
}
#shui table,
#rui table {
width: 100%;
border-collapse: collapse;
}
#shui button {
margin: 0 1em 0 0;
}
#shui .btn {
font-size: 1em;
}
#shui td {
padding: .8em 0;
}
#shui td+td,
#rui td+td {
padding: .2em 0 .2em .5em;
}
@@ -1872,21 +1865,25 @@ html.y #tree.nowrap .ntree a+a:hover {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
#shui td+td,
#rui td+td,
#shui td input[type="text"],
#rui td input[type="text"] {
width: 100%;
}
#shui td.exs input[type="text"] {
width: 3em;
}
#rn_f.m td:first-child {
white-space: nowrap;
}
#rn_f.m td+td {
width: 50%;
}
#rn_f .err td {
background: var(--err-bg);
color: var(--fg-max);
}
#rn_f .err input[readonly] {
#rn_f .err td,
#rn_f .err input[readonly],
#rui .ng input[readonly] {
color: var(--err-fg);
background: var(--err-bg);
}
#rui input[readonly] {
@@ -2670,23 +2667,25 @@ html.b #u2conf a.b:hover {
#u2conf input[type="checkbox"]:checked+label {
position: relative;
cursor: pointer;
background: var(--u2-o-bg);
border-bottom: .2em solid var(--u2-o-b1);
box-shadow: 0 .1em .3em var(--u2-o-sh) inset;
background: var(--btn-bg);
box-shadow: var(--btn-bs);
border-bottom: var(--btn-bb);
text-shadow: 1px 1px 1px #000, 1px -1px 1px #000, -1px -1px 1px #000, -1px 1px 1px #000;
}
#u2conf input[type="checkbox"]:checked+label {
background: var(--u2-o-1-bg);
border-bottom: .2em solid var(--u2-o-1-b1);
box-shadow: 0 .1em .5em var(--u2-o-1-sh);
background: var(--btn-1-bg);
box-shadow: var(--btn-1-bs);
border-bottom: var(--btn-1-bb);
}
#u2conf input[type="checkbox"]+label:hover {
box-shadow: 0 .1em .3em var(--u2-o-h-sh);
border-color: var(--u2-o-h-b1);
background: var(--u2-o-h-bg);
background: var(--btn-h-bg);
box-shadow: var(--btn-h-bs);
border-bottom: var(--btn-h-bb);
}
#u2conf input[type="checkbox"]:checked+label:hover {
background: var(--u2-o-1h-bg);
background: var(--btn-1h-bg);
box-shadow: var(--btn-1h-bs);
border-bottom: var(--btn-1h-bb);
}
#op_up2k.srch #u2conf td:nth-child(2)>*,
#op_up2k.srch #u2conf td:nth-child(3)>* {
@@ -3075,18 +3074,30 @@ html.by #u2cards a.act {
html.cy #wrap {
color: #000;
}
html.cy .mdo a {
background: #f00;
}
html.cy #wrap,
html.cy #acc_info a,
html.cy #op_up2k,
html.cy #files,
html.cy #files a,
html.cy #files tbody div a:last-child {
color: #000;
}
html.cy #u2tab a,
html.cy #u2cards a {
color: #f00;
}
html.cy #unpost a {
color: #ff0;
}
html.cy #barbuf {
filter: hue-rotate(267deg) brightness(0.8) contrast(4);
}
html.cy #pvol {
filter: hue-rotate(4deg) contrast(2.2);
}

View File

@@ -6,7 +6,7 @@
<title>{{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
{{ html_head }}
@@ -67,14 +67,14 @@
<div id="op_up2k" class="opview"></div>
<div id="op_cfg" class="opview opbox opwide"></div>
<h1 id="path">
<a href="#" id="entree">🌲</a>
{%- for n in vpnodes %}
<a href="{{ r }}/{{ n[0] }}">{{ n[1] }}</a>
{%- endfor %}
</h1>
<div id="tree"></div>
<div id="wrap">
@@ -108,21 +108,18 @@
{%- for f in files %}
<tr><td>{{ f.lead }}</td><td><a href="{{ f.href }}">{{ f.name|e }}</a></td><td>{{ f.sz }}</td>
{%- if f.tags is defined %}
{%- for k in taglist %}
<td>{{ f.tags[k] }}</td>
{%- endfor %}
{%- endif %}
<td>{{ f.ext }}</td><td>{{ f.dt }}</td></tr>
{%- if f.tags is defined %}
{%- for k in taglist %}<td>{{ f.tags[k] }}</td>{%- endfor %}
{%- endif %}<td>{{ f.ext }}</td><td>{{ f.dt }}</td></tr>
{%- endfor %}
</tbody>
</table>
<div id="epi" class="logue">{{ "" if sb_lg else logues[1] }}</div>
<h2 id="wfp"><a href="{{ r }}/?h" id="goh">control-panel</a></h2>
<a href="#" id="repl">π</a>
</div>
@@ -135,16 +132,15 @@
<script>
var SR = {{ r|tojson }},
CGV1 = {{ cgv1 }},
CGV = {{ cgv|tojson }},
TS = "{{ ts }}",
dtheme = "{{ dtheme }}",
srvinf = "{{ srv_info }}",
s_name = "{{ s_name }}",
lang = "{{ lang }}",
dfavico = "{{ favico }}",
have_tags_idx = {{ have_tags_idx|tojson }},
have_tags_idx = {{ have_tags_idx }},
sb_lg = "{{ sb_lg }}",
txt_ext = "{{ txt_ext }}",
logues = {{ logues|tojson if sb_lg else "[]" }},
ls0 = {{ ls0|tojson }};

File diff suppressed because it is too large Load Diff

View File

@@ -11,7 +11,6 @@
td{border:1px solid #999;border-width:1px 1px 0 0;padding:0 5px}
a{display:block}
</style>
{{ html_head }}
</head>
<body>
@@ -52,11 +51,11 @@
</tbody>
</table>
{%- if logues[1] %}
<div>{{ logues[1] }}</div><br />
{%- endif %}
<h2><a href="{{ r }}/{{ url_suf }}{{ url_suf and '&amp;' or '?' }}h">control-panel</a></h2>
</body>

BIN
copyparty/web/iiam.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 B

View File

@@ -3,7 +3,7 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
{%- if edit %}
@@ -49,7 +49,7 @@
<div id="mp" class="mdo"></div>
</div>
<a href="#" id="repl">π</a>
{%- if edit %}
<div id="helpbox">
<textarea autocomplete="off">
@@ -125,7 +125,7 @@ write markdown (most html is 🙆 too)
</textarea>
</div>
{%- endif %}
<script>
var SR = {{ r|tojson }},
@@ -159,5 +159,8 @@ try { l.light = drk? 0:1; } catch (ex) { }
{%- if edit %}
<script src="{{ r }}/.cpr/md2.js?_={{ ts }}"></script>
{%- endif %}
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body></html>

View File

@@ -607,10 +607,10 @@ function md_newline() {
var s = linebounds(true),
ln = s.md.substring(s.n1, s.n2),
m1 = /^( *)([0-9]+)(\. +)/.exec(ln),
m2 = /^[ \t>+-]*(\* )?/.exec(ln),
m2 = /^[ \t]*[>+*-]{0,2}[ \t]/.exec(ln),
drop = dom_src.selectionEnd - dom_src.selectionStart;
var pre = m2[0];
var pre = m2 ? m2[0] : '';
if (m1 !== null)
pre = m1[1] + (parseInt(m1[2]) + 1) + m1[3];

View File

@@ -3,7 +3,7 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/mde.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/deps/mini-fa.css?_={{ ts }}">
@@ -53,5 +53,8 @@ try { l.light = drk? 0:1; } catch (ex) { }
<script src="{{ r }}/.cpr/deps/marked.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/deps/easymde.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/mde.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body></html>

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
{{ html_head }}
</head>
@@ -46,6 +46,9 @@
}, 1000);
</script>
{%- endif %}
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

83
copyparty/web/shares.css Normal file
View File

@@ -0,0 +1,83 @@
html {
color: #333;
background: #f7f7f7;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
touch-action: manipulation;
}
#wrap {
margin: 2em auto;
padding: 0 1em 3em 1em;
line-height: 2.3em;
}
#wrap>span {
margin: 0 0 0 1em;
border-bottom: 1px solid #999;
}
li {
margin: 1em 0;
}
a {
color: #047;
background: #fff;
text-decoration: none;
white-space: nowrap;
border-bottom: 1px solid #8ab;
border-radius: .2em;
padding: .2em .6em;
margin: 0 .3em;
}
#wrap td a {
margin: 0;
}
#w {
color: #fff;
background: #940;
border-color: #b70;
}
#repl {
border: none;
background: none;
color: inherit;
padding: 0;
position: fixed;
bottom: .25em;
left: .2em;
}
table {
border-collapse: collapse;
position: relative;
}
th {
top: -1px;
position: sticky;
background: #f7f7f7;
}
#wrap td,
#wrap th {
padding: .3em .6em;
text-align: left;
white-space: nowrap;
}
#wrap td+td+td+td+td+td+td+td {
font-family: var(--font-mono), monospace, monospace;
}
html.z {
background: #222;
color: #ccc;
}
html.z a {
color: #fff;
background: #057;
border-color: #37a;
}
html.z th {
background: #222;
}
html.bz {
color: #bbd;
background: #11121d;
}

79
copyparty/web/shares.html Normal file
View File

@@ -0,0 +1,79 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/shares.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
</head>
<body>
<div id="wrap">
<a id="a" href="{{ r }}/?shares" class="af">refresh</a>
<a id="a" href="{{ r }}/?h" class="af">control-panel</a>
<span>axs = perms (read,write,move,delet)</span>
<span>nf = numFiles (0=dir)</span>
<span>min/hrs = time left</span>
<table id="tab"><thead><tr>
<th>sharekey</th>
<th>delete</th>
<th>pw</th>
<th>source</th>
<th>axs</th>
<th>nf</th>
<th>user</th>
<th>created</th>
<th>expires</th>
<th>min</th>
<th>hrs</th>
<th>add time</th>
</tr></thead><tbody>
{% for k, pw, vp, pr, st, un, t0, t1 in rows %}
<tr>
<td>
<a href="{{ r }}{{ shr }}{{ k }}?qr">qr</a>
<a href="{{ r }}{{ shr }}{{ k }}">{{ k }}</a>
</td>
<td><a href="#" k="{{ k }}">delete</a></td>
<td>{{ "yes" if pw else "--" }}</td>
<td><a href="{{ r }}/{{ vp|e }}">/{{ vp|e }}</a></td>
<td>{{ pr }}</td>
<td>{{ st }}</td>
<td>{{ un|e }}</td>
<td>{{ t0 }}</td>
<td>{{ t1 }}</td>
<td>{{ "inf" if not t1 else "dead" if t1 < now else ((t1 - now) / 60) | round(1) }}</td>
<td>{{ "inf" if not t1 else "dead" if t1 < now else ((t1 - now) / 3600) | round(1) }}</td>
<td></td>
</tr>
{% endfor %}
</tbody></table>
{% if not rows %}
(you don't have any active shares btw)
{% endif %}
<script>
var SR = {{ r|tojson }},
shr="{{ shr }}",
lang="{{ lang }}",
dfavico="{{ favico }}";
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/shares.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

78
copyparty/web/shares.js Normal file
View File

@@ -0,0 +1,78 @@
var t = QSA('a[k]');
for (var a = 0; a < t.length; a++)
t[a].onclick = rm;
function rm() {
var u = SR + shr + uricom_enc(this.getAttribute('k')) + '?eshare=rm',
xhr = new XHR();
xhr.open('POST', u, true);
xhr.onload = xhr.onerror = cb;
xhr.send();
}
function bump() {
var k = this.closest('tr').getElementsByTagName('a')[2].getAttribute('k'),
u = SR + shr + uricom_enc(k) + '?eshare=' + this.value,
xhr = new XHR();
xhr.open('POST', u, true);
xhr.onload = xhr.onerror = cb;
xhr.send();
}
function cb() {
if (this.status !== 200)
return modal.alert('<h6>server error</h6>' + esc(unpre(this.responseText)));
document.location = '?shares';
}
function qr(e) {
ev(e);
var href = this.href,
pw = this.closest('tr').cells[2].textContent;
if (pw.indexOf('yes') < 0)
return showqr(href);
modal.prompt("if you want to bypass the password protection by\nembedding the password into the qr-code, then\ntype the password now, otherwise leave this empty", "", function (v) {
if (v)
href += "&pw=" + v;
showqr(href);
});
}
function showqr(href) {
var vhref = href.replace('?qr&', '?').replace('?qr', '');
modal.alert(esc(vhref) + '<img class="b64" width="100" height="100" src="' + href + '" />');
}
(function() {
var tab = ebi('tab').tBodies[0],
tr = Array.prototype.slice.call(tab.rows, 0);
var buf = [];
for (var a = 0; a < tr.length; a++) {
tr[a].cells[0].getElementsByTagName('a')[0].onclick = qr;
for (var b = 7; b < 9; b++)
buf.push(parseInt(tr[a].cells[b].innerHTML));
}
var ibuf = 0;
for (var a = 0; a < tr.length; a++)
for (var b = 7; b < 9; b++) {
var v = buf[ibuf++];
tr[a].cells[b].innerHTML =
v ? unix2iso(v).replace(' ', ',&nbsp;') : 'never';
}
for (var a = 0; a < tr.length; a++)
tr[a].cells[11].innerHTML =
'<button value="1">1min</button> ' +
'<button value="60">1h</button>';
var btns = QSA('td button'), aa = btns.length;
for (var a = 0; a < aa; a++)
btns[a].onclick = bump;
})();

View File

@@ -53,7 +53,7 @@ a.r {
border-color: #c7a;
}
a.g {
color: #2b0;
color: #0a0;
border-color: #3a0;
box-shadow: 0 .3em 1em #4c0;
}
@@ -90,6 +90,13 @@ table {
text-align: left;
white-space: nowrap;
}
.vols td:empty,
.vols th:empty {
padding: 0;
}
.vols img {
margin: -4px 0;
}
.num {
border-right: 1px solid #bbb;
}
@@ -152,11 +159,13 @@ pre b,
code b {
color: #000;
font-weight: normal;
text-shadow: 0 0 .2em #0f0;
text-shadow: 0 0 .2em #3f3;
border-bottom: 1px solid #090;
}
html.z pre b,
html.z code b {
color: #fff;
border-bottom: 1px solid #9f9;
}
@@ -182,19 +191,35 @@ html.z a.g {
border-color: #af4;
box-shadow: 0 .3em 1em #7d0;
}
form {
line-height: 2.5em;
}
#x,
input {
color: #a50;
background: #fff;
border: 1px solid #a50;
border-radius: .5em;
padding: .5em .7em;
margin: 0 .5em 0 0;
border-radius: .3em;
padding: .25em .6em;
margin: 0 .3em 0 0;
font-size: 1em;
}
input::placeholder {
font-size: 1.2em;
font-style: italic;
letter-spacing: .04em;
opacity: 0.64;
color: #930;
}
#x,
html.z input {
color: #fff;
background: #626;
border-color: #c2c;
}
html.z input::placeholder {
color: #fff;
}
html.z .num {
border-color: #777;
}
@@ -204,3 +229,6 @@ html.bz {
color: #bbd;
background: #11121d;
}
html.bz .vols img {
filter: sepia(0.8) hue-rotate(180deg);
}

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
@@ -14,6 +14,7 @@
<body>
<div id="wrap">
{%- if not in_shr %}
<a id="a" href="{{ r }}/?h" class="af">refresh</a>
<a id="v" href="{{ r }}/?hc" class="af">connect</a>
@@ -21,7 +22,8 @@
<p id="b">howdy stranger &nbsp; <small>(you're not logged in)</small></p>
{%- else %}
<a id="c" href="{{ r }}/?pw=x" class="logout">logout</a>
<p><span id="m">welcome back,</span> <strong>{{ this.uname }}</strong></p>
<p><span id="m">welcome back,</span> <strong>{{ this.uname|e }}</strong></p>
{%- endif %}
{%- endif %}
{%- if msg %}
@@ -30,6 +32,30 @@
</div>
{%- endif %}
{%- if ups %}
<h1 id="aa">incoming files:</h1>
<table class="vols">
<thead><tr><th>%</th><th>speed</th><th>eta</th><th>idle</th><th>dir</th><th>file</th></tr></thead>
<tbody>
{% for u in ups %}
<tr><td>{{ u[0] }}</td><td>{{ u[1] }}</td><td>{{ u[2] }}</td><td>{{ u[3] }}</td><td><a href="{{ u[4] }}">{{ u[5]|e }}</a></td><td>{{ u[6]|e }}</td></tr>
{% endfor %}
</tbody>
</table>
{%- endif %}
{%- if dls %}
<h1 id="ae">active downloads:</h1>
<table class="vols">
<thead><tr><th>%</th><th>sent</th><th>speed</th><th>eta</th><th>idle</th><th></th><th>dir</th><th>file</th></tr></thead>
<tbody>
{% for u in dls %}
<tr><td>{{ u[0] }}</td><td>{{ u[1] }}</td><td>{{ u[2] }}</td><td>{{ u[3] }}</td><td>{{ u[4] }}</td><td>{{ u[5] }}</td><td><a href="{{ u[6] }}">{{ u[7]|e }}</a></td><td>{{ u[8] }}</td></tr>
{% endfor %}
</tbody>
</table>
{%- endif %}
{%- if avol %}
<h1>admin panel:</h1>
<table><tr><td> <!-- hehehe -->
@@ -76,31 +102,64 @@
</ul>
{%- endif %}
<h1 id="cc">client config:</h1>
<ul>
{% if k304 or k304vis %}
{% if k304 %}
<li><a id="h" href="{{ r }}/?k304=n">disable k304</a> (currently enabled)
{%- else %}
<li><a id="i" href="{{ r }}/?k304=y" class="r">enable k304</a> (currently disabled)
{% endif %}
<blockquote id="j">enabling this will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
{% endif %}
<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
</ul>
<h1 id="l">login for more:</h1>
{%- if in_shr %}
<h1 id="z">unlock this share:</h1>
<div>
<form method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
<input type="hidden" name="act" value="login" />
<input type="password" name="cppwd" />
<input type="submit" value="Login" />
<form id="lf" method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
<input type="hidden" id="la" name="act" value="login" />
<input type="password" id="lp" name="cppwd" placeholder=" password" />
<input type="hidden" name="uhash" id="uhash" value="x" />
<input type="submit" id="ls" value="Unlock" />
{% if ahttps %}
<a id="w" href="{{ ahttps }}">switch to https</a>
{% endif %}
</form>
</div>
{%- else %}
<h1 id="l">login for more:</h1>
<div>
<form id="lf" method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
<input type="hidden" id="la" name="act" value="login" />
<input type="password" id="lp" name="cppwd" placeholder=" password" />
<input type="hidden" name="uhash" id="uhash" value="x" />
<input type="submit" id="ls" value="Login" />
{% if chpw %}
<a id="x" href="#">change password</a>
{% endif %}
{% if ahttps %}
<a id="w" href="{{ ahttps }}">switch to https</a>
{% endif %}
</form>
</div>
{%- endif %}
<h1 id="cc">other stuff:</h1>
<ul>
{%- if this.uname != '*' and this.args.shr %}
<li><a id="y" href="{{ r }}/?shares">edit shares</a></li>
{% endif %}
{% if k304 or k304vis %}
{% if k304 %}
<li><a id="h" href="{{ r }}/?cc&setck=k304=n">disable k304</a> (currently enabled)
{%- else %}
<li><a id="i" href="{{ r }}/?cc&setck=k304=y" class="r">enable k304</a> (currently disabled)
{% endif %}
<blockquote id="j">enabling k304 will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
{% endif %}
{% if no304 or no304vis %}
{% if no304 %}
<li><a id="ab" href="{{ r }}/?cc&setck=no304=n">disable no304</a> (currently enabled)
{%- else %}
<li><a id="ac" href="{{ r }}/?cc&setck=no304=y" class="r">enable no304</a> (currently disabled)
{% endif %}
<blockquote id="ad">enabling no304 will disable all caching; try this if k304 wasn't enough. This will waste a huge amount of network traffic!</blockquote></li>
{% endif %}
<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
</ul>
</div>
<a href="#" id="repl">π</a>
{%- if not this.args.nb %}
@@ -118,6 +177,9 @@ document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/splash.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

View File

@@ -9,7 +9,7 @@ var Ls = {
"e2": "leser inn konfigurasjonsfiler på nytt$N(kontoer, volumer, volumbrytere)$Nog kartlegger alle e2ds-volumer$N$Nmerk: endringer i globale parametere$Nkrever en full restart for å ta gjenge",
"f1": "du kan betrakte:",
"g1": "du kan laste opp til:",
"cc1": "klient-konfigurasjon",
"cc1": "brytere og sånt:",
"h1": "skru av k304",
"i1": "skru på k304",
"j1": "k304 bryter tilkoplingen for hver HTTP 304. Dette hjelper mot visse mellomtjenere som kan sette seg fast / plutselig slutter å laste sider, men det reduserer også ytelsen betydelig",
@@ -17,25 +17,85 @@ var Ls = {
"l1": "logg inn:",
"m1": "velkommen tilbake,",
"n1": "404: filen finnes ikke &nbsp;┐( ´ -`)┌",
"o1": 'eller kanskje du ikke har tilgang? prøv å logge inn eller <a href="' + SR + '/?h">gå hjem</a>',
"o1": 'eller kanskje du ikke har tilgang? prøv et passord eller <a href="' + SR + '/?h">gå hjem</a>',
"p1": "403: tilgang nektet &nbsp;~┻━┻",
"q1": 'du må logge inn eller <a href="' + SR + '/?h">gå hjem</a>',
"q1": 'prøv et passord eller <a href="' + SR + '/?h">gå hjem</a>',
"r1": "gå hjem",
".s1": "kartlegg",
"t1": "handling",
"u2": "tid siden noen sist skrev til serveren$N( opplastning / navneendring / ... )$N$N17d = 17 dager$N1h23 = 1 time 23 minutter$N4m56 = 4 minuter 56 sekunder",
"v1": "koble til",
"v2": "bruk denne serveren som en lokal harddisk$N$NADVARSEL: kommer til å vise passordet ditt!",
"v2": "bruk denne serveren som en lokal harddisk",
"w1": "bytt til https",
"x1": "bytt passord",
"y1": "dine delinger",
"z1": "lås opp område:",
"ta1": "du må skrive et nytt passord først",
"ta2": "gjenta for å bekrefte nytt passord:",
"ta3": "fant en skrivefeil; vennligst prøv igjen",
"aa1": "innkommende:",
"ab1": "skru av no304",
"ac1": "skru på no304",
"ad1": "no304 stopper all bruk av cache. Hvis ikke k304 var nok, prøv denne. Vil mangedoble dataforbruk!",
"ae1": "utgående:",
},
"eng": {
"d2": "shows the state of all active threads",
"e2": "reload config files (accounts/volumes/volflags),$Nand rescan all e2ds volumes$N$Nnote: any changes to global settings$Nrequire a full restart to take effect",
"u2": "time since the last server write$N( upload / rename / ... )$N$N17d = 17 days$N1h23 = 1 hour 23 minutes$N4m56 = 4 minutes 56 seconds",
"v2": "use this server as a local HDD$N$NWARNING: this will show your password!",
"v2": "use this server as a local HDD",
"ta1": "fill in your new password first",
"ta2": "repeat to confirm new password:",
"ta3": "found a typo; please try again",
},
"chi": {
"a1": "更新",
"b1": "你好 &nbsp; <small>(你尚未登录)</small>",
"c1": "登出",
"d1": "状态",
"d2": "显示所有活动线程的状态",
"e1": "重新加载配置",
"e2": "重新加载配置文件(账户/卷/卷标),$N并重新扫描所有 e2ds 卷$N$N注意任何全局设置的更改$N都需要完全重启才能生效",
"f1": "你可以查看:",
"g1": "你可以上传到:",
"cc1": "开关等",
"h1": "关闭 k304",
"i1": "开启 k304",
"j1": "k304 会在每个 HTTP 304 时断开连接。这有助于避免某些代理服务器卡住或突然停止加载页面,但也会显著降低性能。",
"k1": "重置设置",
"l1": "登录:",
"m1": "欢迎回来,",
"n1": "404: 文件不存在 &nbsp;┐( ´ -`)┌",
"o1": '或者你可能没有权限?尝试输入密码或 <a href="' + SR + '/?h">回家</a>',
"p1": "403: 访问被拒绝 &nbsp;~┻━┻",
"q1": '尝试输入密码或 <a href="' + SR + '/?h">回家</a>',
"r1": "回家",
".s1": "映射",
"t1": "操作",
"u2": "自上次服务器写入的时间$N( 上传 / 重命名 / ... )$N$N17d = 17 天$N1h23 = 1 小时 23 分钟$N4m56 = 4 分钟 56 秒",
"v1": "连接",
"v2": "将此服务器用作本地硬盘",
"w1": "切换到 https",
"x1": "更改密码",
"y1": "你的分享",
"z1": "解锁区域",
"ta1": "请先输入新密码",
"ta2": "重复以确认新密码:",
"ta3": "发现拼写错误;请重试",
"aa1": "正在接收的文件:", //m
"ab1": "关闭 k304",
"ac1": "开启 k304",
"ad1": "启用 no304 将禁用所有缓存;如果 k304 不够,可以尝试此选项。这将消耗大量的网络流量!", //m
"ae1": "正在下载:", //m
}
},
d = Ls[sread("cpp_lang", ["eng", "nor"]) || lang] || Ls.eng || Ls.nor;
};
if (window.langmod)
langmod();
var d = Ls[sread("cpp_lang", Object.keys(Ls)) || lang] ||
Ls.eng || Ls.nor || Ls.chi;
for (var k in (d || {})) {
var f = k.slice(-1),
@@ -66,3 +126,44 @@ if (!ebi('c') && o.offsetTop + o.offsetHeight < window.innerHeight)
o = ebi('u');
if (o && /[0-9]+$/.exec(o.innerHTML))
o.innerHTML = shumantime(o.innerHTML);
ebi('uhash').value = '' + location.hash;
(function() {
if (!ebi('x'))
return;
var pwi = ebi('lp');
function redo(msg) {
modal.alert(msg, function() {
pwi.value = '';
pwi.focus();
});
}
function mok(v) {
if (v !== pwi.value)
return redo(d.ta3);
pwi.setAttribute('name', 'pw');
ebi('la').value = 'chpw';
ebi('lf').submit();
}
function stars() {
var m = ebi('modali');
function enstars(n) {
setTimeout(function() { m.value = ''; }, n);
}
m.setAttribute('type', 'password');
enstars(17);
enstars(32);
enstars(69);
}
ebi('x').onclick = function (e) {
ev(e);
if (!pwi.value)
return redo(d.ta1);
modal.prompt(d.ta2, "y", mok, null, stars);
};
})();

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<style>ul{padding-left:1.3em}li{margin:.4em 0}</style>
@@ -56,7 +56,7 @@
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
<p>if you want to use the native WebDAV client in windows instead (slow and buggy), first run <a href="{{ r }}/.cpr/a/webdav-cfg.bat">webdav-cfg.bat</a> to remove the 47 MiB filesize limit (also fixes latency and password login), then connect:</p>
<pre>
net use <b>w:</b> http{{ s }}://{{ ep }}/{{ rvp }}{% if accs %} k /user:<b>{{ pw }}</b>{% endif %}
@@ -64,16 +64,7 @@
</div>
<div class="os lin">
<pre>
yum install davfs2
{% if accs %}printf '%s\n' <b>{{ pw }}</b> k | {% endif %}mount -t davfs -ouid=1000 http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b>
</pre>
<p>make it automount on boot:</p>
<pre>
printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>{{ pw }}</b> k" >> /etc/davfs2/secrets
printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b> davfs rw,user,uid=1000,noauto 0 0" >> /etc/fstab
</pre>
<p>or you can use rclone instead, which is much slower but doesn't require root (plus it keeps lastmodified on upload):</p>
<p>rclone (v1.63 or later) is recommended:</p>
<pre>
rclone config create {{ aname }}-dav webdav url=http{{ s }}://{{ rip }}{{ hport }} vendor=owncloud pacer_min_sleep=0.01ms{% if accs %} user=k pass=<b>{{ pw }}</b>{% endif %}
rclone mount --vfs-cache-mode writes --dir-cache-time 5s {{ aname }}-dav:{{ rvp }} <b>mp</b>
@@ -85,6 +76,16 @@
<li>running <code>rclone mount</code> as root? add <code>--allow-other</code></li>
<li>old version of rclone? replace all <code>=</code> with <code>&nbsp;</code> (space)</li>
</ul>
<p>alternatively use davfs2 (requires root, is slower, forgets lastmodified-timestamp on upload):</p>
<pre>
yum install davfs2
{% if accs %}printf '%s\n' <b>{{ pw }}</b> k | {% endif %}mount -t davfs -ouid=1000 http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b>
</pre>
<p>make davfs2 automount on boot:</p>
<pre>
printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>{{ pw }}</b> k" >> /etc/davfs2/secrets
printf '%s\n' "http{{ s }}://{{ ep }}/{{ rvp }} <b>mp</b> davfs rw,user,uid=1000,noauto 0 0" >> /etc/fstab
</pre>
<p>or the emergency alternative (gnome/gui-only):</p>
<!-- gnome-bug: ignores vp -->
<pre>
@@ -104,7 +105,7 @@
<pre>
http{{ s }}://k:<b>{{ pw }}</b>@{{ ep }}/{{ rvp }}
</pre>
{% if s %}
<p><em>replace <code>https</code> with <code>http</code> if it doesn't work</em></p>
{% endif %}
@@ -191,6 +192,7 @@
<h1>partyfuse</h1>
<p>
<a href="{{ r }}/.cpr/a/partyfuse.py">partyfuse.py</a> -- fast, read-only,
needs <a href="{{ r }}/.cpr/deps/fuse.py">fuse.py</a> in the same folder,
<span class="os win">needs <a href="https://winfsp.dev/rel/">winfsp</a></span>
<span class="os lin">doesn't need root</span>
</p>
@@ -207,7 +209,6 @@
{% if args.smb %}
<h1>SMB / CIFS</h1>
<em><a href="https://github.com/SecureAuthCorp/impacket/issues/1433">bug:</a> max ~300 files in each folder</em>
<div class="os win">
<pre>
@@ -244,6 +245,9 @@ document.documentElement.className = (STG && STG.cpp_thm) || "{{ args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/svcs.js?_={{ ts }}"></script>
{%- if js %}
<script src="{{ js }}_={{ ts }}"></script>
{%- endif %}
</body>
</html>

View File

@@ -69,6 +69,18 @@ html {
top: 2em;
bottom: unset;
}
#toastt {
position: absolute;
height: 1px;
top: 1px;
right: 1px;
left: 1px;
animation: toastt var(--tmtime) 0.07s steps(var(--tmstep)) forwards;
transform-origin: right;
}
@keyframes toastt {
to {transform: scaleX(0)}
}
#toast a {
color: inherit;
text-shadow: inherit;
@@ -130,6 +142,9 @@ html {
#toast.inf #toastc {
background: #0be;
}
#toast.inf #toastt {
background: #8ef;
}
#toast.ok {
background: #380;
border-color: #8e4;
@@ -137,6 +152,9 @@ html {
#toast.ok #toastc {
background: #8e4;
}
#toast.ok #toastt {
background: #cf9;
}
#toast.warn {
background: #960;
border-color: #fc0;
@@ -144,6 +162,9 @@ html {
#toast.warn #toastc {
background: #fc0;
}
#toast.warn #toastt {
background: #fe9;
}
#toast.err {
background: #900;
border-color: #d06;
@@ -151,6 +172,9 @@ html {
#toast.err #toastc {
background: #d06;
}
#toast.err #toastt {
background: #f9c;
}
#toast code {
padding: 0 .2em;
background: rgba(0,0,0,0.2);
@@ -184,6 +208,7 @@ html {
padding: 1.5em 2em;
border-width: .5em 0;
}
.logue code,
#modalc code,
#tt code {
color: #eee;
@@ -264,7 +289,11 @@ html.y #tth {
box-shadow: 0 .3em 3em rgba(0,0,0,0.5);
max-width: 50em;
max-height: 30em;
overflow: auto;
overflow-x: auto;
overflow-y: scroll;
}
#modalc.yk {
overflow-y: auto;
}
#modalc td {
text-align: unset;
@@ -288,6 +317,14 @@ html.y #tth {
#modalc a {
color: #07b;
}
#modalc .b64 {
display: block;
margin: .1em auto;
width: 60%;
height: 60%;
background: #999;
background: rgba(128,128,128,0.2);
}
#modalb {
position: sticky;
text-align: right;
@@ -380,6 +417,7 @@ html.y textarea:focus {
}
.mdo pre,
.mdo code,
.mdo code[class*="language-"],
.mdo tt {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;

View File

@@ -17,10 +17,14 @@ function goto_up2k() {
var up2k = null,
up2k_hooks = [],
hws = [],
sha_js = window.WebAssembly ? 'hw' : 'ac', // ff53,c57,sa11
hws_ok = 0,
hws_ng = false,
sha_js = WebAssembly ? 'hw' : 'ac', // ff53,c57,sa11
m = 'will use ' + sha_js + ' instead of native sha512 due to';
try {
if (sread('nosubtle') || window.nosubtle)
throw 'chickenbit';
var cf = crypto.subtle || crypto.webkitSubtle;
cf.digest('SHA-512', new Uint8Array(1)).then(
function (x) { console.log('sha-ok'); up2k = up2k_init(cf); },
@@ -152,12 +156,13 @@ function U2pvis(act, btns, uc, st) {
r.mod0 = null;
var markup = {
'404': '<span class="err">404</span>',
'ERROR': '<span class="err">ERROR</span>',
'OS-error': '<span class="err">OS-error</span>',
'found': '<span class="inf">found</span>',
'YOLO': '<span class="inf">YOLO</span>',
'done': '<span class="ok">done</span>',
'404': '<span class="err">' + L.utl_404 + '</span>',
'ERROR': '<span class="err">' + L.utl_err + '</span>',
'OS-error': '<span class="err">' + L.utl_oserr + '</span>',
'found': '<span class="inf">' + L.utl_found + '</span>',
'defer': '<span class="inf">' + L.utl_defer + '</span>',
'YOLO': '<span class="inf">' + L.utl_yolo + '</span>',
'done': '<span class="ok">' + L.utl_done + '</span>',
};
r.addfile = function (entry, sz, draw) {
@@ -241,7 +246,7 @@ function U2pvis(act, btns, uc, st) {
p = bd * 100.0 / sz,
nb = bd - bd0,
spd = nb / (td / 1000),
eta = (sz - bd) / spd;
eta = spd ? (sz - bd) / spd : 3599;
return [p, s2ms(eta), spd / (1024 * 1024)];
};
@@ -445,9 +450,7 @@ function U2pvis(act, btns, uc, st) {
return;
r.npotato = 0;
var html = [
"<p>files: &nbsp; <b>{0}</b> finished, &nbsp; <b>{1}</b> failed, &nbsp; <b>{2}</b> busy, &nbsp; <b>{3}</b> queued</p>".format(
r.ctr.ok, r.ctr.ng, r.ctr.bz, r.ctr.q)];
var html = [L.u_pott.format(r.ctr.ok, r.ctr.ng, r.ctr.bz, r.ctr.q)];
while (r.head < r.tab.length && has(["ok", "ng"], r.tab[r.head].in))
r.head++;
@@ -602,7 +605,7 @@ function U2pvis(act, btns, uc, st) {
if (nf < 9000)
return go();
modal.confirm('about to show ' + nf + ' files\n\nthis may crash your browser, are you sure?', go, null);
modal.confirm(L.u_bigtab.format(nf), go, null);
};
}
@@ -658,7 +661,9 @@ function Donut(uc, st) {
}
function pos() {
return uc.fsearch ? Math.max(st.bytes.hashed, st.bytes.finished) : st.bytes.finished;
return uc.fsearch ?
Math.max(st.bytes.hashed, st.bytes.finished) :
st.bytes.inflight + st.bytes.finished;
}
r.on = function (ya) {
@@ -717,7 +722,7 @@ function Donut(uc, st) {
sfx();
// firefox may forget that filedrops are user-gestures so it can skip this:
if (uc.upnag && window.Notification && Notification.permission == 'granted')
if (uc.upnag && Notification && Notification.permission == 'granted')
new Notification(uc.nagtxt);
}
@@ -779,8 +784,8 @@ function up2k_init(subtle) {
};
setTimeout(function () {
if (window.WebAssembly && !hws.length)
fetch(SR + '/.cpr/w.hash.js' + CB);
if (WebAssembly && !hws.length)
fetch(SR + '/.cpr/w.hash.js?_=' + TS);
}, 1000);
function showmodal(msg) {
@@ -852,7 +857,13 @@ function up2k_init(subtle) {
setmsg(suggest_up2k, 'msg');
var u2szs = u2sz.split(','),
u2sz_min = parseInt(u2szs[0]),
u2sz_tgt = parseInt(u2szs[1]),
u2sz_max = parseInt(u2szs[2]);
var parallel_uploads = ebi('nthread').value = icfg_get('nthread', u2j),
stitch_tgt = ebi('u2szg').value = icfg_get('u2sz', u2sz_tgt),
uc = {},
fdom_ctr = 0,
biggest_file = 0;
@@ -869,7 +880,7 @@ function up2k_init(subtle) {
bcfg_bind(uc, 'turbo', 'u2turbo', turbolvl > 1, draw_turbo);
bcfg_bind(uc, 'datechk', 'u2tdate', turbolvl < 3, null);
bcfg_bind(uc, 'az', 'u2sort', u2sort.indexOf('n') + 1, set_u2sort);
bcfg_bind(uc, 'hashw', 'hashw', !!window.WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
bcfg_bind(uc, 'hashw', 'hashw', !!WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);
@@ -1034,7 +1045,7 @@ function up2k_init(subtle) {
}
catch (ex) {
document.body.ondragenter = document.body.ondragleave = document.body.ondragover = null;
return modal.alert('your browser does not support drag-and-drop uploading');
return modal.alert(L.u_nodrop);
}
if (btn)
return;
@@ -1101,7 +1112,7 @@ function up2k_init(subtle) {
}
if (!good_files.length && bad_files.length)
return toast.err(30, "that's not a folder!\n\nyour browser is too old,\nplease try dragdrop instead");
return toast.err(30, L.u_notdir);
return read_dirs(null, [], [], good_files, nil_files, bad_files);
}
@@ -1119,7 +1130,7 @@ function up2k_init(subtle) {
if (err)
return modal.alert('sorry, ' + err);
toast.inf(0, 'Scanning files...');
toast.inf(0, L.u_scan);
if ((dz == 'up_dz' && uc.fsearch) || (dz == 'srch_dz' && !uc.fsearch))
tgl_fsearch();
@@ -1207,7 +1218,7 @@ function up2k_init(subtle) {
match = false;
if (match) {
var msg = ['directory iterator got stuck on the following {0} items; good chance your browser is about to spinlock:<ul>'.format(missing.length)];
var msg = [L.u_dirstuck.format(missing.length) + '<ul>'];
for (var a = 0; a < Math.min(20, missing.length); a++)
msg.push('<li>' + esc(missing[a]) + '</li>');
@@ -1278,7 +1289,7 @@ function up2k_init(subtle) {
}
function gotallfiles(good_files, nil_files, bad_files) {
if (toast.txt == 'Scanning files...')
if (toast.txt == L.u_scan)
toast.hide();
if (uc.fsearch && !uc.turbo)
@@ -1347,9 +1358,13 @@ function up2k_init(subtle) {
var evpath = get_evpath(),
draw_each = good_files.length < 50;
if (window.WebAssembly && !hws.length) {
if (WebAssembly && !hws.length) {
for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
hws.push(new Worker(SR + '/.cpr/w.hash.js' + CB));
hws.push(new Worker(SR + '/.cpr/w.hash.js?_=' + TS));
if (!subtle)
for (var a = 0; a < hws.length; a++)
hws[a].postMessage('nosubtle');
console.log(hws.length + " hashers");
}
@@ -1434,7 +1449,7 @@ function up2k_init(subtle) {
if (!actx || actx.state != 'suspended' || toast.visible)
return;
toast.warn(30, "<div onclick=\"start_actx();toast.inf(3,'thanks!')\">please click this text to<br />unlock full upload speed</div>");
toast.warn(30, "<div onclick=\"start_actx();toast.inf(3,'thanks!')\">" + L.u_actx + "</div>");
}, 500);
}
@@ -1476,7 +1491,7 @@ function up2k_init(subtle) {
ev(e);
var txt = linklist();
cliptxt(txt + '\n', function () {
toast.inf(5, txt.split('\n').length + ' links copied to clipboard');
toast.inf(5, un_clip.format(txt.split('\n').length));
});
};
@@ -1736,16 +1751,13 @@ function up2k_init(subtle) {
}
}
var mou_ikkai = false;
if (st.busy.handshake.length &&
st.busy.handshake[0].t_busied < now - 30 * 1000
) {
console.log("retrying stuck handshake");
var t = st.busy.handshake.shift();
st.todo.handshake.unshift(t);
if (st.bytes.inflight && (st.bytes.inflight < 0 || !st.busy.upload.length)) {
console.log('insane inflight ' + st.bytes.inflight);
st.bytes.inflight = 0;
}
var mou_ikkai = false;
var nprev = -1;
for (var a = 0; a < st.todo.upload.length; a++) {
var nf = st.todo.upload[a].nfile;
@@ -1864,10 +1876,12 @@ function up2k_init(subtle) {
function chill(t) {
var now = Date.now();
if ((t.coolmul || 0) < 2 || now - t.cooldown < t.coolmul * 700)
if ((t.coolmul || 0) < 5 || now - t.cooldown < t.coolmul * 700)
t.coolmul = Math.min((t.coolmul || 0.5) * 2, 32);
t.cooldown = Math.max(t.cooldown || 1, Date.now() + t.coolmul * 1000);
var cd = now + 1000 * (t.coolmul + Math.random() * 4 + 2);
t.cooldown = Math.floor(Math.max(cd, t.cooldown || 1));
return t;
}
/////
@@ -1952,7 +1966,7 @@ function up2k_init(subtle) {
pvis.setab(t.n, nchunks);
pvis.move(t.n, 'bz');
if (hws.length && uc.hashw && (nchunks > 1 || document.visibilityState == 'hidden'))
if (hws.length && !hws_ng && uc.hashw && (nchunks > 1 || document.visibilityState == 'hidden'))
// resolving subtle.digest w/o worker takes 1sec on blur if the actx hack breaks
return wexec_hash(t, chunksize, nchunks);
@@ -2061,16 +2075,27 @@ function up2k_init(subtle) {
free = [],
busy = {},
nbusy = 0,
init = 0,
hashtab = {},
mem = (MOBILE ? 128 : 256) * 1024 * 1024;
if (!hws_ok)
init = setTimeout(function() {
hws_ng = true;
toast.warn(30, 'webworkers failed to start\n\nwill be a bit slower due to\nhashing on main-thread');
apop(st.busy.hash, t);
st.todo.hash.unshift(t);
exec_hash();
}, 5000);
for (var a = 0; a < hws.length; a++) {
var w = hws[a];
free.push(w);
w.onmessage = onmsg;
if (init)
w.postMessage('ping');
if (mem > 0)
free.push(w);
mem -= chunksize;
if (mem <= 0)
break;
}
function go_next() {
@@ -2100,6 +2125,12 @@ function up2k_init(subtle) {
d = d.data;
var k = d[0];
if (k == "pong")
if (++hws_ok == hws.length) {
clearTimeout(init);
go_next();
}
if (k == "panic")
return vis_exh(d[1], 'up2k.js', '', '', d[1]);
@@ -2162,7 +2193,8 @@ function up2k_init(subtle) {
tasker();
}
}
go_next();
if (!init)
go_next();
}
/////
@@ -2178,7 +2210,7 @@ function up2k_init(subtle) {
st.busy.head.push(t);
var xhr = new XMLHttpRequest();
xhr.onerror = function () {
xhr.onerror = xhr.ontimeout = function () {
console.log('head onerror, retrying', t.name, t);
if (!toast.visible)
toast.warn(9.98, L.u_enethd + "\n\nfile: " + t.name, t);
@@ -2222,6 +2254,7 @@ function up2k_init(subtle) {
try { orz(e); } catch (ex) { vis_exh(ex + '', 'up2k.js', '', '', ex); }
};
xhr.timeout = 34000;
xhr.open('HEAD', t.purl + uricom_enc(t.name), true);
xhr.send();
}
@@ -2246,8 +2279,11 @@ function up2k_init(subtle) {
if (keepalive)
console.log("sending keepalive handshake", t.name, t);
if (!t.srch && !t.t_handshake)
pvis.seth(t.n, 2, L.u_hs);
var xhr = new XMLHttpRequest();
xhr.onerror = function () {
xhr.onerror = xhr.ontimeout = function () {
if (t.t_busied != me) // t.done ok
return console.log('zombie handshake onerror', t.name, t);
@@ -2256,8 +2292,7 @@ function up2k_init(subtle) {
console.log('handshake onerror, retrying', t.name, t);
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
st.todo.handshake.unshift(chill(t));
t.keepalive = keepalive;
};
var orz = function (e) {
@@ -2270,9 +2305,9 @@ function up2k_init(subtle) {
}
catch (ex) {
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
return toast.err(0, 'Handshake error; will retry...\n\n' + L.badreply + ':\n\n' + unpre(xhr.responseText));
st.todo.handshake.unshift(chill(t));
var txt = t.t_uploading ? L.u_ehsfin : t.srch ? L.u_ehssrch : L.u_ehsinit;
return toast.err(0, txt + '\n\n' + L.badreply + ':\n\n' + unpre(xhr.responseText));
}
t.t_handshake = Date.now();
@@ -2374,11 +2409,39 @@ function up2k_init(subtle) {
var arr = st.todo.upload,
sort = arr.length && arr[arr.length - 1].nfile > t.n;
for (var a = 0; a < t.postlist.length; a++)
if (!t.stitch_sz) {
// keep all connections busy
var bpc = (st.bytes.total - st.bytes.finished) / (parallel_uploads || 1),
ocs = 1024 * 1024,
stp = 1024 * 512,
ccs = ocs;
while (ccs < bpc) {
ocs = ccs;
ccs += stp; if (ccs < bpc) ocs = ccs;
ccs += stp; stp *= 2;
}
ocs = Math.floor(ocs / 1024 / 1024);
t.stitch_sz = Math.min(ocs, stitch_tgt);
}
for (var a = 0; a < t.postlist.length; a++) {
var nparts = [], tbytes = 0, stitch = t.stitch_sz;
if (t.nojoin && t.nojoin - t.postlist.length < 6)
stitch = 1;
--a;
for (var b = 0; b < stitch; b++) {
nparts.push(t.postlist[++a]);
tbytes += chunksize;
if (tbytes + chunksize > stitch * 1024 * 1024 || t.postlist[a + 1] - t.postlist[a] !== 1)
break;
}
arr.push({
'nfile': t.n,
'npart': t.postlist[a]
'nparts': nparts
});
}
t.nojoin = 0;
msg = null;
done = false;
@@ -2387,7 +2450,7 @@ function up2k_init(subtle) {
arr.sort(function (a, b) {
return a.nfile < b.nfile ? -1 :
/* */ a.nfile > b.nfile ? 1 :
a.npart < b.npart ? -1 : 1;
/* */ a.nparts[0] < b.nparts[0] ? -1 : 1;
});
}
@@ -2421,8 +2484,10 @@ function up2k_init(subtle) {
else {
pvis.seth(t.n, 1, "ERROR");
pvis.seth(t.n, 2, L.u_ehstmp, t);
apop(st.busy.handshake, t);
var err = "",
cls = "ERROR",
rsp = unpre(xhr.responseText),
ofs = rsp.lastIndexOf('\nURL: ');
@@ -2433,7 +2498,6 @@ function up2k_init(subtle) {
var penalty = rsp.replace(/.*rate-limit /, "").split(' ')[0];
console.log("rate-limit: " + penalty);
t.cooldown = Date.now() + parseFloat(penalty) * 1000;
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
return;
}
@@ -2452,23 +2516,28 @@ function up2k_init(subtle) {
if (!t.rechecks && (err_pend || err_srcb)) {
t.rechecks = 0;
t.want_recheck = true;
err = L.u_dupdefer;
cls = 'defer';
}
}
if (rsp.indexOf('server HDD is full') + 1)
return toast.err(0, L.u_ehsdf + "\n\n" + rsp.replace(/.*; /, ''));
if (err != "") {
if (!t.t_uploading)
st.bytes.finished += t.size;
pvis.seth(t.n, 1, "ERROR");
pvis.seth(t.n, 1, cls);
pvis.seth(t.n, 2, err);
pvis.move(t.n, 'ng');
apop(st.busy.handshake, t);
tasker();
return;
}
st.todo.handshake.unshift(chill(t));
if (rsp.indexOf('server HDD is full') + 1)
return toast.err(0, L.u_ehsdf + "\n\n" + rsp.replace(/.*; /, ''));
err = t.t_uploading ? L.u_ehsfin : t.srch ? L.u_ehssrch : L.u_ehsinit;
xhrchk(xhr, err + "\n\nfile: " + t.name + "\n\nerror ", "404, target folder not found", "warn", t);
}
@@ -2493,6 +2562,8 @@ function up2k_init(subtle) {
xhr.open('POST', t.purl, true);
xhr.responseType = 'text';
xhr.timeout = 42000 + (t.srch || t.t_uploaded ? 0 :
(t.size / (1048 * 20))); // safededup 20M/s hdd
xhr.send(JSON.stringify(req));
}
@@ -2534,8 +2605,10 @@ function up2k_init(subtle) {
function exec_upload() {
var upt = st.todo.upload.shift(),
t = st.files[upt.nfile],
npart = upt.npart,
tries = 0;
nparts = upt.nparts,
pcar = nparts[0],
pcdr = nparts[nparts.length - 1],
maxsz = (u2sz_max > 1 ? u2sz_max : 2040) * 1024 * 1024;
if (t.done)
return console.log('done; skip chunk', t.name, t);
@@ -2549,45 +2622,82 @@ function up2k_init(subtle) {
pvis.seth(t.n, 1, "🚀 send");
var chunksize = get_chunksize(t.size),
car = npart * chunksize,
cdr = car + chunksize;
car = pcar * chunksize,
cdr = (pcdr + 1) * chunksize;
if (cdr >= t.size)
cdr = t.size;
if (cdr - car <= maxsz)
return upload_sub(t, upt, pcar, pcdr, car, cdr, chunksize, car, []);
var car0 = car, subs = [];
while (car < cdr) {
subs.push([car, Math.min(cdr, car + maxsz)]);
car += maxsz;
}
upload_sub(t, upt, pcar, pcdr, 0, 0, chunksize, car0, subs);
}
function upload_sub(t, upt, pcar, pcdr, car, cdr, chunksize, car0, subs) {
var nparts = upt.nparts,
is_sub = subs.length;
if (is_sub) {
var x = subs.shift();
car = x[0];
cdr = x[1];
}
var snpart = is_sub ? ('' + pcar + '(' + (car-car0) +'+'+ (cdr-car)) :
pcar == pcdr ? pcar : ('' + pcar + '~' + pcdr);
var orz = function (xhr) {
st.bytes.inflight -= xhr.bsent;
var txt = unpre((xhr.response && xhr.response.err) || xhr.responseText);
if (txt.indexOf('upload blocked by x') + 1) {
apop(st.busy.upload, upt);
apop(t.postlist, npart);
for (var a = pcar; a <= pcdr; a++)
apop(t.postlist, a);
pvis.seth(t.n, 1, "ERROR");
pvis.seth(t.n, 2, txt.split(/\n/)[0]);
pvis.move(t.n, 'ng');
return;
}
if (xhr.status == 200) {
pvis.prog(t, npart, cdr - car);
car = car0;
if (subs.length)
return upload_sub(t, upt, pcar, pcdr, 0, 0, chunksize, car0, subs);
var bdone = cdr - car;
for (var a = pcar; a <= pcdr; a++) {
pvis.prog(t, a, Math.min(bdone, chunksize));
bdone -= chunksize;
}
st.bytes.finished += cdr - car;
st.bytes.uploaded += cdr - car;
t.bytes_uploaded += cdr - car;
t.cooldown = t.coolmul = 0;
st.etac.u++;
st.etac.t++;
}
else if (txt.indexOf('already got that') + 1 ||
txt.indexOf('already being written') + 1) {
console.log("ignoring dupe-segment error", t.name, t);
t.nojoin = t.nojoin || t.postlist.length;
console.log("ignoring dupe-segment with backoff", t.nojoin, t.name, t);
if (!toast.visible && st.todo.upload.length < 4)
toast.inf(10, L.u_cbusy);
}
else {
xhrchk(xhr, L.u_cuerr2.format(npart, Math.ceil(t.size / chunksize), t.name), "404, target folder not found (???)", "warn", t);
xhrchk(xhr, L.u_cuerr2.format(snpart, Math.ceil(t.size / chunksize), t.name), "404, target folder not found (???)", "warn", t);
chill(t);
}
orz2(xhr);
}
var orz2 = function (xhr) {
apop(st.busy.upload, upt);
apop(t.postlist, npart);
for (var a = pcar; a <= pcdr; a++)
apop(t.postlist, a);
if (!t.postlist.length) {
t.t_uploaded = Date.now();
pvis.seth(t.n, 1, 'verifying');
@@ -2601,37 +2711,63 @@ function up2k_init(subtle) {
btot = Math.floor(st.bytes.total / 1024 / 1024);
xhr.upload.onprogress = function (xev) {
var nb = xev.loaded;
st.bytes.inflight += nb - xhr.bsent;
var nb = xev.loaded,
db = nb - xhr.bsent;
if (!db)
return;
st.bytes.inflight += db;
xhr.bsent = nb;
pvis.prog(t, npart, nb);
xhr.timeout = 64000 + Date.now() - xhr.t0;
pvis.prog(t, pcar, nb);
};
xhr.onload = function (xev) {
try { orz(xhr); } catch (ex) { vis_exh(ex + '', 'up2k.js', '', '', ex); }
};
xhr.onerror = function (xev) {
xhr.onerror = xhr.ontimeout = function (xev) {
if (crashed)
return;
st.bytes.inflight -= (xhr.bsent || 0);
xhr.bsent = 0;
if (!toast.visible)
toast.warn(9.98, L.u_cuerr.format(npart, Math.ceil(t.size / chunksize), t.name), t);
toast.warn(9.98, L.u_cuerr.format(snpart, Math.ceil(t.size / chunksize), t.name), t);
console.log('chunkpit onerror,', ++tries, t.name, t);
t.nojoin = t.nojoin || t.postlist.length; // maybe rproxy postsize limit
console.log('chunkpit onerror,', t.name, t);
orz2(xhr);
};
var chashes = [],
ctxt = t.hash[pcar],
plen = Math.floor(192 / nparts.length);
plen = plen > 9 ? 9 : plen < 2 ? 2 : plen;
for (var a = pcar + 1; a <= pcdr; a++)
chashes.push(t.hash[a].slice(0, plen));
if (chashes.length)
ctxt += ',' + plen + ',' + chashes.join('');
xhr.open('POST', t.purl, true);
xhr.setRequestHeader("X-Up2k-Hash", t.hash[npart]);
xhr.setRequestHeader("X-Up2k-Hash", ctxt);
xhr.setRequestHeader("X-Up2k-Wark", t.wark);
if (is_sub)
xhr.setRequestHeader("X-Up2k-Subc", car - car0);
xhr.setRequestHeader("X-Up2k-Stat", "{0}/{1}/{2}/{3} {4}/{5} {6}".format(
pvis.ctr.ok, pvis.ctr.ng, pvis.ctr.bz, pvis.ctr.q, btot, btot - bfin,
st.eta.t.split(' ').pop()));
st.eta.t.indexOf('/s, ')+1 ? st.eta.t.split(' ').pop() : 'x'));
xhr.setRequestHeader('Content-Type', 'application/octet-stream');
if (xhr.overrideMimeType)
xhr.overrideMimeType('Content-Type', 'application/octet-stream');
xhr.bsent = 0;
xhr.t0 = Date.now();
xhr.timeout = 42000;
xhr.responseType = 'text';
xhr.send(t.fobj.slice(car, cdr));
}
@@ -2732,13 +2868,34 @@ function up2k_init(subtle) {
if (parallel_uploads > 16)
parallel_uploads = 16;
if (parallel_uploads > 7)
if (parallel_uploads > 6)
toast.warn(10, L.u_maxconn);
else if (toast.txt == L.u_maxconn)
toast.hide();
obj.value = parallel_uploads;
bumpthread({ "target": 1 });
}
var read_u2sz = function () {
var el = ebi('u2szg'), n = parseInt(el.value);
stitch_tgt = n = (
isNaN(n) ? u2sz_tgt :
n < u2sz_min ? u2sz_min :
n > u2sz_max ? u2sz_max : n
);
if (n == u2sz_tgt) sdrop('u2sz'); else swrite('u2sz', n);
if (el.value != n) el.value = n;
};
ebi('u2szg').addEventListener('blur', read_u2sz);
ebi('u2szg').onkeydown = function (e) {
if (anymod(e)) return;
var n = e.code == 'ArrowUp' ? 1 : e.code == 'ArrowDown' ? -1 : 0;
if (!n) return;
this.value = parseInt(this.value) + n;
read_u2sz();
}
function tgl_fsearch() {
set_fsearch(!uc.fsearch);
}
@@ -2950,7 +3107,7 @@ function up2k_init(subtle) {
}
function set_hashw() {
if (!window.WebAssembly) {
if (!WebAssembly) {
bcfg_set('hashw', uc.hashw = false);
toast.err(10, L.u_nowork);
}
@@ -2967,7 +3124,7 @@ function up2k_init(subtle) {
nopenag();
}
if (!window.Notification || !HTTPS)
if (!Notification || !HTTPS)
return nopenag();
if (en && Notification.permission == 'default')
@@ -2989,7 +3146,7 @@ function up2k_init(subtle) {
};
}
if (uc.upnag && (!window.Notification || Notification.permission != 'granted'))
if (uc.upnag && (!Notification || Notification.permission != 'granted'))
bcfg_set('upnag', uc.upnag = false);
ebi('nthread_add').onclick = function (e) {

View File

@@ -5,10 +5,17 @@ if (!window.console || !console.log)
"log": function (msg) { }
};
if (!Object.assign)
Object.assign = function (a, b) {
for (var k in b)
a[k] = b[k];
};
if (window.CGV1)
Object.assign(window, window.CGV1);
if (window.CGV)
for (var k in CGV)
window[k] = CGV[k];
Object.assign(window, window.CGV);
var wah = '',
@@ -16,7 +23,6 @@ var wah = '',
NOAC = 'autocorrect="off" autocapitalize="off"',
L, tt, treectl, thegrid, up2k, asmCrypto, hashwasm, vbar, marked,
T0 = Date.now(),
CB = '?_=' + Math.floor(T0 / 1000).toString(36),
R = SR.slice(1),
RS = R ? "/" + R : "",
HALFMAX = 8192 * 8192 * 8192 * 8192,
@@ -52,8 +58,6 @@ catch (ex) {
}
try {
CB = '?' + document.currentScript.src.split('?').pop();
if (navigator.userAgentData.mobile)
MOBILE = true;
@@ -130,13 +134,13 @@ if ((document.location + '').indexOf(',rej,') + 1)
try {
console.hist = [];
var CMAXHIST = 100;
var CMAXHIST = MOBILE ? 9000 : 44000;
var hook = function (t) {
var orig = console[t].bind(console),
cfun = function () {
console.hist.push(Date.now() + ' ' + t + ': ' + Array.from(arguments).join(', '));
if (console.hist.length > CMAXHIST)
console.hist = console.hist.slice(CMAXHIST / 2);
console.hist = console.hist.slice(CMAXHIST / 4);
orig.apply(console, arguments);
};
@@ -182,7 +186,7 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
if (url.indexOf('easymde.js') + 1)
return; // clicking the preview pane
if (url.indexOf('deps/marked.js') + 1 && !window.WebAssembly)
if (url.indexOf('deps/marked.js') + 1 && !WebAssembly)
return; // ff<52
crashed = true;
@@ -476,6 +480,24 @@ function crc32(str) {
}
function randstr(len) {
var ret = '';
try {
var ar = new Uint32Array(Math.floor((len + 3) / 4));
crypto.getRandomValues(ar);
for (var a = 0; a < ar.length; a++)
ret += ('000' + ar[a].toString(36)).slice(-4);
return ret.slice(0, len);
}
catch (ex) {
console.log('using unsafe randstr because ' + ex);
while (ret.length < len)
ret += ('000' + Math.floor(Math.random() * 1679616).toString(36)).slice(-4);
return ret.slice(0, len);
}
}
function clmod(el, cls, add) {
if (!el)
return false;
@@ -520,6 +542,14 @@ function clgot(el, cls) {
}
function setcvar(k, v) {
try {
document.documentElement.style.setProperty(k, v);
}
catch (e) { }
}
var ANIM = true;
try {
var mq = window.matchMedia('(prefers-reduced-motion: reduce)');
@@ -740,6 +770,15 @@ function vjoin(p1, p2) {
}
function addq(url, q) {
var uh = url.split('#', 1),
u = uh[0],
h = uh.length == 1 ? '' : '#' + uh[1];
return u + (u.indexOf('?') < 0 ? '?' : '&') + (q === undefined ? '' : q) + h;
}
function uricom_enc(txt, do_fb_enc) {
try {
return encodeURIComponent(txt);
@@ -842,6 +881,8 @@ if (window.Number && Number.isFinite)
function f2f(val, nd) {
// 10.toFixed(1) returns 10.00 for certain values of 10
if (!isNum(val))
val = 999;
val = (val * Math.pow(10, nd)).toFixed(0).split('.')[0];
return nd ? (val.slice(0, -nd) || '0') + '.' + val.slice(-nd) : val;
}
@@ -898,15 +939,18 @@ function shumantime(v, long) {
function lhumantime(v) {
var t = shumantime(v, 1),
tp = t.replace(/([a-z])/g, " $1 ").split(/ /g).slice(0, -1);
var t = shumantime(v, 1);
if (/[0-9]$/.exec(t))
t += 's';
var tp = t.replace(/([a-z])/g, " $1 ").split(/ /g).slice(0, -1);
if (!L || tp.length < 2 || tp[1].indexOf('$') + 1)
return t;
var ret = '';
for (var a = 0; a < tp.length; a += 2)
ret += tp[a] + ' ' + L['ht_' + tp[a + 1]].replace(tp[a] == 1 ? /!.*/ : /!/, '') + L.ht_and;
ret += tp[a] + ' ' + L['ht_' + tp[a + 1] + (tp[a]==1?1:2)] + L.ht_and;
return ret.slice(0, -L.ht_and.length);
}
@@ -1390,10 +1434,10 @@ var tt = (function () {
o = ctr.querySelectorAll('*[tt]');
for (var a = o.length - 1; a >= 0; a--) {
o[a].onfocus = _cshow;
o[a].onblur = _hide;
o[a].onmouseenter = _dshow;
o[a].onmouseleave = _hide;
o[a].addEventListener('focus', _cshow);
o[a].addEventListener('blur', _hide);
o[a].addEventListener('mouseenter', _dshow);
o[a].addEventListener('mouseleave', _hide);
}
r.hide();
}
@@ -1469,7 +1513,7 @@ var toast = (function () {
clmod(obj, 'vis');
r.visible = false;
r.tag = obj;
if (!window.WebAssembly)
if (!WebAssembly)
te = setTimeout(function () {
obj.className = 'hide';
}, 500);
@@ -1492,13 +1536,26 @@ var toast = (function () {
if (sec)
te = setTimeout(r.hide, sec * 1000);
if (same && delta < 1000)
if (same && delta < 1000) {
var tb = ebi('toastt');
if (tb) {
tb.style.animation = 'none';
tb.offsetHeight;
tb.style.animation = null;
}
return;
}
if (txt.indexOf('<body>') + 1)
txt = txt.slice(0, txt.indexOf('<')) + ' [...]';
obj.innerHTML = '<a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
var html = '';
if (sec) {
setcvar('--tmtime', (sec - 0.15) + 's');
setcvar('--tmstep', Math.floor(sec * 20));
html += '<div id="toastt"></div>';
}
obj.innerHTML = html + '<a href="#" id="toastc">x</a><div id="toastb">' + lf2br(txt) + '</div>';
obj.className = cl;
sec += obj.offsetWidth;
obj.className += ' vis';
@@ -1530,9 +1587,12 @@ var modal = (function () {
var r = {},
q = [],
o = null,
scrolling = null,
cb_up = null,
cb_ok = null,
cb_ng = null,
sel_0 = 0,
sel_1 = 0,
tok, tng, prim, sec, ok_cancel;
r.load = function () {
@@ -1548,6 +1608,7 @@ var modal = (function () {
r.nofocus = 0;
r.show = function (html) {
tt.hide();
o = mknod('div', 'modal');
o.innerHTML = '<table><tr><td><div id="modalc">' + html + '</div></td></tr></table>';
document.body.appendChild(o);
@@ -1566,11 +1627,12 @@ var modal = (function () {
(inp || a).focus();
if (inp)
setTimeout(function () {
inp.setSelectionRange(0, inp.value.length, "forward");
inp.setSelectionRange(sel_0, sel_1, "forward");
}, 0);
document.addEventListener('focus', onfocus);
document.addEventListener('selectionchange', onselch);
timer.add(scrollchk, 1);
timer.add(onfocus);
if (cb_up)
setTimeout(cb_up, 1);
@@ -1578,6 +1640,8 @@ var modal = (function () {
r.hide = function () {
timer.rm(onfocus);
timer.rm(scrollchk);
scrolling = null;
try {
ebi('modal-ok').removeEventListener('blur', onblur);
}
@@ -1596,13 +1660,28 @@ var modal = (function () {
r.hide();
if (cb_ok)
cb_ok(v);
}
};
var ng = function (e) {
ev(e);
r.hide();
if (cb_ng)
cb_ng(null);
}
};
var scrollchk = function () {
if (scrolling === true)
return;
var o = ebi('modalc'),
vis = o.offsetHeight,
all = o.scrollHeight,
nsc = 8 + vis < all;
if (scrolling !== nsc)
clmod(o, 'yk', !nsc);
scrolling = nsc;
};
var onselch = function () {
try {
@@ -1689,16 +1768,18 @@ var modal = (function () {
r.show(html);
}
r.prompt = function (html, v, cok, cng, fun) {
r.prompt = function (html, v, cok, cng, fun, so0, so1) {
q.push(function () {
_prompt(lf2br(html), v, cok, cng, fun);
_prompt(lf2br(html), v, cok, cng, fun, so0, so1);
});
next();
}
var _prompt = function (html, v, cok, cng, fun) {
var _prompt = function (html, v, cok, cng, fun, so0, so1) {
cb_ok = cok;
cb_ng = cng === undefined ? cok : null;
cb_up = fun;
sel_0 = so0 || 0;
sel_1 = so1 === undefined ? v.length : so1;
html += '<input id="modali" type="text" ' + NOAC + ' /><div id="modalb">' + ok_cancel + '</div>';
r.show(html);
@@ -1885,7 +1966,7 @@ function md_thumbs(md) {
float = has(flags, 'l') ? 'left' : has(flags, 'r') ? 'right' : '';
if (!/[?&]cache/.exec(url))
url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=i';
url = addq(url, 'cache=i');
md[a] = '<a href="' + url + '" class="mdth mdth' + float.slice(0, 1) + '"><img src="' + url + '&th=w" alt="' + alt + '" /></a>' + md[a].slice(o2 + 1);
}
@@ -2014,6 +2095,9 @@ function xhrchk(xhr, prefix, e404, lvl, tag) {
if (xhr.status == 404)
return toast.err(0, prefix + e404 + suf, tag);
if (!xhr.status && !errtxt)
return toast.err(0, prefix + L.xhr0);
if (is_cf && (xhr.status == 403 || xhr.status == 503)) {
var now = Date.now(), td = now - cf_cha_t;
if (td < 15000)

View File

@@ -20,6 +20,7 @@ catch (ex) {
function load_fb() {
subtle = null;
importScripts('deps/sha512.hw.js');
console.log('using fallback hasher');
}
@@ -29,6 +30,12 @@ var reader = null,
onmessage = (d) => {
if (d.data == 'nosubtle')
return load_fb();
if (d.data == 'ping')
return postMessage(['pong']);
if (busy)
return postMessage(["panic", 'worker got another task while busy']);

View File

@@ -5,9 +5,6 @@ a living list of upcoming features / fixes / changes, very roughly in order of p
* maybe resumable downloads (chrome-only, jank api)
* maybe checksum validation (return sha512 of requested range in responses, and probably also warks)
* [github issue #64](https://github.com/9001/copyparty/issues/64) - dirkeys 2nd season
* popular feature request, finally time to refactor browser.js i suppose...
* [github issue #37](https://github.com/9001/copyparty/issues/37) - upload PWA
* or [maybe not](https://arstechnica.com/tech-policy/2024/02/apple-under-fire-for-disabling-iphone-web-apps-eu-asks-developers-to-weigh-in/), or [maybe](https://arstechnica.com/gadgets/2024/03/apple-changes-course-will-keep-iphone-eu-web-apps-how-they-are-in-ios-17-4/)

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,7 @@
## devnotes toc
* top
* [future plans](#future-plans) - some improvement ideas
* [future ideas](#future-ideas) - list of dreams which will probably never happen
* [design](#design)
* [up2k](#up2k) - quick outline of the up2k protocol
* [why not tus](#why-not-tus) - I didn't know about [tus](https://tus.io/)
@@ -12,6 +12,8 @@
* [write](#write)
* [admin](#admin)
* [general](#general)
* [event hooks](#event-hooks) - on writing your own [hooks](../README.md#event-hooks)
* [hook effects](#hook-effects) - hooks can cause intentional side-effects
* [assumptions](#assumptions)
* [mdns](#mdns)
* [sfx repack](#sfx-repack) - reduce the size of an sfx by removing features
@@ -20,13 +22,14 @@
* [just the sfx](#just-the-sfx)
* [build from release tarball](#build-from-release-tarball) - uses the included prebuilt webdeps
* [complete release](#complete-release)
* [todo](#todo) - roughly sorted by priority
* [debugging](#debugging)
* [music playback halting on phones](#music-playback-halting-on-phones) - mostly fine on android
* [discarded ideas](#discarded-ideas)
# future plans
# future ideas
some improvement ideas
list of dreams which will probably never happen
* the JS is a mess -- a ~~preact~~ rewrite would be nice
* preferably without build dependencies like webpack/babel/node.js, maybe a python thing to assemble js files into main.js
@@ -54,8 +57,8 @@ quick outline of the up2k protocol, see [uploading](https://github.com/9001/cop
* server creates the `wark`, an identifier for this upload
* `sha512( salt + filesize + chunk_hashes )`
* and a sparse file is created for the chunks to drop into
* client uploads each chunk
* header entries for the chunk-hash and wark
* client sends a series of POSTs, with one or more consecutive chunks in each
* header entries for the chunk-hashes (comma-separated) and wark
* server writes chunks into place based on the hash
* client does another handshake with the hashlist; server replies with OK or a list of chunks to reupload
@@ -133,10 +136,16 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?zip=utf-8` | ...as a zip file |
| GET | `?zip` | ...as a WinXP-compatible zip file |
| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
| GET | `?tar&w` | pregenerate webp thumbnails |
| GET | `?tar&j` | pregenerate jpg thumbnails |
| GET | `?tar&p` | pregenerate audio waveforms |
| GET | `?shares` | list your shared files/folders |
| GET | `?dls` | show active downloads (do this as admin) |
| GET | `?ups` | show recent uploads from your IP |
| GET | `?ups&filter=f` | ...where URL contains `f` |
| GET | `?mime=foo` | specify return mimetype `foo` |
| GET | `?v` | render markdown file at URL |
| GET | `?v` | open image/video/audio in mediaplayer |
| GET | `?txt` | get file at URL as plaintext |
| GET | `?txt=iso-8859-1` | ...with specific charset |
| GET | `?th` | get image/video at URL as thumbnail |
@@ -155,6 +164,7 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| method | params | result |
|--|--|--|
| POST | `?copy=/foo/bar` | copy the file/folder at URL to /foo/bar |
| POST | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
| method | params | body | result |
@@ -165,8 +175,12 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| mPOST | | `f=FILE` | upload `FILE` into the folder at URL |
| mPOST | `?j` | `f=FILE` | ...and reply with json |
| mPOST | `?replace` | `f=FILE` | ...and overwrite existing files |
| mPOST | `?media` | `f=FILE` | ...and return medialink (not hotlink) |
| mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
| POST | `?delete` | | delete URL recursively |
| POST | `?eshare=rm` | | stop sharing a file/folder |
| POST | `?eshare=3` | | set expiration to 3 minutes |
| jPOST | `?share` | (complicated) | create temp URL for file/folder |
| jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
| uPOST | | `msg=foo` | send message `foo` into server log |
| mPOST | | `act=tput`, `body=TEXT` | overwrite markdown document at URL |
@@ -196,6 +210,38 @@ upload modifiers:
| method | params | result |
|--|--|--|
| GET | `?pw=x` | logout |
| GET | `?grid` | ui: show grid-view |
| GET | `?imgs` | ui: show grid-view with thumbnails |
| GET | `?grid=0` | ui: show list-view |
| GET | `?imgs=0` | ui: show list-view |
| GET | `?thumb` | ui, grid-mode: show thumbnails |
| GET | `?thumb=0` | ui, grid-mode: show icons |
# event hooks
on writing your own [hooks](../README.md#event-hooks)
## hook effects
hooks can cause intentional side-effects, such as redirecting an upload into another location, or creating+indexing additional files, or deleting existing files, by returning json on stdout
* `reloc` can redirect uploads before/after uploading has finished, based on filename, extension, file contents, uploader ip/name etc.
* `idx` informs copyparty about a new file to index as a consequence of this upload
* `del` tells copyparty to delete an unrelated file by vpath
for these to take effect, the hook must be defined with the `c1` flag; see example [reloc-by-ext](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/reloc-by-ext.py)
a subset of effect types are available for a subset of hook types,
* most hook types (xbu/xau/xbr/xar/xbd/xad/xm) support `idx` and `del` for all http protocols (up2k / basic-uploader / webdav), but not ftp/tftp/smb
* most hook types will abort/reject the action if the hook returns nonzero, assuming flag `c` is given, see examples [reject-extension](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/reject-extension.py) and [reject-mimetype](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/reject-mimetype.py)
* `xbu` supports `reloc` for all http protocols (up2k / basic-uploader / webdav), but not ftp/tftp/smb
* `xau` supports `reloc` for basic-uploader / webdav only, not up2k or ftp/tftp/smb
* so clients like sharex are supported, but not dragdrop into browser
to trigger indexing of files `/foo/1.txt` and `/foo/bar/2.txt`, a hook can `print(json.dumps({"idx":{"vp":["/foo/1.txt","/foo/bar/2.txt"]}}))` (and replace "idx" with "del" to delete instead)
* note: paths starting with `/` are absolute URLs, but you can also do `../3.txt` relative to the destination folder of each uploaded file
# assumptions
@@ -241,6 +287,7 @@ the rest is mostly optional; if you need a working env for vscode or similar
python3 -m venv .venv
. .venv/bin/activate
pip install jinja2 strip_hints # MANDATORY
pip install argon2-cffi # password hashing
pip install mutagen # audio metadata
pip install pyftpdlib # ftp server
pip install partftpy # tftp server
@@ -301,19 +348,26 @@ in the `scripts` folder:
* run `./rls.sh 1.2.3` which uploads to pypi + creates github release + sfx
# todo
# debugging
roughly sorted by priority
## music playback halting on phones
* nothing! currently
mostly fine on android, but still haven't find a way to massage iphones into behaving well
* conditionally starting/stopping mp.fau according to mp.au.readyState <3 or <4 doesn't help
* loop=true doesn't work, and manually looping mp.fau from an onended also doesn't work (it does nothing)
* assigning fau.currentTime in a timer doesn't work, as safari merely pretends to assign it
* on ios 16.7.7, mp.fau can sometimes make everything visibly work correctly, but no audio is actually hitting the speakers
can be reproduced with `--no-sendfile --s-wr-sz 8192 --s-wr-slp 0.3 --rsp-slp 6` and then play a collection of small audio files with the screen off, `ffmpeg -i track01.cdda.flac -c:a libopus -b:a 128k -segment_time 12 -f segment smol-%02d.opus`
## discarded ideas
* reduce up2k roundtrips
* start from a chunk index and just go
* terminate client on bad data
* not worth the effort, just throw enough conncetions at it
* optimization attempts which didn't improve performance
* remove brokers / multiprocessing stuff; https://github.com/9001/copyparty/tree/no-broker
* reduce the nesting / indirections in `HttpCli` / `httpcli.py`
* nearly zero benefit from stuff like replacing all the `self.conn.hsrv` with a local `hsrv` variable
* single sha512 across all up2k chunks?
* crypto.subtle cannot into streaming, would have to use hashwasm, expensive
* separate sqlite table per tag

View File

@@ -0,0 +1,45 @@
the following setup appears to work (copyparty starts, accepts uploads, is able to persist config)
tested on debian 12 using [portainer-ce](https://docs.portainer.io/start/install-ce/server/docker/linux) with [docker-ce](https://docs.docker.com/engine/install/debian/) as root (not rootless)
before making the container, first `mkdir /etc/copyparty /srv/pub` which will be bind-mounts into the container
> both `/etc/copyparty` and `/srv/pub` are examples; you can change them if you'd like
put your copyparty config files directly into `/etc/copyparty` and the files to share inside `/srv/pub`
on first startup, copyparty will create a subfolder inside `/etc/copyparty` called `copyparty` where it puts some runtime state; for example replacing `/etc/copyparty/copyparty/cert.pem` with another TLS certificate is a quick and dirty way to get valid HTTPS (if you really want copyparty to handle that and not a reverse-proxy)
## in portainer:
```
environments -> local -> containers -> add container:
name = copyparty-ac
registry = docker hub
image = copyparty/ac
always pull = no
manual network port publishing:
3923 to 3923 [TCP]
advanced -> command & logging:
console = interactive & tty
advanced -> volumes -> map additional volume:
container = /cfg [Bind]
host = /etc/copyparty [Writable]
advanced -> volumes -> map additional volume:
container = /w [Bind]
host = /srv/pub [Writable]
```
notes:
* `/cfg` is where copyparty expects to find its config files; `/etc/copyparty` is just an example mapping to that
* `/w` is where copyparty expects to find the folder to share; `/srv/pub` is just an example mapping to that
* the volumes must be bind-mounts to avoid permission issues (or so the theory goes)

View File

@@ -16,7 +16,7 @@ open up notepad and save the following as `c:\users\you\documents\party.conf` (f
```yaml
[global]
lo: ~/logs/cpp-%Y-%m%d.xz # log to c:\users\you\logs\
e2dsa, e2ts, no-dedup, z # sets 4 flags; see expl.
e2dsa, e2ts, z # sets 3 flags; see explanation
p: 80, 443 # listen on ports 80 and 443, not 3923
theme: 2 # default theme: protonmail-monokai
lang: nor # default language: viking
@@ -46,11 +46,10 @@ open up notepad and save the following as `c:\users\you\documents\party.conf` (f
### config explained: [global]
the `[global]` section accepts any config parameters you can see when running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
the `[global]` section accepts any config parameters [listed here](https://ocv.me/copyparty/helptext.html), also viewable by running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts -z -p 80,443 --theme 2 --lang nor`
* `lo: ~/logs/cpp-%Y-%m%d.xz` writes compressed logs (the compression will make them delayed)
* `e2dsa` enables the upload deduplicator and file indexer, which enables searching
* `e2dsa` enables the file indexer, which enables searching and upload-undo
* `e2ts` enables music metadata indexing, making albums / titles etc. searchable too
* `no-dedup` writes full dupes to disk instead of symlinking, since lots of windows software doesn't handle symlinks well
* but the improved upload speed from `e2dsa` is not affected
* `z` enables zeroconf, making the server available at `http://HOSTNAME.local/` from any other machine in the LAN
* `p: 80,443` listens on the ports `80` and `443` instead of the default `3923`

210
docs/logo.svg Normal file
View File

@@ -0,0 +1,210 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
width="300mm"
height="207mm"
viewBox="0 0 300 207"
version="1.1"
id="svg1"
inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<title
id="title1">copyparty_logo</title>
<defs
id="defs1">
<linearGradient
inkscape:collect="always"
id="linearGradient1">
<stop
style="stop-color:#ffcc55;stop-opacity:1"
offset="0"
id="stop1" />
<stop
style="stop-color:#ffcc00;stop-opacity:1"
offset="0.2"
id="stop2" />
<stop
style="stop-color:#ff8800;stop-opacity:1"
offset="1"
id="stop3" />
</linearGradient>
<linearGradient
inkscape:collect="always"
xlink:href="#linearGradient1"
id="linearGradient2"
x1="15"
y1="15"
x2="15"
y2="143"
gradientUnits="userSpaceOnUse" />
</defs>
<metadata
id="metadata5">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title>copyparty_logo</dc:title>
<dc:source>github.com/9001/copyparty</dc:source>
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:groupmode="layer"
id="layer1"
inkscape:label="kassett">
<rect
style="fill:#333333"
id="rect1"
width="300"
height="205"
x="0"
y="0"
rx="12"
ry="12" />
<rect
style="fill:url(#linearGradient2)"
id="rect2"
width="270"
height="128"
x="15"
y="15"
rx="8"
ry="8" />
<rect
style="fill:#333333"
id="rect3"
width="172"
height="52"
x="64"
y="72"
rx="26"
ry="26" />
<circle
style="fill:#cccccc"
id="circle1"
cx="91"
cy="98"
r="18" />
<circle
style="fill:#cccccc"
id="circle2"
cx="209"
cy="98"
r="18" />
<path
style="fill:#737373;stroke-width:1px"
d="m 48,207 10,-39 c 1.79,-6.2 5.6,-7.8 12,-8 60,-1 100,-1 160,0 6.4,0.2 10,1.8 12,8 l 10,39 z"
id="path1"
sodipodi:nodetypes="ccccccc" />
</g>
<g
inkscape:groupmode="layer"
id="layer3"
inkscape:label="tekst"
style="display:none">
<text
xml:space="preserve"
style="font-size:38.8056px;line-height:1.25;font-family:Akbar;-inkscape-font-specification:Akbar;letter-spacing:3.70417px;word-spacing:0px;fill:#333333"
x="47.153069"
y="55.548954"
id="text1"><tspan
sodipodi:role="line"
id="tspan1"
x="47.153069"
y="55.548954"
style="-inkscape-font-specification:Akbar"
rotate="0 0">copyparty</tspan></text>
</g>
<g
inkscape:groupmode="layer"
id="layer4"
inkscape:label="stensatt">
<path
d="m 63.5,50.9 q -0.85,0.93 -4.73,2.3 -3.6,1.3 -4.4,1.3 -3.3,0 -5.1,-2.1 -1.75,-2 -1.75,-5.36 0,-4.6 3.76,-7.64 3.3,-2.7 7.3,-2.7 0.4,0 0.93,0.74 0.54,0.7 0.54,1.16 0,2.06 -2.2,2.7 -1.36,0.4 -4.04,1.16 -2.2,1.16 -2.2,4.4 0,3.2 2.9,3.2 0.85,0 0.85,0 0.54,0 1.44,-0.16 1.1,-0.23 2.9,-0.74 1.8,-0.54 2.13,-0.54 0.4,0 1.75,0.6 z"
style="fill:#333333"
id="path11" />
<path
d="m 87.6,45 q 0,4.2 -3.7,6.95 -3.2,2.3 -6.87,2.3 -3.4,0 -6,-2.6 -2.5,-2.6 -2.5,-6 0,-3.6 3.14,-6.64 3.2,-3 6.8,-3 3.5,0 6.3,2.76 2.83,2.76 2.83,6.25 z m -3.4,0.16 q 0,-2.25 -1.75,-3.7 -1.7,-1.5 -4,-1.5 -0.1,0 -1.6,1.6 -1.44,1.55 -2.44,1.55 -0.6,0 -0.8,-0.3 -1.16,2.3 -1.16,3 0,2.25 2.13,3.4 1.6,0.9 3.6,0.9 2,0 3.76,-1.1 2.25,-1.4 2.25,-3.84 z"
style="fill:#333333"
id="path12" />
<path
d="m 112.8,46.8 q 0,2.8 -1.9,4.4 -1.8,1.5 -4.7,1.5 -0.7,0 -2.7,-0.4 -1.9,-0.4 -2.6,-0.4 -2.1,0 -2.1,2.64 0,0.85 0.23,2.6 0.2,1.75 0.2,2.6 0,1.9 -0.77,2.83 -1.44,0 -3,-0.85 -1.46,-9.5 -1.46,-12 0,-3.65 1.75,-8.1 2.37,-6.05 6.45,-6.05 3.7,0 7.3,4.1 3.3,3.84 3.3,7.14 z m -3.8,0.2 q -0.6,-2.2 -2.6,-4.4 -2.3,-2.5 -4.3,-2.5 -1.3,0 -2.33,2.2 -0.9,1.8 -0.9,3.26 0,0.47 0.38,1.24 0.43,0.8 0.85,0.8 1.1,0 3.2,0.3 2.1,0.3 3.2,0.3 0.3,0 1.3,-0.4 1,-0.47 1.3,-0.74 z"
style="fill:#333333"
id="path13" />
<path
d="m 133,40 q -2.1,4.1 -3.2,7 -0.1,0.3 -1.6,4.5 -0.4,1.36 -1,4.2 -0.5,2.83 -1,4.2 -1,2.83 -2.3,2.64 -1.4,-0.2 -1.6,-1.6 0,-0.2 0,-0.5 0,-0.16 0.3,-1.5 1,-5.04 1,-6.44 0,-0.54 -0.1,-0.74 -1.4,-2.44 -4.1,-7.4 -2.7,-4.97 -2.4,-7.7 1.5,-1.36 2.1,-1.36 0.4,0 1.1,0.6 0.6,0.6 0.7,1.1 0.8,6.2 4.9,11.1 1,-1.8 1.8,-4.04 0.5,-1.4 1.6,-4.15 1.9,-4.46 3.4,-4.46 0.2,0 0.4,0.1 0.9,0.3 1.3,2.8 z"
style="fill:#333333"
id="path14" />
<path
d="m 157.5,48 q 0,2.8 -1.9,4.4 -1.8,1.5 -4.7,1.5 -0.7,0 -2.7,-0.4 -1.9,-0.4 -2.6,-0.4 -2,0 -2,2.64 0,0.85 0.2,2.6 0.2,1.75 0.2,2.6 0,1.9 -0.7,2.83 -1.5,0 -3,-0.85 -1.5,-9.5 -1.5,-11.95 0,-3.65 1.8,-8.1 2.3,-6.05 6.4,-6.05 3.7,0 7.2,4.1 3.3,3.84 3.3,7.14 z m -3.8,0.2 q -0.6,-2.2 -2.6,-4.4 -2.3,-2.5 -4.3,-2.5 -1.3,0 -2.3,2.2 -0.9,1.8 -0.9,3.26 0,0.47 0.4,1.24 0.4,0.8 0.8,0.8 1.1,0 3.2,0.3 2.1,0.3 3.2,0.3 0.3,0 1.3,-0.4 1,-0.47 1.3,-0.74 z"
style="fill:#333333"
id="path15" />
<path
d="m 182,53.3 q 0,0.9 -0.6,1.5 -0.6,0.6 -1.4,0.6 -1.6,0 -3,-0.9 -1.4,-0.93 -2.1,-2.3 -0.7,-0.1 -1.5,0.85 -0.9,1.16 -1.1,1.24 -1.2,0.54 -3.9,0.54 -2.2,0 -3.9,-2.44 -1.5,-2.13 -1.5,-4 0,-3.4 3.4,-6.4 3.2,-2.9 6.7,-2.9 0.9,0 1.7,0.6 0.8,0.6 0.8,1.44 0,0.54 -0.4,1.1 2.4,0.9 2.4,2.83 0,0.35 -0.1,1.05 -0.1,0.7 -0.1,1.05 0,0.4 0.1,0.6 0.5,1.3 2.5,3.4 1.9,1.9 1.9,2.2 z m -8.1,-10.1 q -0.4,0 -1.1,-0.1 -0.8,-0.16 -1.1,-0.16 -1.3,0 -3.2,1.94 -1.9,1.94 -1.9,3.3 0,0.8 0.7,1.8 0.9,1.3 2.2,1.3 2.6,0 3.5,-2.9 0.5,-2.6 1,-5.16 z"
style="fill:#333333"
id="path16" />
<path
d="m 203.8,42.4 q -0.4,0.4 -1.5,0.4 -0.9,0 -2.5,-0.3 -1.7,-0.3 -2.5,-0.3 -4.7,0 -5.5,6.9 -0.3,3.1 -0.4,3.3 -0.4,1 -1.7,2.3 h -1.1 q -0.7,-1.2 -1.3,-4.1 -0.6,-2.76 -0.6,-4.27 0,-1.16 0.1,-1.5 0.2,-0.54 1,-0.54 0.3,0 0.6,0.3 0.4,0.3 0.4,0.3 1.9,-3.53 3.1,-4.6 1.8,-1.7 5.1,-1.7 1.4,0 3.6,0.9 2.8,1.16 3.3,2.8 z"
style="fill:#333333"
id="path17" />
<path
d="m 229.5,37.16 q 0.3,0.8 0.3,1.44 0,1.86 -2.4,1.86 -1,0 -3.5,-0.5 -2.5,-0.54 -3.4,-0.54 -1.3,0 -1.5,0.1 -0.4,0.2 -0.4,1.2 0,2.2 0.6,6.9 0.7,5.86 1.6,6.13 -0.4,0.35 -0.4,1.1 -1.2,0.7 -2.6,0.7 -1.4,0 -2,-3.9 -0.2,-1.36 -0.5,-7.76 -0.2,-4.6 -0.8,-5.5 -0.3,-0.47 -4.3,-0.35 -1,0 -1.6,0.1 -0.5,0 -0.3,0 -0.8,0 -1.2,-0.7 -0.5,-1.3 -0.5,-1.4 0,-1.44 4.1,-2 1.6,-0.16 4.7,-0.5 0,-0.85 -0.1,-2.56 0,-1.75 0,-2.6 0,-4.35 2.1,-4.35 0.5,0 1.1,0.6 0.6,0.6 0.6,1.1 v 7.9 q 1.1,1.2 5,1.7 3.9,0.5 5.3,1.86 z"
style="fill:#333333"
id="path18" />
<path
d="m 251.2,40.2 q -2,4.1 -3.2,7 -0.1,0.3 -1.5,4.5 -0.5,1.36 -1,4.2 -0.5,2.83 -1,4.2 -1,2.83 -2.4,2.64 -1.4,-0.2 -1.5,-1.6 -0.1,-0.2 -0.1,-0.5 0,-0.16 0.3,-1.5 1.1,-5.04 1.1,-6.44 0,-0.54 -0.1,-0.74 -1.4,-2.44 -4.1,-7.4 -2.7,-4.97 -2.4,-7.7 1.4,-1.36 2.1,-1.36 0.4,0 1,0.6 0.6,0.6 0.7,1.1 0.9,6.2 4.9,11.1 1,-1.8 1.9,-4.04 0.5,-1.4 1.6,-4.15 1.8,-4.46 3.4,-4.46 0.2,0 0.4,0.1 0.8,0.3 1.2,2.8 z"
style="fill:#333333"
id="path19" />
</g>
<g
inkscape:groupmode="layer"
id="layer5"
inkscape:label="tagger">
<g
id="g1">
<path
id="path4"
style="fill:#333333"
d="m 111.4,83.335 -9.526,5.5 2.5,4.33 9.526,-5.5 z m -33.775,19.5 -9.526,5.5 2.5,4.33 9.526,-5.5 z"
sodipodi:nodetypes="cccccccccc" />
<path
id="path5"
style="fill:#333333"
d="M 88.5,73 V 84 h 5 V 73 Z m 0,39 v 11 h 5 V 112 Z"
sodipodi:nodetypes="cccccccccc" />
<path
id="path6"
style="fill:#333333"
d="m 68.1,87.665 9.526,5.5 2.5,-4.33 -9.526,-5.5 z m 33.775,19.5 9.527,5.5 2.5,-4.33 -9.527,-5.5 z"
sodipodi:nodetypes="cccccccccc" />
</g>
<g
id="g2"
transform="rotate(30,150,318.19)">
<path
id="path7"
style="fill:#333333"
d="m 111.4,83.335 -9.526,5.5 2.5,4.33 9.526,-5.5 z m -33.775,19.5 -9.526,5.5 2.5,4.33 9.526,-5.5 z"
sodipodi:nodetypes="cccccccccc" />
<path
id="path8"
style="fill:#333333"
d="M 88.5,73 V 84 h 5 V 73 Z m 0,39 v 11 h 5 V 112 Z"
sodipodi:nodetypes="cccccccccc" />
<path
id="path9"
style="fill:#333333"
d="m 68.1,87.665 9.526,5.5 2.5,-4.33 -9.526,-5.5 z m 33.775,19.5 9.527,5.5 2.5,-4.33 -9.527,-5.5 z"
sodipodi:nodetypes="cccccccccc" />
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 8.3 KiB

View File

@@ -141,6 +141,9 @@ find -maxdepth 1 -printf '%s %p\n' | sort -n | awk '!/-([0-9a-zA-Z_-]{11})\.(mkv
# unique stacks in a stackdump
f=a; rm -rf stacks; mkdir stacks; grep -E '^#' $f | while IFS= read -r n; do awk -v n="$n" '!$0{o=0} o; $0==n{o=1}' <$f >stacks/f; h=$(sha1sum <stacks/f | cut -c-16); mv stacks/f stacks/$h-"$n"; done ; find stacks/ | sort | uniq -cw24
# find unused css variables
cat browser.css | sed -r 's/(var\()/\n\1/g' | awk '{sub(/:/," ")} $1~/^--/{d[$1]=1} /var\(/{sub(/.*var\(/,"");sub(/\).*/,"");u[$1]=1} END{for (x in u) delete d[x]; for (x in d) print x}' | tr '\n' '|'
##
## sqlite3 stuff
@@ -221,6 +224,11 @@ sox -DnV -r8000 -b8 -c1 /dev/shm/a.wav synth 1.1 sin 400 vol 0.02
# play icon calibration pics
for w in 150 170 190 210 230 250; do for h in 130 150 170 190 210; do /c/Program\ Files/ImageMagick-7.0.11-Q16-HDRI/magick.exe convert -size ${w}x${h} xc:brown -fill orange -draw "circle $((w/2)),$((h/2)) $((w/2)),$((h/3))" $w-$h.png; done; done
# compress chiptune modules
mkdir gz; for f in *.*; do pigz -c11 -I100 <"$f" >gz/"$f"gz; touch -r "$f" gz/"$f"gz; done
mkdir xz; for f in *.*; do xz -cz9 <"$f" >xz/"$f"xz; touch -r "$f" xz/"$f"xz; done
mkdir z; for f in *.*; do 7z a -tzip -mx=9 -mm=lzma "z/${f}z" "$f" && touch -r "$f" z/"$f"z; done
##
## vscode
@@ -247,6 +255,11 @@ cat copyparty/httpcli.py | awk '/^[^a-zA-Z0-9]+def / {printf "%s\n%s\n\n", f, pl
# create a folder with symlinks to big files
for d in /usr /var; do find $d -type f -size +30M 2>/dev/null; done | while IFS= read -r x; do ln -s "$x" big/; done
# up2k worst-case testfiles: create 64 GiB (256 x 256 MiB) of sparse files; each file takes 1 MiB disk space; each 1 MiB chunk is globally unique
for f in {0..255}; do echo $f; truncate -s 256M $f; b1=$(printf '%02x' $f); for o in {0..255}; do b2=$(printf '%02x' $o); printf "\x$b1\x$b2" | dd of=$f bs=2 seek=$((o*1024*1024)) conv=notrunc 2>/dev/null; done; done
# create 6.06G file with 16 bytes of unique data at start+end of each 32M chunk
sz=6509559808; truncate -s $sz f; csz=33554432; sz=$((sz/16)); step=$((csz/16)); ofs=0; while [ $ofs -lt $sz ]; do dd if=/dev/urandom of=f bs=16 count=2 seek=$ofs conv=notrunc iflag=fullblock; [ $ofs = 0 ] && ofs=$((ofs+step-1)) || ofs=$((ofs+step)); done
# py2 on osx
brew install python@2
pip install virtualenv

View File

@@ -1,82 +1,71 @@
# recipe for building an exe with nuitka (extreme jank edition)
#
# NOTE: win7 and win10 builds both work on win10 but
# on win7 they immediately c0000005 in kernelbase.dll
#
# first install python-3.6.8-amd64.exe
# [x] add to path
#
NOTE: copyparty runs SLOWER when compiled with nuitka;
just use copyparty-sfx.py and/or pyinstaller instead
( the sfx and the pyinstaller EXEs are equally fast if you
have the latest jinja2 installed, but the older jinja that
comes bundled with the sfx is slightly faster yet )
roughly, copyparty-sfx.py is 6% faster than copyparty.exe
(win10-pyinstaller), and copyparty.exe is 10% faster than
nuitka, making copyparty-sfx.py 17% faster than nuitka
NOTE: every time a nuitka-compiled copyparty.exe is launched,
it will show the windows firewall prompt since nuitka will
pick a new unique location in %TEMP% to unpack an exe into,
unlike pyinstaller which doesn't fork itself on startup...
might be fixable by configuring nuitka differently, idk
NOTE: nuitka EXEs are larger than pyinstaller ones;
a minimal nuitka build of just the sfx (with its bundled
dependencies) was already the same size as the pyinstaller
copyparty.exe which also includes Mutagen and Pillow
NOTE: nuitka takes a lot longer to build than pyinstaller
(due to actual compilation of course, but still)
NOTE: binaries built with nuitka cannot run on windows7,
even when compiled with python 3.6 on windows 7 itself
NOTE: `--python-flags=-m` is the magic sauce to
correctly compile `from .util import Daemon`
(which otherwise only explodes at runtime)
NOTE: `--deployment` doesn't seem to affect performance
########################################################################
# copypaste the rest of this file into cmd
rem from pypi
cd \users\ed\downloads
python -m pip install --user Nuitka-0.6.14.7.tar.gz
rem https://github.com/brechtsanders/winlibs_mingw/releases/download/10.2.0-11.0.0-8.0.0-r5/winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\
copy c:\users\ed\downloads\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
rem https://github.com/ccache/ccache/releases/download/v3.7.12/ccache-3.7.12-windows-32.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\
copy c:\users\ed\downloads\ccache-3.7.12-windows-32.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\ccache-3.7.12-windows-32.zip
python -m pip install --user -U nuitka
rem https://dependencywalker.com/depends22_x64.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\
copy c:\users\ed\downloads\depends22_x64.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\depends22_x64.zip
cd %homedrive%
cd %homepath%\downloads
cd \
rd /s /q %appdata%\..\local\temp\pe-copyparty
cd \users\ed\downloads
python copyparty-sfx.py -h
cd %appdata%\..\local\temp\pe-copyparty\copyparty
rd /s /q copypuitka
mkdir copypuitka
cd copypuitka
python
import os, re
os.rename('../dep-j2/jinja2', '../jinja2')
os.rename('../dep-j2/markupsafe', '../markupsafe')
rd /s /q %temp%\pe-copyparty
python ..\copyparty-sfx.py --version
print("# nuitka dies if .__init__.stuff is imported")
with open('__init__.py','r',encoding='utf-8') as f:
t1 = f.read()
move %temp%\pe-copyparty\copyparty .\
move %temp%\pe-copyparty\partftpy .\
move %temp%\pe-copyparty\ftp\pyftpdlib .\
move %temp%\pe-copyparty\j2\jinja2 .\
move %temp%\pe-copyparty\j2\markupsafe .\
with open('util.py','r',encoding='utf-8') as f:
t2 = f.read().split('\n')[3:]
rd /s /q %temp%\pe-copyparty
t2 = [x for x in t2 if 'from .__init__' not in x]
t = t1 + '\n'.join(t2)
with open('__init__.py','w',encoding='utf-8') as f:
f.write('\n')
python -m nuitka ^
--onefile --deployment --python-flag=-m ^
--include-package=markupsafe ^
--include-package=jinja2 ^
--include-package=partftpy ^
--include-package=pyftpdlib ^
--include-data-dir=copyparty\web=copyparty\web ^
--include-data-dir=copyparty\res=copyparty\res ^
--run copyparty
with open('util.py','w',encoding='utf-8') as f:
f.write(t)
print("# local-imports fail, prefix module names")
ptn = re.compile(r'^( *from )(\.[^ ]+ import .*)')
for d, _, fs in os.walk('.'):
for f in fs:
fp = os.path.join(d, f)
if not fp.endswith('.py'):
continue
t = ''
with open(fp,'r',encoding='utf-8') as f:
for ln in [x.rstrip('\r\n') for x in f]:
m = ptn.match(ln)
if not m:
t += ln + '\n'
continue
p1, p2 = m.groups()
t += "{}copyparty{}\n".format(p1, p2).replace("__init__", "util")
with open(fp,'w',encoding='utf-8') as f:
f.write(t)
exit()
cd ..
rd /s /q bout & python -m nuitka --standalone --onefile --windows-onefile-tempdir --python-flag=no_site --assume-yes-for-downloads --include-data-dir=copyparty\web=copyparty\web --include-data-dir=copyparty\res=copyparty\res --run --output-dir=bout --mingw64 --include-package=markupsafe --include-package=jinja2 copyparty

View File

@@ -1,52 +0,0 @@
pyoxidizer doesn't crosscompile yet so need to build in a windows vm,
luckily possible to do mostly airgapped (https-proxy for crates)
none of this is version-specific but doing absolute links just in case
(only exception is py3.8 which is the final win7 ver)
# deps (download on linux host):
https://www.python.org/ftp/python/3.10.7/python-3.10.7-amd64.exe
https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.22.0/pyoxidizer-0.22.0-x86_64-pc-windows-msvc.zip
https://github.com/upx/upx/releases/download/v3.96/upx-3.96-win64.zip
https://static.rust-lang.org/dist/rust-1.61.0-x86_64-pc-windows-msvc.msi
https://github.com/indygreg/python-build-standalone/releases/download/20220528/cpython-3.8.13%2B20220528-i686-pc-windows-msvc-static-noopt-full.tar.zst
# need cl.exe, prefer 2017 -- download on linux host:
https://visualstudio.microsoft.com/downloads/?q=build+tools
https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-history#release-dates-and-build-numbers
https://aka.ms/vs/15/release/vs_buildtools.exe # 2017
https://aka.ms/vs/16/release/vs_buildtools.exe # 2019
https://aka.ms/vs/17/release/vs_buildtools.exe # 2022
https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2017
# use disposable w10 vm to prep offline installer; xfer to linux host with firefox to copyparty
vs_buildtools-2017.exe --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools --add Microsoft.VisualStudio.Component.Windows10SDK.17763 --layout c:\msbt2017 --lang en-us
# need two proxies on host; s5s or ssh for msys2(socks5), and tinyproxy for rust(http)
UP=- python3 socks5server.py 192.168.123.1 4321
ssh -vND 192.168.123.1:4321 localhost
git clone https://github.com/tinyproxy/tinyproxy.git
./autogen.sh
./configure --prefix=/home/ed/pe/tinyproxy
make -j24 install
printf '%s\n' >cfg "Port 4380" "Listen 192.168.123.1"
./tinyproxy -dccfg
https://github.com/msys2/msys2-installer/releases/download/2022-09-04/msys2-x86_64-20220904.exe
export all_proxy=socks5h://192.168.123.1:4321
# if chat dies after auth (2 messages) it probably failed dns, note the h in socks5h to tunnel dns
pacman -Syuu
pacman -S git patch mingw64/mingw-w64-x86_64-zopfli
cd /c && curl -k https://192.168.123.1:3923/ro/ox/msbt2017/?tar | tar -xv
first install certs from msbt/certificates then admin-cmd `vs_buildtools.exe --noweb`,
default selection (vc++2017-v15.9-v14.16, vc++redist, vc++bt-core) += win10sdk (for io.h)
install rust without documentation, python 3.10, put upx and pyoxidizer into ~/bin,
[cmd.exe] python -m pip install --user -U wheel-0.37.1.tar.gz strip-hints-0.1.10.tar.gz
p=192.168.123.1:4380; export https_proxy=$p; export http_proxy=$p
# and with all of the one-time-setup out of the way,
mkdir /c/d; cd /c/d && curl -k https://192.168.123.1:3923/cpp/gb?pw=wark > gb && git clone gb copyparty
cd /c/d/copyparty/ && curl -k https://192.168.123.1:3923/cpp/patch?pw=wark | patch -p1
cd /c/d/copyparty/scripts && CARGO_HTTP_CHECK_REVOKE=false PATH=/c/Users/$USER/AppData/Local/Programs/Python/Python310:/c/Users/$USER/bin:"$(cygpath "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86"):$PATH" ./make-sfx.sh ox ultra

View File

@@ -47,3 +47,44 @@ and if you want to have a monospace font in the fancy markdown editor, do this:
NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative
# `<head>`
to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
if you give it the value `@ASDF` it will try to open a file named ASDF and send the text within
if the value starts with `%` it will assume a jinja2 template and expand it; the template has access to the `HttpCli` object through a property named `this` as well as everything in `j2a` and the stuff added by `self.j2s`; see [browser.html](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.html) for inspiration or look under the hood in [httpcli.py](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/httpcli.py)
# translations
add your own translations by using the english or norwegian one from `browser.js` as a template
the easy way is to open up and modify `browser.js` in your own installation; depending on how you installed copyparty it might be named `browser.js.gz` instead, in which case just decompress it, restart copyparty, and start editing it anyways
you will be delighted to see inline html in the translation strings; to help prevent syntax errors, there is [a very jank linux script](https://github.com/9001/copyparty/blob/hovudstraum/scripts/tlcheck.sh) which is slightly better than nothing -- just beware the false-positives, so even if it complains it's not necessarily wrong/bad
if you're running `copyparty-sfx.py` then you'll find it at `/tmp/pe-copyparty.1000/copyparty/web` (on linux) or `%TEMP%\pe-copyparty\copyparty\web` (on windows)
* make sure to keep backups of your work religiously! since that location is volatile af
## translations (docker-friendly)
if editing `browser.js` is inconvenient in your setup, for example if you're running in docker, then you can instead do this:
* if you have python, go to the `scripts` folder and run `./tl.py fra Français` to generate a `tl.js` which is perfect for translating to French, using the three-letter language code `fra`
* if you do not have python, you can also just grab `tl.js` from the scripts folder, but I'll probably forget to keep that up to date... and then you'll have to find/replace all `"eng"` and `Ls.eng` to your three-letter language code
* put your `tl.js` inside a folder that is being shared by your copyparty, preferably the webroot
* run copyparty with the argument `--html-head='<script src="/tl.js"></script>'`
* if you placed `tl.js` in the webroot then you're all good, but if you put it somewhere else then change `/tl.js` accordingly
* if you are running copyparty with config files, you can do this:
```yaml
[global]
html-head: <script src="/tl.js"></script>
```
you can now edit `tl.js` and press CTRL-SHIFT-R in the browser to see your changes take effect as you go
if you want to contribute your translation back to the project (please do!) then you'll want to...
* grab all of the text inside your `var tl_cpanel = {` and add it to the translations inside `copyparty/web/splash.js` in the repo
* and the text inside your `var tl_browser = {` and add that to the translations inside `copyparty/web/browser.js` in the repo

View File

@@ -20,6 +20,7 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* 💾 = what copyparty offers as an alternative
* 🔵 = similarities
* ⚠️ = disadvantages (something copyparty does "better")
* 🔥 = hazards
## toc
@@ -37,7 +38,7 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [another matrix](#another-matrix)
* [reviews](#reviews)
* [copyparty](#copyparty)
* [hfs2](#hfs2)
* [hfs2](#hfs2) 🔥
* [hfs3](#hfs3)
* [nextcloud](#nextcloud)
* [seafile](#seafile)
@@ -48,6 +49,7 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [filebrowser](#filebrowser)
* [filegator](#filegator)
* [sftpgo](#sftpgo)
* [arozos](#arozos)
* [updog](#updog)
* [goshs](#goshs)
* [gimme-that](#gimme-that)
@@ -56,7 +58,9 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [h5ai](#h5ai)
* [autoindex](#autoindex)
* [miniserve](#miniserve)
* [pingvin-share](#pingvin-share)
* [briefly considered](#briefly-considered)
* [notes](#notes)
# recommendations
@@ -82,8 +86,8 @@ the table headers in the matrixes below are the different softwares, with a quic
the softwares,
* `a` = [copyparty](https://github.com/9001/copyparty)
* `b` = [hfs2](https://rejetto.com/hfs/)
* `c` = [hfs3](https://github.com/rejetto/hfs)
* `b` = [hfs2](https://github.com/rejetto/hfs2/) 🔥
* `c` = [hfs3](https://rejetto.com/hfs/)
* `d` = [nextcloud](https://github.com/nextcloud/server)
* `e` = [seafile](https://github.com/haiwen/seafile)
* `f` = [rclone](https://github.com/rclone/rclone), specifically `rclone serve webdav .`
@@ -93,6 +97,7 @@ the softwares,
* `j` = [filebrowser](https://github.com/filebrowser/filebrowser)
* `k` = [filegator](https://github.com/filegator/filegator)
* `l` = [sftpgo](https://github.com/drakkan/sftpgo)
* `m` = [arozos](https://github.com/tobychui/arozos)
some softwares not in the matrixes,
* [updog](#updog)
@@ -103,6 +108,7 @@ some softwares not in the matrixes,
* [h5ai](#h5ai)
* [autoindex](#autoindex)
* [miniserve](#miniserve)
* [pingvin-share](#pingvin-share)
symbol legend,
* `█` = absolutely
@@ -113,22 +119,22 @@ symbol legend,
## general
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | |
| runs on iOS | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | |
| android app | | | | █ | █ | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | | |
| runs on iOS | | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ | |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ | |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | | █ |
| android app | | | | █ | █ | | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | | |
* `zero setup` = you can get a mostly working setup by just launching the app, without having to install any software or configure whatever
* `a`/copyparty remarks:
@@ -140,37 +146,41 @@ symbol legend,
* `f`/rclone must be started with the command `rclone serve webdav .` or similar
* `h`/chibisafe has undocumented windows support
* `i`/sftpgo must be launched with a command
* `m`/arozos has partial windows support
## file transfer
*the thing that copyparty is actually kinda good at*
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ |
| download folder as tar | █ | | | | | | | | | | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | |
| resumable uploads | █ | | | | | | | | █ | | █ | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | |
| upload acceleration | █ | | | | | | | | █ | | █ | |
| upload verification | █ | | | █ | █ | | | | █ | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | |
| keep last-modified time | █ | | | █ | █ | █ | | | | | | |
| upload rules | | | | | | | | | | | | |
| ┗ max disk usage | █ | █ | | | | | | | █ | | | █ |
| ┗ max filesize | █ | | | | | | | | | | | |
| ┗ max items in folder | █ | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | █ | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | |
| ┗ randomize filename | █ | | | | | | | | | | | |
| ┗ mimetype reject-list | | | | | | | | | • | | | |
| ┗ extension reject-list | | | | | | | | █ | • | | | |
| checksums provided | | | | █ | █ | | | | | | | |
| cloud storage backend | | | | | | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ | |
| download folder as tar | █ | | | | | | | | | | | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | | █ |
| resumable uploads | █ | | | | | | | | █ | | █ | | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | | █ |
| upload acceleration | █ | | | | | | | | █ | | █ | | |
| upload verification | █ | | | █ | █ | | | | █ | | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | | |
| CTRL-V from device | █ | | | █ | | | | | | | | | |
| race the beam ("p2p") | █ | | | | | | | | | | | | |
| keep last-modified time | █ | | | █ | █ | | | | | | | █ | |
| upload rules | | | | | | | | | | | | | |
| ┗ max disk usage | █ | █ | █ | | | | | | | | | | |
| ┗ max filesize | █ | | | | | | | █ | | | █ | █ | █ |
| ┗ max items in folder | █ | | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | █ | | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | | |
| ┗ randomize filename | | | | | | | | █ | | | | | |
| ┗ mimetype reject-list | | | | | | | | | | | | | • |
| ┗ extension reject-list | | | | | | | | | | | | | • |
| ┗ upload routing | █ | | | | | | | | | | | | |
| checksums provided | | | | █ | █ | | | | █ | | | | |
| cloud storage backend | | | | █ | █ | █ | | | | | █ | █ | |
* `upload segmenting` = files are sliced into chunks, making it possible to upload files larger than 100 MiB on cloudflare for example
@@ -178,6 +188,13 @@ symbol legend,
* `upload verification` = uploads are checksummed or otherwise confirmed to have been transferred correctly
* `CTRL-V from device` = press CTRL-C in Windows Explorer (or whatever) and paste into the webbrowser to upload it
* `race the beam` = files can be downloaded while they're still uploading; downloaders are slowed down such that the uploader is always ahead
* `upload routing` = depending on filetype / contents / uploader etc., the file can be redirected to another location or otherwise transformed; mitigates limitations such as [sharex#3992](https://github.com/ShareX/ShareX/issues/3992)
* copyparty example: [reloc-by-ext](https://github.com/9001/copyparty/tree/hovudstraum/bin/hooks#before-upload)
* `checksums provided` = when downloading a file from the server, the file's checksum is provided for verification client-side
* `cloud storage backend` = able to serve files from (and write to) s3 or similar cloud services; `` means the software can do this with some help from `rclone mount` as a bridge
@@ -192,26 +209,27 @@ symbol legend,
* resumable/segmented uploads only over SFTP, not over HTTP
* upload rules are totals only, not over time
* can probably do extension/mimetype rejection similar to copyparty
* `m`/arozos download-as-zip is not streaming; it creates the full zipfile before download can start, and fails on big folders
## protocols and client support
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ |
| serve ftp (tcp) | █ | | | | | █ | | | | | | █ |
| serve ftps (tls) | █ | | | | | █ | | | | | | █ |
| serve tftp (udp) | █ | | | | | | | | | | | |
| serve sftp (ssh) | | | | | | █ | | | | | | █ |
| serve smb/cifs | | | | | | █ | | | | | | |
| serve dlna | | | | | | █ | | | | | | |
| listen on unix-socket | | | | █ | █ | | █ | █ | █ | | █ | █ |
| zeroconf | █ | | | | | | | | | | | |
| supports netscape 4 | | | | | | █ | | | | | • | |
| ...internet explorer 6 | | █ | | █ | | █ | | | | | • | |
| mojibake filenames | █ | | | • | • | █ | █ | • | | • | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ | █ |
| serve ftp (tcp) | █ | | | | | █ | | | | | | █ | █ |
| serve ftps (tls) | █ | | | | | █ | | | | | | █ | |
| serve tftp (udp) | █ | | | | | | | | | | | | |
| serve sftp (ssh) | | | | | | █ | | | | | | █ | █ |
| serve smb/cifs | | | | | | █ | | | | | | | |
| serve dlna | | | | | | █ | | | | | | | |
| listen on unix-socket | | | | █ | █ | | █ | █ | █ | | █ | █ | |
| zeroconf | █ | | | | | | | | | | | | █ |
| supports netscape 4 | | | | | | █ | | | | | • | | |
| ...internet explorer 6 | | █ | | █ | | █ | | | | | • | | |
| mojibake filenames | █ | | | • | • | █ | █ | • | | • | | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | | |
* `webdav` = protocol convenient for mounting a remote server as a local filesystem; see zeroconf:
* `zeroconf` = the server announces itself on the LAN, [automatically appearing](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png) on other zeroconf-capable devices
@@ -222,61 +240,66 @@ symbol legend,
* extremely minimal samba/cifs server
* netscape 4 / ie6 support is mostly listed as a joke altho some people have actually found it useful ([ie4 tho](https://user-images.githubusercontent.com/241032/118192791-fb31fe00-b446-11eb-9647-898ea8efc1f7.png))
* `l`/sftpgo translates mojibake filenames into valid utf-8 (information loss)
* `m`/arozos has readonly-support for older browsers; no uploading
## server configuration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | |
| same-port http / https | █ | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | | • | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | | █ |
| same-port http / https | █ | | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ | |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ | |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ | |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | | • | | • |
* `folder-rproxy` = reverse-proxying without dedicating an entire (sub)domain, using a subfolder instead
* `l`/sftpgo:
* config: users must be added through gui / api calls
* `m`/arozos:
* configuration is primarily through GUI
* reverse-proxy is not guaranteed to see the correct client IP
## server capabilities
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ |
| single-sign-on | | | | █ | █ | | | | • | | | |
| token auth | | | | █ | █ | | | █ | | | | |
| 2fa | | | | █ | █ | | | | | | | █ |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | |
| unmap subfolders | █ | | | | | | █ | | | █ | | • |
| index.html blocks list | | | | | | | █ | | | • | | |
| write-only folders | █ | | | | | | | | | | █ | █ |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | |
| file action event hooks | █ | | | | | | | | | █ | | █ |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | |
| full sync | | | | █ | █ | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ |
| dyndns updater | | █ | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | |
| curl-friendly ls | █ | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ | |
| single-sign-on | | | | █ | █ | | | | • | | | | |
| token auth | | | | █ | █ | | | █ | | | | | █ |
| 2fa | | | | █ | █ | | | | | | | █ | |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | | █ |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | | █ |
| unmap subfolders | █ | | | | | | █ | | | █ | | • | |
| index.html blocks list | | | | | | | █ | | | • | | | |
| write-only folders | █ | | | | | | | | | | █ | █ | |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ | |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | | |
| file action event hooks | █ | | | | | | | | | █ | | █ | • |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | | |
| full sync | | | | █ | █ | | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ | |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ | • |
| dyndns updater | | █ | | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | | █ |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ | • |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | | █ |
| prometheus metrics | █ | | | █ | | | | | | | | | |
| curl-friendly ls | █ | | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | █ | █ | • | | | | | |
* `unmap subfolders` = "shadowing"; mounting a local folder in the middle of an existing filesystem tree in order to disable access below that path
* `files stored as-is` = uploaded files are trivially readable from the server HDD, not sliced into chunks or in weird folder structures or anything like that
@@ -286,6 +309,7 @@ symbol legend,
* `file action event hooks` = run script before/after upload, move, rename, ...
* `one-way folder sync` = like rsync, optionally deleting unexpected files at target
* `full sync` = stateful, dropbox-like sync
* `speed throttle` = rate limiting (per ip, per user, per connection, anything like that)
* `curl-friendly ls` = returns a [sortable plaintext folder listing](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png) when curled
* `curl-friendly upload` = uploading with curl is just `curl -T some.bin http://.../`
* `a`/copyparty remarks:
@@ -302,49 +326,51 @@ symbol legend,
* `l`/sftpgo:
* `file action event hooks` also include on-download triggers
* `upload tracking / log` in main logfile
* `m`/arozos:
* `2fa` maybe possible through LDAP/Oauth
## client features
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | |
| themes | █ | █ | | █ | | | | | █ | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | |
| multi-column sorting | █ | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | |
| ┗ audio spectrograms | █ | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | |
| ┗ gapless playback | █ | | | | | | | | • | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | |
| ┗ video transcoding | | | | | | | | | █ | | | |
| audio BPM detector | █ | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | |
| search by bpm / key | █ | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | |
| search in file contents | | | | █ | █ | | | | █ | | | |
| search by custom parser | █ | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | |
| markdown viewer | █ | | | | █ | | | | █ | | | |
| markdown editor | █ | | | | █ | | | | █ | | | |
| readme.md in listing | █ | | | █ | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | | █ |
| themes | █ | █ | | █ | | | | | █ | | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | | |
| multi-column sorting | █ | | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | | █ |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | | █ |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ audio spectrograms | █ | | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ gapless playback | █ | | | | | | | | • | | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | | █ |
| ┗ video transcoding | | | | | | | | | █ | | | | |
| audio BPM detector | █ | | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | | |
| search by bpm / key | █ | | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | | |
| search in file contents | | | | █ | █ | | | | █ | | | | |
| search by custom parser | █ | | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | | █ |
| markdown viewer | █ | | | | █ | | | | █ | | | | █ |
| markdown editor | █ | | | | █ | | | | █ | | | | █ |
| readme.md in listing | █ | | | █ | | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | | █ |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | | █ |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | | █ |
* `single-page app` = multitasking; possible to continue navigating while uploading
* `audio player » os-integration` = use the [lockscreen](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) or [media hotkeys](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) to play/pause, prev/next song
@@ -353,21 +379,25 @@ symbol legend,
* `undo recent uploads` = accounts without delete permissions have a time window where they can undo their own uploads
* `a`/copyparty has teeny-tiny skips playing gapless albums depending on audio codec (opus best)
* `b`/hfs2 has a very basic directory tree view, not showing sibling folders
* `c`/hfs3 remarks:
* audio playback does not continue into next song
* `f`/rclone can do some file management (mkdir, rename, delete) when hosting througn webdav
* `j`/filebrowser has a plaintext viewer/editor
* `j`/filebrowser remarks:
* audio playback does not continue into next song
* plaintext viewer/editor
* `k`/filegator directory tree is a modal window
## integration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | |
| flameshot | | | | | | █ | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | | |
| flameshot | | | | | | █ | | | | | | | |
* sharex `` = yes, but does not provide example sharex config
* `a`/copyparty remarks:
@@ -393,11 +423,16 @@ symbol legend,
| filebrowser | go | █ apl2 | 20 MB |
| filegator | php | █ mit | • |
| sftpgo | go | ‼ agpl | 44 MB |
| arozos | go | ░ gpl3 | 531 MB |
| updog | python | █ mit | 17 MB |
| goshs | go | █ mit | 11 MB |
| gimme-that | python | █ mit | 4.8 MB |
| ass | ts | █ isc | • |
| linx | go | ░ gpl3 | 20 MB |
| h5ai | php | █ mit | • |
| autoindex | go | █ mpl2 | 11 MB |
| miniserve | rust | █ mit | 2 MB |
| pingvin-share | go | █ bsd2 | 487 MB |
* `size` = binary (if available) or installed size of program and its dependencies
* copyparty size is for the [standalone python](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) file; the [windows exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) is **6 MiB**
@@ -409,6 +444,7 @@ symbol legend,
* 💾 are what copyparty offers as an alternative
* 🔵 are similarities
* ⚠️ are disadvantages (something copyparty does "better")
* 🔥 are hazards
## [copyparty](https://github.com/9001/copyparty)
* resumable uploads which are verified server-side
@@ -416,8 +452,9 @@ symbol legend,
* both of the above are surprisingly uncommon features
* very cross-platform (python, no dependencies)
## [hfs2](https://rejetto.com/hfs/)
* the OG, the legend
## [hfs2](https://github.com/rejetto/hfs2/)
* the OG, the legend (now replaced by [hfs3](#hfs3))
* 🔥 hfs2 is dead and dangerous! unfixed RCE: [info](https://github.com/rejetto/hfs2/issues/44), [info](https://github.com/drapid/hfs/issues/3), [info](https://asec.ahnlab.com/en/67650/)
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ windows-only
@@ -425,18 +462,30 @@ symbol legend,
* vfs with gui config, per-volume permissions
* starting to show its age, hence the rewrite:
## [hfs3](https://github.com/rejetto/hfs)
## [hfs3](https://rejetto.com/hfs/)
* nodejs; cross-platform
* vfs with gui config, per-volume permissions
* still early development, let's revisit later
* tested locally, v0.53.2 on archlinux
* 🔵 uploads are resumable
* ⚠️ uploads are not segmented; max upload size 100 MiB on cloudflare
* ⚠️ uploads are not accelerated (copyparty is 3x faster across the atlantic)
* ⚠️ uploads are not integrity-checked
* ⚠️ copies the file after upload; need twice filesize free disk space
* ⚠️ uploading small files is decent; `107` files per sec (copyparty does `670`/sec, 6x faster)
* ⚠️ doesn't support crazy filenames
* ✅ config GUI
* ✅ download counter
* ✅ watch active connections
* ✅ plugins
## [nextcloud](https://github.com/nextcloud/server)
* php, mariadb
* tested locally, [linuxserver/nextcloud](https://hub.docker.com/r/linuxserver/nextcloud) v30.0.2 (sqlite)
* ⚠️ [isolated on-disk file hierarchy] in per-user folders
* not that bad, can probably be remedied with bindmounts or maybe symlinks
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ uploading small files is slow; `2.2` files per sec (copyparty does `87`/sec), tested locally with [linuxserver/nextcloud](https://hub.docker.com/r/linuxserver/nextcloud) (sqlite)
* ⚠️ uploading small files is slow; `4` files per sec (copyparty does `670`/sec, 160x faster)
* ⚠️ no write-only / upload-only folders
* ⚠️ http/webdav only; no ftp, zeroconf
* ⚠️ less awesome music player
@@ -452,11 +501,12 @@ symbol legend,
## [seafile](https://github.com/haiwen/seafile)
* c, mariadb
* tested locally, [official container](https://manual.seafile.com/latest/docker/deploy_seafile_with_docker/) v11.0.13
* ⚠️ [isolated on-disk file hierarchy](https://manual.seafile.com/maintain/seafile_fsck/), incompatible with other software
* *much worse than nextcloud* in that regard
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ uploading small files is slow; `2.7` files per sec (copyparty does `87`/sec), tested locally with [official container](https://manual.seafile.com/docker/deploy_seafile_with_docker/)
* ⚠️ uploading small files is slow; `4.7` files per sec (copyparty does `670`/sec, 140x faster)
* ⚠️ no write-only / upload-only folders
* ⚠️ big folders cannot be zip-downloaded
* ⚠️ http/webdav only; no ftp, zeroconf
@@ -480,8 +530,11 @@ symbol legend,
## [dufs](https://github.com/sigoden/dufs)
* rust; cross-platform (windows, linux, macos)
* tested locally, v0.43.0 on archlinux (plain binary)
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ across the atlantic, copyparty is 3x faster
* ⚠️ uploading small files is decent; `97` files per sec (copyparty does `670`/sec, 7x faster)
* ⚠️ doesn't support crazy filenames
* ✅ per-url access control (copyparty is per-volume)
* 🔵 basic but really snappy ui
@@ -504,12 +557,14 @@ symbol legend,
* ✅ token auth (api keys)
## [kodbox](https://github.com/kalcaddle/kodbox)
* this thing is insane
* this thing is insane (but is getting competition from [arozos](#arozos))
* php; [docker](https://hub.docker.com/r/kodcloud/kodbox)
* 🔵 *upload segmenting, acceleration, and integrity checking!*
* ⚠️ but uploads are not resumable(?)
* ⚠️ not portable
* ⚠️ isolated on-disk file hierarchy, incompatible with other software
* ⚠️ uploading small files to copyparty is 16x faster
* ⚠️ uploading large files to copyparty is 3x faster
* ⚠️ http/webdav only; no ftp or zeroconf
* ⚠️ some parts of the GUI are in chinese
* ✅ fantastic ui/ux
@@ -522,8 +577,12 @@ symbol legend,
## [filebrowser](https://github.com/filebrowser/filebrowser)
* go; cross-platform (windows, linux, mac)
* ⚠️ uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* tested locally, v2.31.2 on archlinux (plain binary)
* 🔵 uploads are resumable and segmented
* 🔵 multiple files are uploaded in parallel, but...
* ⚠️ big files are not accelerated (copyparty is 5x faster across the atlantic)
* ⚠️ uploads are not integrity-checked
* ⚠️ uploading small files is decent; `69` files per sec (copyparty does `670`/sec, 9x faster)
* ⚠️ http only; no webdav / ftp / zeroconf
* ⚠️ doesn't support crazy filenames
* ⚠️ no directory tree nav
@@ -533,12 +592,14 @@ symbol legend,
* ⚠️ but no directory tree for navigation
* ✅ user signup
* ✅ command runner / remote shell
* 🔵 supposed to have write-only folders but couldn't get it to work
* ✅ more efficient; can handle around twice as much simultaneous traffic
## [filegator](https://github.com/filegator/filegator)
* go; cross-platform (windows, linux, mac)
* php; cross-platform (windows, linux, mac)
* 🔵 *it has upload segmenting and acceleration*
* ⚠️ but uploads are still not integrity-checked
* ⚠️ on copyparty, uploads are 40x faster
* compared to the official filegator docker example which might be bad
* ⚠️ http only; no webdav / ftp / zeroconf
* ⚠️ does not support symlinks
* ⚠️ expensive download-as-zip feature
@@ -549,6 +610,7 @@ symbol legend,
* go; cross-platform (windows, linux, mac)
* ⚠️ http uploads not resumable / accelerated / integrity-checked
* ⚠️ on cloudflare: max upload size 100 MiB
* ⚠️ across the atlantic, copyparty is 2.5x faster
* 🔵 sftp uploads are resumable
* ⚠️ web UI is very minimal + a bit slow
* ⚠️ no thumbnails / image viewer / audio player
@@ -556,7 +618,9 @@ symbol legend,
* ⚠️ no filesystem indexing / search
* ⚠️ doesn't run on phones, tablets
* ⚠️ no zeroconf (mdns/ssdp)
* ⚠️ impractical directory URLs
* ⚠️ AGPL licensed
* 🔵 uploading small files is fast; `340` files per sec (copyparty does `670`/sec)
* 🔵 ftp, ftps, webdav
* ✅ sftp server
* ✅ settings gui
@@ -569,6 +633,26 @@ symbol legend,
* ✅ on-download event hook (otherwise same as copyparty)
* ✅ more extensive permissions control
## [arozos](https://github.com/tobychui/arozos)
* big suite of applications similar to [kodbox](#kodbox), copyparty is better at downloading/uploading/music/indexing but arozos has other advantages
* go; primarily linux (limited support for windows)
* ⚠️ needs root
* ⚠️ uploads not resumable / integrity-checked
* ⚠️ uploading small files to copyparty is 2.7x faster
* ⚠️ uploading large files to copyparty is at least 10% faster
* arozos is websocket-based, 512 KiB chunks; writes each chunk to separate files and then merges
* copyparty splices directly into the final file; faster and better for the HDD and filesystem
* ⚠️ across the atlantic, uploading to copyparty is 6x faster
* ⚠️ no directory tree navpane; not as easy to navigate
* ⚠️ download-as-zip is not streaming; creates a temp.file on the server
* ⚠️ not self-contained (pulls from jsdelivr)
* ⚠️ has an audio player, but supports less filetypes
* ⚠️ limited support for configuring real-ip detection
* ✅ sftp server
* ✅ settings gui
* ✅ good-looking gui
* ✅ an IDE, msoffice viewer, rich host integration, much more
## [updog](https://github.com/sc0tfree/updog)
* python; cross-platform
* basic directory listing with upload feature
@@ -651,7 +735,31 @@ symbol legend,
* 🔵 upload, tar/zip download, qr-code
* ✅ faster at loading huge folders
## [pingvin-share](https://github.com/stonith404/pingvin-share)
* node; linux (docker)
* mainly for uploads, not a general file server
* 🔵 uploads are segmented (avoids cloudflare size limit)
* 🔵 segments are written directly to target file (HDD-friendly)
* ⚠️ uploads not resumable after a browser or laptop crash
* ⚠️ uploads are not accelerated / integrity-checked
* ⚠️ across the atlantic, copyparty is 3x faster
* measured with chunksize 96 MiB; pingvin's default 10 MiB is much slower
* ⚠️ can't upload folders with subfolders
* ⚠️ no upload ETA
* 🔵 expiration times, shares, upload-undo
* ✅ config + user-registration gui
* ✅ built-in OpenID and LDAP support
* 💾 [IdP middleware](https://github.com/9001/copyparty#identity-providers) and config-files
* ✅ probably more than one person who understands the code
# briefly considered
* [pydio](https://github.com/pydio/cells): python/agpl3, looks great, fantastic ux -- but needs mariadb, systemwide install
* [gossa](https://github.com/pldubouilh/gossa): go/mit, minimalistic, basic file upload, text editor, mkdir and rename (no delete/move)
# notes
* high-latency connections (cross-atlantic uploads) can be accurately simulated with `tc qdisc add dev eth0 root netem delay 100ms`

View File

@@ -1,48 +0,0 @@
# builds win7-i386 exe on win10-ltsc-1809(17763.316)
# see docs/pyoxidizer.txt
def make_exe():
dist = default_python_distribution(flavor="standalone_static", python_version="3.8")
policy = dist.make_python_packaging_policy()
policy.allow_files = True
policy.allow_in_memory_shared_library_loading = True
#policy.bytecode_optimize_level_zero = True
#policy.include_distribution_sources = False # error instantiating embedded Python interpreter: during initializing Python main: init_fs_encoding: failed to get the Python codec of the filesystem encoding
policy.include_distribution_resources = False
policy.include_non_distribution_sources = False
policy.include_test = False
python_config = dist.make_python_interpreter_config()
#python_config.module_search_paths = ["$ORIGIN/lib"]
python_config.run_module = "copyparty"
exe = dist.to_python_executable(
name="copyparty",
config=python_config,
packaging_policy=policy,
)
exe.windows_runtime_dlls_mode = "never"
exe.windows_subsystem = "console"
exe.add_python_resources(exe.read_package_root(
path="sfx",
packages=[
"copyparty",
"jinja2",
"markupsafe",
"pyftpdlib",
"python-magic",
]
))
return exe
def make_embedded_resources(exe):
return exe.to_embedded_resources()
def make_install(exe):
files = FileManifest()
files.add_python_resource("copyparty", exe)
return files
register_target("exe", make_exe)
register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
register_target("install", make_install, depends=["exe"], default=True)
resolve_targets()

View File

@@ -25,6 +25,7 @@ classifiers = [
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: Jython",
"Programming Language :: Python :: Implementation :: PyPy",
@@ -49,7 +50,7 @@ thumbnails2 = ["pyvips"]
audiotags = ["mutagen"]
ftpd = ["pyftpdlib"]
ftps = ["pyftpdlib", "pyopenssl"]
tftpd = ["partftpy>=0.3.1"]
tftpd = ["partftpy>=0.4.0"]
pwhash = ["argon2-cffi"]
[project.scripts]

View File

@@ -3,7 +3,7 @@ WORKDIR /z
ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
ver_hashwasm=4.10.0 \
ver_marked=4.3.0 \
ver_dompf=3.0.9 \
ver_dompf=3.1.7 \
ver_mde=2.18.0 \
ver_codemirror=5.65.16 \
ver_fontawesome=5.13.0 \
@@ -24,7 +24,9 @@ ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
# the scp url is regular latin from https://fonts.googleapis.com/css2?family=Source+Code+Pro&display=swap
RUN mkdir -p /z/dist/no-pk \
&& wget https://fonts.gstatic.com/s/sourcecodepro/v11/HI_SiYsKILxRpg3hIP6sJ7fM7PqlPevW.woff2 -O scp.woff2 \
&& apk add cmake make g++ git bash npm patch wget tar pigz brotli gzip unzip python3 python3-dev py3-brotli \
&& apk add \
bash brotli cmake make g++ git gzip lame npm patch pigz \
python3 python3-dev py3-brotli sox tar unzip wget \
&& rm -f /usr/lib/python3*/EXTERNALLY-MANAGED \
&& wget https://github.com/openpgpjs/asmcrypto.js/archive/$ver_asmcrypto.tar.gz -O asmcrypto.tgz \
&& wget https://github.com/markedjs/marked/archive/v$ver_marked.tar.gz -O marked.tgz \
@@ -35,6 +37,7 @@ RUN mkdir -p /z/dist/no-pk \
&& wget https://github.com/google/zopfli/archive/zopfli-$ver_zopfli.tar.gz -O zopfli.tgz \
&& wget https://github.com/Daninet/hash-wasm/releases/download/v$ver_hashwasm/hash-wasm@$ver_hashwasm.zip -O hash-wasm.zip \
&& wget https://github.com/PrismJS/prism/archive/refs/tags/v$ver_prism.tar.gz -O prism.tgz \
&& wget https://files.pythonhosted.org/packages/04/0b/4506cb2e831cea4b0214d3625430e921faaa05a7fb520458c75a2dbd2152/fusepy-3.0.1.tar.gz -O fusepy.tgz \
&& (mkdir hash-wasm \
&& cd hash-wasm \
&& unzip ../hash-wasm.zip) \
@@ -54,10 +57,16 @@ RUN mkdir -p /z/dist/no-pk \
&& npm i gulp-cli -g ) \
&& tar -xf dompurify.tgz \
&& tar -xf prism.tgz \
&& tar -xf fusepy.tgz \
&& unzip fontawesome.zip \
&& tar -xf zopfli.tgz
COPY busy-mp3.sh /z/
RUN /z/busy-mp3.sh \
&& mv -v /dev/shm/busy.mp3.gz /z/dist
# build fonttools (which needs zopfli)
RUN tar -xf zopfli.tgz \
&& cd zopfli* \
@@ -151,6 +160,18 @@ RUN cd /z/dist \
&& rmdir no-pk
# build fusepy
COPY uncomment.py /z
RUN mv /z/fusepy-3.0.1/fuse.py /z/dist/f1 \
&& cd /z/dist \
&& python3 /z/uncomment.py f1 \
&& sed -ri '/self.__critical_exception = e/d' f1 \
&& awk '/^log =/{s=0} !s; /^from traceback im/{s=1;print"from functools import partial";print"basestring = str"}' <f1 >f2 \
&& awk '/LoggingMixIn:/{exit} --s<0;/self.use_ns = getattr/{s=7}' <f2 >f1 \
&& awk "/if _machine =/{s=0} /'(mips|ppc|ppc64)'/{s=1} !s" <f1 >f2 \
&& rm f1 && mv f2 fuse.py
# git diff -U2 --no-index marked-1.1.0-orig/ marked-1.1.0-edit/ -U2 | sed -r '/^index /d;s`^(diff --git a/)[^/]+/(.* b/)[^/]+/`\1\2`; s`^(---|\+\+\+) ([ab]/)[^/]+/`\1 \2`' > ../dev/copyparty/scripts/deps-docker/marked-ln.patch
# d=/home/ed/dev/copyparty/scripts/deps-docker/; tar -cf ../x . && ssh root@$bip "cd $d && tar -xv >&2 && make >&2 && tar -cC ../../copyparty/web deps" <../x | (cd ../../copyparty/web/; cat > the.tgz; tar -xvf the.tgz; rm the.tgz)
# gzip -dkf ../dev/copyparty/copyparty/web/deps/deps/marked.full.js.gz && diff -NarU2 ../dev/copyparty/copyparty/web/deps/{,deps/}marked.full.js

View File

@@ -4,8 +4,10 @@ vend := $(self)/../../copyparty/web/deps
# prefers podman-docker (optionally rootless) over actual docker/moby
all:
cp -pv ../uncomment.py .
docker build -t build-copyparty-deps .
rm -rf $(vend)
mkdir $(vend)

61
scripts/deps-docker/busy-mp3.sh Executable file
View File

@@ -0,0 +1,61 @@
#!/bin/bash
set -e
cat >/dev/null <<EOF
a frame is 1152 samples
1sec @ 48000 = 41.66 frames
11 frames = 12672 samples = 0.264 sec
22 frames = 25344 samples = 0.528 sec
EOF
fast=1
fast=
echo
mkdir -p /dev/shm/$1
cd /dev/shm/$1
find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete
min=99999999
for freq in 425; do # {400..500}
for vol in 0; do # {10..30}
for kbps in 32; do
for fdur in 1124; do # {800..1200}
for fdu2 in 1042; do # {800..1200}
for ftyp in h; do # q h t l p
for ofs1 in 9214; do # {0..32768}
for ofs2 in 0; do # {0..4096}
for ofs3 in 0; do # {0..4096}
for nores in --nores; do # '' --nores
f=a.b$kbps$nores-f$freq-v$vol-$ftyp$fdur-$fdu2-o$ofs1-$ofs2-$ofs3
sox -r48000 -Dn -r48000 -b16 -c2 -t raw s1.pcm synth 25344s sin $freq vol 0.$vol fade $ftyp ${fdur}s 25344s ${fdu2}s
sox -r48000 -Dn -r48000 -b16 -c2 -t raw s0.pcm synth 12672s sin $freq vol 0
tail -c +$ofs1 s0.pcm >s0a.pcm
tail -c +$ofs2 s0.pcm >s0b.pcm
tail -c +$ofs3 s0.pcm >s0c.pcm
cat s{0a,1,0,0b,1,0c}.pcm > a.pcm
lame --silent -r -s 48 --bitwidth 16 --signed a.pcm -m j --resample 48 -b $kbps -q 0 $nores $f.mp3
if [ $fast ]
then gzip -c9 <$f.mp3 >$f.mp3.gz
else pigz -c11 -I1 <$f.mp3 >$f.mp3.gz
fi
sz=$(wc -c <$f.mp3.gz)
printf '\033[A%d %s\033[K\n' $sz $f
[ $sz -le $((min+10)) ] && echo
[ $sz -le $min ] && echo && min=$sz
done;done;done;done;done;done;done;done;done;done
true
f=a.b32--nores-f425-v0-h1124-1042-o9214-0-0.mp3
[ $fast ] &&
pigz -c11 -I1 <$f >busy.mp3.gz ||
mv $f.gz busy.mp3.gz
sz=$(wc -c <busy.mp3.gz)
[ "$sz" -eq 106 ] &&
echo busy.mp3 built successfully ||
echo WARNING: unexpected size of busy.mp3
find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete

View File

@@ -5,19 +5,16 @@ LABEL org.opencontainers.image.url="https://github.com/9001/copyparty" \
org.opencontainers.image.licenses="MIT" \
org.opencontainers.image.title="copyparty-ac" \
org.opencontainers.image.description="copyparty with Pillow and FFmpeg (image/audio/video thumbnails, audio transcoding, media tags)"
ENV PYTHONPYCACHEPREFIX=/tmp/pyc \
XDG_CONFIG_HOME=/cfg
ENV XDG_CONFIG_HOME=/cfg
RUN apk --no-cache add !pyc \
wget \
py3-argon2-cffi py3-pillow \
ffmpeg \
&& rm -rf /tmp/pyc \
&& mkdir /cfg /w \
&& chmod 777 /cfg /w \
&& echo % /cfg > initcfg
tzdata wget \
py3-jinja2 py3-argon2-cffi py3-pillow \
ffmpeg
COPY i/dist/copyparty-sfx.py innvikler.sh ./
RUN ash innvikler.sh && rm innvikler.sh
COPY i/dist/copyparty-sfx.py ./
WORKDIR /w
EXPOSE 3923
ENTRYPOINT ["python3", "/z/copyparty-sfx.py", "--no-crt", "-c", "/z/initcfg"]
ENTRYPOINT ["python3", "-m", "copyparty", "--no-crt", "-c", "/z/initcfg"]

Some files were not shown because too many files have changed in this diff Show More