Compare commits
87 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
391caca043 | ||
|
|
171ce348d6 | ||
|
|
c2cc729135 | ||
|
|
e7e71b76f0 | ||
|
|
a2af61cf6f | ||
|
|
e111edd5e4 | ||
|
|
3375377371 | ||
|
|
0ced020c67 | ||
|
|
c0d7aa9e4a | ||
|
|
e5b3d2a312 | ||
|
|
7b4a794981 | ||
|
|
86a859de17 | ||
|
|
b3aaa7bd0f | ||
|
|
a90586e6a8 | ||
|
|
807f272895 | ||
|
|
f050647b43 | ||
|
|
73baebbd16 | ||
|
|
f327f698b9 | ||
|
|
8164910fe8 | ||
|
|
3498644055 | ||
|
|
d31116b54c | ||
|
|
aced110cdf | ||
|
|
e9ab6aec77 | ||
|
|
15b261c861 | ||
|
|
970badce66 | ||
|
|
64304a9d65 | ||
|
|
d1983553d2 | ||
|
|
6b15df3bcd | ||
|
|
730b1fff71 | ||
|
|
c3add751e5 | ||
|
|
9da2dbdc1c | ||
|
|
977f09c470 | ||
|
|
4d0c6a8802 | ||
|
|
5345565037 | ||
|
|
be38c27c64 | ||
|
|
82a0401099 | ||
|
|
33bea1b663 | ||
|
|
f083acd46d | ||
|
|
5aacd15272 | ||
|
|
cb7674b091 | ||
|
|
3899c7ad56 | ||
|
|
d2debced09 | ||
|
|
b86c0ddc48 | ||
|
|
ba36f33bd8 | ||
|
|
49368a10ba | ||
|
|
ac1568cacf | ||
|
|
862ca3439d | ||
|
|
fdd4f9f2aa | ||
|
|
aa2dc49ebe | ||
|
|
cc23b7ee74 | ||
|
|
f6f9fc5a45 | ||
|
|
26c8589399 | ||
|
|
c2469935cb | ||
|
|
5e7c20955e | ||
|
|
967fa38108 | ||
|
|
280fe8e36b | ||
|
|
03ca96ccc3 | ||
|
|
b5b8a2c9d5 | ||
|
|
0008832730 | ||
|
|
c9b385db4b | ||
|
|
c951b66ae0 | ||
|
|
de735f3a45 | ||
|
|
19161425f3 | ||
|
|
c69e8d5bf4 | ||
|
|
3d3bce2788 | ||
|
|
1cb0dc7f8e | ||
|
|
cd5c56e601 | ||
|
|
8c979905e4 | ||
|
|
4d69f15f48 | ||
|
|
083f6572f7 | ||
|
|
4e7dd75266 | ||
|
|
3eb83f449b | ||
|
|
d31f69117b | ||
|
|
f5f9e3ac97 | ||
|
|
598d6c598c | ||
|
|
744727087a | ||
|
|
f93212a665 | ||
|
|
6dade82d2c | ||
|
|
6b737bf1d7 | ||
|
|
94dbd70677 | ||
|
|
527ae0348e | ||
|
|
79629c430a | ||
|
|
908dd61be5 | ||
|
|
88f77b8cca | ||
|
|
1e846657d1 | ||
|
|
ce70f62a88 | ||
|
|
bca0cdbb62 |
134
README.md
134
README.md
@@ -46,7 +46,7 @@ turn your phone or raspi into a portable file server with resumable uploads/down
|
||||
* [markdown viewer](#markdown-viewer) - and there are *two* editors
|
||||
* [other tricks](#other-tricks)
|
||||
* [searching](#searching) - search by size, date, path/name, mp3-tags, ...
|
||||
* [server config](#server-config)
|
||||
* [server config](#server-config) - using arguments or config files, or a mix of both
|
||||
* [file indexing](#file-indexing)
|
||||
* [upload rules](#upload-rules) - set upload rules using volume flags
|
||||
* [compress uploads](#compress-uploads) - files can be autocompressed on upload
|
||||
@@ -60,11 +60,17 @@ turn your phone or raspi into a portable file server with resumable uploads/down
|
||||
* [up2k](#up2k) - quick outline of the up2k protocol, see [uploading](#uploading) for the web-client
|
||||
* [why chunk-hashes](#why-chunk-hashes) - a single sha512 would be better, right?
|
||||
* [performance](#performance) - defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
|
||||
* [client-side](#client-side) - when uploading files
|
||||
* [security](#security) - some notes on hardening
|
||||
* [gotchas](#gotchas) - behavior that might be unexpected
|
||||
* [recovering from crashes](#recovering-from-crashes)
|
||||
* [client crashes](#client-crashes)
|
||||
* [frefox wsod](#frefox-wsod) - firefox 87 can crash during uploads
|
||||
* [HTTP API](#HTTP-API)
|
||||
* [read](#read)
|
||||
* [write](#write)
|
||||
* [admin](#admin)
|
||||
* [general](#general)
|
||||
* [dependencies](#dependencies) - mandatory deps
|
||||
* [optional dependencies](#optional-dependencies) - install these to enable bonus features
|
||||
* [install recommended deps](#install-recommended-deps)
|
||||
@@ -153,7 +159,7 @@ feature summary
|
||||
* browser
|
||||
* ☑ [navpane](#navpane) (directory tree sidebar)
|
||||
* ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
|
||||
* ☑ audio player (with OS media controls)
|
||||
* ☑ audio player (with OS media controls and opus transcoding)
|
||||
* ☑ image gallery with webm player
|
||||
* ☑ [thumbnails](#thumbnails)
|
||||
* ☑ ...of images using Pillow
|
||||
@@ -218,6 +224,7 @@ some improvement ideas
|
||||
* Windows: python 2.7 cannot index non-ascii filenames with `-e2d`
|
||||
* Windows: python 2.7 cannot handle filenames with mojibake
|
||||
* `--th-ff-jpg` may fix video thumbnails on some FFmpeg versions (macos, some linux)
|
||||
* `--th-ff-swr` may fix audio thumbnails on some FFmpeg versions
|
||||
|
||||
## general bugs
|
||||
|
||||
@@ -304,6 +311,7 @@ the browser has the following hotkeys (always qwerty)
|
||||
* `B` toggle breadcrumbs / [navpane](#navpane)
|
||||
* `I/K` prev/next folder
|
||||
* `M` parent folder (or unexpand current)
|
||||
* `V` toggle folders / textfiles in the navpane
|
||||
* `G` toggle list / [grid view](#thumbnails)
|
||||
* `T` toggle thumbnails / icons
|
||||
* `ctrl-X` cut selected files/folders
|
||||
@@ -315,6 +323,10 @@ the browser has the following hotkeys (always qwerty)
|
||||
* ctrl+`Up/Down` move cursor and scroll viewport
|
||||
* `Space` toggle file selection
|
||||
* `Ctrl-A` toggle select all
|
||||
* when a textfile is open:
|
||||
* `I/K` prev/next textfile
|
||||
* `S` toggle selection of open file
|
||||
* `M` close textfile
|
||||
* when playing audio:
|
||||
* `J/L` prev/next song
|
||||
* `U/O` skip 10sec back/forward
|
||||
@@ -366,6 +378,8 @@ press `g` to toggle grid-view instead of the file listing, and `t` toggles icon
|
||||
|
||||
it does static images with Pillow and uses FFmpeg for video files, so you may want to `--no-thumb` or maybe just `--no-vthumb` depending on how dangerous your users are
|
||||
|
||||
audio files are covnerted into spectrograms using FFmpeg unless you `--no-athumb` (and some FFmpeg builds may need `--th-ff-swr`)
|
||||
|
||||
images with the following names (see `--th-covers`) become the thumbnail of the folder they're in: `folder.png`, `folder.jpg`, `cover.png`, `cover.jpg`
|
||||
|
||||
in the grid/thumbnail view, if the audio player panel is open, songs will start playing when clicked
|
||||
@@ -421,6 +435,8 @@ see [up2k](#up2k) for details on how it works
|
||||
|
||||
**protip:** you can avoid scaring away users with [docs/minimal-up2k.html](docs/minimal-up2k.html) which makes it look [much simpler](https://user-images.githubusercontent.com/241032/118311195-dd6ca380-b4ef-11eb-86f3-75a3ff2e1332.png)
|
||||
|
||||
**protip:** if you enable `favicon` in the `[⚙️] settings` tab (by typing something into the textbox), the icon in the browser tab will indicate upload progress
|
||||
|
||||
the up2k UI is the epitome of polished inutitive experiences:
|
||||
* "parallel uploads" specifies how many chunks to upload at the same time
|
||||
* `[🏃]` analysis of other files should continue while one is uploading
|
||||
@@ -546,6 +562,8 @@ and there are *two* editors
|
||||
|
||||
* you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
|
||||
|
||||
* get a plaintext file listing by adding `?ls=t` to a URL, or a compact colored one with `?ls=v` (for unix terminals)
|
||||
|
||||
* if you are using media hotkeys to switch songs and are getting tired of seeing the OSD popup which Windows doesn't let you disable, consider https://ocv.me/dev/?media-osd-bgone.ps1
|
||||
|
||||
* click the bottom-left `π` to open a javascript prompt for debugging
|
||||
@@ -574,6 +592,10 @@ add the argument `-e2ts` to also scan/index tags from music files, which brings
|
||||
|
||||
# server config
|
||||
|
||||
using arguments or config files, or a mix of both:
|
||||
* config files (`-c some.conf`) can set additional commandline arguments; see [./docs/example.conf](docs/example.conf)
|
||||
|
||||
|
||||
## file indexing
|
||||
|
||||
file indexing relies on two database tables, the up2k filetree (`-e2d`) and the metadata tags (`-e2t`), stored in `.hist/up2k.db`. Configuration can be done through arguments, volume flags, or a mix of both.
|
||||
@@ -716,7 +738,7 @@ that'll run the command `notify-send` with the path to the uploaded file as the
|
||||
|
||||
note that it will only trigger on new unique files, not dupes
|
||||
|
||||
and it will occupy the parsing threads, so fork anything expensive, or if you want to intentionally queue/singlethread you can combine it with `--no-mtag-mt`
|
||||
and it will occupy the parsing threads, so fork anything expensive, or if you want to intentionally queue/singlethread you can combine it with `--mtag-mt 1`
|
||||
|
||||
if this becomes popular maybe there should be a less janky way to do it actually
|
||||
|
||||
@@ -845,8 +867,6 @@ hashwasm would solve the streaming issue but reduces hashing speed for sha512 (x
|
||||
|
||||
defaults are usually fine - expect `8 GiB/s` download, `1 GiB/s` upload
|
||||
|
||||
you can ignore the `cannot efficiently use multiple CPU cores` message, very unlikely to be a problem
|
||||
|
||||
below are some tweaks roughly ordered by usefulness:
|
||||
|
||||
* `-q` disables logging and can help a bunch, even when combined with `-lo` to redirect logs to file
|
||||
@@ -860,6 +880,21 @@ below are some tweaks roughly ordered by usefulness:
|
||||
...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
|
||||
|
||||
|
||||
## client-side
|
||||
|
||||
when uploading files,
|
||||
|
||||
* chrome is recommended, at least compared to firefox:
|
||||
* up to 90% faster when hashing, especially on SSDs
|
||||
* up to 40% faster when uploading over extremely fast internets
|
||||
* but [up2k.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py) can be 40% faster than chrome again
|
||||
|
||||
* if you're cpu-bottlenecked, or the browser is maxing a cpu core:
|
||||
* up to 30% faster uploads if you hide the upload status list by switching away from the `[🚀]` up2k ui-tab (or closing it)
|
||||
* switching to another browser-tab also works, the favicon will update every 10 seconds in that case
|
||||
* unlikely to be a problem, but can happen when uploding many small files, or your internet is too fast, or PC too slow
|
||||
|
||||
|
||||
# security
|
||||
|
||||
some notes on hardening
|
||||
@@ -903,6 +938,83 @@ however you can hit `F12` in the up2k tab and use the devtools to see how far yo
|
||||
`await fetch('/inc', {method:'PUT', body:JSON.stringify(ng,null,1)})`
|
||||
|
||||
|
||||
# HTTP API
|
||||
|
||||
* table-column `params` = URL parameters; `?foo=bar&qux=...`
|
||||
* table-column `body` = POST payload
|
||||
* method `jPOST` = json post
|
||||
* method `mPOST` = multipart post
|
||||
* method `uPOST` = url-encoded post
|
||||
* `FILE` = conventional HTTP file upload entry (rfc1867 et al, filename in `Content-Disposition`)
|
||||
|
||||
authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
|
||||
|
||||
## read
|
||||
|
||||
| method | params | result |
|
||||
|--|--|--|
|
||||
| GET | `?ls` | list files/folders at URL as JSON |
|
||||
| GET | `?ls&dots` | list files/folders at URL as JSON, including dotfiles |
|
||||
| GET | `?ls=t` | list files/folders at URL as plaintext |
|
||||
| GET | `?ls=v` | list files/folders at URL, terminal-formatted |
|
||||
| GET | `?b` | list files/folders at URL as simplified HTML |
|
||||
| GET | `?tree=.` | list one level of subdirectories inside URL |
|
||||
| GET | `?tree` | list one level of subdirectories for each level until URL |
|
||||
| GET | `?tar` | download everything below URL as a tar file |
|
||||
| GET | `?zip=utf-8` | download everything below URL as a zip file |
|
||||
| GET | `?ups` | show recent uploads from your IP |
|
||||
| GET | `?ups&filter=f` | ...where URL contains `f` |
|
||||
| GET | `?mime=foo` | specify return mimetype `foo` |
|
||||
| GET | `?raw` | get markdown file at URL as plaintext |
|
||||
| GET | `?txt` | get file at URL as plaintext |
|
||||
| GET | `?txt=iso-8859-1` | ...with specific charset |
|
||||
| GET | `?th` | get image/video at URL as thumbnail |
|
||||
| GET | `?th=opus` | convert audio file to 128kbps opus |
|
||||
|
||||
| method | body | result |
|
||||
|--|--|--|
|
||||
| jPOST | `{"q":"foo"}` | do a server-wide search; see the `[🔎]` search tab `raw` field for syntax |
|
||||
|
||||
| method | params | body | result |
|
||||
|--|--|--|--|
|
||||
| jPOST | `?tar` | `["foo","bar"]` | download folders `foo` and `bar` inside URL as a tar file |
|
||||
|
||||
## write
|
||||
|
||||
| method | params | result |
|
||||
|--|--|--|
|
||||
| GET | `?move=/foo/bar` | move/rename the file/folder at URL to /foo/bar |
|
||||
|
||||
| method | params | body | result |
|
||||
|--|--|--|--|
|
||||
| PUT | | (binary data) | upload into file at URL |
|
||||
| PUT | `?gz` | (binary data) | compress with gzip and write into file at URL |
|
||||
| PUT | `?xz` | (binary data) | compress with xz and write into file at URL |
|
||||
| mPOST | | `act=bput`, `f=FILE` | upload `FILE` into the folder at URL |
|
||||
| mPOST | `?j` | `act=bput`, `f=FILE` | ...and reply with json |
|
||||
| mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
|
||||
| GET | `?delete` | | delete URL recursively |
|
||||
| jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
|
||||
| uPOST | | `msg=foo` | send message `foo` into server log |
|
||||
| mPOST | | `act=tput`, `body=TEXT` | overwrite markdown document at URL |
|
||||
|
||||
server behavior of `msg` can be reconfigured with `--urlform`
|
||||
|
||||
## admin
|
||||
|
||||
| method | params | result |
|
||||
|--|--|--|
|
||||
| GET | `?reload=cfg` | reload config files and rescan volumes |
|
||||
| GET | `?scan` | initiate a rescan of the volume which provides URL |
|
||||
| GET | `?stack` | show a stacktrace of all threads |
|
||||
|
||||
## general
|
||||
|
||||
| method | params | result |
|
||||
|--|--|--|
|
||||
| GET | `?pw=x` | logout |
|
||||
|
||||
|
||||
# dependencies
|
||||
|
||||
mandatory deps:
|
||||
@@ -919,7 +1031,7 @@ enable music tags:
|
||||
|
||||
enable [thumbnails](#thumbnails) of...
|
||||
* **images:** `Pillow` (requires py2.7 or py3.5+)
|
||||
* **videos:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
|
||||
* **videos/audio:** `ffmpeg` and `ffprobe` somewhere in `$PATH`
|
||||
* **HEIF pictures:** `pyheif-pillow-opener` (requires Linux or a C compiler)
|
||||
* **AVIF pictures:** `pillow-avif-plugin`
|
||||
|
||||
@@ -953,13 +1065,15 @@ pls note that `copyparty-sfx.sh` will fail if you rename `copyparty-sfx.py` to `
|
||||
reduce the size of an sfx by removing features
|
||||
|
||||
if you don't need all the features, you can repack the sfx and save a bunch of space; all you need is an sfx and a copy of this repo (nothing else to download or build, except if you're on windows then you need msys2 or WSL)
|
||||
* `525k` size of original sfx.py as of v0.11.30
|
||||
* `315k` after `./scripts/make-sfx.sh re no-ogv`
|
||||
* `223k` after `./scripts/make-sfx.sh re no-ogv no-cm`
|
||||
* `584k` size of original sfx.py as of v1.1.0
|
||||
* `392k` after `./scripts/make-sfx.sh re no-ogv`
|
||||
* `310k` after `./scripts/make-sfx.sh re no-ogv no-cm`
|
||||
* `269k` after `./scripts/make-sfx.sh re no-ogv no-cm no-hl`
|
||||
|
||||
the features you can opt to drop are
|
||||
* `ogv`.js, the opus/vorbis decoder which is needed by apple devices to play foss audio files, saves ~192k
|
||||
* `cm`/easymde, the "fancy" markdown editor, saves ~92k
|
||||
* `cm`/easymde, the "fancy" markdown editor, saves ~82k
|
||||
* `hl`, prism, the syntax hilighter, saves ~41k
|
||||
* `fnt`, source-code-pro, the monospace font, saves ~9k
|
||||
* `dd`, the custom mouse cursor for the media player tray tab, saves ~2k
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import print_function, unicode_literals
|
||||
|
||||
"""
|
||||
up2k.py: upload to copyparty
|
||||
2021-10-12, v0.9, ed <irc.rizon.net>, MIT-Licensed
|
||||
2021-10-31, v0.11, ed <irc.rizon.net>, MIT-Licensed
|
||||
https://github.com/9001/copyparty/blob/hovudstraum/bin/up2k.py
|
||||
|
||||
- dependencies: requests
|
||||
@@ -250,9 +250,10 @@ def walkdirs(tops):
|
||||
"""recursive statdir for a list of tops, yields [top, relpath, stat]"""
|
||||
sep = "{0}".format(os.sep).encode("ascii")
|
||||
for top in tops:
|
||||
stop = top
|
||||
if top[-1:] == sep:
|
||||
stop = os.path.dirname(top.rstrip(sep))
|
||||
stop = top.rstrip(sep)
|
||||
else:
|
||||
stop = os.path.dirname(top)
|
||||
|
||||
if os.path.isdir(top):
|
||||
for ap, inf in walkdir(top):
|
||||
@@ -454,6 +455,7 @@ class Ctl(object):
|
||||
self.nbytes = nbytes
|
||||
|
||||
if ar.td:
|
||||
requests.packages.urllib3.disable_warnings()
|
||||
req_ses.verify = False
|
||||
if ar.te:
|
||||
req_ses.verify = ar.te
|
||||
|
||||
@@ -30,6 +30,7 @@ however if your copyparty is behind a reverse-proxy, you may want to use [`share
|
||||
# OS integration
|
||||
init-scripts to start copyparty as a service
|
||||
* [`systemd/copyparty.service`](systemd/copyparty.service) runs the sfx normally
|
||||
* [`rc/copyparty`](rc/copyparty) runs sfx normally on freebsd, create a `copyparty` user
|
||||
* [`systemd/prisonparty.service`](systemd/prisonparty.service) runs the sfx in a chroot
|
||||
* [`openrc/copyparty`](openrc/copyparty)
|
||||
|
||||
|
||||
31
contrib/rc/copyparty
Normal file
31
contrib/rc/copyparty
Normal file
@@ -0,0 +1,31 @@
|
||||
#!/bin/sh
|
||||
#
|
||||
# PROVIDE: copyparty
|
||||
# REQUIRE: networking
|
||||
# KEYWORD:
|
||||
|
||||
. /etc/rc.subr
|
||||
|
||||
name="copyparty"
|
||||
rcvar="copyparty_enable"
|
||||
copyparty_user="copyparty"
|
||||
copyparty_args="-e2dsa -v /storage:/storage:r" # change as you see fit
|
||||
copyparty_command="/usr/local/bin/python3.8 /usr/local/copyparty/copyparty-sfx.py ${copyparty_args}"
|
||||
pidfile="/var/run/copyparty/${name}.pid"
|
||||
command="/usr/sbin/daemon"
|
||||
command_args="-P ${pidfile} -r -f ${copyparty_command}"
|
||||
|
||||
stop_postcmd="copyparty_shutdown"
|
||||
|
||||
copyparty_shutdown()
|
||||
{
|
||||
if [ -e "${pidfile}" ]; then
|
||||
echo "Stopping supervising daemon."
|
||||
kill -s TERM `cat ${pidfile}`
|
||||
fi
|
||||
}
|
||||
|
||||
load_rc_config $name
|
||||
: ${copyparty_enable:=no}
|
||||
|
||||
run_rc_command "$1"
|
||||
@@ -3,10 +3,16 @@
|
||||
#
|
||||
# installation:
|
||||
# cp -pv copyparty.service /etc/systemd/system && systemctl enable --now copyparty
|
||||
# restorecon -vr /etc/systemd/system/copyparty.service
|
||||
# firewall-cmd --permanent --add-port={80,443,3923}/tcp
|
||||
# firewall-cmd --reload
|
||||
#
|
||||
# you may want to:
|
||||
# change '/usr/bin/python' to another interpreter
|
||||
# change '/usr/bin/python3' to another interpreter
|
||||
# change '/mnt::rw' to another location or permission-set
|
||||
# remove '-p 80,443,3923' to only listen on port 3923
|
||||
# add '-i 127.0.0.1' to only allow local connections
|
||||
# add '--use-fpool' if uploading into nfs locations
|
||||
#
|
||||
# with `Type=notify`, copyparty will signal systemd when it is ready to
|
||||
# accept connections; correctly delaying units depending on copyparty.
|
||||
@@ -14,11 +20,8 @@
|
||||
# python disabling line-buffering, so messages are out-of-order:
|
||||
# https://user-images.githubusercontent.com/241032/126040249-cb535cc7-c599-4931-a796-a5d9af691bad.png
|
||||
#
|
||||
# enable line-buffering for realtime logging (slight performance cost):
|
||||
# modify ExecStart and prefix it with `/usr/bin/stdbuf -oL` like so:
|
||||
# ExecStart=/usr/bin/stdbuf -oL /usr/bin/python3 [...]
|
||||
# but some systemd versions require this instead (higher performance cost):
|
||||
# inside the [Service] block, add the following line:
|
||||
# if you remove -q to enable logging, you may also want to remove the
|
||||
# following line to enable buffering (slightly better performance):
|
||||
# Environment=PYTHONUNBUFFERED=x
|
||||
|
||||
[Unit]
|
||||
@@ -27,8 +30,10 @@ Description=copyparty file server
|
||||
[Service]
|
||||
Type=notify
|
||||
SyslogIdentifier=copyparty
|
||||
ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -v /mnt::rw
|
||||
Environment=PYTHONUNBUFFERED=x
|
||||
ExecReload=/bin/kill -s USR1 $MAINPID
|
||||
ExecStartPre=/bin/bash -c 'mkdir -p /run/tmpfiles.d/ && echo "x /tmp/pe-copyparty*" > /run/tmpfiles.d/copyparty.conf'
|
||||
ExecStart=/usr/bin/python3 /usr/local/bin/copyparty-sfx.py -q -p 80,443,3923 -v /mnt::rw
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
@@ -20,7 +20,7 @@ import threading
|
||||
import traceback
|
||||
from textwrap import dedent
|
||||
|
||||
from .__init__ import E, WINDOWS, VT100, PY2, unicode
|
||||
from .__init__ import E, WINDOWS, ANYWIN, VT100, PY2, unicode
|
||||
from .__version__ import S_VERSION, S_BUILD_DT, CODENAME
|
||||
from .svchub import SvcHub
|
||||
from .util import py_desc, align_tab, IMPLICATIONS, ansi_re
|
||||
@@ -186,6 +186,32 @@ def configure_ssl_ciphers(al):
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
def args_from_cfg(cfg_path):
|
||||
ret = []
|
||||
skip = False
|
||||
with open(cfg_path, "rb") as f:
|
||||
for ln in [x.decode("utf-8").strip() for x in f]:
|
||||
if not ln:
|
||||
skip = False
|
||||
continue
|
||||
|
||||
if ln.startswith("#"):
|
||||
continue
|
||||
|
||||
if not ln.startswith("-"):
|
||||
continue
|
||||
|
||||
if skip:
|
||||
continue
|
||||
|
||||
try:
|
||||
ret.extend(ln.split(" ", 1))
|
||||
except:
|
||||
ret.append(ln)
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
def sighandler(sig=None, frame=None):
|
||||
msg = [""] * 5
|
||||
for th in threading.enumerate():
|
||||
@@ -208,6 +234,8 @@ def run_argparse(argv, formatter):
|
||||
except:
|
||||
fk_salt = "hunter2"
|
||||
|
||||
cores = os.cpu_count() if hasattr(os, "cpu_count") else 4
|
||||
|
||||
sects = [
|
||||
[
|
||||
"accounts",
|
||||
@@ -333,7 +361,7 @@ def run_argparse(argv, formatter):
|
||||
ap2 = ap.add_argument_group('general options')
|
||||
ap2.add_argument("-c", metavar="PATH", type=u, action="append", help="add config file")
|
||||
ap2.add_argument("-nc", metavar="NUM", type=int, default=64, help="max num clients")
|
||||
ap2.add_argument("-j", metavar="CORES", type=int, default=1, help="max num cpu cores")
|
||||
ap2.add_argument("-j", metavar="CORES", type=int, default=1, help="max num cpu cores, 0=all")
|
||||
ap2.add_argument("-a", metavar="ACCT", type=u, action="append", help="add account, USER:PASS; example [ed:wark")
|
||||
ap2.add_argument("-v", metavar="VOL", type=u, action="append", help="add volume, SRC:DST:FLAG; example [.::r], [/mnt/nas/music:/music:r:aed")
|
||||
ap2.add_argument("-ed", action="store_true", help="enable ?dots")
|
||||
@@ -395,23 +423,30 @@ def run_argparse(argv, formatter):
|
||||
ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|\?th=[wj]$", help="dont log URLs matching")
|
||||
|
||||
ap2 = ap.add_argument_group('admin panel options')
|
||||
ap2.add_argument("--no-reload", action="store_true", help="disable ?reload=cfg (reload users/volumes/volflags from config file)")
|
||||
ap2.add_argument("--no-rescan", action="store_true", help="disable ?scan (volume reindexing)")
|
||||
ap2.add_argument("--no-stack", action="store_true", help="disable ?stack (list all stacks)")
|
||||
|
||||
ap2 = ap.add_argument_group('thumbnail options')
|
||||
ap2.add_argument("--no-thumb", action="store_true", help="disable all thumbnails")
|
||||
ap2.add_argument("--no-athumb", action="store_true", help="disable audio thumbnails (spectrograms)")
|
||||
ap2.add_argument("--no-vthumb", action="store_true", help="disable video thumbnails")
|
||||
ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res")
|
||||
ap2.add_argument("--th-mt", metavar="CORES", type=int, default=0, help="max num cpu cores to use, 0=all")
|
||||
ap2.add_argument("--th-mt", metavar="CORES", type=int, default=cores, help="num cpu cores to use for generating thumbnails")
|
||||
ap2.add_argument("--th-no-crop", action="store_true", help="dynamic height; show full image")
|
||||
ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
|
||||
ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
|
||||
ap2.add_argument("--th-ff-jpg", action="store_true", help="force jpg for video thumbs")
|
||||
ap2.add_argument("--th-ff-swr", action="store_true", help="use swresample instead of soxr for audio thumbs")
|
||||
ap2.add_argument("--th-poke", metavar="SEC", type=int, default=300, help="activity labeling cooldown")
|
||||
ap2.add_argument("--th-clean", metavar="SEC", type=int, default=43200, help="cleanup interval; 0=disabled")
|
||||
ap2.add_argument("--th-maxage", metavar="SEC", type=int, default=604800, help="max folder age")
|
||||
ap2.add_argument("--th-covers", metavar="N,N", type=u, default="folder.png,folder.jpg,cover.png,cover.jpg", help="folder thumbnails to stat for")
|
||||
|
||||
ap2 = ap.add_argument_group('transcoding options')
|
||||
ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
|
||||
ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete transcode output after SEC seconds")
|
||||
|
||||
ap2 = ap.add_argument_group('general db options')
|
||||
ap2.add_argument("-e2d", action="store_true", help="enable up2k database")
|
||||
ap2.add_argument("-e2ds", action="store_true", help="enable up2k db-scanner, sets -e2d")
|
||||
@@ -419,7 +454,6 @@ def run_argparse(argv, formatter):
|
||||
ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs)")
|
||||
ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching paths during e2ds folder scans")
|
||||
ap2.add_argument("--no-idx", metavar="PTN", type=u, help="regex: disable indexing of matching paths during e2ds folder scans")
|
||||
ap2.add_argument("--re-int", metavar="SEC", type=int, default=30, help="disk rescan check interval")
|
||||
ap2.add_argument("--re-maxage", metavar="SEC", type=int, default=0, help="disk rescan volume interval, 0=off, can be set per-volume with the 'scan' volflag")
|
||||
ap2.add_argument("--srch-time", metavar="SEC", type=int, default=30, help="search deadline")
|
||||
|
||||
@@ -428,8 +462,8 @@ def run_argparse(argv, formatter):
|
||||
ap2.add_argument("-e2ts", action="store_true", help="enable metadata scanner, sets -e2t")
|
||||
ap2.add_argument("-e2tsr", action="store_true", help="rescan all metadata, sets -e2ts")
|
||||
ap2.add_argument("--no-mutagen", action="store_true", help="use FFprobe for tags instead")
|
||||
ap2.add_argument("--no-mtag-mt", action="store_true", help="disable tag-read parallelism")
|
||||
ap2.add_argument("--no-mtag-ff", action="store_true", help="never use FFprobe as tag reader")
|
||||
ap2.add_argument("--mtag-mt", metavar="CORES", type=int, default=cores, help="num cpu cores to use for tag scanning")
|
||||
ap2.add_argument("-mtm", metavar="M=t,t,t", type=u, action="append", help="add/replace metadata mapping")
|
||||
ap2.add_argument("-mte", metavar="M,M,M", type=u, help="tags to index/display (comma-sep.)",
|
||||
default="circle,album,.tn,artist,title,.bpm,key,.dur,.q,.vq,.aq,vc,ac,res,.fps,ahash,vhash")
|
||||
@@ -440,6 +474,7 @@ def run_argparse(argv, formatter):
|
||||
ap2 = ap.add_argument_group('ui options')
|
||||
ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
|
||||
ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
|
||||
ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
|
||||
|
||||
ap2 = ap.add_argument_group('debug options')
|
||||
ap2.add_argument("--no-sendfile", action="store_true", help="disable sendfile")
|
||||
@@ -482,7 +517,12 @@ def main(argv=None):
|
||||
if HAVE_SSL:
|
||||
ensure_cert()
|
||||
|
||||
deprecated = [["-e2s", "-e2ds"]]
|
||||
for k, v in zip(argv, argv[1:]):
|
||||
if k == "-c":
|
||||
supp = args_from_cfg(v)
|
||||
argv.extend(supp)
|
||||
|
||||
deprecated = []
|
||||
for dk, nk in deprecated:
|
||||
try:
|
||||
idx = argv.index(dk)
|
||||
@@ -494,6 +534,12 @@ def main(argv=None):
|
||||
argv[idx] = nk
|
||||
time.sleep(2)
|
||||
|
||||
try:
|
||||
if len(argv) == 1 and (ANYWIN or not os.geteuid()):
|
||||
argv.extend(["-p80,443,3923", "--ign-ebind"])
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
al = run_argparse(argv, RiceFormatter)
|
||||
except AssertionError:
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# coding: utf-8
|
||||
|
||||
VERSION = (1, 0, 12)
|
||||
CODENAME = "sufficient"
|
||||
BUILD_DT = (2021, 10, 24)
|
||||
VERSION = (1, 1, 1)
|
||||
CODENAME = "opus"
|
||||
BUILD_DT = (2021, 11, 8)
|
||||
|
||||
S_VERSION = ".".join(map(str, VERSION))
|
||||
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)
|
||||
|
||||
@@ -526,8 +526,27 @@ class AuthSrv(object):
|
||||
|
||||
yield prev, True
|
||||
|
||||
def _map_volume(self, src, dst, mount, daxs, mflags):
|
||||
if dst in mount:
|
||||
m = "multiple filesystem-paths mounted at [/{}]:\n [{}]\n [{}]"
|
||||
self.log(m.format(dst, mount[dst], src), c=1)
|
||||
raise Exception("invalid config")
|
||||
|
||||
if src in mount.values():
|
||||
m = "warning: filesystem-path [{}] mounted in multiple locations:"
|
||||
m = m.format(src)
|
||||
for v in [k for k, v in mount.items() if v == src] + [dst]:
|
||||
m += "\n /{}".format(v)
|
||||
|
||||
self.log(m, c=3)
|
||||
|
||||
mount[dst] = src
|
||||
daxs[dst] = AXS()
|
||||
mflags[dst] = {}
|
||||
|
||||
def _parse_config_file(self, fd, acct, daxs, mflags, mount):
|
||||
# type: (any, str, dict[str, AXS], any, str) -> None
|
||||
skip = False
|
||||
vol_src = None
|
||||
vol_dst = None
|
||||
self.line_ctr = 0
|
||||
@@ -537,6 +556,11 @@ class AuthSrv(object):
|
||||
vol_src = None
|
||||
vol_dst = None
|
||||
|
||||
if skip:
|
||||
if not ln:
|
||||
skip = False
|
||||
continue
|
||||
|
||||
if not ln or ln.startswith("#"):
|
||||
continue
|
||||
|
||||
@@ -544,6 +568,8 @@ class AuthSrv(object):
|
||||
if ln.startswith("u "):
|
||||
u, p = ln[2:].split(":", 1)
|
||||
acct[u] = p
|
||||
elif ln.startswith("-"):
|
||||
skip = True # argv
|
||||
else:
|
||||
vol_src = ln
|
||||
continue
|
||||
@@ -556,9 +582,7 @@ class AuthSrv(object):
|
||||
# cfg files override arguments and previous files
|
||||
vol_src = bos.path.abspath(vol_src)
|
||||
vol_dst = vol_dst.strip("/")
|
||||
mount[vol_dst] = vol_src
|
||||
daxs[vol_dst] = AXS()
|
||||
mflags[vol_dst] = {}
|
||||
self._map_volume(vol_src, vol_dst, mount, daxs, mflags)
|
||||
continue
|
||||
|
||||
try:
|
||||
@@ -597,7 +621,7 @@ class AuthSrv(object):
|
||||
if uname == "":
|
||||
uname = "*"
|
||||
|
||||
for un in uname.split(","):
|
||||
for un in uname.replace(",", " ").strip().split():
|
||||
if "r" in lvl:
|
||||
axs.uread[un] = 1
|
||||
|
||||
@@ -663,9 +687,7 @@ class AuthSrv(object):
|
||||
# print("\n".join([src, dst, perms]))
|
||||
src = bos.path.abspath(src)
|
||||
dst = dst.strip("/")
|
||||
mount[dst] = src
|
||||
daxs[dst] = AXS()
|
||||
mflags[dst] = {}
|
||||
self._map_volume(src, dst, mount, daxs, mflags)
|
||||
|
||||
for x in perms.split(":"):
|
||||
lvl, uname = x.split(",", 1) if "," in x else [x, ""]
|
||||
@@ -986,7 +1008,7 @@ class AuthSrv(object):
|
||||
v, _ = vfs.get("/", "*", False, True)
|
||||
if self.warn_anonwrite and os.getcwd() == v.realpath:
|
||||
self.warn_anonwrite = False
|
||||
msg = "anyone can read/write the current directory: {}"
|
||||
msg = "anyone can read/write the current directory: {}\n"
|
||||
self.log(msg.format(v.realpath), c=1)
|
||||
except Pebkac:
|
||||
self.warn_anonwrite = True
|
||||
|
||||
@@ -21,6 +21,10 @@ def getsize(p):
|
||||
return os.path.getsize(fsenc(p))
|
||||
|
||||
|
||||
def isfile(p):
|
||||
return os.path.isfile(fsenc(p))
|
||||
|
||||
|
||||
def isdir(p):
|
||||
return os.path.isdir(fsenc(p))
|
||||
|
||||
|
||||
@@ -62,6 +62,11 @@ class BrokerMp(object):
|
||||
|
||||
procs.pop()
|
||||
|
||||
def reload(self):
|
||||
self.log("broker", "reloading")
|
||||
for _, proc in enumerate(self.procs):
|
||||
proc.q_pend.put([0, "reload", []])
|
||||
|
||||
def collector(self, proc):
|
||||
"""receive message from hub in other process"""
|
||||
while True:
|
||||
|
||||
@@ -29,7 +29,7 @@ class MpWorker(object):
|
||||
# we inherited signal_handler from parent,
|
||||
# replace it with something harmless
|
||||
if not FAKE_MP:
|
||||
for sig in [signal.SIGINT, signal.SIGTERM]:
|
||||
for sig in [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]:
|
||||
signal.signal(sig, self.signal_handler)
|
||||
|
||||
# starting to look like a good idea
|
||||
@@ -69,6 +69,11 @@ class MpWorker(object):
|
||||
sys.exit(0)
|
||||
return
|
||||
|
||||
elif dest == "reload":
|
||||
self.logw("mpw.asrv reloading")
|
||||
self.asrv.reload()
|
||||
self.logw("mpw.asrv reloaded")
|
||||
|
||||
elif dest == "listen":
|
||||
self.httpsrv.listen(args[0], args[1])
|
||||
|
||||
|
||||
@@ -21,10 +21,13 @@ class BrokerThr(object):
|
||||
|
||||
# instantiate all services here (TODO: inheritance?)
|
||||
self.httpsrv = HttpSrv(self, None)
|
||||
self.reload = self.noop
|
||||
|
||||
def shutdown(self):
|
||||
# self.log("broker", "shutting down")
|
||||
self.httpsrv.shutdown()
|
||||
|
||||
def noop(self):
|
||||
pass
|
||||
|
||||
def put(self, want_retval, dest, *args):
|
||||
|
||||
@@ -298,7 +298,7 @@ class HttpCli(object):
|
||||
|
||||
# default to utf8 html if no content-type is set
|
||||
if not mime:
|
||||
mime = self.out_headers.get("Content-Type", "text/html; charset=UTF-8")
|
||||
mime = self.out_headers.get("Content-Type", "text/html; charset=utf-8")
|
||||
|
||||
self.out_headers["Content-Type"] = mime
|
||||
|
||||
@@ -419,6 +419,9 @@ class HttpCli(object):
|
||||
return self.scanvol()
|
||||
|
||||
if not self.vpath:
|
||||
if "reload" in self.uparam:
|
||||
return self.handle_reload()
|
||||
|
||||
if "stack" in self.uparam:
|
||||
return self.tx_stack()
|
||||
|
||||
@@ -1461,12 +1464,15 @@ class HttpCli(object):
|
||||
else:
|
||||
self.permit_caching()
|
||||
|
||||
if "txt" in self.uparam:
|
||||
mime = "text/plain; charset={}".format(self.uparam["txt"] or "utf-8")
|
||||
elif "mime" in self.uparam:
|
||||
mime = self.uparam.get("mime")
|
||||
else:
|
||||
mime = guess_mime(req_path)
|
||||
|
||||
self.out_headers["Accept-Ranges"] = "bytes"
|
||||
self.send_headers(
|
||||
length=upper - lower,
|
||||
status=status,
|
||||
mime=guess_mime(req_path),
|
||||
)
|
||||
self.send_headers(length=upper - lower, status=status, mime=mime)
|
||||
|
||||
logmsg += unicode(status) + logtail
|
||||
|
||||
@@ -1711,7 +1717,7 @@ class HttpCli(object):
|
||||
|
||||
vn, _ = self.asrv.vfs.get(self.vpath, self.uname, True, True)
|
||||
|
||||
args = [self.asrv.vfs.all_vols, [vn.vpath]]
|
||||
args = [self.asrv.vfs.all_vols, [vn.vpath], False]
|
||||
|
||||
x = self.conn.hsrv.broker.put(True, "up2k.rescan", *args)
|
||||
x = x.get()
|
||||
@@ -1721,6 +1727,20 @@ class HttpCli(object):
|
||||
|
||||
raise Pebkac(500, x)
|
||||
|
||||
def handle_reload(self):
|
||||
act = self.uparam.get("reload")
|
||||
if act != "cfg":
|
||||
raise Pebkac(400, "only config files ('cfg') can be reloaded rn")
|
||||
|
||||
if not [x for x in self.wvol if x in self.rvol]:
|
||||
raise Pebkac(403, "not allowed for user " + self.uname)
|
||||
|
||||
if self.args.no_reload:
|
||||
raise Pebkac(403, "the reload feature is disabled in server config")
|
||||
|
||||
x = self.conn.hsrv.broker.put(True, "reload")
|
||||
return self.redirect("", "?h", x.get(), "return to", False)
|
||||
|
||||
def tx_stack(self):
|
||||
if not [x for x in self.wvol if x in self.rvol]:
|
||||
raise Pebkac(403, "not allowed for user " + self.uname)
|
||||
@@ -1856,6 +1876,64 @@ class HttpCli(object):
|
||||
)
|
||||
self.loud_reply(x.get())
|
||||
|
||||
def tx_ls(self, ls):
|
||||
dirs = ls["dirs"]
|
||||
files = ls["files"]
|
||||
arg = self.uparam["ls"]
|
||||
if arg in ["v", "t", "txt"]:
|
||||
try:
|
||||
biggest = max(ls["files"] + ls["dirs"], key=itemgetter("sz"))["sz"]
|
||||
except:
|
||||
biggest = 0
|
||||
|
||||
if arg == "v":
|
||||
fmt = "\033[0;7;36m{{}} {{:>{}}}\033[0m {{}}"
|
||||
nfmt = "{}"
|
||||
biggest = 0
|
||||
f2 = "".join(
|
||||
"{}{{}}".format(x)
|
||||
for x in [
|
||||
"\033[7m",
|
||||
"\033[27m",
|
||||
"",
|
||||
"\033[0;1m",
|
||||
"\033[0;36m",
|
||||
"\033[0m",
|
||||
]
|
||||
)
|
||||
ctab = {"B": 6, "K": 5, "M": 1, "G": 3}
|
||||
for lst in [dirs, files]:
|
||||
for x in lst:
|
||||
a = x["dt"].replace("-", " ").replace(":", " ").split(" ")
|
||||
x["dt"] = f2.format(*list(a))
|
||||
sz = humansize(x["sz"], True)
|
||||
x["sz"] = "\033[0;3{}m{:>5}".format(ctab.get(sz[-1:], 0), sz)
|
||||
else:
|
||||
fmt = "{{}} {{:{},}} {{}}"
|
||||
nfmt = "{:,}"
|
||||
|
||||
fmt = fmt.format(len(nfmt.format(biggest)))
|
||||
ret = [
|
||||
"# {}: {}".format(x, ls[x])
|
||||
for x in ["acct", "perms", "srvinf"]
|
||||
if x in ls
|
||||
]
|
||||
ret += [
|
||||
fmt.format(x["dt"], x["sz"], x["name"])
|
||||
for y in [dirs, files]
|
||||
for x in y
|
||||
]
|
||||
ret = "\n".join(ret)
|
||||
mime = "text/plain; charset=utf-8"
|
||||
else:
|
||||
[x.pop(k) for k in ["name", "dt"] for y in [dirs, files] for x in y]
|
||||
|
||||
ret = json.dumps(ls)
|
||||
mime = "application/json"
|
||||
|
||||
self.reply(ret.encode("utf-8", "replace") + b"\n", mime=mime)
|
||||
return True
|
||||
|
||||
def tx_browser(self):
|
||||
vpath = ""
|
||||
vpnodes = [["", "/"]]
|
||||
@@ -2014,6 +2092,7 @@ class HttpCli(object):
|
||||
"def_hcols": [],
|
||||
"have_up2k_idx": ("e2d" in vn.flags),
|
||||
"have_tags_idx": ("e2t" in vn.flags),
|
||||
"have_acode": (not self.args.no_acode),
|
||||
"have_mv": (not self.args.no_mv),
|
||||
"have_del": (not self.args.no_del),
|
||||
"have_zip": (not self.args.no_zip),
|
||||
@@ -2027,9 +2106,7 @@ class HttpCli(object):
|
||||
}
|
||||
if not self.can_read:
|
||||
if is_ls:
|
||||
ret = json.dumps(ls_ret)
|
||||
self.reply(ret.encode("utf-8", "replace"), mime="application/json")
|
||||
return True
|
||||
return self.tx_ls(ls_ret)
|
||||
|
||||
if not stat.S_ISDIR(st.st_mode):
|
||||
return self.tx_404(True)
|
||||
@@ -2126,6 +2203,8 @@ class HttpCli(object):
|
||||
|
||||
try:
|
||||
ext = "---" if is_dir else fn.rsplit(".", 1)[1]
|
||||
if len(ext) > 16:
|
||||
ext = ext[:16]
|
||||
except:
|
||||
ext = "%"
|
||||
|
||||
@@ -2204,13 +2283,26 @@ class HttpCli(object):
|
||||
f["tags"] = {}
|
||||
|
||||
if is_ls:
|
||||
[x.pop(k) for k in ["name", "dt"] for y in [dirs, files] for x in y]
|
||||
ls_ret["dirs"] = dirs
|
||||
ls_ret["files"] = files
|
||||
ls_ret["taglist"] = taglist
|
||||
ret = json.dumps(ls_ret)
|
||||
self.reply(ret.encode("utf-8", "replace"), mime="application/json")
|
||||
return True
|
||||
return self.tx_ls(ls_ret)
|
||||
|
||||
doc = self.uparam.get("doc") if self.can_read else None
|
||||
if doc:
|
||||
doc = unquotep(doc.replace("+", " "))
|
||||
j2a["docname"] = doc
|
||||
if next((x for x in files if x["name"] == doc), None):
|
||||
with open(os.path.join(abspath, doc), "rb") as f:
|
||||
doc = f.read().decode("utf-8", "replace")
|
||||
else:
|
||||
self.log("doc 404: [{}]".format(doc), c=6)
|
||||
doc = "( textfile not found )"
|
||||
|
||||
j2a["doc"] = doc
|
||||
|
||||
if not self.conn.hsrv.prism:
|
||||
j2a["no_prism"] = True
|
||||
|
||||
for d in dirs:
|
||||
d["name"] += "/"
|
||||
@@ -2220,6 +2312,7 @@ class HttpCli(object):
|
||||
j2a["files"] = dirs + files
|
||||
j2a["logues"] = logues
|
||||
j2a["taglist"] = taglist
|
||||
j2a["txt_ext"] = self.args.textfiles.replace(",", " ")
|
||||
|
||||
if "mth" in vn.flags:
|
||||
j2a["def_hcols"] = vn.flags["mth"].split(",")
|
||||
|
||||
@@ -39,7 +39,7 @@ class HttpConn(object):
|
||||
self.u2fh = hsrv.u2fh
|
||||
|
||||
enth = HAVE_PIL and not self.args.no_thumb
|
||||
self.thumbcli = ThumbCli(hsrv.broker) if enth else None
|
||||
self.thumbcli = ThumbCli(hsrv) if enth else None
|
||||
self.ico = Ico(self.args)
|
||||
|
||||
self.t0 = time.time()
|
||||
|
||||
@@ -50,10 +50,9 @@ class HttpSrv(object):
|
||||
self.log = broker.log
|
||||
self.asrv = broker.asrv
|
||||
|
||||
nsuf = "-{}".format(nid) if nid else ""
|
||||
nsuf2 = "-n{}-i{:x}".format(nid, os.getpid()) if nid else ""
|
||||
nsuf = "-n{}-i{:x}".format(nid, os.getpid()) if nid else ""
|
||||
|
||||
self.name = "hsrv" + nsuf2
|
||||
self.name = "hsrv" + nsuf
|
||||
self.mutex = threading.Lock()
|
||||
self.stopping = False
|
||||
|
||||
@@ -61,6 +60,7 @@ class HttpSrv(object):
|
||||
self.tp_ncli = 0 # fading
|
||||
self.tp_time = None # latest worker collect
|
||||
self.tp_q = None if self.args.no_htp else queue.LifoQueue()
|
||||
self.t_periodic = None
|
||||
|
||||
self.u2fh = FHC()
|
||||
self.srvs = []
|
||||
@@ -76,6 +76,7 @@ class HttpSrv(object):
|
||||
x: env.get_template(x + ".html")
|
||||
for x in ["splash", "browser", "browser2", "msg", "md", "mde"]
|
||||
}
|
||||
self.prism = os.path.exists(os.path.join(E.mod, "web", "deps", "prism.js.gz"))
|
||||
|
||||
cert_path = os.path.join(E.cfg, "cert.pem")
|
||||
if bos.path.exists(cert_path):
|
||||
@@ -93,10 +94,6 @@ class HttpSrv(object):
|
||||
if self.args.log_thrs:
|
||||
start_log_thrs(self.log, self.args.log_thrs, nid)
|
||||
|
||||
t = threading.Thread(target=self.periodic, name="hsrv-pt" + nsuf)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
def start_threads(self, n):
|
||||
self.tp_nthr += n
|
||||
if self.args.log_htp:
|
||||
@@ -120,7 +117,7 @@ class HttpSrv(object):
|
||||
|
||||
def periodic(self):
|
||||
while True:
|
||||
time.sleep(2 if self.tp_ncli else 10)
|
||||
time.sleep(2 if self.tp_ncli or self.ncli else 10)
|
||||
with self.mutex:
|
||||
self.u2fh.clean()
|
||||
if self.tp_q:
|
||||
@@ -128,6 +125,10 @@ class HttpSrv(object):
|
||||
if self.tp_nthr > self.tp_ncli + 8:
|
||||
self.stop_threads(4)
|
||||
|
||||
if not self.ncli and not self.u2fh.cache and self.tp_nthr <= 8:
|
||||
self.t_periodic = None
|
||||
return
|
||||
|
||||
def listen(self, sck, nlisteners):
|
||||
ip, port = sck.getsockname()
|
||||
self.srvs.append(sck)
|
||||
@@ -146,7 +147,12 @@ class HttpSrv(object):
|
||||
fno = srv_sck.fileno()
|
||||
msg = "subscribed @ {}:{} f{}".format(ip, port, fno)
|
||||
self.log(self.name, msg)
|
||||
self.broker.put(False, "cb_httpsrv_up")
|
||||
|
||||
def fun():
|
||||
self.broker.put(False, "cb_httpsrv_up")
|
||||
|
||||
threading.Thread(target=fun).start()
|
||||
|
||||
while not self.stopping:
|
||||
if self.args.log_conn:
|
||||
self.log(self.name, "|%sC-ncli" % ("-" * 1,), c="1;30")
|
||||
@@ -186,6 +192,16 @@ class HttpSrv(object):
|
||||
|
||||
with self.mutex:
|
||||
self.ncli += 1
|
||||
if not self.t_periodic:
|
||||
name = "hsrv-pt"
|
||||
if self.nid:
|
||||
name += "-{}".format(self.nid)
|
||||
|
||||
t = threading.Thread(target=self.periodic, name=name)
|
||||
self.t_periodic = t
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
if self.tp_q:
|
||||
self.tp_time = self.tp_time or now
|
||||
self.tp_ncli = max(self.tp_ncli, self.ncli)
|
||||
|
||||
@@ -413,6 +413,9 @@ class MTag(object):
|
||||
return r1
|
||||
|
||||
def get_mutagen(self, abspath):
|
||||
if not bos.path.isfile(abspath):
|
||||
return {}
|
||||
|
||||
import mutagen
|
||||
|
||||
try:
|
||||
@@ -458,10 +461,16 @@ class MTag(object):
|
||||
return self.normalize_tags(ret, md)
|
||||
|
||||
def get_ffprobe(self, abspath):
|
||||
if not bos.path.isfile(abspath):
|
||||
return {}
|
||||
|
||||
ret, md = ffprobe(abspath)
|
||||
return self.normalize_tags(ret, md)
|
||||
|
||||
def get_bin(self, parsers, abspath):
|
||||
if not bos.path.isfile(abspath):
|
||||
return {}
|
||||
|
||||
pypath = os.path.abspath(os.path.dirname(os.path.dirname(__file__)))
|
||||
pypath = [str(pypath)] + [str(x) for x in sys.path if x]
|
||||
pypath = str(os.pathsep.join(pypath))
|
||||
|
||||
@@ -18,6 +18,7 @@ from .authsrv import AuthSrv
|
||||
from .tcpsrv import TcpSrv
|
||||
from .up2k import Up2k
|
||||
from .th_srv import ThumbSrv, HAVE_PIL, HAVE_WEBP
|
||||
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
|
||||
|
||||
|
||||
class SvcHub(object):
|
||||
@@ -36,7 +37,9 @@ class SvcHub(object):
|
||||
self.argv = argv
|
||||
self.logf = None
|
||||
self.stop_req = False
|
||||
self.reload_req = False
|
||||
self.stopping = False
|
||||
self.reloading = False
|
||||
self.stop_cond = threading.Condition()
|
||||
self.retcode = 0
|
||||
self.httpsrv_up = 0
|
||||
@@ -88,11 +91,22 @@ class SvcHub(object):
|
||||
"thumb", msg.format(" " * 37, os.path.basename(sys.executable)), c=3
|
||||
)
|
||||
|
||||
if not args.no_acode and args.no_thumb:
|
||||
msg = "setting --no-acode because --no-thumb (sorry)"
|
||||
self.log("thumb", msg, c=6)
|
||||
args.no_acode = True
|
||||
|
||||
if not args.no_acode and (not HAVE_FFMPEG or not HAVE_FFPROBE):
|
||||
msg = "setting --no-acode because either FFmpeg or FFprobe is not available"
|
||||
self.log("thumb", msg, c=6)
|
||||
args.no_acode = True
|
||||
|
||||
args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
|
||||
|
||||
# decide which worker impl to use
|
||||
if self.check_mp_enable():
|
||||
from .broker_mp import BrokerMp as Broker
|
||||
else:
|
||||
self.log("root", "cannot efficiently use multiple CPU cores")
|
||||
from .broker_thr import BrokerThr as Broker
|
||||
|
||||
self.broker = Broker(self)
|
||||
@@ -104,16 +118,16 @@ class SvcHub(object):
|
||||
if not failed:
|
||||
return
|
||||
|
||||
m = "{}/{} workers failed to start"
|
||||
m = m.format(failed, expected)
|
||||
self.log("root", m, 1)
|
||||
|
||||
if self.args.ign_ebind_all:
|
||||
return
|
||||
|
||||
if self.args.ign_ebind and self.tcpsrv.srv:
|
||||
return
|
||||
|
||||
m = "{}/{} workers failed to start"
|
||||
m = m.format(failed, expected)
|
||||
self.log("root", m, 1)
|
||||
|
||||
self.retcode = 1
|
||||
os.kill(os.getpid(), signal.SIGTERM)
|
||||
|
||||
@@ -122,6 +136,7 @@ class SvcHub(object):
|
||||
if self.httpsrv_up != self.broker.num_workers:
|
||||
return
|
||||
|
||||
time.sleep(0.1) # purely cosmetic dw
|
||||
self.log("root", "workers OK\n")
|
||||
self.up2k.init_vols()
|
||||
|
||||
@@ -182,7 +197,11 @@ class SvcHub(object):
|
||||
thr.daemon = True
|
||||
thr.start()
|
||||
|
||||
for sig in [signal.SIGINT, signal.SIGTERM]:
|
||||
sigs = [signal.SIGINT, signal.SIGTERM]
|
||||
if not ANYWIN:
|
||||
sigs.append(signal.SIGUSR1)
|
||||
|
||||
for sig in sigs:
|
||||
signal.signal(sig, self.signal_handler)
|
||||
|
||||
# macos hangs after shutdown on sigterm with while-sleep,
|
||||
@@ -206,18 +225,45 @@ class SvcHub(object):
|
||||
else:
|
||||
self.stop_thr()
|
||||
|
||||
def reload(self):
|
||||
if self.reloading:
|
||||
return "cannot reload; already in progress"
|
||||
|
||||
self.reloading = True
|
||||
t = threading.Thread(target=self._reload)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
return "reload initiated"
|
||||
|
||||
def _reload(self):
|
||||
self.log("root", "reload scheduled")
|
||||
with self.up2k.mutex:
|
||||
self.asrv.reload()
|
||||
self.up2k.reload()
|
||||
self.broker.reload()
|
||||
|
||||
self.reloading = False
|
||||
|
||||
def stop_thr(self):
|
||||
while not self.stop_req:
|
||||
with self.stop_cond:
|
||||
self.stop_cond.wait(9001)
|
||||
|
||||
if self.reload_req:
|
||||
self.reload_req = False
|
||||
self.reload()
|
||||
|
||||
self.shutdown()
|
||||
|
||||
def signal_handler(self, sig, frame):
|
||||
if self.stopping:
|
||||
return
|
||||
|
||||
self.stop_req = True
|
||||
if sig == signal.SIGUSR1:
|
||||
self.reload_req = True
|
||||
else:
|
||||
self.stop_req = True
|
||||
|
||||
with self.stop_cond:
|
||||
self.stop_cond.notify_all()
|
||||
|
||||
@@ -349,10 +395,11 @@ class SvcHub(object):
|
||||
|
||||
def check_mp_enable(self):
|
||||
if self.args.j == 1:
|
||||
self.log("root", "multiprocessing disabled by argument -j 1;")
|
||||
self.log("svchub", "multiprocessing disabled by argument -j 1")
|
||||
return False
|
||||
|
||||
if mp.cpu_count() <= 1:
|
||||
self.log("svchub", "only one CPU detected; multiprocessing disabled")
|
||||
return False
|
||||
|
||||
try:
|
||||
@@ -367,6 +414,7 @@ class SvcHub(object):
|
||||
return True
|
||||
else:
|
||||
self.log("svchub", err)
|
||||
self.log("svchub", "cannot efficiently use multiple CPU cores")
|
||||
return False
|
||||
|
||||
def sd_notify(self):
|
||||
|
||||
@@ -21,6 +21,29 @@ class TcpSrv(object):
|
||||
|
||||
self.stopping = False
|
||||
|
||||
self.srv = []
|
||||
self.nsrv = 0
|
||||
ok = {}
|
||||
for ip in self.args.i:
|
||||
ok[ip] = []
|
||||
for port in self.args.p:
|
||||
self.nsrv += 1
|
||||
try:
|
||||
self._listen(ip, port)
|
||||
ok[ip].append(port)
|
||||
except Exception as ex:
|
||||
if self.args.ign_ebind or self.args.ign_ebind_all:
|
||||
m = "could not listen on {}:{}: {}"
|
||||
self.log("tcpsrv", m.format(ip, port, ex), c=3)
|
||||
else:
|
||||
raise
|
||||
|
||||
if not self.srv and not self.args.ign_ebind_all:
|
||||
raise Exception("could not listen on any of the given interfaces")
|
||||
|
||||
if self.nsrv != len(self.srv):
|
||||
self.log("tcpsrv", "")
|
||||
|
||||
ip = "127.0.0.1"
|
||||
eps = {ip: "local only"}
|
||||
nonlocals = [x for x in self.args.i if x != ip]
|
||||
@@ -34,6 +57,9 @@ class TcpSrv(object):
|
||||
m = "available @ http://{}:{}/ (\033[33m{}\033[0m)"
|
||||
for ip, desc in sorted(eps.items(), key=lambda x: x[1]):
|
||||
for port in sorted(self.args.p):
|
||||
if port not in ok.get(ip, ok.get("0.0.0.0", [])):
|
||||
continue
|
||||
|
||||
msgs.append(m.format(ip, port, desc))
|
||||
|
||||
if msgs:
|
||||
@@ -41,23 +67,6 @@ class TcpSrv(object):
|
||||
for m in msgs:
|
||||
self.log("tcpsrv", m)
|
||||
|
||||
self.srv = []
|
||||
self.nsrv = 0
|
||||
for ip in self.args.i:
|
||||
for port in self.args.p:
|
||||
self.nsrv += 1
|
||||
try:
|
||||
self._listen(ip, port)
|
||||
except Exception as ex:
|
||||
if self.args.ign_ebind or self.args.ign_ebind_all:
|
||||
m = "could not listen on {}:{}: {}"
|
||||
self.log("tcpsrv", m.format(ip, port, ex), c=1)
|
||||
else:
|
||||
raise
|
||||
|
||||
if not self.srv and not self.args.ign_ebind_all:
|
||||
raise Exception("could not listen on any of the given interfaces")
|
||||
|
||||
def _listen(self, ip, port):
|
||||
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
|
||||
@@ -4,28 +4,44 @@ from __future__ import print_function, unicode_literals
|
||||
import os
|
||||
|
||||
from .util import Cooldown
|
||||
from .th_srv import thumb_path, THUMBABLE, FMT_FF
|
||||
from .th_srv import thumb_path, THUMBABLE, FMT_FFV, FMT_FFA
|
||||
from .bos import bos
|
||||
|
||||
|
||||
class ThumbCli(object):
|
||||
def __init__(self, broker):
|
||||
self.broker = broker
|
||||
self.args = broker.args
|
||||
self.asrv = broker.asrv
|
||||
def __init__(self, hsrv):
|
||||
self.broker = hsrv.broker
|
||||
self.log_func = hsrv.log
|
||||
self.args = hsrv.args
|
||||
self.asrv = hsrv.asrv
|
||||
|
||||
# cache on both sides for less broker spam
|
||||
self.cooldown = Cooldown(self.args.th_poke)
|
||||
|
||||
def log(self, msg, c=0):
|
||||
self.log_func("thumbcli", msg, c)
|
||||
|
||||
def get(self, ptop, rem, mtime, fmt):
|
||||
ext = rem.rsplit(".")[-1].lower()
|
||||
if ext not in THUMBABLE:
|
||||
return None
|
||||
|
||||
is_vid = ext in FMT_FF
|
||||
is_vid = ext in FMT_FFV
|
||||
if is_vid and self.args.no_vthumb:
|
||||
return None
|
||||
|
||||
want_opus = fmt == "opus"
|
||||
is_au = ext in FMT_FFA
|
||||
if is_au:
|
||||
if want_opus:
|
||||
if self.args.no_acode:
|
||||
return None
|
||||
else:
|
||||
if self.args.no_athumb:
|
||||
return None
|
||||
elif want_opus:
|
||||
return None
|
||||
|
||||
if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg"]:
|
||||
return os.path.join(ptop, rem)
|
||||
|
||||
@@ -33,10 +49,14 @@ class ThumbCli(object):
|
||||
fmt = "w"
|
||||
|
||||
if fmt == "w":
|
||||
if self.args.th_no_webp or (is_vid and self.args.th_ff_jpg):
|
||||
if self.args.th_no_webp or ((is_vid or is_au) and self.args.th_ff_jpg):
|
||||
fmt = "j"
|
||||
|
||||
histpath = self.asrv.vfs.histtab[ptop]
|
||||
histpath = self.asrv.vfs.histtab.get(ptop)
|
||||
if not histpath:
|
||||
self.log("no histpath for [{}]".format(ptop))
|
||||
return None
|
||||
|
||||
tpath = thumb_path(histpath, rem, mtime, fmt)
|
||||
ret = None
|
||||
try:
|
||||
@@ -53,6 +73,11 @@ class ThumbCli(object):
|
||||
if self.cooldown.poke(tdir):
|
||||
self.broker.put(False, "thumbsrv.poke", tdir)
|
||||
|
||||
if want_opus:
|
||||
# audio files expire individually
|
||||
if self.cooldown.poke(tpath):
|
||||
self.broker.put(False, "thumbsrv.poke", tpath)
|
||||
|
||||
return ret
|
||||
|
||||
x = self.broker.put(True, "thumbsrv.get", ptop, rem, mtime, fmt)
|
||||
|
||||
@@ -10,7 +10,7 @@ import threading
|
||||
import subprocess as sp
|
||||
|
||||
from .__init__ import PY2, unicode
|
||||
from .util import fsenc, vsplit, runcmd, Queue, Cooldown, BytesIO, min_ex
|
||||
from .util import fsenc, vsplit, statdir, runcmd, Queue, Cooldown, BytesIO, min_ex
|
||||
from .bos import bos
|
||||
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
|
||||
|
||||
@@ -50,7 +50,8 @@ except:
|
||||
# https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html
|
||||
# ffmpeg -formats
|
||||
FMT_PIL = "bmp dib gif icns ico jpg jpeg jp2 jpx pcx png pbm pgm ppm pnm sgi tga tif tiff webp xbm dds xpm"
|
||||
FMT_FF = "av1 asf avi flv m4v mkv mjpeg mjpg mpg mpeg mpg2 mpeg2 h264 avc mts h265 hevc mov 3gp mp4 ts mpegts nut ogv ogm rm vob webm wmv"
|
||||
FMT_FFV = "av1 asf avi flv m4v mkv mjpeg mjpg mpg mpeg mpg2 mpeg2 h264 avc mts h265 hevc mov 3gp mp4 ts mpegts nut ogv ogm rm vob webm wmv"
|
||||
FMT_FFA = "aac m4a ogg opus flac alac mp3 mp2 ac3 dts wma ra wav aif aiff au alaw ulaw mulaw amr gsm ape tak tta wv"
|
||||
|
||||
if HAVE_HEIF:
|
||||
FMT_PIL += " heif heifs heic heics"
|
||||
@@ -58,7 +59,9 @@ if HAVE_HEIF:
|
||||
if HAVE_AVIF:
|
||||
FMT_PIL += " avif avifs"
|
||||
|
||||
FMT_PIL, FMT_FF = [{x: True for x in y.split(" ") if x} for y in [FMT_PIL, FMT_FF]]
|
||||
FMT_PIL, FMT_FFV, FMT_FFA = [
|
||||
{x: True for x in y.split(" ") if x} for y in [FMT_PIL, FMT_FFV, FMT_FFA]
|
||||
]
|
||||
|
||||
|
||||
THUMBABLE = {}
|
||||
@@ -67,7 +70,8 @@ if HAVE_PIL:
|
||||
THUMBABLE.update(FMT_PIL)
|
||||
|
||||
if HAVE_FFMPEG and HAVE_FFPROBE:
|
||||
THUMBABLE.update(FMT_FF)
|
||||
THUMBABLE.update(FMT_FFV)
|
||||
THUMBABLE.update(FMT_FFA)
|
||||
|
||||
|
||||
def thumb_path(histpath, rem, mtime, fmt):
|
||||
@@ -86,9 +90,13 @@ def thumb_path(histpath, rem, mtime, fmt):
|
||||
h = hashlib.sha512(fsenc(fn)).digest()
|
||||
fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
|
||||
|
||||
return "{}/th/{}/{}.{:x}.{}".format(
|
||||
histpath, rd, fn, int(mtime), "webp" if fmt == "w" else "jpg"
|
||||
)
|
||||
if fmt == "opus":
|
||||
cat = "ac"
|
||||
else:
|
||||
fmt = "webp" if fmt == "w" else "jpg"
|
||||
cat = "th"
|
||||
|
||||
return "{}/{}/{}/{}.{:x}.{}".format(histpath, cat, rd, fn, int(mtime), fmt)
|
||||
|
||||
|
||||
class ThumbSrv(object):
|
||||
@@ -105,9 +113,7 @@ class ThumbSrv(object):
|
||||
self.mutex = threading.Lock()
|
||||
self.busy = {}
|
||||
self.stopping = False
|
||||
self.nthr = self.args.th_mt
|
||||
if not self.nthr:
|
||||
self.nthr = os.cpu_count() if hasattr(os, "cpu_count") else 4
|
||||
self.nthr = max(1, self.args.th_mt)
|
||||
|
||||
self.q = Queue(self.nthr * 4)
|
||||
for n in range(self.nthr):
|
||||
@@ -117,7 +123,8 @@ class ThumbSrv(object):
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
if not self.args.no_vthumb and (not HAVE_FFMPEG or not HAVE_FFPROBE):
|
||||
want_ff = not self.args.no_vthumb or not self.args.no_athumb
|
||||
if want_ff and (not HAVE_FFMPEG or not HAVE_FFPROBE):
|
||||
missing = []
|
||||
if not HAVE_FFMPEG:
|
||||
missing.append("FFmpeg")
|
||||
@@ -125,12 +132,12 @@ class ThumbSrv(object):
|
||||
if not HAVE_FFPROBE:
|
||||
missing.append("FFprobe")
|
||||
|
||||
msg = "cannot create video thumbnails because some of the required programs are not available: "
|
||||
msg = "cannot create audio/video thumbnails because some of the required programs are not available: "
|
||||
msg += ", ".join(missing)
|
||||
self.log(msg, c=3)
|
||||
|
||||
if self.args.th_clean:
|
||||
t = threading.Thread(target=self.cleaner, name="thumb-cleaner")
|
||||
t = threading.Thread(target=self.cleaner, name="thumb.cln")
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
@@ -147,7 +154,11 @@ class ThumbSrv(object):
|
||||
return not self.nthr
|
||||
|
||||
def get(self, ptop, rem, mtime, fmt):
|
||||
histpath = self.asrv.vfs.histtab[ptop]
|
||||
histpath = self.asrv.vfs.histtab.get(ptop)
|
||||
if not histpath:
|
||||
self.log("no histpath for [{}]".format(ptop))
|
||||
return None
|
||||
|
||||
tpath = thumb_path(histpath, rem, mtime, fmt)
|
||||
abspath = os.path.join(ptop, rem)
|
||||
cond = threading.Condition(self.mutex)
|
||||
@@ -183,6 +194,7 @@ class ThumbSrv(object):
|
||||
try:
|
||||
st = bos.stat(tpath)
|
||||
if st.st_size:
|
||||
self.poke(tpath)
|
||||
return tpath
|
||||
except:
|
||||
pass
|
||||
@@ -201,8 +213,13 @@ class ThumbSrv(object):
|
||||
if not bos.path.exists(tpath):
|
||||
if ext in FMT_PIL:
|
||||
fun = self.conv_pil
|
||||
elif ext in FMT_FF:
|
||||
elif ext in FMT_FFV:
|
||||
fun = self.conv_ffmpeg
|
||||
elif ext in FMT_FFA:
|
||||
if tpath.endswith(".opus"):
|
||||
fun = self.conv_opus
|
||||
else:
|
||||
fun = self.conv_spec
|
||||
|
||||
if fun:
|
||||
try:
|
||||
@@ -328,8 +345,10 @@ class ThumbSrv(object):
|
||||
]
|
||||
|
||||
cmd += [fsenc(tpath)]
|
||||
# self.log((b" ".join(cmd)).decode("utf-8"))
|
||||
self._run_ff(cmd)
|
||||
|
||||
def _run_ff(self, cmd):
|
||||
# self.log((b" ".join(cmd)).decode("utf-8"))
|
||||
ret, sout, serr = runcmd(cmd)
|
||||
if ret != 0:
|
||||
m = "FFmpeg failed (probably a corrupt video file):\n"
|
||||
@@ -337,16 +356,81 @@ class ThumbSrv(object):
|
||||
self.log(m, c="1;30")
|
||||
raise sp.CalledProcessError(ret, (cmd[0], b"...", cmd[-1]))
|
||||
|
||||
def conv_spec(self, abspath, tpath):
|
||||
ret, _ = ffprobe(abspath)
|
||||
if "ac" not in ret:
|
||||
raise Exception("not audio")
|
||||
|
||||
fc = "[0:a:0]aresample=48000{},showspectrumpic=s=640x512,crop=780:544:70:50[o]"
|
||||
|
||||
if self.args.th_ff_swr:
|
||||
fco = ":filter_size=128:cutoff=0.877"
|
||||
else:
|
||||
fco = ":resampler=soxr"
|
||||
|
||||
fc = fc.format(fco)
|
||||
|
||||
# fmt: off
|
||||
cmd = [
|
||||
b"ffmpeg",
|
||||
b"-nostdin",
|
||||
b"-v", b"error",
|
||||
b"-hide_banner",
|
||||
b"-i", fsenc(abspath),
|
||||
b"-filter_complex", fc.encode("utf-8"),
|
||||
b"-map", b"[o]"
|
||||
]
|
||||
# fmt: on
|
||||
|
||||
if tpath.endswith(".jpg"):
|
||||
cmd += [
|
||||
b"-q:v",
|
||||
b"6", # default=??
|
||||
]
|
||||
else:
|
||||
cmd += [
|
||||
b"-q:v",
|
||||
b"50", # default=75
|
||||
b"-compression_level:v",
|
||||
b"6", # default=4, 0=fast, 6=max
|
||||
]
|
||||
|
||||
cmd += [fsenc(tpath)]
|
||||
self._run_ff(cmd)
|
||||
|
||||
def conv_opus(self, abspath, tpath):
|
||||
if self.args.no_acode:
|
||||
raise Exception("disabled in server config")
|
||||
|
||||
ret, _ = ffprobe(abspath)
|
||||
if "ac" not in ret:
|
||||
raise Exception("not audio")
|
||||
|
||||
# fmt: off
|
||||
cmd = [
|
||||
b"ffmpeg",
|
||||
b"-nostdin",
|
||||
b"-v", b"error",
|
||||
b"-hide_banner",
|
||||
b"-i", fsenc(abspath),
|
||||
b"-map", b"0:a:0",
|
||||
b"-c:a", b"libopus",
|
||||
b"-b:a", b"128k",
|
||||
fsenc(tpath)
|
||||
]
|
||||
# fmt: on
|
||||
|
||||
self._run_ff(cmd)
|
||||
|
||||
def poke(self, tdir):
|
||||
if not self.poke_cd.poke(tdir):
|
||||
return
|
||||
|
||||
ts = int(time.time())
|
||||
try:
|
||||
p1 = os.path.dirname(tdir)
|
||||
p2 = os.path.dirname(p1)
|
||||
for dp in [tdir, p1, p2]:
|
||||
bos.utime(dp, (ts, ts))
|
||||
for _ in range(4):
|
||||
bos.utime(tdir, (ts, ts))
|
||||
tdir = os.path.dirname(tdir)
|
||||
except:
|
||||
pass
|
||||
|
||||
@@ -366,25 +450,36 @@ class ThumbSrv(object):
|
||||
self.log("\033[Jcln ok; rm {} dirs".format(ndirs))
|
||||
|
||||
def clean(self, histpath):
|
||||
thumbpath = os.path.join(histpath, "th")
|
||||
ret = 0
|
||||
for cat in ["th", "ac"]:
|
||||
ret += self._clean(histpath, cat, None)
|
||||
|
||||
return ret
|
||||
|
||||
def _clean(self, histpath, cat, thumbpath):
|
||||
if not thumbpath:
|
||||
thumbpath = os.path.join(histpath, cat)
|
||||
|
||||
# self.log("cln {}".format(thumbpath))
|
||||
maxage = self.args.th_maxage
|
||||
exts = ["jpg", "webp"] if cat == "th" else ["opus"]
|
||||
maxage = getattr(self.args, cat + "_maxage")
|
||||
now = time.time()
|
||||
prev_b64 = None
|
||||
prev_fp = None
|
||||
try:
|
||||
ents = bos.listdir(thumbpath)
|
||||
ents = statdir(self.log, not self.args.no_scandir, False, thumbpath)
|
||||
ents = sorted(list(ents))
|
||||
except:
|
||||
return 0
|
||||
|
||||
ndirs = 0
|
||||
for f in sorted(ents):
|
||||
for f, inf in ents:
|
||||
fp = os.path.join(thumbpath, f)
|
||||
cmp = fp.lower().replace("\\", "/")
|
||||
|
||||
# "top" or b64 prefix/full (a folder)
|
||||
if len(f) <= 3 or len(f) == 24:
|
||||
age = now - bos.path.getmtime(fp)
|
||||
age = now - inf.st_mtime
|
||||
if age > maxage:
|
||||
with self.mutex:
|
||||
safe = True
|
||||
@@ -398,16 +493,15 @@ class ThumbSrv(object):
|
||||
self.log("rm -rf [{}]".format(fp))
|
||||
shutil.rmtree(fp, ignore_errors=True)
|
||||
else:
|
||||
ndirs += self.clean(fp)
|
||||
self._clean(histpath, cat, fp)
|
||||
|
||||
continue
|
||||
|
||||
# thumb file
|
||||
try:
|
||||
b64, ts, ext = f.split(".")
|
||||
if len(b64) != 24 or len(ts) != 8 or ext not in ["jpg", "webp"]:
|
||||
if len(b64) != 24 or len(ts) != 8 or ext not in exts:
|
||||
raise Exception()
|
||||
|
||||
ts = int(ts, 16)
|
||||
except:
|
||||
if f != "dir.txt":
|
||||
self.log("foreign file in thumbs dir: [{}]".format(fp), 1)
|
||||
@@ -418,6 +512,10 @@ class ThumbSrv(object):
|
||||
self.log("rm replaced [{}]".format(fp))
|
||||
bos.unlink(prev_fp)
|
||||
|
||||
if cat != "th" and inf.st_mtime + maxage < now:
|
||||
self.log("rm expired [{}]".format(fp))
|
||||
bos.unlink(fp)
|
||||
|
||||
prev_b64 = b64
|
||||
prev_fp = fp
|
||||
|
||||
|
||||
@@ -67,7 +67,11 @@ class U2idx(object):
|
||||
if cur:
|
||||
return cur
|
||||
|
||||
histpath = self.asrv.vfs.histtab[ptop]
|
||||
histpath = self.asrv.vfs.histtab.get(ptop)
|
||||
if not histpath:
|
||||
self.log("no histpath for [{}]".format(ptop))
|
||||
return None
|
||||
|
||||
db_path = os.path.join(histpath, "up2k.db")
|
||||
if not bos.path.exists(db_path):
|
||||
return None
|
||||
|
||||
@@ -63,10 +63,12 @@ class Up2k(object):
|
||||
|
||||
# state
|
||||
self.mutex = threading.Lock()
|
||||
self.rescan_cond = threading.Condition()
|
||||
self.hashq = Queue()
|
||||
self.tagq = Queue()
|
||||
self.n_hashq = 0
|
||||
self.n_tagq = 0
|
||||
self.gid = 0
|
||||
self.volstate = {}
|
||||
self.need_rescan = {}
|
||||
self.dupesched = {}
|
||||
@@ -113,6 +115,12 @@ class Up2k(object):
|
||||
t.daemon = True
|
||||
t.start()
|
||||
|
||||
def reload(self):
|
||||
self.gid += 1
|
||||
self.log("reload #{} initiated".format(self.gid))
|
||||
all_vols = self.asrv.vfs.all_vols
|
||||
self.rescan(all_vols, list(all_vols.keys()), True)
|
||||
|
||||
def deferred_init(self):
|
||||
all_vols = self.asrv.vfs.all_vols
|
||||
have_e2d = self.init_indexes(all_vols)
|
||||
@@ -131,9 +139,11 @@ class Up2k(object):
|
||||
thr.start()
|
||||
|
||||
if self.mtag:
|
||||
thr = threading.Thread(target=self._tagger, name="up2k-tagger")
|
||||
thr.daemon = True
|
||||
thr.start()
|
||||
for n in range(max(1, self.args.mtag_mt)):
|
||||
name = "tagger-{}".format(n)
|
||||
thr = threading.Thread(target=self._tagger, name=name)
|
||||
thr.daemon = True
|
||||
thr.start()
|
||||
|
||||
thr = threading.Thread(target=self._run_all_mtp, name="up2k-mtp-init")
|
||||
thr.daemon = True
|
||||
@@ -165,15 +175,15 @@ class Up2k(object):
|
||||
}
|
||||
return json.dumps(ret, indent=4)
|
||||
|
||||
def rescan(self, all_vols, scan_vols):
|
||||
if hasattr(self, "pp"):
|
||||
def rescan(self, all_vols, scan_vols, wait):
|
||||
if not wait and hasattr(self, "pp"):
|
||||
return "cannot initiate; scan is already in progress"
|
||||
|
||||
args = (all_vols, scan_vols)
|
||||
t = threading.Thread(
|
||||
target=self.init_indexes,
|
||||
args=args,
|
||||
name="up2k-rescan-{}".format(scan_vols[0]),
|
||||
name="up2k-rescan-{}".format(scan_vols[0] if scan_vols else "all"),
|
||||
)
|
||||
t.daemon = True
|
||||
t.start()
|
||||
@@ -181,9 +191,23 @@ class Up2k(object):
|
||||
|
||||
def _sched_rescan(self):
|
||||
volage = {}
|
||||
cooldown = 0
|
||||
timeout = time.time() + 3
|
||||
while True:
|
||||
time.sleep(self.args.re_int)
|
||||
timeout = max(timeout, cooldown)
|
||||
wait = max(0.1, timeout + 0.1 - time.time())
|
||||
with self.rescan_cond:
|
||||
self.rescan_cond.wait(wait)
|
||||
|
||||
now = time.time()
|
||||
if now < cooldown:
|
||||
continue
|
||||
|
||||
if hasattr(self, "pp"):
|
||||
cooldown = now + 5
|
||||
continue
|
||||
|
||||
timeout = now + 9001
|
||||
with self.mutex:
|
||||
for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
|
||||
maxage = vol.flags.get("scan")
|
||||
@@ -193,14 +217,18 @@ class Up2k(object):
|
||||
if vp not in volage:
|
||||
volage[vp] = now
|
||||
|
||||
if now - volage[vp] >= maxage:
|
||||
deadline = volage[vp] + maxage
|
||||
if deadline <= now:
|
||||
self.need_rescan[vp] = 1
|
||||
|
||||
timeout = min(timeout, deadline)
|
||||
|
||||
vols = list(sorted(self.need_rescan.keys()))
|
||||
self.need_rescan = {}
|
||||
|
||||
if vols:
|
||||
err = self.rescan(self.asrv.vfs.all_vols, vols)
|
||||
cooldown = now + 10
|
||||
err = self.rescan(self.asrv.vfs.all_vols, vols, False)
|
||||
if err:
|
||||
for v in vols:
|
||||
self.need_rescan[v] = True
|
||||
@@ -222,8 +250,11 @@ class Up2k(object):
|
||||
if not cur:
|
||||
continue
|
||||
|
||||
lifetime = int(lifetime)
|
||||
timeout = min(timeout, now + lifetime)
|
||||
|
||||
nrm = 0
|
||||
deadline = time.time() - int(lifetime)
|
||||
deadline = time.time() - lifetime
|
||||
q = "select rd, fn from up where at > 0 and at < ? limit 100"
|
||||
while True:
|
||||
with self.mutex:
|
||||
@@ -240,12 +271,22 @@ class Up2k(object):
|
||||
if vp:
|
||||
fvp = "{}/{}".format(vp, fvp)
|
||||
|
||||
self._handle_rm(LEELOO_DALLAS, None, fvp, True)
|
||||
self._handle_rm(LEELOO_DALLAS, None, fvp)
|
||||
nrm += 1
|
||||
|
||||
if nrm:
|
||||
self.log("{} files graduated in {}".format(nrm, vp))
|
||||
|
||||
if timeout < 10:
|
||||
continue
|
||||
|
||||
q = "select at from up where at > 0 order by at limit 1"
|
||||
with self.mutex:
|
||||
hits = cur.execute(q).fetchone()
|
||||
|
||||
if hits:
|
||||
timeout = min(timeout, now + lifetime - (now - hits[0]))
|
||||
|
||||
def _vis_job_progress(self, job):
|
||||
perc = 100 - (len(job["need"]) * 100.0 / len(job["hash"]))
|
||||
path = os.path.join(job["ptop"], job["prel"], job["name"])
|
||||
@@ -269,6 +310,16 @@ class Up2k(object):
|
||||
return True, ret
|
||||
|
||||
def init_indexes(self, all_vols, scan_vols=None):
|
||||
gid = self.gid
|
||||
while hasattr(self, "pp") and gid == self.gid:
|
||||
time.sleep(0.1)
|
||||
|
||||
if gid != self.gid:
|
||||
return
|
||||
|
||||
if gid:
|
||||
self.log("reload #{} running".format(self.gid))
|
||||
|
||||
self.pp = ProgressPrinter()
|
||||
vols = all_vols.values()
|
||||
t0 = time.time()
|
||||
@@ -399,7 +450,11 @@ class Up2k(object):
|
||||
return have_e2d
|
||||
|
||||
def register_vpath(self, ptop, flags):
|
||||
histpath = self.asrv.vfs.histtab[ptop]
|
||||
histpath = self.asrv.vfs.histtab.get(ptop)
|
||||
if not histpath:
|
||||
self.log("no histpath for [{}]".format(ptop))
|
||||
return None
|
||||
|
||||
db_path = os.path.join(histpath, "up2k.db")
|
||||
if ptop in self.registry:
|
||||
try:
|
||||
@@ -700,7 +755,7 @@ class Up2k(object):
|
||||
return n_add, n_rm, False
|
||||
|
||||
mpool = False
|
||||
if self.mtag.prefer_mt and not self.args.no_mtag_mt:
|
||||
if self.mtag.prefer_mt and self.args.mtag_mt > 1:
|
||||
mpool = self._start_mpool()
|
||||
|
||||
conn = sqlite3.connect(db_path, timeout=15)
|
||||
@@ -767,10 +822,11 @@ class Up2k(object):
|
||||
return ret
|
||||
|
||||
def _run_all_mtp(self):
|
||||
gid = self.gid
|
||||
t0 = time.time()
|
||||
for ptop, flags in self.flags.items():
|
||||
if "mtp" in flags:
|
||||
self._run_one_mtp(ptop)
|
||||
self._run_one_mtp(ptop, gid)
|
||||
|
||||
td = time.time() - t0
|
||||
msg = "mtp finished in {:.2f} sec ({})"
|
||||
@@ -781,7 +837,10 @@ class Up2k(object):
|
||||
if "OFFLINE" not in self.volstate[k]:
|
||||
self.volstate[k] = "online, idle"
|
||||
|
||||
def _run_one_mtp(self, ptop):
|
||||
def _run_one_mtp(self, ptop, gid):
|
||||
if gid != self.gid:
|
||||
return
|
||||
|
||||
entags = self.entags[ptop]
|
||||
|
||||
parsers = {}
|
||||
@@ -814,6 +873,9 @@ class Up2k(object):
|
||||
in_progress = {}
|
||||
while True:
|
||||
with self.mutex:
|
||||
if gid != self.gid:
|
||||
break
|
||||
|
||||
q = "select w from mt where k = 't:mtp' limit ?"
|
||||
warks = cur.execute(q, (batch_sz,)).fetchall()
|
||||
warks = [x[0] for x in warks]
|
||||
@@ -933,9 +995,7 @@ class Up2k(object):
|
||||
def _start_mpool(self):
|
||||
# mp.pool.ThreadPool and concurrent.futures.ThreadPoolExecutor
|
||||
# both do crazy runahead so lets reinvent another wheel
|
||||
nw = os.cpu_count() if hasattr(os, "cpu_count") else 4
|
||||
if self.args.no_mtag_mt:
|
||||
nw = 1
|
||||
nw = max(1, self.args.mtag_mt)
|
||||
|
||||
if self.pending_tags is None:
|
||||
self.log("using {}x {}".format(nw, self.mtag.backend))
|
||||
@@ -998,7 +1058,10 @@ class Up2k(object):
|
||||
except Exception as ex:
|
||||
msg = "failed to read tags from {}:\n{}"
|
||||
self.log(msg.format(abspath, ex), c=3)
|
||||
return
|
||||
return 0
|
||||
|
||||
if not bos.path.isfile(abspath):
|
||||
return 0
|
||||
|
||||
if entags:
|
||||
tags = {k: v for k, v in tags.items() if k in entags}
|
||||
@@ -1501,7 +1564,7 @@ class Up2k(object):
|
||||
ok = {}
|
||||
ng = {}
|
||||
for vp in vpaths:
|
||||
a, b, c = self._handle_rm(uname, ip, vp, False)
|
||||
a, b, c = self._handle_rm(uname, ip, vp)
|
||||
n_files += a
|
||||
for k in b:
|
||||
ok[k] = 1
|
||||
@@ -1514,7 +1577,7 @@ class Up2k(object):
|
||||
|
||||
return "deleted {} files (and {}/{} folders)".format(n_files, ok, ok + ng)
|
||||
|
||||
def _handle_rm(self, uname, ip, vpath, rm_topdir):
|
||||
def _handle_rm(self, uname, ip, vpath):
|
||||
try:
|
||||
permsets = [[True, False, False, True]]
|
||||
vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
|
||||
@@ -1585,7 +1648,7 @@ class Up2k(object):
|
||||
|
||||
bos.unlink(abspath)
|
||||
|
||||
rm = rmdirs(self.log_func, scandir, True, atop, 1 if rm_topdir else 0)
|
||||
rm = rmdirs(self.log_func, scandir, True, atop, 1)
|
||||
return n_files, rm[0], rm[1]
|
||||
|
||||
def handle_mv(self, uname, svp, dvp):
|
||||
@@ -1664,6 +1727,9 @@ class Up2k(object):
|
||||
# folders are too scary, schedule rescan of both vols
|
||||
self.need_rescan[svn.vpath] = 1
|
||||
self.need_rescan[dvn.vpath] = 1
|
||||
with self.rescan_cond:
|
||||
self.rescan_cond.notify_all()
|
||||
|
||||
return "k"
|
||||
|
||||
c1, w, ftime, fsize, ip, at = self._find_from_vpath(svn.realpath, srem)
|
||||
@@ -1926,7 +1992,8 @@ class Up2k(object):
|
||||
self.snap_prev = {}
|
||||
while True:
|
||||
time.sleep(self.snap_persist_interval)
|
||||
self.do_snapshot()
|
||||
if not hasattr(self, "pp"):
|
||||
self.do_snapshot()
|
||||
|
||||
def do_snapshot(self):
|
||||
with self.mutex:
|
||||
@@ -1935,7 +2002,10 @@ class Up2k(object):
|
||||
|
||||
def _snap_reg(self, ptop, reg):
|
||||
now = time.time()
|
||||
histpath = self.asrv.vfs.histtab[ptop]
|
||||
histpath = self.asrv.vfs.histtab.get(ptop)
|
||||
if not histpath:
|
||||
return
|
||||
|
||||
rm = [x for x in reg.values() if now - x["poke"] > self.snap_discard_interval]
|
||||
if rm:
|
||||
m = "dropping {} abandoned uploads in {}".format(len(rm), ptop)
|
||||
|
||||
@@ -100,10 +100,24 @@ IMPLICATIONS = [
|
||||
|
||||
|
||||
MIMES = {
|
||||
"md": "text/plain; charset=UTF-8",
|
||||
"md": "text/plain",
|
||||
"txt": "text/plain",
|
||||
"js": "text/javascript",
|
||||
"opus": "audio/ogg; codecs=opus",
|
||||
"webp": "image/webp",
|
||||
"mp3": "audio/mpeg",
|
||||
"m4a": "audio/mp4",
|
||||
"jpg": "image/jpeg",
|
||||
}
|
||||
for ln in """text css html csv
|
||||
application json wasm xml pdf rtf zip
|
||||
image webp jpeg png gif bmp
|
||||
audio aac ogg wav
|
||||
video webm mp4 mpeg
|
||||
font woff woff2 otf ttf
|
||||
""".splitlines():
|
||||
k, vs = ln.split(" ", 1)
|
||||
for v in vs.strip().split():
|
||||
MIMES[v] = "{}/{}".format(k, v)
|
||||
|
||||
|
||||
REKOBO_KEY = {
|
||||
@@ -445,7 +459,7 @@ def log_thrs(log, ival, name):
|
||||
tv = [x.name for x in threading.enumerate()]
|
||||
tv = [
|
||||
x.split("-")[0]
|
||||
if x.startswith("httpconn-") or x.startswith("thumb-")
|
||||
if x.split("-")[0] in ["httpconn", "thumb", "tagger"]
|
||||
else "listen"
|
||||
if "-listen-" in x
|
||||
else x
|
||||
@@ -1224,6 +1238,7 @@ def statdir(logger, scandir, lstat, top):
|
||||
def rmdirs(logger, scandir, lstat, top, depth):
|
||||
if not os.path.exists(fsenc(top)) or not os.path.isdir(fsenc(top)):
|
||||
top = os.path.dirname(top)
|
||||
depth -= 1
|
||||
|
||||
dirs = statdir(logger, scandir, lstat, top)
|
||||
dirs = [x[0] for x in dirs if stat.S_ISDIR(x[1].st_mode)]
|
||||
@@ -1279,11 +1294,18 @@ def guess_mime(url, fallback="application/octet-stream"):
|
||||
except:
|
||||
return fallback
|
||||
|
||||
ret = MIMES.get(ext) or mimetypes.guess_type(url)[0] or fallback
|
||||
ret = MIMES.get(ext)
|
||||
|
||||
if not ret:
|
||||
x = mimetypes.guess_type(url)
|
||||
ret = "application/{}".format(x[1]) if x[1] else x[0]
|
||||
|
||||
if not ret:
|
||||
ret = fallback
|
||||
|
||||
if ";" not in ret:
|
||||
if ret.startswith("text/") or ret.endswith("/javascript"):
|
||||
ret += "; charset=UTF-8"
|
||||
ret += "; charset=utf-8"
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
@@ -331,7 +331,7 @@ window.baguetteBox = (function () {
|
||||
|
||||
function tglsel() {
|
||||
var thumb = currentGallery[currentIndex].imageElement,
|
||||
name = vsplit(thumb.href)[1],
|
||||
name = vsplit(thumb.href)[1].split('?')[0],
|
||||
files = msel.getall();
|
||||
|
||||
for (var a = 0; a < files.length; a++)
|
||||
@@ -345,7 +345,7 @@ window.baguetteBox = (function () {
|
||||
function selbg() {
|
||||
var img = vidimg(),
|
||||
thumb = currentGallery[currentIndex].imageElement,
|
||||
name = vsplit(thumb.href)[1],
|
||||
name = vsplit(thumb.href)[1].split('?')[0],
|
||||
files = msel.getsel(),
|
||||
sel = false;
|
||||
|
||||
@@ -530,9 +530,7 @@ window.baguetteBox = (function () {
|
||||
if (options.bodyClass && document.body.classList)
|
||||
document.body.classList.remove(options.bodyClass);
|
||||
|
||||
var h = ebi('bbox-halp');
|
||||
if (h)
|
||||
h.parentNode.removeChild(h);
|
||||
qsr('#bbox-halp');
|
||||
|
||||
if (options.afterHide)
|
||||
options.afterHide();
|
||||
@@ -590,8 +588,7 @@ window.baguetteBox = (function () {
|
||||
|
||||
image.addEventListener(is_vid ? 'loadedmetadata' : 'load', function () {
|
||||
// Remove loader element
|
||||
var spinner = QS('#baguette-img-' + index + ' .bbox-spinner');
|
||||
figure.removeChild(spinner);
|
||||
qsr('#baguette-img-' + index + ' .bbox-spinner');
|
||||
if (!options.async && callback)
|
||||
callback();
|
||||
});
|
||||
|
||||
@@ -23,7 +23,7 @@ html, body {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
pre, code, tt {
|
||||
pre, code, tt, #doc, #doc>code {
|
||||
font-family: 'scp', monospace, monospace;
|
||||
}
|
||||
#path,
|
||||
@@ -31,9 +31,8 @@ pre, code, tt {
|
||||
font-size: 1em;
|
||||
}
|
||||
#path {
|
||||
color: #aca;
|
||||
color: #ccc;
|
||||
text-shadow: 1px 1px 0 #000;
|
||||
font-variant: small-caps;
|
||||
font-weight: normal;
|
||||
display: inline-block;
|
||||
padding: .35em .5em .2em .5em;
|
||||
@@ -205,6 +204,12 @@ a, #files tbody div a:last-child {
|
||||
#repl {
|
||||
padding: .33em;
|
||||
}
|
||||
#files a.doc {
|
||||
color: #666;
|
||||
}
|
||||
#files a.doc.bri {
|
||||
color: #f5a;
|
||||
}
|
||||
#files tbody a.play {
|
||||
color: #e70;
|
||||
padding: .2em;
|
||||
@@ -507,6 +512,11 @@ html.light #wfm a:not(.en) {
|
||||
box-shadow: 0 -.15em .2em #000 inset;
|
||||
padding-bottom: .3em;
|
||||
}
|
||||
#ops a svg {
|
||||
width: 1.75em;
|
||||
height: 1.75em;
|
||||
margin: -.5em -.3em;
|
||||
}
|
||||
#ops {
|
||||
margin: 1.7em 1.5em 0 1.5em;
|
||||
padding: .3em .6em;
|
||||
@@ -663,6 +673,17 @@ input.eq_gain {
|
||||
border-bottom: 1px solid #111;
|
||||
overflow: hidden;
|
||||
}
|
||||
#treepar {
|
||||
z-index: 1;
|
||||
position: fixed;
|
||||
left: -.75em;
|
||||
width: calc(var(--nav-sz) - 0.5em);
|
||||
border-bottom: 1px solid #444;
|
||||
overflow: hidden;
|
||||
}
|
||||
#treepar.off {
|
||||
display: none;
|
||||
}
|
||||
#tree, #treeh {
|
||||
border-radius: 0 .3em 0 0;
|
||||
}
|
||||
@@ -727,37 +748,46 @@ input.eq_gain {
|
||||
#tree li:last-child {
|
||||
border-bottom: none;
|
||||
}
|
||||
#treeul a.hl {
|
||||
#tree ul a.sel {
|
||||
background: #111;
|
||||
box-shadow: -.8em 0 0 #c37 inset;
|
||||
color: #fff;
|
||||
}
|
||||
#tree ul a.hl {
|
||||
color: #400;
|
||||
background: #fc4;
|
||||
text-shadow: none;
|
||||
}
|
||||
#treeul a {
|
||||
#tree ul a.par {
|
||||
color: #fff;
|
||||
}
|
||||
#tree ul a {
|
||||
border-radius: .3em;
|
||||
display: inline-block;
|
||||
}
|
||||
#treeul a+a {
|
||||
.ntree a+a {
|
||||
width: calc(100% - 2em);
|
||||
line-height: 1em;
|
||||
}
|
||||
#tree.nowrap #treeul li {
|
||||
#tree.nowrap li {
|
||||
min-height: 1.4em;
|
||||
white-space: nowrap;
|
||||
}
|
||||
#tree.nowrap #treeul a+a:hover {
|
||||
#tree.nowrap .ntree a+a:hover {
|
||||
background: rgba(16, 16, 16, 0.67);
|
||||
min-width: calc(var(--nav-sz) - 2em);
|
||||
width: auto;
|
||||
}
|
||||
html.light #tree.nowrap #treeul a+a:hover {
|
||||
html.light #tree.nowrap .ntree a+a:hover {
|
||||
background: rgba(255, 255, 255, 0.67);
|
||||
color: #000;
|
||||
}
|
||||
#treeul a+a:hover {
|
||||
#docul a:hover,
|
||||
#tree .ntree a+a:hover {
|
||||
background: #181818;
|
||||
color: #fff;
|
||||
}
|
||||
#treeul a:first-child {
|
||||
.ntree a:first-child {
|
||||
font-family: 'scp', monospace, monospace;
|
||||
font-size: 1.2em;
|
||||
line-height: 0;
|
||||
@@ -793,17 +823,17 @@ html.light #tree.nowrap #treeul a+a:hover {
|
||||
#files td:nth-child(2n) {
|
||||
color: #f5a;
|
||||
}
|
||||
#files tr.play td,
|
||||
#files tr.play div a {
|
||||
#files tbody tr.play td,
|
||||
#files tbody tr.play div a {
|
||||
background: #fc4;
|
||||
border-color: transparent;
|
||||
color: #400;
|
||||
text-shadow: none;
|
||||
}
|
||||
#files tr.play a {
|
||||
#files tbody tr.play a {
|
||||
color: inherit;
|
||||
}
|
||||
#files tr.play a:hover {
|
||||
#files tbody tr.play a:hover {
|
||||
color: #300;
|
||||
background: #fea;
|
||||
}
|
||||
@@ -840,11 +870,13 @@ html.light #tree.nowrap #treeul a+a:hover {
|
||||
border-bottom: 1px solid #555;
|
||||
}
|
||||
#thumbs,
|
||||
#au_fullpre,
|
||||
#au_osd_cv,
|
||||
#u2tdate {
|
||||
opacity: .3;
|
||||
}
|
||||
#griden.on+#thumbs,
|
||||
#au_preload.on+#au_fullpre,
|
||||
#au_os_ctl.on+#au_osd_cv,
|
||||
#u2turbo.on+#u2tdate {
|
||||
opacity: 1;
|
||||
@@ -852,30 +884,31 @@ html.light #tree.nowrap #treeul a+a:hover {
|
||||
#wraptree.on+#hovertree {
|
||||
display: none;
|
||||
}
|
||||
#ghead {
|
||||
.ghead {
|
||||
border-radius: .3em;
|
||||
padding: .2em .5em;
|
||||
line-height: 2.3em;
|
||||
margin-bottom: 1em;
|
||||
margin-bottom: 1.5em;
|
||||
}
|
||||
#ghead {
|
||||
position: sticky;
|
||||
top: -.3em;
|
||||
z-index: 1;
|
||||
}
|
||||
html.light #ghead {
|
||||
html.light .ghead {
|
||||
background: #f7f7f7;
|
||||
border-color: #ddd;
|
||||
}
|
||||
#ghead .btn {
|
||||
.ghead .btn {
|
||||
position: relative;
|
||||
top: 0;
|
||||
}
|
||||
#ghead>span {
|
||||
.ghead>span {
|
||||
white-space: pre;
|
||||
padding-left: .3em;
|
||||
}
|
||||
#ggrid {
|
||||
padding-top: .5em;
|
||||
margin: 0 -.5em;
|
||||
margin: -.2em -.5em;
|
||||
}
|
||||
#ggrid>a>span {
|
||||
overflow: hidden;
|
||||
@@ -984,6 +1017,53 @@ html.light #rui {
|
||||
padding: 0;
|
||||
font-size: 1.5em;
|
||||
}
|
||||
#doc {
|
||||
background: none;
|
||||
overflow: visible;
|
||||
margin: -1em 0 .5em 0;
|
||||
padding: 1em 0 1em 0;
|
||||
}
|
||||
#docul {
|
||||
position: relative;
|
||||
}
|
||||
#docul li.bn {
|
||||
text-align: center;
|
||||
padding: .5em;
|
||||
}
|
||||
#doc.prism {
|
||||
padding-left: 3em;
|
||||
}
|
||||
#doc>code {
|
||||
background: none;
|
||||
box-shadow: none;
|
||||
z-index: 1;
|
||||
}
|
||||
#doc.mdo {
|
||||
white-space: normal;
|
||||
font-family: sans-serif;
|
||||
}
|
||||
#doc.prism * {
|
||||
line-height: 1.5em;
|
||||
}
|
||||
#doc .line-highlight {
|
||||
border-radius: .3em;
|
||||
box-shadow: 0 0 .5em #333;
|
||||
background: linear-gradient(90deg, #111, #222);
|
||||
}
|
||||
html.light #doc .line-highlight {
|
||||
box-shadow: 0 0 .5em #ccc;
|
||||
background: linear-gradient(90deg, #fff, #eee);
|
||||
}
|
||||
#docul li {
|
||||
margin: 0;
|
||||
}
|
||||
#tree #docul a {
|
||||
display: block;
|
||||
}
|
||||
#seldoc.sel {
|
||||
color: #fff;
|
||||
background: #925;
|
||||
}
|
||||
#pvol,
|
||||
#barbuf,
|
||||
#barpos,
|
||||
@@ -1038,7 +1118,7 @@ html,
|
||||
.opbox,
|
||||
#path,
|
||||
#srch_form,
|
||||
#ghead {
|
||||
.ghead {
|
||||
background: #2b2b2b;
|
||||
border: 1px solid #333;
|
||||
box-shadow: 0 0 .3em #111;
|
||||
@@ -1047,7 +1127,8 @@ html,
|
||||
background: #282828;
|
||||
}
|
||||
#tree,
|
||||
#treeh {
|
||||
#treeh,
|
||||
#treepar {
|
||||
background: #2b2b2b;
|
||||
}
|
||||
#wtoggle,
|
||||
@@ -1105,18 +1186,24 @@ html.light #ops,
|
||||
html.light .opbox,
|
||||
html.light #path,
|
||||
html.light #srch_form,
|
||||
html.light #ghead,
|
||||
html.light .ghead,
|
||||
html.light #u2etas {
|
||||
background: #f7f7f7;
|
||||
box-shadow: 0 0 .3em #ccc;
|
||||
border-color: #f7f7f7;
|
||||
}
|
||||
html.light #wrap.doc {
|
||||
background: #f7f7f7;
|
||||
}
|
||||
html.light #ops a.act {
|
||||
box-shadow: 0 .2em .2em #ccc;
|
||||
background: #fff;
|
||||
border-color: #07a;
|
||||
padding-top: .4em;
|
||||
}
|
||||
html.light #ops svg circle {
|
||||
stroke: black;
|
||||
}
|
||||
html.light #op_cfg h3 {
|
||||
border-color: #ccc;
|
||||
}
|
||||
@@ -1146,21 +1233,25 @@ html.light #acc_info {
|
||||
html.light #srv_info span {
|
||||
color: #777;
|
||||
}
|
||||
html.light #treeul a+a {
|
||||
html.light #tree .ntree a+a {
|
||||
background: inherit;
|
||||
color: #06a;
|
||||
}
|
||||
html.light #treeul a.hl {
|
||||
html.light #tree ul a.hl {
|
||||
background: #07a;
|
||||
color: #fff;
|
||||
}
|
||||
html.light #treeul a.hl:hover {
|
||||
html.light #tree ul a.par {
|
||||
color: #000;
|
||||
}
|
||||
html.light #tree ul a.hl:hover {
|
||||
background: #059;
|
||||
}
|
||||
html.light #tree li {
|
||||
html.light #tree li,
|
||||
html.light #tree #treepar {
|
||||
border-color: #f7f7f7 #fff #ddd #fff;
|
||||
}
|
||||
html.light #treeul a:hover {
|
||||
html.light #tree ul a:hover {
|
||||
background: #fff;
|
||||
}
|
||||
html.light #tree ul {
|
||||
@@ -1208,6 +1299,12 @@ html.light #files tbody a.play {
|
||||
html.light #files tbody a.play.act {
|
||||
color: #90c;
|
||||
}
|
||||
html.light #files a.doc {
|
||||
color: #bbb;
|
||||
}
|
||||
html.light #files a.doc.bri {
|
||||
color: #d38;
|
||||
}
|
||||
html.light #files tr.play td {
|
||||
background: #fc5;
|
||||
border-color: #eb1;
|
||||
@@ -1275,6 +1372,7 @@ html.light #files td div span {
|
||||
color: #000;
|
||||
}
|
||||
html.light #path {
|
||||
color: #777;
|
||||
background: #f7f7f7;
|
||||
text-shadow: none;
|
||||
box-shadow: 0 0 .3em #bbb;
|
||||
@@ -1292,13 +1390,15 @@ html.light #path a:hover {
|
||||
html.light #files tbody div a {
|
||||
color: #d38;
|
||||
}
|
||||
html.light #docul a:hover,
|
||||
html.light #files a:hover,
|
||||
html.light #files tr.sel a:hover {
|
||||
color: #000;
|
||||
background: #fff;
|
||||
text-decoration: underline;
|
||||
}
|
||||
html.light #treeh {
|
||||
html.light #treeh,
|
||||
html.light #treepar {
|
||||
background: #f7f7f7;
|
||||
border-color: #ddd;
|
||||
}
|
||||
@@ -1306,14 +1406,14 @@ html.light #tree {
|
||||
border-color: #ddd;
|
||||
box-shadow: 0 0 1em #ddd;
|
||||
background: #f7f7f7;
|
||||
scrollbar-color: #a70 #ddd;
|
||||
scrollbar-color: #490 #ddd;
|
||||
}
|
||||
html.light #tree::-webkit-scrollbar-track,
|
||||
html.light #tree::-webkit-scrollbar {
|
||||
background: #ddd;
|
||||
}
|
||||
#tree::-webkit-scrollbar-thumb {
|
||||
background: #da0;
|
||||
html.light #tree::-webkit-scrollbar-thumb {
|
||||
background: #490;
|
||||
}
|
||||
|
||||
|
||||
@@ -1696,8 +1796,9 @@ html.light #u2err.err {
|
||||
box-shadow: .4em .4em 0 #111;
|
||||
}
|
||||
#u2conf.ww #u2btn {
|
||||
font-size: 1.3em;
|
||||
margin-right: .5em;
|
||||
line-height: 1em;
|
||||
padding: .5em 0;
|
||||
margin: -1.5em .5em -3em 0;
|
||||
}
|
||||
#op_up2k.srch #u2btn {
|
||||
background: linear-gradient(to bottom, #ca3 0%, #fd8 50%, #fc6 51%, #b92 100%);
|
||||
@@ -1705,8 +1806,9 @@ html.light #u2err.err {
|
||||
color: #333;
|
||||
}
|
||||
#u2conf #u2btn {
|
||||
margin: -2.4em 0;
|
||||
padding: .8em 0;
|
||||
padding: .6em 0;
|
||||
margin: -2em 0;
|
||||
font-size: 1.25em;
|
||||
width: 100%;
|
||||
max-width: 12em;
|
||||
display: inline-block;
|
||||
@@ -1771,6 +1873,7 @@ html.light #u2err.err {
|
||||
display: none;
|
||||
}
|
||||
#u2etas.o .o {
|
||||
display: inherit;
|
||||
display: unset;
|
||||
}
|
||||
#u2etaw {
|
||||
@@ -1830,6 +1933,11 @@ html.light #u2err.err {
|
||||
#u2cards span {
|
||||
color: #fff;
|
||||
}
|
||||
#u2cards > a:nth-child(4) > span {
|
||||
display: inline-block;
|
||||
text-align: center;
|
||||
min-width: 1.3em;
|
||||
}
|
||||
#u2conf {
|
||||
margin: 1em auto;
|
||||
width: 30em;
|
||||
|
||||
@@ -76,6 +76,12 @@
|
||||
|
||||
<div id="wrap">
|
||||
|
||||
{%- if doc %}
|
||||
<div id="bdoc"><pre>{{ doc|e }}</pre></div>
|
||||
{%- else %}
|
||||
<div id="bdoc"></div>
|
||||
{%- endif %}
|
||||
|
||||
<div id="pro" class="logue">{{ logues[0] }}</div>
|
||||
|
||||
<table id="files">
|
||||
@@ -130,10 +136,13 @@
|
||||
def_hcols = {{ def_hcols|tojson }},
|
||||
have_up2k_idx = {{ have_up2k_idx|tojson }},
|
||||
have_tags_idx = {{ have_tags_idx|tojson }},
|
||||
have_acode = {{ have_acode|tojson }},
|
||||
have_mv = {{ have_mv|tojson }},
|
||||
have_del = {{ have_del|tojson }},
|
||||
have_unpost = {{ have_unpost|tojson }},
|
||||
have_zip = {{ have_zip|tojson }},
|
||||
txt_ext = "{{ txt_ext }}",
|
||||
{% if no_prism %}no_prism = 1,{% endif %}
|
||||
readme = {{ readme|tojson }};
|
||||
|
||||
document.documentElement.setAttribute("class", localStorage.lightmode == 1 ? "light" : "dark");
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -164,10 +164,7 @@ function copydom(src, dst, lv) {
|
||||
|
||||
|
||||
function md_plug_err(ex, js) {
|
||||
var errbox = ebi('md_errbox');
|
||||
if (errbox)
|
||||
errbox.parentNode.removeChild(errbox);
|
||||
|
||||
qsr('#md_errbox');
|
||||
if (!ex)
|
||||
return;
|
||||
|
||||
@@ -183,7 +180,7 @@ function md_plug_err(ex, js) {
|
||||
o.textContent = lns[ln - 1];
|
||||
}
|
||||
}
|
||||
errbox = mknod('div');
|
||||
var errbox = mknod('div');
|
||||
errbox.setAttribute('id', 'md_errbox');
|
||||
errbox.style.cssText = 'position:absolute;top:0;left:0;padding:1em .5em;background:#2b2b2b;color:#fc5'
|
||||
errbox.textContent = msg;
|
||||
@@ -381,8 +378,7 @@ function convert_markdown(md_text, dest_dom) {
|
||||
|
||||
|
||||
function init_toc() {
|
||||
var loader = ebi('ml');
|
||||
loader.parentNode.removeChild(loader);
|
||||
qsr('#ml');
|
||||
|
||||
var anchors = []; // list of toc entries, complex objects
|
||||
var anchor = null; // current toc node
|
||||
|
||||
@@ -65,8 +65,7 @@ var mde = (function () {
|
||||
mde.codemirror.on("change", function () {
|
||||
md_changed(mde);
|
||||
});
|
||||
var loader = ebi('ml');
|
||||
loader.parentNode.removeChild(loader);
|
||||
qsr('#ml');
|
||||
return mde;
|
||||
})();
|
||||
|
||||
|
||||
@@ -25,10 +25,22 @@ a {
|
||||
color: #047;
|
||||
background: #fff;
|
||||
text-decoration: none;
|
||||
border-bottom: 1px solid #aaa;
|
||||
border-bottom: 1px solid #8ab;
|
||||
border-radius: .2em;
|
||||
padding: .2em .8em;
|
||||
}
|
||||
a+a {
|
||||
margin-left: .5em;
|
||||
}
|
||||
.refresh,
|
||||
.logout {
|
||||
float: right;
|
||||
margin: -.2em 0 0 .5em;
|
||||
}
|
||||
.logout {
|
||||
color: #c04;
|
||||
border-color: #c7a;
|
||||
}
|
||||
#repl {
|
||||
border: none;
|
||||
background: none;
|
||||
@@ -42,6 +54,7 @@ table {
|
||||
.vols th {
|
||||
padding: .3em .6em;
|
||||
text-align: left;
|
||||
white-space: nowrap;
|
||||
}
|
||||
.num {
|
||||
border-right: 1px solid #bbb;
|
||||
@@ -81,6 +94,10 @@ html.dark a {
|
||||
background: #057;
|
||||
border-color: #37a;
|
||||
}
|
||||
html.dark .logout {
|
||||
background: #804;
|
||||
border-color: #c28;
|
||||
}
|
||||
html.dark input {
|
||||
color: #fff;
|
||||
background: #626;
|
||||
|
||||
@@ -12,9 +12,12 @@
|
||||
|
||||
<body>
|
||||
<div id="wrap">
|
||||
<a href="/?h" class="refresh">refresh</a>
|
||||
|
||||
{%- if this.uname == '*' %}
|
||||
<p>howdy stranger <small>(you're not logged in)</small></p>
|
||||
{%- else %}
|
||||
<a href="/?pw=x" class="logout">logout</a>
|
||||
<p>welcome back, <strong>{{ this.uname }}</strong></p>
|
||||
{%- endif %}
|
||||
|
||||
@@ -46,7 +49,8 @@
|
||||
</table>
|
||||
</td></tr></table>
|
||||
<div class="btns">
|
||||
<a href="/?stack">dump stack</a>
|
||||
<a href="/?stack" tt="shows the state of all active threads">dump stack</a>
|
||||
<a href="/?reload=cfg" tt="reload config files (accounts/volumes/volflags),$Nand rescan all e2ds volumes">reload cfg</a>
|
||||
</div>
|
||||
{%- endif %}
|
||||
|
||||
@@ -85,5 +89,6 @@ if (localStorage.lightmode != 1)
|
||||
|
||||
</script>
|
||||
<script src="/.cpr/util.js?_={{ ts }}"></script>
|
||||
<script>tt.init();</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -79,7 +79,8 @@ html {
|
||||
}
|
||||
#toast.vis {
|
||||
right: 1.3em;
|
||||
transform: unset;
|
||||
transform: inherit;
|
||||
transform: initial;
|
||||
}
|
||||
#toast.vis #toastc {
|
||||
left: -2em;
|
||||
|
||||
@@ -30,7 +30,10 @@ catch (ex) {
|
||||
try {
|
||||
up2k = up2k_init(false);
|
||||
}
|
||||
catch (ex) { }
|
||||
catch (ex) {
|
||||
console.log('up2k init failed:', ex);
|
||||
toast.err(10, 'could not initialze up2k\n\n' + basenames(ex));
|
||||
}
|
||||
}
|
||||
treectl.onscroll();
|
||||
|
||||
@@ -210,14 +213,14 @@ function U2pvis(act, btns) {
|
||||
};
|
||||
|
||||
r.setat = function (nfile, blocktab) {
|
||||
r.tab[nfile].cb = blocktab;
|
||||
var fo = r.tab[nfile], bd = 0;
|
||||
|
||||
var bd = 0;
|
||||
for (var a = 0; a < blocktab.length; a++)
|
||||
bd += blocktab[a];
|
||||
|
||||
r.tab[nfile].bd = bd;
|
||||
r.tab[nfile].bd0 = bd;
|
||||
fo.bd = bd;
|
||||
fo.bd0 = bd;
|
||||
fo.cb = blocktab;
|
||||
};
|
||||
|
||||
r.perc = function (bd, bd0, sz, t0) {
|
||||
@@ -329,8 +332,7 @@ function U2pvis(act, btns) {
|
||||
r.head++;
|
||||
|
||||
if (!bz_act) {
|
||||
var tr = ebi("f" + nfile);
|
||||
tr.parentNode.removeChild(tr);
|
||||
qsr("#f" + nfile);
|
||||
}
|
||||
}
|
||||
else return;
|
||||
@@ -349,9 +351,7 @@ function U2pvis(act, btns) {
|
||||
last = parseInt(last.getAttribute('id').slice(1));
|
||||
|
||||
while (r.head - first > r.wsz) {
|
||||
var obj = ebi('f' + (first++));
|
||||
if (obj)
|
||||
obj.parentNode.removeChild(obj);
|
||||
qsr('#f' + (first++));
|
||||
}
|
||||
while (last - r.tail < r.wsz && last < r.tab.length - 2) {
|
||||
var obj = ebi('f' + (++last));
|
||||
@@ -477,6 +477,86 @@ function U2pvis(act, btns) {
|
||||
}
|
||||
|
||||
|
||||
function Donut(uc, st) {
|
||||
var r = this,
|
||||
el = null,
|
||||
psvg = null,
|
||||
o = 20 * 2 * Math.PI,
|
||||
optab = QS('#ops a[data-dest="up2k"]');
|
||||
|
||||
optab.setAttribute('ico', optab.textContent);
|
||||
|
||||
function svg(v) {
|
||||
var ico = v !== undefined,
|
||||
bg = ico ? '#333' : 'transparent',
|
||||
fg = '#fff',
|
||||
fsz = 52,
|
||||
rc = 32;
|
||||
|
||||
if (r.eta && (r.eta > 99 || (uc.fsearch ? st.time.hashing : st.time.uploading) < 20))
|
||||
r.eta = null;
|
||||
|
||||
if (r.eta) {
|
||||
if (r.eta < 10) {
|
||||
fg = '#fa0';
|
||||
fsz = 72;
|
||||
}
|
||||
rc = 8;
|
||||
}
|
||||
|
||||
return (
|
||||
'<svg version="1.1" viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg">\n' +
|
||||
(ico ? '<rect width="100%" height="100%" rx="' + rc + '" fill="#333" />\n' :
|
||||
'<circle stroke="white" stroke-width="6" r="3" cx="32" cy="32" />\n') +
|
||||
(r.eta ? (
|
||||
'<text x="55%" y="58%" dominant-baseline="middle" text-anchor="middle"' +
|
||||
' font-family="sans-serif" font-weight="bold" font-size="' + fsz + 'px"' +
|
||||
' fill="' + fg + '">' + r.eta + '</text></svg>'
|
||||
) : (
|
||||
'<circle class="donut" stroke="white" fill="' + bg +
|
||||
'" stroke-dashoffset="' + (ico ? v : o) + '" stroke-dasharray="' + o + ' ' + o +
|
||||
'" transform="rotate(270 32 32)" stroke-width="12" r="20" cx="32" cy="32" /></svg>'
|
||||
))
|
||||
);
|
||||
}
|
||||
|
||||
function pos() {
|
||||
return uc.fsearch ? Math.max(st.bytes.hashed, st.bytes.finished) : st.bytes.finished;
|
||||
}
|
||||
|
||||
r.on = function (ya) {
|
||||
r.fc = 99;
|
||||
r.eta = null;
|
||||
r.base = pos();
|
||||
optab.innerHTML = ya ? svg() : optab.getAttribute('ico');
|
||||
el = QS('#ops a .donut');
|
||||
if (!ya)
|
||||
favico.upd();
|
||||
};
|
||||
r.do = function () {
|
||||
if (!el)
|
||||
return;
|
||||
|
||||
var t = st.bytes.total - r.base,
|
||||
v = pos() - r.base,
|
||||
ofs = el.style.strokeDashoffset = o - o * v / t;
|
||||
|
||||
if (favico.txt) {
|
||||
if (++r.fc < 10 && r.eta && r.eta > 99)
|
||||
return;
|
||||
|
||||
var s = svg(ofs);
|
||||
if (s == psvg || (r.eta === null && r.fc < 10))
|
||||
return;
|
||||
|
||||
favico.upd('', s);
|
||||
psvg = s;
|
||||
r.fc = 0;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
function fsearch_explain(n) {
|
||||
if (n)
|
||||
return toast.inf(60, 'your access to this folder is Read-Only\n\n' + (acct == '*' ? 'you are currently not logged in' : 'you are currently logged in as "' + acct + '"'));
|
||||
@@ -623,7 +703,8 @@ function up2k_init(subtle) {
|
||||
});
|
||||
}
|
||||
|
||||
var pvis = new U2pvis("bz", '#u2cards');
|
||||
var pvis = new U2pvis("bz", '#u2cards'),
|
||||
donut = new Donut(uc, st);
|
||||
|
||||
var bobslice = null;
|
||||
if (window.File)
|
||||
@@ -942,6 +1023,7 @@ function up2k_init(subtle) {
|
||||
"lmod": lmod / 1000,
|
||||
"purl": fdir,
|
||||
"done": false,
|
||||
"bytes_uploaded": 0,
|
||||
"hash": []
|
||||
},
|
||||
key = entry.name + '\n' + entry.size;
|
||||
@@ -990,11 +1072,9 @@ function up2k_init(subtle) {
|
||||
for (var a = 0; a < st.files.length; a++) {
|
||||
var t = st.files[a];
|
||||
if (t.done && t.name) {
|
||||
var tr = ebi('f' + t.n);
|
||||
if (!tr)
|
||||
if (!qsr('#f' + t.n))
|
||||
continue;
|
||||
|
||||
tr.parentNode.removeChild(tr);
|
||||
t.name = undefined;
|
||||
}
|
||||
}
|
||||
@@ -1066,6 +1146,7 @@ function up2k_init(subtle) {
|
||||
continue;
|
||||
}
|
||||
|
||||
donut.eta = eta;
|
||||
if (etaskip)
|
||||
continue;
|
||||
|
||||
@@ -1175,6 +1256,8 @@ function up2k_init(subtle) {
|
||||
window[(is_busy ? "add" : "remove") +
|
||||
"EventListener"]("beforeunload", warn_uploader_busy);
|
||||
|
||||
donut.on(is_busy);
|
||||
|
||||
if (!is_busy) {
|
||||
var k = uc.fsearch ? 'searches' : 'uploads',
|
||||
ks = uc.fsearch ? 'Search' : 'Upload',
|
||||
@@ -1196,9 +1279,11 @@ function up2k_init(subtle) {
|
||||
toast.err(t, '{0} {1}'.format(ks, tng));
|
||||
|
||||
timer.rm(etafun);
|
||||
timer.rm(donut.do);
|
||||
op_minh = 0;
|
||||
}
|
||||
else {
|
||||
timer.add(donut.do);
|
||||
timer.add(etafun, false);
|
||||
ebi('u2etas').style.textAlign = 'left';
|
||||
}
|
||||
@@ -1346,7 +1431,6 @@ function up2k_init(subtle) {
|
||||
function exec_hash() {
|
||||
var t = st.todo.hash.shift();
|
||||
st.busy.hash.push(t);
|
||||
t.bytes_uploaded = 0;
|
||||
|
||||
var bpend = 0,
|
||||
nchunk = 0,
|
||||
|
||||
@@ -18,6 +18,15 @@ var ebi = document.getElementById.bind(document),
|
||||
mknod = document.createElement.bind(document);
|
||||
|
||||
|
||||
function qsr(sel) {
|
||||
var el = QS(sel);
|
||||
if (el)
|
||||
el.parentNode.removeChild(el);
|
||||
|
||||
return el;
|
||||
}
|
||||
|
||||
|
||||
// error handler for mobile devices
|
||||
function esc(txt) {
|
||||
return txt.replace(/[&"<>]/g, function (c) {
|
||||
@@ -71,7 +80,7 @@ try {
|
||||
catch (ex) {
|
||||
if (console.stdlog)
|
||||
console.log = console.stdlog;
|
||||
console.log(ex);
|
||||
console.log('console capture failed', ex);
|
||||
}
|
||||
var crashed = false, ignexd = {};
|
||||
function vis_exh(msg, url, lineNo, columnNo, error) {
|
||||
@@ -163,7 +172,6 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
|
||||
catch (e) {
|
||||
document.body.innerHTML = html.join('\n');
|
||||
}
|
||||
throw 'fatal_err';
|
||||
}
|
||||
function ignex(all) {
|
||||
var o = ebi('exbox');
|
||||
@@ -727,7 +735,7 @@ function hist_replace(url) {
|
||||
|
||||
function sethash(hv) {
|
||||
if (window.history && history.replaceState) {
|
||||
hist_replace(document.location.pathname + '#' + hv);
|
||||
hist_replace(document.location.pathname + document.location.search + '#' + hv);
|
||||
}
|
||||
else {
|
||||
document.location.hash = hv;
|
||||
@@ -1057,15 +1065,22 @@ var modal = (function () {
|
||||
}
|
||||
|
||||
function onkey(e) {
|
||||
if (e.code == 'Enter') {
|
||||
var a = ebi('modal-ng');
|
||||
if (a && document.activeElement == a)
|
||||
var k = e.code,
|
||||
eok = ebi('modal-ok'),
|
||||
eng = ebi('modal-ng'),
|
||||
ae = document.activeElement;
|
||||
|
||||
if (k == 'Space' && ae && (ae === eok || ae === eng))
|
||||
k = 'Enter';
|
||||
|
||||
if (k == 'Enter') {
|
||||
if (ae && ae == eng)
|
||||
return ng();
|
||||
|
||||
return ok();
|
||||
}
|
||||
|
||||
if (e.code == 'Escape')
|
||||
if (k == 'Escape')
|
||||
return ng();
|
||||
}
|
||||
|
||||
@@ -1142,6 +1157,7 @@ function repl_load() {
|
||||
if (!ret.length)
|
||||
ret = [
|
||||
'var v=Object.keys(localStorage); v.sort(); JSON.stringify(v)',
|
||||
"for (var a of QSA('#files a[id]')) a.setAttribute('download','')",
|
||||
'console.hist.slice(-10).join("\\n")'
|
||||
];
|
||||
|
||||
@@ -1213,28 +1229,31 @@ if (ebi('repl'))
|
||||
ebi('repl').onclick = repl;
|
||||
|
||||
|
||||
var svg_decl = '<?xml version="1.0" encoding="UTF-8"?>\n';
|
||||
|
||||
|
||||
var favico = (function () {
|
||||
var r = {};
|
||||
r.en = true;
|
||||
r.tag = null;
|
||||
|
||||
function gx(txt) {
|
||||
return (
|
||||
'<?xml version="1.0" encoding="UTF-8"?>\n' +
|
||||
'<svg version="1.1" viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg"><g>\n' +
|
||||
return (svg_decl +
|
||||
'<svg version="1.1" viewBox="0 0 64 64" xmlns="http://www.w3.org/2000/svg">\n' +
|
||||
(r.bg ? '<rect width="100%" height="100%" rx="16" fill="#' + r.bg + '" />\n' : '') +
|
||||
'<text x="50%" y="55%" dominant-baseline="middle" text-anchor="middle"' +
|
||||
' font-family="sans-serif" font-weight="bold" font-size="64px"' +
|
||||
' fill="#' + r.fg + '">' + txt + '</text></g></svg>'
|
||||
' fill="#' + r.fg + '">' + txt + '</text></svg>'
|
||||
);
|
||||
}
|
||||
|
||||
r.upd = function () {
|
||||
var i = QS('link[rel="icon"]'), b64;
|
||||
r.upd = function (txt, svg) {
|
||||
if (!r.txt)
|
||||
return;
|
||||
|
||||
var b64;
|
||||
try {
|
||||
b64 = btoa(gx(r.txt));
|
||||
b64 = btoa(svg ? svg_decl + svg : gx(r.txt));
|
||||
}
|
||||
catch (ex) {
|
||||
b64 = encodeURIComponent(r.txt).replace(/%([0-9A-F]{2})/g,
|
||||
@@ -1243,12 +1262,12 @@ var favico = (function () {
|
||||
b64 = btoa(gx(unescape(encodeURIComponent(r.txt))));
|
||||
}
|
||||
|
||||
if (!i) {
|
||||
i = mknod('link');
|
||||
i.rel = 'icon';
|
||||
document.head.appendChild(i);
|
||||
if (!r.tag) {
|
||||
r.tag = mknod('link');
|
||||
r.tag.rel = 'icon';
|
||||
document.head.appendChild(r.tag);
|
||||
}
|
||||
i.href = 'data:image/svg+xml;base64,' + b64;
|
||||
r.tag.href = 'data:image/svg+xml;base64,' + b64;
|
||||
};
|
||||
|
||||
r.init = function () {
|
||||
|
||||
@@ -29,7 +29,7 @@ point `--css-browser` to one of these by URL:
|
||||
* notes on using rclone as a fuse client/server
|
||||
|
||||
## [`example.conf`](example.conf)
|
||||
* example config file for `-c` (supports accounts, volumes, and volume-flags)
|
||||
* example config file for `-c`
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
# append some arguments to the commandline;
|
||||
# the first space in a line counts as a separator,
|
||||
# any additional spaces are part of the value
|
||||
-e2dsa
|
||||
-e2ts
|
||||
-i 127.0.0.1
|
||||
|
||||
# create users:
|
||||
# u username:password
|
||||
u ed:123
|
||||
@@ -24,7 +31,8 @@ rw ed
|
||||
r k
|
||||
rw ed
|
||||
|
||||
# this does the same thing:
|
||||
# this does the same thing,
|
||||
# and will cause an error on startup since /priv is already taken:
|
||||
./priv
|
||||
/priv
|
||||
r ed k
|
||||
|
||||
@@ -38,6 +38,13 @@ para() { for s in 1 2 3 4 5 6 7 8 12 16 24 32 48 64; do echo $s; for r in {1..4}
|
||||
avg() { awk 'function pr(ncsz) {if (nsmp>0) {printf "%3s %s\n", csz, sum/nsmp} csz=$1;sum=0;nsmp=0} {sub(/\r$/,"")} /^[0-9]+$/ {pr($1);next} / MiB/ {sub(/ MiB.*/,"");sub(/.* /,"");sum+=$1;nsmp++} END {pr(0)}' "$1"; }
|
||||
|
||||
|
||||
##
|
||||
## time between first and last upload
|
||||
|
||||
python3 -um copyparty -nw -v srv::rw -i 127.0.0.1 2>&1 | tee log
|
||||
cat log | awk '!/"purl"/{next} {s=$1;sub(/[^m]+m/,"");gsub(/:/," ");t=60*(60*$1+$2)+$3} !a{a=t;sa=s} {b=t;sb=s} END {print b-a,sa,sb}'
|
||||
|
||||
|
||||
##
|
||||
## bad filenames
|
||||
|
||||
|
||||
@@ -10,14 +10,41 @@ set -e
|
||||
# (and those are usually linux so bash is good inaff)
|
||||
# (but that said this even has macos support)
|
||||
#
|
||||
# bundle will look like:
|
||||
# -rwxr-xr-x 0 ed ed 183808 Nov 19 00:43 copyparty
|
||||
# -rw-r--r-- 0 ed ed 491318 Nov 19 00:40 copyparty-extras/copyparty-0.5.4.tar.gz
|
||||
# -rwxr-xr-x 0 ed ed 30254 Nov 17 23:58 copyparty-extras/copyparty-fuse.py
|
||||
# -rwxr-xr-x 0 ed ed 481403 Nov 19 00:40 copyparty-extras/sfx-full/copyparty-sfx.sh
|
||||
# -rwxr-xr-x 0 ed ed 506043 Nov 19 00:40 copyparty-extras/sfx-full/copyparty-sfx.py
|
||||
# -rwxr-xr-x 0 ed ed 167699 Nov 19 00:43 copyparty-extras/sfx-lite/copyparty-sfx.sh
|
||||
# -rwxr-xr-x 0 ed ed 183808 Nov 19 00:43 copyparty-extras/sfx-lite/copyparty-sfx.py
|
||||
# output summary (filesizes and contents):
|
||||
#
|
||||
# 535672 copyparty-extras/sfx-full/copyparty-sfx.sh
|
||||
# 550760 copyparty-extras/sfx-full/copyparty-sfx.py
|
||||
# `- original unmodified sfx from github
|
||||
#
|
||||
# 572923 copyparty-extras/sfx-full/copyparty-sfx-gz.py
|
||||
# `- unmodified but recompressed from bzip2 to gzip
|
||||
#
|
||||
# 341792 copyparty-extras/sfx-ent/copyparty-sfx.sh
|
||||
# 353975 copyparty-extras/sfx-ent/copyparty-sfx.py
|
||||
# 376934 copyparty-extras/sfx-ent/copyparty-sfx-gz.py
|
||||
# `- removed iOS ogg/opus/vorbis audio decoder,
|
||||
# removed the audio tray mouse cursor,
|
||||
# "enterprise edition"
|
||||
#
|
||||
# 259288 copyparty-extras/sfx-lite/copyparty-sfx.sh
|
||||
# 270004 copyparty-extras/sfx-lite/copyparty-sfx.py
|
||||
# 293159 copyparty-extras/sfx-lite/copyparty-sfx-gz.py
|
||||
# `- also removed the codemirror markdown editor
|
||||
# and the text-viewer syntax hilighting,
|
||||
# only essential features remaining
|
||||
#
|
||||
# 646297 copyparty-extras/copyparty-1.0.14.tar.gz
|
||||
# 4823 copyparty-extras/copyparty-repack.sh
|
||||
# `- source files from github
|
||||
#
|
||||
# 23663 copyparty-extras/up2k.py
|
||||
# `- standalone utility to upload or search for files
|
||||
#
|
||||
# 32280 copyparty-extras/copyparty-fuse.py
|
||||
# `- standalone to mount a URL as a local read-only filesystem
|
||||
#
|
||||
# 270004 copyparty
|
||||
# `- minimal binary, same as sfx-lite/copyparty-sfx.py
|
||||
|
||||
|
||||
command -v gnutar && tar() { gnutar "$@"; }
|
||||
@@ -54,6 +81,7 @@ cache="$od/.copyparty-repack.cache"
|
||||
# fallback to awk (sorry)
|
||||
awk -F\" '/"browser_download_url".*(\.tar\.gz|-sfx\.)/ {print$4}'
|
||||
) |
|
||||
grep -E '(sfx\.(sh|py)|tar\.gz)$' |
|
||||
tee /dev/stderr |
|
||||
tr -d '\r' | tr '\n' '\0' |
|
||||
xargs -0 bash -c 'dl_files "$@"' _
|
||||
@@ -64,7 +92,7 @@ cache="$od/.copyparty-repack.cache"
|
||||
|
||||
# move src into copyparty-extras/,
|
||||
# move sfx into copyparty-extras/sfx-full/
|
||||
mkdir -p copyparty-extras/sfx-{full,lite}
|
||||
mkdir -p copyparty-extras/sfx-{full,ent,lite}
|
||||
mv copyparty-sfx.* copyparty-extras/sfx-full/
|
||||
mv copyparty-*.tar.gz copyparty-extras/
|
||||
|
||||
@@ -112,14 +140,17 @@ repack() {
|
||||
}
|
||||
|
||||
repack sfx-full "re gz no-sh"
|
||||
repack sfx-lite "re no-ogv no-cm"
|
||||
repack sfx-lite "re no-ogv no-cm gz no-sh"
|
||||
repack sfx-ent "re no-dd no-ogv"
|
||||
repack sfx-ent "re no-dd no-ogv gz no-sh"
|
||||
repack sfx-lite "re no-dd no-ogv no-cm no-hl"
|
||||
repack sfx-lite "re no-dd no-ogv no-cm no-hl gz no-sh"
|
||||
|
||||
|
||||
# move fuse client into copyparty-extras/,
|
||||
# move fuse and up2k clients into copyparty-extras/,
|
||||
# copy lite-sfx.py to ./copyparty,
|
||||
# delete extracted source code
|
||||
( cd copyparty-extras/
|
||||
mv copyparty-*/bin/up2k.py .
|
||||
mv copyparty-*/bin/copyparty-fuse.py .
|
||||
cp -pv sfx-lite/copyparty-sfx.py ../copyparty
|
||||
rm -rf copyparty-{0..9}*.*.*{0..9}
|
||||
|
||||
@@ -45,6 +45,12 @@ RUN mkdir -p /z/dist/no-pk \
|
||||
&& tar -xf zopfli.tgz
|
||||
|
||||
|
||||
# todo
|
||||
# https://cdn.jsdelivr.net/gh/highlightjs/cdn-release/build/highlight.min.js
|
||||
# https://cdn.jsdelivr.net/gh/highlightjs/cdn-release/build/styles/default.min.css
|
||||
# https://prismjs.com/download.html#themes=prism-funky&languages=markup+css+clike+javascript+autohotkey+bash+basic+batch+c+csharp+cpp+cmake+diff+docker+go+ini+java+json+kotlin+latex+less+lisp+lua+makefile+objectivec+perl+powershell+python+r+jsx+ruby+rust+sass+scss+sql+swift+systemd+toml+typescript+vbnet+verilog+vhdl+yaml&plugins=line-highlight+line-numbers+autolinker
|
||||
|
||||
|
||||
# build fonttools (which needs zopfli)
|
||||
RUN tar -xf zopfli.tgz \
|
||||
&& cd zopfli* \
|
||||
|
||||
@@ -19,9 +19,11 @@ help() { exec cat <<'EOF'
|
||||
# `no-ogv` saves ~192k by removing the opus/vorbis audio codecs
|
||||
# (only affects apple devices; everything else has native support)
|
||||
#
|
||||
# `no-cm` saves ~92k by removing easymde/codemirror
|
||||
# `no-cm` saves ~82k by removing easymde/codemirror
|
||||
# (the fancy markdown editor)
|
||||
#
|
||||
# `no-hl` saves ~41k by removing syntax hilighting in the text viewer
|
||||
#
|
||||
# `no-fnt` saves ~9k by removing the source-code-pro font
|
||||
# (browsers will try to use 'Consolas' instead)
|
||||
#
|
||||
@@ -75,6 +77,7 @@ while [ ! -z "$1" ]; do
|
||||
gz) use_gz=1 ; ;;
|
||||
no-ogv) no_ogv=1 ; ;;
|
||||
no-fnt) no_fnt=1 ; ;;
|
||||
no-hl) no_hl=1 ; ;;
|
||||
no-dd) no_dd=1 ; ;;
|
||||
no-cm) no_cm=1 ; ;;
|
||||
no-sh) do_sh= ; ;;
|
||||
@@ -226,6 +229,9 @@ rm have
|
||||
tmv "$f"
|
||||
}
|
||||
|
||||
[ $no_hl ] &&
|
||||
rm -rf copyparty/web/deps/prism*
|
||||
|
||||
[ $no_fnt ] && {
|
||||
rm -f copyparty/web/deps/scp.woff2
|
||||
f=copyparty/web/ui.css
|
||||
|
||||
@@ -57,6 +57,9 @@ copyparty/web/deps/ogv-demuxer-ogg-wasm.js,
|
||||
copyparty/web/deps/ogv-demuxer-ogg-wasm.wasm,
|
||||
copyparty/web/deps/ogv-worker-audio.js,
|
||||
copyparty/web/deps/ogv.js,
|
||||
copyparty/web/deps/prism.js,
|
||||
copyparty/web/deps/prism.css,
|
||||
copyparty/web/deps/prismd.css,
|
||||
copyparty/web/deps/scp.woff2,
|
||||
copyparty/web/deps/sha512.ac.js,
|
||||
copyparty/web/deps/sha512.hw.js,
|
||||
|
||||
@@ -47,12 +47,13 @@ class Cfg(Namespace):
|
||||
mtp=[],
|
||||
mte="a",
|
||||
mth="",
|
||||
textfiles="",
|
||||
hist=None,
|
||||
no_idx=None,
|
||||
no_hash=None,
|
||||
js_browser=None,
|
||||
css_browser=None,
|
||||
**{k: False for k in "e2d e2ds e2dsa e2t e2ts e2tsr".split()}
|
||||
**{k: False for k in "e2d e2ds e2dsa e2t e2ts e2tsr no_acode".split()}
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -17,7 +17,8 @@ from copyparty import util
|
||||
|
||||
class Cfg(Namespace):
|
||||
def __init__(self, a=None, v=None, c=None):
|
||||
ex = {k: False for k in "nw e2d e2ds e2dsa e2t e2ts e2tsr".split()}
|
||||
ex = "nw e2d e2ds e2dsa e2t e2ts e2tsr no_logues no_readme no_acode"
|
||||
ex = {k: False for k in ex.split()}
|
||||
ex2 = {
|
||||
"mtp": [],
|
||||
"mte": "a",
|
||||
@@ -28,8 +29,6 @@ class Cfg(Namespace):
|
||||
"js_browser": None,
|
||||
"css_browser": None,
|
||||
"no_voldump": True,
|
||||
"no_logues": False,
|
||||
"no_readme": False,
|
||||
"re_maxage": 0,
|
||||
"rproxy": 0,
|
||||
}
|
||||
|
||||
@@ -113,6 +113,7 @@ class VSock(object):
|
||||
class VHttpSrv(object):
|
||||
def __init__(self):
|
||||
self.broker = NullBroker()
|
||||
self.prism = None
|
||||
|
||||
aliases = ["splash", "browser", "browser2", "msg", "md", "mde"]
|
||||
self.j2 = {x: J2_FILES for x in aliases}
|
||||
|
||||
Reference in New Issue
Block a user