Compare commits

...

82 Commits

Author SHA1 Message Date
ed
f00b939402 v1.13.3 2024-06-01 23:24:35 +00:00
ed
bef9617638 u2c.exe: explain that https is disabled 2024-06-01 22:26:47 +00:00
ed
692175f5b0 md-editor autoindent was duplicating hr markers
only keep characters `>+-*` if there's less than three of them,
and discard entire prefix if there's more

markdown spec only cares about exactly-one or three-or-more, but
let's keep pairs in case anyone use that as unconventional markup
2024-06-01 20:56:15 +00:00
ed
5ad65450c4 more intuitive df option/volflag, closes #83 2024-06-01 01:15:34 +00:00
ed
60c96f990a ux: hide video ui + floor seekbar text
* hide lightbox buttons when a video is playing

* move audio seekbar text to the bottom, so it
   hides less of the waveform and minute-markers
2024-06-01 00:35:44 +00:00
ed
07b2bf1104 better support for 700+ connections
when there was more than ~700 active connections,
* sendfile (non-https downloads) could fail
* mdns and ssdp could fail to reinitialize on network changes

...because `select` can't handle FDs higher than 512 on windows
(1024 on linux/macos), so prefer `poll` where possible (linux/macos)

but apple keeps breaking and unbreaking `poll` in macos,
so use `--no-poll` if necessary to force `select` instead
2024-05-31 23:31:32 +00:00
ed
ac1bc232a9 black 2024-05-31 08:57:33 +00:00
ed
5919607ad0 sanitize fs-paths in archive error summary
also gets rid of a dumb debug print i forgot
2024-05-30 23:55:37 +00:00
ed
07ea629ca5 keep most tags during audio transcode
metadata is no longer discarded when transcoding to opus or mp3;
this was a good idea back when the transcodes were only used by
the webplayer, but now that folders can be batch-downloaded with
on-the-fly transcoding, it makes sense to keep most of the tags

individual tags are discarded if its value exceeds 1023 letters

this should mainly affect the following:
* traktor beatmaps, size usually somewhere around 100 KiB
* non-standard cover-art embeddings, size around 250 KiB
* XMP (project data from adobe premiere), around 48 KiB
2024-05-30 23:46:56 +00:00
ed
b629d18df6 print helpful warning if unix env is inhospitable
thx kipu you're the best
2024-05-11 18:34:41 +00:00
ed
566cbb6507 update pkgs to 1.13.2 2024-05-10 15:04:33 +00:00
ed
400d700845 v1.13.2 2024-05-10 14:31:50 +00:00
ed
82ce6862ee option to use pngquant for smaller waveform PNGs 2024-05-10 13:06:02 +00:00
ed
38e4fdfe03 batch-convert audio waveforms with ?tar&p 2024-05-10 12:55:35 +00:00
ed
c04662798d play compressed s3xmodit chiptunes
adds support for playing gz, xz, and zip-compressed tracker files

using the de-facto naming convention for compressed modules;

* mod: mdz, mdgz, mdxz
* s3m: s3z, s3gz, s3xz
* xm: xmz, xmgz, xmxz
* it: itz, itgz, itxz
2024-05-10 12:45:17 +00:00
ed
19d156ff4e option to add custom UI translations 2024-05-09 23:09:45 +00:00
ed
87c60a1ec9 ensure OS signals hit main-thread as intended;
use sigmasks to block SIGINT, SIGTERM, SIGUSR1 from all other threads

also initiate shutdown by calling sighandler directly,
in case this misses anything and that is still unreliable
(discovered by `--exit=idx` being noop once in a blue moon)
2024-05-09 22:28:16 +00:00
ed
2c92dab165 fix small annoyances,
* mute exception on early shutdown
* sfx: give the utime thread a name
2024-05-09 14:17:53 +00:00
ed
5c1e23907d og: append full original filename as url suffix 2024-05-09 13:18:15 +00:00
ed
925c7f0a57 in gridview, assume .ts files are video, not typescript 2024-05-08 22:20:29 +00:00
ed
feed08deb2 doc: export --help to html and link it 2024-05-08 22:01:58 +00:00
ed
560d7b6672 option to add or change mimetype mappings 2024-05-08 21:12:14 +00:00
ed
565daee98b fix mimetype detection for uppercase file extensions 2024-05-08 20:08:11 +00:00
ed
e396c5c2b5 only drop index caches if necessary;
prevents having to rebuild covers due to unrelated changes
2024-05-08 20:03:51 +00:00
ed
1ee2cdd089 update pkgs to 1.13.1 2024-05-06 01:11:01 +00:00
ed
beacedab50 v1.13.1 2024-05-06 00:29:15 +00:00
ed
25139a4358 qr-code: better fallback ip when no default-route 2024-05-05 23:36:05 +00:00
ed
f8491970fd remember url-hash during login from 403 2024-05-05 22:37:41 +00:00
ed
da091aec85 "volume" is too overloaded, make it --au-vol instead 2024-05-05 21:27:07 +00:00
ed
e9eb5affcd and option to set default audio/video volume 2024-05-05 19:10:29 +00:00
ed
c1918bc36c expand tcolor early to avoid listing in volume props 2024-05-05 18:52:02 +00:00
ed
fdda567f50 ux: add "this folder is empty" banner 2024-05-05 18:44:36 +00:00
ed
603d0ed72b misc: messages, docs, ie4 / win311 support
* docker: improve config-not-found warning message
* readme: mention markdown variable expansion
* basic-browser: use zip=crc to support ie4 / win-3.11
2024-05-05 17:32:50 +00:00
ed
b15a4ef79f failed attempt at making images load on android-discord 2024-05-05 14:16:22 +00:00
ed
48a6789d36 use --og-title as fallback if template gives blank result 2024-05-05 11:25:52 +00:00
ed
36f2c446af opengraph stuff:
* template-based title formatting
* picture embeds are no longer ant-sized
* `--og-color` sets accent color; default #333
* `--og-s-title` forces default title, ignoring e2t
* add a music indicator to song titles because discord doesn't
2024-05-03 00:11:40 +00:00
ed
69517e4624 add general-purpose query-string parcelling;
currently only being used to workaround discord discarding
query strings in opengraph tags, but i'm sure there will be
plenty more wonderful usecases for this atrocity
2024-05-02 22:49:27 +00:00
ed
ea270ab9f2 add og / opengraph / discord embeds 2024-05-01 23:40:56 +00:00
ed
b6cf2d3089 --html-head can take a filepath and/or jinja2 2024-05-01 20:24:18 +00:00
ed
e8db3dd37f fix tests on windows 2024-04-25 22:25:38 +00:00
ed
27485a4cb1 add pyz builder 2024-04-24 23:45:01 +00:00
ed
253a414443 better ctrl-v upload ux 2024-04-24 23:49:34 +02:00
ed
f6e693f0f5 reevaluate support for sparse files periodically
if a given filesystem were to disappear (e.g. removable storage)
followed by another filesystem appearing at the same location,
this would not get noticed by up2k in a timely manner

fix this by discarding the mtab cache after `--mtab-age` seconds and
rebuild it from scratch, unless the previous values are definitely
correct (as indicated by identical output from `/bin/mount`)

probably reduces windows performance by an acceptable amount
2024-04-24 21:18:26 +00:00
ed
c5f7cfc355 upload files/images with CTRL-V (from explorer etc.) 2024-04-23 19:46:54 +00:00
ed
bc2c1e427a config-reset forgot the dots cookie 2024-04-23 19:39:43 +00:00
ed
95d9e693c6 d2d should disable search/unpost even if db exists 2024-04-22 18:55:13 +00:00
ed
70a3cf36d1 pipe: only flush FDs when necessary
should give higher performance on servers with slow storage
2024-04-21 23:53:04 +00:00
ed
aa45fccf11 update pkgs to 1.13.0 2024-04-20 22:48:16 +00:00
ed
42d00050c1 v1.13.0 2024-04-20 22:32:50 +00:00
ed
4bb0e6e75a pipe: windows: make it safe with aggressive flushing 2024-04-20 22:15:08 +00:00
ed
2f7f9de3f5 pipe: optimize (1 GiB/s @ ryzen5-4500U) 2024-04-20 20:13:31 +00:00
ed
f31ac90932 less confusing help-text for --re-dhash 2024-04-20 16:42:56 +00:00
ed
439cb7f85b u2c: add --ow (previously part of --dr) 2024-04-20 16:36:10 +00:00
ed
af193ee834 keep up2k state integrity on abort 2024-04-20 16:13:32 +00:00
ed
c06126cc9d pipe: add volflag to disable 2024-04-19 23:54:23 +00:00
ed
897ffbbbd0 pipe: add to docs 2024-04-19 00:02:28 +00:00
ed
8244d3b4fc pipe: add tapering to keep tcp alive 2024-04-18 23:10:37 +00:00
ed
74266af6d1 pipe: warn when trying to download a .PARTIAL
and fix file sorting indicators on firefox
2024-04-18 23:10:11 +00:00
ed
8c552f1ad1 windows: fix upload-abort 2024-04-18 23:08:05 +00:00
ed
bf5850785f add opt-out from storing uploader IPs 2024-04-18 17:16:00 +00:00
ed
feecb3e0b8 up2k: fix put-hasher dying + a harmless race
* hasher thread could die if a client would rapidly
   upload and delete files (so very unlikely)

* two unprotected calls to register_vpath which was
   almost-definitely safe because the volumes
   already existed in the registry
2024-04-18 16:43:38 +00:00
ed
08d8c82167 PoC: ongoing uploads can be downloaded in lockstep 2024-04-18 00:10:54 +00:00
ed
5239e7ac0c separate registry mutex for faster access
also fix a harmless toctou in handle_json where clients
could get stuck hanging for a bit longer than necessary
2024-04-18 00:07:56 +00:00
ed
9937c2e755 add ArozOS to comparison 2024-04-16 21:00:47 +00:00
ed
f1e947f37d rehost deps from a flaky server 2024-04-12 21:49:01 +00:00
ed
a70a49b9c9 update pkgs to 1.12.2 2024-04-12 21:25:21 +00:00
ed
fe700dcf1a v1.12.2 2024-04-12 21:10:02 +00:00
ed
c8e3ed3aae retry failed renames on windows
theoretical issue which nobody has ran into yet,
probably because nobody uses this on windows
2024-04-12 20:38:30 +00:00
ed
b8733653a3 fix audio transcoding with filekeys 2024-04-11 21:54:15 +00:00
ed
b772a4f8bb fix wordwrap of buttons on ios 2024-04-11 21:31:40 +00:00
ed
9e5253ef87 ie11: restore load-bearing thing 2024-04-11 20:53:15 +00:00
ed
7b94e4edf3 configurable basic-auth preference;
adds options `--bauth-last` to lower the preference for
taking the basic-auth password in case of conflict,
and `--no-bauth` to entirely disable basic-authentication

if a client is providing multiple passwords, for example when
"logged in" with one password (the `cppwd` cookie) and switching
to another account by also sending a PW header/url-param, then
the default evaluation order to determine which password to use is:

url-param `pw`, header `pw`, basic-auth header, cookie (cppwd/cppws)

so if a client supplies a basic-auth header, it will ignore the cookie
and use the basic-auth password instead, which usually makes sense

but this can become a problem if you have other webservers running
on the same domain which also support basic-authentication

--bauth-last is a good choice for cooperating with such services, as
--no-bauth currently breaks support for the android app...
2024-04-11 20:15:49 +00:00
ed
da26ec36ca add password placeholder on login page
was easy to assume you were supposed to put a username there
2024-04-11 19:31:02 +00:00
ed
443acf2f8b update nuitka notes 2024-04-10 22:04:43 +00:00
ed
6c90e3893d update pkgs to 1.12.1 2024-04-09 23:53:43 +00:00
ed
ea002ee71d v1.12.1 2024-04-09 23:34:31 +00:00
ed
ab18893cd2 update deps 2024-04-09 23:25:54 +00:00
ed
844d16b9e5 bbox: scrollwheel for prev/next pic
inspired by d385305f5e
2024-04-09 20:39:07 +00:00
ed
989cc613ef fix tree-rendering when history-popping into bbox
plus misc similar technically-incorrect addq usages;
most of these don't matter in practice since they'll
never get a url with a hash, but makes the intent clear

and make sure hashes never get passed around
like they're part of a dirkey, harmless as it is
2024-04-09 19:54:15 +00:00
ed
4f0cad5468 fix bbox destructor, closes #81 for real 2024-04-09 19:10:55 +00:00
ed
f89de6b35d preloading too aggressive, chill a bit 2024-04-09 18:44:23 +00:00
ed
e0bcb88ee7 update pkgs to 1.12.0 2024-04-06 20:56:52 +00:00
66 changed files with 2414 additions and 679 deletions

1
.gitignore vendored
View File

@@ -12,6 +12,7 @@ copyparty.egg-info/
/dist/
/py2/
/sfx*
/pyz/
/unt/
/log/

View File

@@ -10,13 +10,16 @@ turn almost any device into a file server with resumable uploads/downloads using
📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer)
🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)
## readme toc
* top
* [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
* [at home](#at-home) - make it accessible over the internet
* [on servers](#on-servers) - you may also want these, especially on servers
* [features](#features)
* [features](#features) - also see [comparison to similar software](./docs/versus.md)
* [testimonials](#testimonials) - small collection of user feedback
* [motivations](#motivations) - project goals / philosophy
* [notes](#notes) - general notes
@@ -37,12 +40,14 @@ turn almost any device into a file server with resumable uploads/downloads using
* [file-search](#file-search) - dropping files into the browser also lets you see if they exist on the server
* [unpost](#unpost) - undo/delete accidental uploads
* [self-destruct](#self-destruct) - uploads can be given a lifetime
* [race the beam](#race-the-beam) - download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm))
* [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
* [markdown viewer](#markdown-viewer) - and there are *two* editors
* [markdown vars](#markdown-vars) - dynamic docs with serverside variable expansion
* [other tricks](#other-tricks)
* [searching](#searching) - search by size, date, path/name, mp3-tags, ...
* [server config](#server-config) - using arguments or config files, or a mix of both
@@ -56,6 +61,7 @@ turn almost any device into a file server with resumable uploads/downloads using
* [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
* [smb server](#smb-server) - unsafe, slow, not recommended for wan
* [browser ux](#browser-ux) - tweaking the ui
* [opengraph](#opengraph) - discord and social-media embeds
* [file indexing](#file-indexing) - enables dedup and music search ++
* [exclude-patterns](#exclude-patterns) - to save some time
* [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
@@ -104,8 +110,9 @@ turn almost any device into a file server with resumable uploads/downloads using
* [dependencies](#dependencies) - mandatory deps
* [optional dependencies](#optional-dependencies) - install these to enable bonus features
* [optional gpl stuff](#optional-gpl-stuff)
* [sfx](#sfx) - the self-contained "binary"
* [sfx](#sfx) - the self-contained "binary" (recommended!)
* [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
* [zipapp](#zipapp) - another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)
* [install on android](#install-on-android)
* [reporting bugs](#reporting-bugs) - ideas for context to include, and where to submit them
* [devnotes](#devnotes) - for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
@@ -119,6 +126,7 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
* or install [on arch](#arch-package) [on NixOS](#nixos-module) [through nix](#nix-package)
* or if you are on android, [install copyparty in termux](#install-on-android)
* or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
* docker has all deps built-in, so skip this step:
@@ -126,7 +134,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
* **Alpine:** `apk add py3-pillow ffmpeg`
* **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
* **MacOS:** `port install py-Pillow ffmpeg`
* **MacOS** (alternative): `brew install pillow ffmpeg`
@@ -147,6 +155,17 @@ some recommended options:
* see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
### at home
make it accessible over the internet by starting a [cloudflare quicktunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/) like so:
first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) and then start the tunnel with `cloudflared tunnel --url http://127.0.0.1:3923`
as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you
since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
### on servers
you may also want these, especially on servers:
@@ -170,6 +189,8 @@ firewall-cmd --reload
## features
also see [comparison to similar software](./docs/versus.md)
* backend stuff
* ☑ IPv6
* ☑ [multiprocessing](#performance) (actual multithreading)
@@ -192,6 +213,7 @@ firewall-cmd --reload
* ☑ write-only folders
* ☑ [unpost](#unpost): undo/delete accidental uploads
* ☑ [self-destruct](#self-destruct) (specified server-side or client-side)
* ☑ [race the beam](#race-the-beam) (almost like peer-to-peer)
* ☑ symlink/discard duplicates (content-matching)
* download
* ☑ single files in browser
@@ -208,6 +230,7 @@ firewall-cmd --reload
* ☑ ...of videos using FFmpeg
* ☑ ...of audio (spectrograms) using FFmpeg
* ☑ cache eviction (max-age; maybe max-size eventually)
* ☑ multilingual UI (english, norwegian, [add your own](./docs/rice/#translations)))
* ☑ SPA (browse while uploading)
* server indexing
* ☑ [locate files by contents](#file-search)
@@ -216,9 +239,11 @@ firewall-cmd --reload
* client support
* ☑ [folder sync](#folder-sync)
* ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
* ☑ [opengraph](#opengraph) (discord embeds)
* markdown
* ☑ [viewer](#markdown-viewer)
* ☑ editor (sure why not)
* ☑ [variables](#markdown-vars)
PS: something missing? post any crazy ideas you've got as a [feature request](https://github.com/9001/copyparty/issues/new?assignees=9001&labels=enhancement&template=feature_request.md) or [discussion](https://github.com/9001/copyparty/discussions/new?category=ideas) 🤙
@@ -388,7 +413,7 @@ configuring accounts/volumes with arguments:
`-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
permissions:
* `r` (read): browse folder contents, download files, download as zip/tar
* `r` (read): browse folder contents, download files, download as zip/tar, see filekeys/dirkeys
* `w` (write): upload files, move files *into* this folder
* `m` (move): move files/folders *from* this folder
* `d` (delete): delete files/folders
@@ -590,15 +615,21 @@ you can also zip a selection of files or folders by clicking them in the browser
cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
* super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images (`&p` for audio waveforms)
* can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
## uploading
drag files/folders into the web-browser to upload (or use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy))
drag files/folders into the web-browser to upload
this initiates an upload using `up2k`; there are two uploaders available:
dragdrop is the recommended way, but you may also:
* select some files (not folders) in your file explorer and press CTRL-V inside the browser window
* use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy)
* upload using [curl or sharex](#client-examples)
when uploading files through dragdrop or CTRL-V, this initiates an upload using `up2k`; there are two browser-based uploaders available:
* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
* `[🚀] up2k`, the good / fancy one
@@ -617,7 +648,7 @@ up2k has several advantages:
> it is perfectly safe to restart / upgrade copyparty while someone is uploading to it!
> all known up2k clients will resume just fine 💪
see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
see [up2k](./docs/devnotes.md#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
@@ -683,6 +714,13 @@ clients can specify a shorter expiration time using the [up2k ui](#uploading) --
specifying a custom expiration time client-side will affect the timespan in which unposts are permitted, so keep an eye on the estimates in the up2k ui
### race the beam
download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)) -- it's almost like peer-to-peer
requires the file to be uploaded using up2k (which is the default drag-and-drop uploader), alternatively the command-line program
## file manager
cut/paste, rename, and delete files/folders (if you have permission)
@@ -818,6 +856,13 @@ other notes,
* the document preview has a max-width which is the same as an A4 paper when printed
### markdown vars
dynamic docs with serverside variable expansion to replace stuff like `{{self.ip}}` with the client's IP, or `{{srv.htime}}` with the current time on the server
see [./srv/expand/](./srv/expand/) for usage and examples
## other tricks
* you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
@@ -866,6 +911,8 @@ using arguments or config files, or a mix of both:
**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with `--help` to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in `--help-flags` can be used in volumes as volflags.
* if running in docker/podman, try this: `docker run --rm -it copyparty/ac --help`
* or see this (probably outdated): https://ocv.me/copyparty/helptext.html
* or if you prefer plaintext, https://ocv.me/copyparty/helptext.txt
## zeroconf
@@ -1042,6 +1089,23 @@ tweaking the ui
* to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Cirle,tags/.tn,tags/Artist,tags/Title,href`
* to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`
see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag, or to add your own translation
## opengraph
discord and social-media embeds
can be enabled globally with `--og` or per-volume with volflag `og`
note that this disables hotlinking because the opengraph spec demands it; to sneak past this intentional limitation, you can enable opengraph selectively by user-agent, for example `--og-ua '(Discord|Twitter|Slack)bot'` (or volflag `og_ua`)
you can also hotlink files regardless by appending `?raw` to the url
NOTE: because discord (and maybe others) strip query args such as `?raw` in opengraph tags, any links which require a filekey or dirkey will not work
if you want to entirely replace the copyparty response with your own jinja2 template, give the template filepath to `--og-tpl` or volflag `og_tpl` (all members of `HttpCli` are available through the `this` object)
## file indexing
@@ -1954,7 +2018,7 @@ these are standalone programs and will never be imported / evaluated by copypart
# sfx
the self-contained "binary" [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
the self-contained "binary" (recommended!) [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
@@ -1981,6 +2045,16 @@ meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/d
then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) every once ina while if you can afford the size
## zipapp
another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) has less features, requires python 3.7 or newer, worse compression, and more importantly is unable to benefit from more recent versions of jinja2 and such (which makes it less secure)... lots of drawbacks with this one really -- but it *may* just work if the regular sfx fails to start because the computer is messed up in certain funky ways, so it's worth a shot if all else fails
run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails
it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
# install on android
install [Termux](https://termux.com/) + its companion app `Termux:API` (see [ocv.me/termux](https://ocv.me/termux/)) and then copy-paste this into Termux (long-tap) all at once:

View File

@@ -231,7 +231,7 @@ install_vamp() {
cd "$td"
echo '#include <vamp-sdk/Plugin.h>' | g++ -x c++ -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
printf '\033[33mcould not find the vamp-sdk, building from source\033[0m\n'
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/2691/vamp-plugin-sdk-2.10.0.tar.gz)
(dl_files yolo https://ocv.me/mirror/vamp-plugin-sdk-2.10.0.tar.gz)
sha512sum -c <(
echo "153b7f2fa01b77c65ad393ca0689742d66421017fd5931d216caa0fcf6909355fff74706fabbc062a3a04588a619c9b515a1dae00f21a57afd97902a355c48ed -"
) <vamp-plugin-sdk-2.10.0.tar.gz
@@ -247,7 +247,7 @@ install_vamp() {
cd "$td"
have_beatroot || {
printf '\033[33mcould not find the vamp beatroot plugin, building from source\033[0m\n'
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/885/beatroot-vamp-v1.0.tar.gz)
(dl_files yolo https://ocv.me/mirror/beatroot-vamp-v1.0.tar.gz)
sha512sum -c <(
echo "1f444d1d58ccf565c0adfe99f1a1aa62789e19f5071e46857e2adfbc9d453037bc1c4dcb039b02c16240e9b97f444aaff3afb625c86aa2470233e711f55b6874 -"
) <beatroot-vamp-v1.0.tar.gz

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env python3
from __future__ import print_function, unicode_literals
S_VERSION = "1.15"
S_BUILD_DT = "2024-02-18"
S_VERSION = "1.18"
S_BUILD_DT = "2024-06-01"
"""
u2c.py: upload to copyparty
@@ -79,12 +79,21 @@ req_ses = requests.Session()
class Daemon(threading.Thread):
def __init__(self, target, name=None, a=None):
# type: (Any, Any, Any) -> None
threading.Thread.__init__(self, target=target, args=a or (), name=name)
def __init__(self, target, name = None, a = None):
threading.Thread.__init__(self, name=name)
self.a = a or ()
self.fun = target
self.daemon = True
self.start()
def run(self):
try:
signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM])
except:
pass
self.fun(*self.a)
class File(object):
"""an up2k upload task; represents a single file"""
@@ -563,7 +572,7 @@ def handshake(ar, file, search):
else:
if ar.touch:
req["umod"] = True
if ar.dr:
if ar.ow:
req["replace"] = True
headers = {"Content-Type": "text/plain"} # <=1.5.1 compat
@@ -1135,11 +1144,12 @@ source file/folder selection uses rsync syntax, meaning that:
ap.add_argument("url", type=unicode, help="server url, including destination folder")
ap.add_argument("files", type=unicode, nargs="+", help="files and/or folders to process")
ap.add_argument("-v", action="store_true", help="verbose")
ap.add_argument("-a", metavar="PASSWORD", help="password or $filepath")
ap.add_argument("-a", metavar="PASSWD", help="password or $filepath")
ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
ap.add_argument("-x", type=unicode, metavar="REGEX", default="", help="skip file if filesystem-abspath matches REGEX, example: '.*/\\.hist/.*'")
ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
ap.add_argument("--version", action="store_true", help="show version and exit")
ap = app.add_argument_group("compatibility")
@@ -1148,12 +1158,12 @@ source file/folder selection uses rsync syntax, meaning that:
ap = app.add_argument_group("folder sync")
ap.add_argument("--dl", action="store_true", help="delete local files after uploading")
ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally")
ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally (implies --ow)")
ap.add_argument("--drd", action="store_true", help="delete remote files during upload instead of afterwards; reduces peak disk space usage, but will reupload instead of detecting renames")
ap = app.add_argument_group("performance tweaks")
ap.add_argument("-j", type=int, metavar="THREADS", default=4, help="parallel connections")
ap.add_argument("-J", type=int, metavar="THREADS", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
ap.add_argument("-j", type=int, metavar="CONNS", default=2, help="parallel connections")
ap.add_argument("-J", type=int, metavar="CORES", default=hcores, help="num cpu-cores to use for hashing; set 0 or 1 for single-core hashing")
ap.add_argument("-nh", action="store_true", help="disable hashing while uploading")
ap.add_argument("-ns", action="store_true", help="no status panel (for slow consoles and macos)")
ap.add_argument("--cd", type=float, metavar="SEC", default=5, help="delay before reattempting a failed handshake/upload")
@@ -1161,7 +1171,7 @@ source file/folder selection uses rsync syntax, meaning that:
ap.add_argument("-z", action="store_true", help="ZOOMIN' (skip uploading files if they exist at the destination with the ~same last-modified timestamp, so same as yolo / turbo with date-chk but even faster)")
ap = app.add_argument_group("tls")
ap.add_argument("-te", metavar="PEM_FILE", help="certificate to expect/verify")
ap.add_argument("-te", metavar="PATH", help="path to ca.pem or cert.pem to expect/verify")
ap.add_argument("-td", action="store_true", help="disable certificate check")
# fmt: on
@@ -1178,6 +1188,9 @@ source file/folder selection uses rsync syntax, meaning that:
if ar.drd:
ar.dr = True
if ar.dr:
ar.ow = True
for k in "dl dr drd".split():
errs = []
if ar.safe and getattr(ar, k):
@@ -1195,6 +1208,14 @@ source file/folder selection uses rsync syntax, meaning that:
ar.url = ar.url.rstrip("/") + "/"
if "://" not in ar.url:
ar.url = "http://" + ar.url
if "https://" in ar.url.lower():
try:
import ssl, zipfile
except:
t = "ERROR: https is not available for some reason; please use http"
print("\n\n %s\n\n" % (t,))
raise
if ar.a and ar.a.startswith("$"):
fn = ar.a[1:]

View File

@@ -1,6 +1,6 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.11.2"
pkgver="1.13.2"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
arch=("any")
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" )
sha256sums=("0b37641746d698681691ea9e7070096404afc64a42d3d4e96cc4e036074fded9")
sha256sums=("39937526aab77f4d78f19d16ed5c0eae9ac28f658e772ae9b54fff5281161c77")
build() {
cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.11.2/copyparty-sfx.py",
"version": "1.11.2",
"hash": "sha256-3nIHLM4xJ9RQH3ExSGvBckHuS40IdzyREAtMfpJmfug="
"url": "https://github.com/9001/copyparty/releases/download/v1.13.2/copyparty-sfx.py",
"version": "1.13.2",
"hash": "sha256-5vvhbiZtgW/km9csq9iYezCaS6wAsLn1qVXDjl6gvwU="
}

98
copyparty/__main__.py Executable file → Normal file
View File

@@ -13,6 +13,7 @@ import base64
import locale
import os
import re
import select
import socket
import sys
import threading
@@ -43,11 +44,13 @@ from .util import (
DEF_MTH,
IMPLICATIONS,
JINJA_VER,
MIMES,
PARTFTPY_VER,
PY_DESC,
PYFTPD_VER,
SQLITE_VER,
UNPLICATIONS,
Daemon,
align_tab,
ansi_re,
dedent,
@@ -171,8 +174,10 @@ def init_E(EE: EnvParams) -> None:
(os.environ.get, "TMP"),
(unicode, "/tmp"),
]
errs = []
for chk in [os.listdir, os.mkdir]:
for pf, pa in paths:
for npath, (pf, pa) in enumerate(paths):
p = ""
try:
p = pf(pa)
# print(chk.__name__, p, pa)
@@ -185,9 +190,20 @@ def init_E(EE: EnvParams) -> None:
if not os.path.isdir(p):
os.mkdir(p)
if npath > 1:
t = "Using [%s] for config; filekeys/dirkeys will change on every restart. Consider setting XDG_CONFIG_HOME or giving the unix-user a ~/.config/"
errs.append(t % (p,))
elif errs:
errs.append("Using [%s] instead" % (p,))
if errs:
print("WARNING: " + ". ".join(errs))
return p # type: ignore
except:
pass
except Exception as ex:
if p and npath < 2:
t = "Unable to store config in [%s] due to %r"
errs.append(t % (p, ex))
raise Exception("could not find a writable path for config")
@@ -470,6 +486,16 @@ def disable_quickedit() -> None:
cmode(True, mode | 4)
def sfx_tpoke(top: str):
files = [os.path.join(dp, p) for dp, dd, df in os.walk(top) for p in dd + df]
while True:
t = int(time.time())
for f in [top] + files:
os.utime(f, (t, t))
time.sleep(78123)
def showlic() -> None:
p = os.path.join(E.mod, "res", "COPYING.txt")
if not os.path.exists(p):
@@ -820,7 +846,7 @@ def build_flags_desc():
v = v.replace("\n", "\n ")
ret += "\n \033[36m{}\033[35m {}".format(k, v)
return ret + "\033[0m"
return ret
# fmt: off
@@ -838,6 +864,8 @@ def add_general(ap, nc, srvname):
ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-form POSTs; see \033[33m--help-urlform\033[0m")
ap2.add_argument("--wintitle", metavar="TXT", type=u, default="cpp @ $pub", help="server terminal title, for example [\033[32m$ip-10.1.2.\033[0m] or [\033[32m$ip-]")
ap2.add_argument("--name", metavar="TXT", type=u, default=srvname, help="server name (displayed topleft in browser and in mDNS)")
ap2.add_argument("--mime", metavar="EXT=MIME", type=u, action="append", help="map file \033[33mEXT\033[0mension to \033[33mMIME\033[0mtype, for example [\033[32mjpg=image/jpeg\033[0m]")
ap2.add_argument("--mimes", action="store_true", help="list default mimetype mapping and exit")
ap2.add_argument("--license", action="store_true", help="show licenses and exit")
ap2.add_argument("--version", action="store_true", help="show versions and exit")
@@ -856,9 +884,11 @@ def add_qr(ap, tty):
def add_fs(ap):
ap2 = ap.add_argument_group("filesystem options")
rm_re_def = "5/0.1" if ANYWIN else "0/0"
rm_re_def = "15/0.1" if ANYWIN else "0/0"
ap2.add_argument("--rm-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be deleted because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=rm_retry)")
ap2.add_argument("--mv-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be renamed because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=mv_retry)")
ap2.add_argument("--iobuf", metavar="BYTES", type=int, default=256*1024, help="file I/O buffer-size; if your volumes are on a network drive, try increasing to \033[32m524288\033[0m or even \033[32m4194304\033[0m (and let me know if that improves your performance)")
ap2.add_argument("--mtab-age", metavar="SEC", type=int, default=60, help="rebuild mountpoint cache every \033[33mSEC\033[0m to keep track of sparse-files support; keep low on servers with removable media")
def add_upload(ap):
@@ -882,7 +912,7 @@ def add_upload(ap):
ap2.add_argument("--rand", action="store_true", help="force randomized filenames, \033[33m--nrand\033[0m chars long (volflag=rand)")
ap2.add_argument("--nrand", metavar="NUM", type=int, default=9, help="randomized filenames length (volflag=nrand)")
ap2.add_argument("--magic", action="store_true", help="enable filetype detection on nameless uploads (volflag=magic)")
ap2.add_argument("--df", metavar="GiB", type=float, default=0, help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests")
ap2.add_argument("--df", metavar="GiB", type=u, default="0", help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests; assumes gigabytes unless a unit suffix is given: [\033[32m256m\033[0m], [\033[32m4\033[0m], [\033[32m2T\033[0m] (volflag=df)")
ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16 for cross-atlantic (max=64)")
@@ -949,6 +979,8 @@ def add_auth(ap):
ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
ap2.add_argument("--no-bauth", action="store_true", help="disable basic-authentication support; do not accept passwords from the 'Authenticate' header at all. NOTE: This breaks support for the android app")
ap2.add_argument("--bauth-last", action="store_true", help="keeps basic-authentication enabled, but only as a last-resort; if a cookie is also provided then the cookie wins")
def add_zeroconf(ap):
@@ -1088,6 +1120,8 @@ def add_optouts(ap):
ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
def add_safety(ap):
@@ -1188,7 +1222,8 @@ def add_thumbnail(ap):
ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,m4a,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,tak,tta,ulaw,wav,wma,wv,xm,xpk", help="audio formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,mdgz,mdxz,mdz,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz", help="audio formats to decompress before passing to ffmpeg")
def add_transcoding(ap):
@@ -1213,7 +1248,7 @@ def add_db_general(ap, hcores):
ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
ap2.add_argument("--re-dhash", action="store_true", help="rebuild the cache if it gets out of sync (for example crash on startup during metadata scanning)")
ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice -- only useful for offloading uploads to a cloud service or something (volflag=noforget)")
ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see \033[33m--help-dbd\033[0m (volflag=dbd)")
ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
@@ -1251,19 +1286,38 @@ def add_txt(ap):
ap2.add_argument("--exp-lg", metavar="V,V,V", type=u, default=DEF_EXP, help="comma/space-separated list of placeholders to expand in prologue/epilogue files (volflag=exp_lg)")
def add_og(ap):
ap2 = ap.add_argument_group('og / open graph / discord-embed options')
ap2.add_argument("--og", action="store_true", help="disable hotlinking and return an html document instead; this is required by open-graph, but can also be useful on its own (volflag=og)")
ap2.add_argument("--og-ua", metavar="RE", type=u, default="", help="only disable hotlinking / engage OG behavior if the useragent matches regex \033[33mRE\033[0m (volflag=og_ua)")
ap2.add_argument("--og-tpl", metavar="PATH", type=u, default="", help="do not return the regular copyparty html, but instead load the jinja2 template at \033[33mPATH\033[0m (if path contains 'EXT' then EXT will be replaced with the requested file's extension) (volflag=og_tpl)")
ap2.add_argument("--og-no-head", action="store_true", help="do not automatically add OG entries into <head> (useful if you're doing this yourself in a template or such) (volflag=og_no_head)")
ap2.add_argument("--og-th", metavar="FMT", type=u, default="jf3", help="thumbnail format; j=jpeg, jf=jpeg-uncropped, jf3=jpeg-uncropped-large, w=webm, ... (volflag=og_th)")
ap2.add_argument("--og-title", metavar="TXT", type=u, default="", help="fallback title if there is nothing in the \033[33m-e2t\033[0m database (volflag=og_title)")
ap2.add_argument("--og-title-a", metavar="T", type=u, default="🎵 {{ artist }} - {{ title }}", help="audio title format; takes any metadata key (volflag=og_title_a)")
ap2.add_argument("--og-title-v", metavar="T", type=u, default="{{ title }}", help="video title format; takes any metadata key (volflag=og_title_v)")
ap2.add_argument("--og-title-i", metavar="T", type=u, default="{{ title }}", help="image title format; takes any metadata key (volflag=og_title_i)")
ap2.add_argument("--og-s-title", action="store_true", help="force default title; do not read from tags (volflag=og_s_title)")
ap2.add_argument("--og-desc", metavar="TXT", type=u, default="", help="description text; same for all files, disable with [\033[32m-\033[0m] (volflag=og_desc)")
ap2.add_argument("--og-site", metavar="TXT", type=u, default="", help="sitename; defaults to \033[33m--name\033[0m, disable with [\033[32m-\033[0m] (volflag=og_site)")
ap2.add_argument("--tcolor", metavar="RGB", type=u, default="333", help="accent color (3 or 6 hex digits); may also affect safari and/or android-chrome (volflag=tcolor)")
ap2.add_argument("--uqe", action="store_true", help="query-string parceling; translate a request for \033[33m/foo/.uqe/BASE64\033[0m into \033[33m/foo?TEXT\033[0m, or \033[33m/foo/?TEXT\033[0m if the first character in \033[33mTEXT\033[0m is a slash. Automatically enabled for \033[33m--og\033[0m")
def add_ui(ap, retry):
ap2 = ap.add_argument_group('ui options')
ap2.add_argument("--grid", action="store_true", help="show grid/thumbnails by default (volflag=grid)")
ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language; one of the following: \033[32meng nor\033[0m")
ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use (0..7)")
ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
ap2.add_argument("--au-vol", metavar="0-100", type=int, default=50, choices=range(0, 101), help="default audio/video volume percent")
ap2.add_argument("--sort", metavar="C,C,C", type=u, default="href", help="default sort order, comma-separated column IDs (see header tooltips), prefix with '-' for descending. Examples: \033[32mhref -href ext sz ts tags/Album tags/.tn\033[0m (volflag=sort)")
ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages; can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template (volflag=html_head)")
ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
@@ -1282,6 +1336,8 @@ def add_debug(ap):
ap2 = ap.add_argument_group('debug options')
ap2.add_argument("--vc", action="store_true", help="verbose config file parser (explain config)")
ap2.add_argument("--cgen", action="store_true", help="generate config file from current config (best-effort; probably buggy)")
if hasattr(select, "poll"):
ap2.add_argument("--no-poll", action="store_true", help="kernel-bug workaround: disable poll; use select instead (limits max num clients to ~700)")
ap2.add_argument("--no-sendfile", action="store_true", help="kernel-bug workaround: disable sendfile; do a safe and slow read-send-loop instead")
ap2.add_argument("--no-scandir", action="store_true", help="kernel-bug workaround: disable scandir; do a listdir + stat on each file instead")
ap2.add_argument("--no-fastboot", action="store_true", help="wait for initial filesystem indexing before accepting client requests")
@@ -1351,6 +1407,7 @@ def run_argparse(
add_hooks(ap)
add_stats(ap)
add_txt(ap)
add_og(ap)
add_ui(ap, retry)
add_admin(ap)
add_logging(ap)
@@ -1379,18 +1436,22 @@ def run_argparse(
k2 = "help_" + k.replace("-", "_")
if vars(ret)[k2]:
lprint("# %s help page (%s)" % (k, h))
lprint(t + "\033[0m")
lprint(t.rstrip() + "\033[0m")
sys.exit(0)
return ret
def main(argv: Optional[list[str]] = None) -> None:
def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
time.strptime("19970815", "%Y%m%d") # python#7980
if WINDOWS:
os.system("rem") # enables colors
init_E(E)
if rsrc: # pyz
E.mod = rsrc
if argv is None:
argv = sys.argv
@@ -1414,9 +1475,19 @@ def main(argv: Optional[list[str]] = None) -> None:
showlic()
sys.exit(0)
if "--mimes" in argv:
print("\n".join("%8s %s" % (k, v) for k, v in sorted(MIMES.items())))
sys.exit(0)
if EXE:
print("pybin: {}\n".format(pybin), end="")
for n, zs in enumerate(argv):
if zs.startswith("--sfx-tpoke="):
Daemon(sfx_tpoke, "sfx-tpoke", (zs.split("=", 1)[1],))
argv.pop(n)
break
ensure_locale()
ensure_webdeps()
@@ -1477,7 +1548,7 @@ def main(argv: Optional[list[str]] = None) -> None:
if hard > 0: # -1 == infinite
nc = min(nc, int(hard / 4))
except:
nc = 512
nc = 486 # mdns/ssdp restart headroom; select() maxfd is 512 on windows
retry = False
for fmtr in [RiceFormatter, RiceFormatter, Dodge11874, BasicDodge11874]:
@@ -1570,6 +1641,9 @@ def main(argv: Optional[list[str]] = None) -> None:
if not hasattr(os, "sendfile"):
al.no_sendfile = True
if not hasattr(select, "poll"):
al.no_poll = True
# signal.signal(signal.SIGINT, sighandler)
SvcHub(al, dal, argv, "".join(printed)).run()

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 12, 0)
CODENAME = "locksmith"
BUILD_DT = (2024, 4, 6)
VERSION = (1, 13, 3)
CODENAME = "race the beam"
BUILD_DT = (2024, 6, 1)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -17,7 +17,9 @@ from .bos import bos
from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
from .pwhash import PWHash
from .util import (
EXTS,
IMPLICATIONS,
MIMES,
SQLITE_VER,
UNPLICATIONS,
UTC,
@@ -1442,6 +1444,7 @@ class AuthSrv(object):
elif "" not in mount:
# there's volumes but no root; make root inaccessible
vfs = VFS(self.log_func, "", "", AXS(), {})
vfs.flags["tcolor"] = self.args.tcolor
vfs.flags["d2d"] = True
maxdepth = 0
@@ -1614,11 +1617,14 @@ class AuthSrv(object):
use = True
lim.nosub = True
zs = vol.flags.get("df") or (
"{}g".format(self.args.df) if self.args.df else ""
)
if zs:
zs = vol.flags.get("df") or self.args.df or ""
if zs not in ("", "0"):
use = True
try:
_ = float(zs)
zs = "%sg" % (zs)
except:
pass
lim.dfl = unhumanize(zs)
zs = vol.flags.get("sz")
@@ -1727,7 +1733,11 @@ class AuthSrv(object):
if self.args.e2d or "e2ds" in vol.flags:
vol.flags["e2d"] = True
for ga, vf in [["no_hash", "nohash"], ["no_idx", "noidx"]]:
for ga, vf in [
["no_hash", "nohash"],
["no_idx", "noidx"],
["og_ua", "og_ua"],
]:
if vf in vol.flags:
ptn = re.compile(vol.flags.pop(vf))
else:
@@ -1764,13 +1774,21 @@ class AuthSrv(object):
if k in vol.flags:
vol.flags[k] = float(vol.flags[k])
try:
zs1, zs2 = vol.flags["rm_retry"].split("/")
vol.flags["rm_re_t"] = float(zs1)
vol.flags["rm_re_r"] = float(zs2)
except:
t = 'volume "/%s" has invalid rm_retry [%s]'
raise Exception(t % (vol.vpath, vol.flags.get("rm_retry")))
for k in ("mv_re", "rm_re"):
try:
zs1, zs2 = vol.flags[k + "try"].split("/")
vol.flags[k + "_t"] = float(zs1)
vol.flags[k + "_r"] = float(zs2)
except:
t = 'volume "/%s" has invalid %stry [%s]'
raise Exception(t % (vol.vpath, k, vol.flags.get(k + "try")))
if vol.flags.get("og"):
self.args.uqe = True
zs = str(vol.flags.get("tcolor", "")).lstrip("#")
if len(zs) == 3: # fc5 => ffcc55
vol.flags["tcolor"] = "".join([x * 2 for x in zs])
for k1, k2 in IMPLICATIONS:
if k1 in vol.flags:
@@ -2052,6 +2070,13 @@ class AuthSrv(object):
self.re_pwd = re.compile(zs)
# to ensure it propagates into tcpsrv with mp on
if self.args.mime:
for zs in self.args.mime:
ext, mime = zs.split("=", 1)
MIMES[ext] = mime
EXTS.update({v: k for k, v in MIMES.items()})
def setup_pwhash(self, acct: dict[str, str]) -> None:
self.ah = PWHash(self.args)
if not self.ah.on:
@@ -2409,7 +2434,7 @@ def expand_config_file(
if not cnames:
t = "warning: tried to read config-files from folder '%s' but it does not contain any "
if names:
t += ".conf files; the following files were ignored: %s"
t += ".conf files; the following files/subfolders were ignored: %s"
t = t % (fp, ", ".join(names[:8]))
else:
t += "files at all"

View File

@@ -57,11 +57,8 @@ class BrokerMp(object):
def shutdown(self) -> None:
self.log("broker", "shutting down")
for n, proc in enumerate(self.procs):
thr = threading.Thread(
target=proc.q_pend.put((0, "shutdown", [])),
name="mp-shutdown-{}-{}".format(n, len(self.procs)),
)
thr.start()
name = "mp-shut-%d-%d" % (n, len(self.procs))
Daemon(proc.q_pend.put, name, ((0, "shutdown", []),))
with self.mutex:
procs = self.procs

View File

@@ -6,7 +6,8 @@ import os
import shutil
import time
from .util import Netdev, runcmd
from .__init__ import ANYWIN
from .util import Netdev, runcmd, wrename, wunlink
HAVE_CFSSL = True
@@ -14,6 +15,12 @@ if True: # pylint: disable=using-constant-test
from .util import RootLogger
if ANYWIN:
VF = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
else:
VF = {"mv_re_t": 0, "rm_re_t": 0}
def ensure_cert(log: "RootLogger", args) -> None:
"""
the default cert (and the entire TLS support) is only here to enable the
@@ -105,8 +112,12 @@ def _gen_ca(log: "RootLogger", args):
raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
bname = os.path.join(args.crt_dir, "ca")
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
try:
wunlink(log, bname + ".key", VF)
except:
pass
wrename(log, bname + "-key.pem", bname + ".key", VF)
wunlink(log, bname + ".csr", VF)
log("cert", "new ca OK", 2)
@@ -185,11 +196,11 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
bname = os.path.join(args.crt_dir, "srv")
try:
os.unlink(bname + ".key")
wunlink(log, bname + ".key", VF)
except:
pass
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
wrename(log, bname + "-key.pem", bname + ".key", VF)
wunlink(log, bname + ".csr", VF)
with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:
ca = f.read()

View File

@@ -16,6 +16,7 @@ def vf_bmap() -> dict[str, str]:
"no_dedup": "copydupes",
"no_dupe": "nodupe",
"no_forget": "noforget",
"no_pipe": "nopipe",
"no_robots": "norobots",
"no_thumb": "dthumb",
"no_vthumb": "dvthumb",
@@ -38,6 +39,9 @@ def vf_bmap() -> dict[str, str]:
"magic",
"no_sb_md",
"no_sb_lg",
"og",
"og_no_head",
"og_s_title",
"rand",
"xdev",
"xlink",
@@ -60,11 +64,23 @@ def vf_vmap() -> dict[str, str]:
}
for k in (
"dbd",
"html_head",
"lg_sbf",
"md_sbf",
"nrand",
"og_desc",
"og_site",
"og_th",
"og_title",
"og_title_a",
"og_title_v",
"og_title_i",
"og_tpl",
"og_ua",
"mv_retry",
"rm_retry",
"sort",
"tcolor",
"unlist",
"u2abort",
"u2ts",
@@ -79,7 +95,6 @@ def vf_cmap() -> dict[str, str]:
for k in (
"exp_lg",
"exp_md",
"html_head",
"mte",
"mth",
"mtp",
@@ -175,6 +190,7 @@ flagcats = {
"dvthumb": "disables video thumbnails",
"dathumb": "disables audio thumbnails (spectrograms)",
"dithumb": "disables image thumbnails",
"pngquant": "compress audio waveforms 33% better",
"thsize": "thumbnail res; WxH",
"crop": "center-cropping (y/n/fy/fn)",
"th3x": "3x resolution (y/n/fy/fn)",
@@ -199,7 +215,7 @@ flagcats = {
"grid": "show grid/thumbnails by default",
"sort": "default sort order",
"unlist": "dont list files matching REGEX",
"html_head=TXT": "includes TXT in the <head>",
"html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
"robots": "allows indexing by search engines (default)",
"norobots": "kindly asks search engines to leave",
"no_sb_md": "disable js sandbox for markdown files",
@@ -214,6 +230,7 @@ flagcats = {
"dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
"fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
"fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
"mv_retry": "ms-windows: timeout for renaming busy files",
"rm_retry": "ms-windows: timeout for deleting busy files",
"davauth": "ask webdav clients to login for all folders",
"davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",

View File

@@ -1,6 +1,7 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import re
import time
@@ -17,20 +18,26 @@ if True: # pylint: disable=using-constant-test
class Fstab(object):
def __init__(self, log: "RootLogger"):
def __init__(self, log: "RootLogger", args: argparse.Namespace):
self.log_func = log
self.warned = False
self.trusted = False
self.tab: Optional[VFS] = None
self.oldtab: Optional[VFS] = None
self.srctab = "a"
self.cache: dict[str, str] = {}
self.age = 0.0
self.maxage = args.mtab_age
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("fstab", msg, c)
def get(self, path: str) -> str:
if len(self.cache) > 9000:
self.age = time.time()
now = time.time()
if now - self.age > self.maxage or len(self.cache) > 9000:
self.age = now
self.oldtab = self.tab or self.oldtab
self.tab = None
self.cache = {}
@@ -75,7 +82,7 @@ class Fstab(object):
self.trusted = False
def build_tab(self) -> None:
self.log("building tab")
self.log("inspecting mtab for changes")
sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
if MACOS:
@@ -84,6 +91,7 @@ class Fstab(object):
ptn = re.compile(sptn)
so, _ = chkcmd(["mount"])
tab1: list[tuple[str, str]] = []
atab = []
for ln in so.split("\n"):
m = ptn.match(ln)
if not m:
@@ -91,6 +99,15 @@ class Fstab(object):
zs1, zs2 = m.groups()
tab1.append((str(zs1), str(zs2)))
atab.append(ln)
# keep empirically-correct values if mounttab unchanged
srctab = "\n".join(sorted(atab))
if srctab == self.srctab:
self.tab = self.oldtab
return
self.log("mtab has changed; reevaluating support for sparse files")
tab1.sort(key=lambda x: (len(x[0]), x[0]))
path1, fs1 = tab1[0]
@@ -99,6 +116,7 @@ class Fstab(object):
tab.add(fs, path.lstrip("/"))
self.tab = tab
self.srctab = srctab
def relabel(self, path: str, nval: str) -> None:
assert self.tab
@@ -133,7 +151,9 @@ class Fstab(object):
self.trusted = True
except:
# prisonparty or other restrictive environment
self.log("failed to build tab:\n{}".format(min_ex()), 3)
if not self.warned:
self.warned = True
self.log("failed to build tab:\n{}".format(min_ex()), 3)
self.build_fallback()
assert self.tab

View File

@@ -36,6 +36,7 @@ from .bos import bos
from .star import StreamTar
from .sutil import StreamArc, gfilter
from .szip import StreamZip
from .up2k import up2k_chunksize
from .util import unquote # type: ignore
from .util import (
APPLESAN_RE,
@@ -83,12 +84,16 @@ from .util import (
sanitize_vpath,
sendfile_kern,
sendfile_py,
ub64dec,
ub64enc,
ujoin,
undot,
unescape_cookie,
unquotep,
vjoin,
vol_san,
vsplit,
wrename,
wunlink,
yieldfile,
)
@@ -126,6 +131,7 @@ class HttpCli(object):
self.ico = conn.ico # mypy404
self.thumbcli = conn.thumbcli # mypy404
self.u2fh = conn.u2fh # mypy404
self.pipes = conn.pipes # mypy404
self.log_func = conn.log_func # mypy404
self.log_src = conn.log_src # mypy404
self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
@@ -214,6 +220,13 @@ class HttpCli(object):
ka["favico"] = self.args.favico
ka["s_name"] = self.args.bname
ka["s_doctitle"] = self.args.doctitle
ka["tcolor"] = self.vn.flags["tcolor"]
zso = self.vn.flags.get("html_head")
if zso:
ka["this"] = self
self._build_html_head(zso, ka)
ka["html_head"] = self.html_head
return tpl.render(**ka) # type: ignore
@@ -361,6 +374,21 @@ class HttpCli(object):
if "&" in self.req and "?" not in self.req:
self.hint = "did you mean '?' instead of '&'"
if self.args.uqe and "/.uqe/" in self.req:
try:
vpath, query = self.req.split("?")[0].split("/.uqe/")
query = query.split("/")[0] # discard trailing junk
# (usually a "filename" to trick discord into behaving)
query = ub64dec(query.encode("utf-8")).decode("utf-8", "replace")
if query.startswith("/"):
self.req = "%s/?%s" % (vpath, query[1:])
else:
self.req = "%s?%s" % (vpath, query)
except Exception as ex:
t = "bad uqe in request [%s]: %r" % (self.req, ex)
self.loud_reply(t, status=400)
return False
# split req into vpath + uparam
uparam = {}
if "?" not in self.req:
@@ -427,7 +455,8 @@ class HttpCli(object):
cookie_pw = ""
if len(uparam) > 10 or len(cookies) > 50:
raise Pebkac(400, "u wot m8")
self.loud_reply("u wot m8", status=400)
return False
self.uparam = uparam
self.cookies = cookies
@@ -443,7 +472,11 @@ class HttpCli(object):
zso = self.headers.get("authorization")
bauth = ""
if zso:
if (
zso
and not self.args.no_bauth
and (not cookie_pw or not self.args.bauth_last)
):
try:
zb = zso.split(" ")[1].encode("ascii")
zs = base64.b64decode(zb).decode("utf-8")
@@ -711,6 +744,30 @@ class HttpCli(object):
or ("; Trident/" in self.ua and not k304)
)
def _build_html_head(self, maybe_html: Any, kv: dict[str, Any]) -> bool:
html = str(maybe_html)
is_jinja = html[:2] in "%@%"
if is_jinja:
html = html.replace("%", "", 1)
if html.startswith("@"):
with open(html[1:], "rb") as f:
html = f.read().decode("utf-8")
if html.startswith("%"):
html = html[1:]
is_jinja = True
if is_jinja:
with self.conn.hsrv.mutex:
if html not in self.conn.hsrv.j2:
j2env = jinja2.Environment()
tpl = j2env.from_string(html)
self.conn.hsrv.j2[html] = tpl
html = self.conn.hsrv.j2[html].render(**kv)
self.html_head += html + "\n"
def send_headers(
self,
length: Optional[int],
@@ -1800,7 +1857,7 @@ class HttpCli(object):
f, fn = zfw["orz"]
path2 = os.path.join(fdir, fn2)
atomic_move(path, path2)
atomic_move(self.log, path, path2, vfs.flags)
fn = fn2
path = path2
@@ -1881,7 +1938,9 @@ class HttpCli(object):
self.reply(t.encode("utf-8"), 201, headers=h)
return True
def bakflip(self, f: typing.BinaryIO, ofs: int, sz: int, sha: str) -> None:
def bakflip(
self, f: typing.BinaryIO, ofs: int, sz: int, sha: str, flags: dict[str, Any]
) -> None:
if not self.args.bak_flips or self.args.nw:
return
@@ -1909,7 +1968,7 @@ class HttpCli(object):
if nrem:
self.log("bakflip truncated; {} remains".format(nrem), 1)
atomic_move(fp, fp + ".trunc")
atomic_move(self.log, fp, fp + ".trunc", flags)
else:
self.log("bakflip ok", 2)
@@ -2175,7 +2234,7 @@ class HttpCli(object):
if sha_b64 != chash:
try:
self.bakflip(f, cstart[0], post_sz, sha_b64)
self.bakflip(f, cstart[0], post_sz, sha_b64, vfs.flags)
except:
self.log("bakflip failed: " + min_ex())
@@ -2242,6 +2301,10 @@ class HttpCli(object):
def handle_login(self) -> bool:
assert self.parser
pwd = self.parser.require("cppwd", 64)
try:
uhash = self.parser.require("uhash", 256)
except:
uhash = ""
self.parser.drop()
self.out_headerlist = [
@@ -2254,6 +2317,11 @@ class HttpCli(object):
dst += self.ourlq()
uhash = uhash.lstrip("#")
if uhash not in ("", "-"):
dst += "&" if "?" in dst else "?"
dst += "_=1#" + html_escape(uhash, True, True)
msg = self.get_pwd_cookie(pwd)
html = self.j2s("msg", h1=msg, h2='<a href="' + dst + '">ack</a>', redir=dst)
self.reply(html.encode("utf-8"))
@@ -2527,7 +2595,7 @@ class HttpCli(object):
raise
if not nullwrite:
atomic_move(tabspath, abspath)
atomic_move(self.log, tabspath, abspath, vfs.flags)
tabspath = ""
@@ -2767,7 +2835,7 @@ class HttpCli(object):
hidedir(dp)
except:
pass
bos.rename(fp, os.path.join(mdir, ".hist", mfile2))
wrename(self.log, fp, os.path.join(mdir, ".hist", mfile2), vfs.flags)
assert self.parser.gen
p_field, _, p_data = next(self.parser.gen)
@@ -2922,17 +2990,42 @@ class HttpCli(object):
return txt
def tx_file(self, req_path: str) -> bool:
def tx_file(self, req_path: str, ptop: Optional[str] = None) -> bool:
status = 200
logmsg = "{:4} {} ".format("", self.req)
logtail = ""
if ptop is not None:
try:
dp, fn = os.path.split(req_path)
tnam = fn + ".PARTIAL"
if self.args.dotpart:
tnam = "." + tnam
ap_data = os.path.join(dp, tnam)
st_data = bos.stat(ap_data)
if not st_data.st_size:
raise Exception("partial is empty")
x = self.conn.hsrv.broker.ask("up2k.find_job_by_ap", ptop, req_path)
job = json.loads(x.get())
if not job:
raise Exception("not found in registry")
self.pipes.set(req_path, job)
except Exception as ex:
self.log("will not pipe [%s]; %s" % (ap_data, ex), 6)
ptop = None
#
# if request is for foo.js, check if we have foo.js.gz
file_ts = 0.0
editions: dict[str, tuple[str, int]] = {}
for ext in ("", ".gz"):
if ptop is not None:
sz = job["size"]
file_ts = job["lmod"]
editions["plain"] = (ap_data, sz)
break
try:
fs_path = req_path + ext
st = bos.stat(fs_path)
@@ -3089,13 +3182,25 @@ class HttpCli(object):
self.send_headers(length=upper - lower, status=status, mime=mime)
return True
if ptop is not None:
return self.tx_pipe(
ptop, req_path, ap_data, job, lower, upper, status, mime, logmsg
)
ret = True
with open_func(*open_args) as f:
self.send_headers(length=upper - lower, status=status, mime=mime)
sendfun = sendfile_kern if use_sendfile else sendfile_py
remains = sendfun(
self.log, lower, upper, f, self.s, self.args.s_wr_sz, self.args.s_wr_slp
self.log,
lower,
upper,
f,
self.s,
self.args.s_wr_sz,
self.args.s_wr_slp,
not self.args.no_poll,
)
if remains > 0:
@@ -3108,6 +3213,143 @@ class HttpCli(object):
return ret
def tx_pipe(
self,
ptop: str,
req_path: str,
ap_data: str,
job: dict[str, Any],
lower: int,
upper: int,
status: int,
mime: str,
logmsg: str,
) -> bool:
M = 1048576
self.send_headers(length=upper - lower, status=status, mime=mime)
wr_slp = self.args.s_wr_slp
wr_sz = self.args.s_wr_sz
file_size = job["size"]
chunk_size = up2k_chunksize(file_size)
num_need = -1
data_end = 0
remains = upper - lower
broken = False
spins = 0
tier = 0
tiers = ["uncapped", "reduced speed", "one byte per sec"]
while lower < upper and not broken:
with self.u2mutex:
job = self.pipes.get(req_path)
if not job:
x = self.conn.hsrv.broker.ask("up2k.find_job_by_ap", ptop, req_path)
job = json.loads(x.get())
if job:
self.pipes.set(req_path, job)
if not job:
t = "pipe: OK, upload has finished; yeeting remainder"
self.log(t, 2)
data_end = file_size
break
if num_need != len(job["need"]) and data_end - lower < 8 * M:
num_need = len(job["need"])
data_end = 0
for cid in job["hash"]:
if cid in job["need"]:
break
data_end += chunk_size
t = "pipe: can stream %.2f MiB; requested range is %.2f to %.2f"
self.log(t % (data_end / M, lower / M, upper / M), 6)
with self.u2mutex:
if data_end > self.u2fh.aps.get(ap_data, data_end):
try:
fhs = self.u2fh.cache[ap_data].all_fhs
for fh in fhs:
fh.flush()
self.u2fh.aps[ap_data] = data_end
self.log("pipe: flushed %d up2k-FDs" % (len(fhs),))
except Exception as ex:
self.log("pipe: u2fh flush failed: %r" % (ex,))
if lower >= data_end:
if data_end:
t = "pipe: uploader is too slow; aborting download at %.2f MiB"
self.log(t % (data_end / M))
raise Pebkac(416, "uploader is too slow")
raise Pebkac(416, "no data available yet; please retry in a bit")
slack = data_end - lower
if slack >= 8 * M:
ntier = 0
winsz = M
bufsz = wr_sz
slp = wr_slp
else:
winsz = max(40, int(M * (slack / (12 * M))))
base_rate = M if not wr_slp else wr_sz / wr_slp
if winsz > base_rate:
ntier = 0
bufsz = wr_sz
slp = wr_slp
elif winsz > 300:
ntier = 1
bufsz = winsz // 5
slp = 0.2
else:
ntier = 2
bufsz = winsz = slp = 1
if tier != ntier:
tier = ntier
self.log("moved to tier %d (%s)" % (tier, tiers[tier]))
try:
with open(ap_data, "rb", self.args.iobuf) as f:
f.seek(lower)
page = f.read(min(winsz, data_end - lower, upper - lower))
if not page:
raise Exception("got 0 bytes (EOF?)")
except Exception as ex:
self.log("pipe: read failed at %.2f MiB: %s" % (lower / M, ex), 3)
with self.u2mutex:
self.pipes.c.pop(req_path, None)
spins += 1
if spins > 3:
raise Pebkac(500, "file became unreadable")
time.sleep(2)
continue
spins = 0
pofs = 0
while pofs < len(page):
if slp:
time.sleep(slp)
try:
buf = page[pofs : pofs + bufsz]
self.s.sendall(buf)
zi = len(buf)
remains -= zi
lower += zi
pofs += zi
except:
broken = True
break
if lower < upper and not broken:
with open(req_path, "rb") as f:
remains = sendfile_py(self.log, lower, upper, f, self.s, wr_sz, wr_slp)
spd = self._spd((upper - lower) - remains)
if self.do_log:
self.log("{}, {}".format(logmsg, spd))
return not broken
def tx_zip(
self,
fmt: str,
@@ -3177,7 +3419,7 @@ class HttpCli(object):
# for f in fgen: print(repr({k: f[k] for k in ["vp", "ap"]}))
cfmt = ""
if self.thumbcli and not self.args.no_bacode:
for zs in ("opus", "mp3", "w", "j"):
for zs in ("opus", "mp3", "w", "j", "p"):
if zs in self.ouparam or uarg == zs:
cfmt = zs
@@ -3187,7 +3429,7 @@ class HttpCli(object):
bgen = packer(
self.log,
self.args,
self.asrv,
fgen,
utf8="utf" in uarg,
pre_crc="crc" in uarg,
@@ -3308,7 +3550,6 @@ class HttpCli(object):
targs = {
"r": self.args.SR if self.is_vproxied else "",
"ts": self.conn.hsrv.cachebuster(),
"html_head": self.html_head,
"edit": "edit" in self.uparam,
"title": html_escape(self.vpath, crlf=True),
"lastmod": int(ts_md * 1000),
@@ -3319,6 +3560,13 @@ class HttpCli(object):
"md": boundary,
"arg_base": arg_base,
}
zfv = self.vn.flags.get("html_head")
if zfv:
targs["this"] = self
self._build_html_head(zfv, targs)
targs["html_head"] = self.html_head
zs = template.render(**targs).encode("utf-8", "replace")
html = zs.split(boundary.encode("utf-8"))
if len(html) != 2:
@@ -3434,8 +3682,6 @@ class HttpCli(object):
self.reply(zb, mime="text/plain; charset=utf-8")
return True
self.html_head += self.vn.flags.get("html_head", "")
html = self.j2s(
"splash",
this=self,
@@ -3480,7 +3726,7 @@ class HttpCli(object):
return True
def set_cfg_reset(self) -> bool:
for k in ("k304", "js", "idxh", "cppwd", "cppws"):
for k in ("k304", "js", "idxh", "dots", "cppwd", "cppws"):
cookie = gencookie(k, "x", self.args.R, False)
self.out_headerlist.append(("Set-Cookie", cookie))
@@ -3691,7 +3937,7 @@ class HttpCli(object):
allvols = [x for x in allvols if "e2d" in x.flags]
for vol in allvols:
cur = idx.get_cur(vol.realpath)
cur = idx.get_cur(vol)
if not cur:
continue
@@ -3745,7 +3991,7 @@ class HttpCli(object):
if not allvols:
ret = [{"kinshi": 1}]
jtxt = '{"u":%s,"c":%s}' % (uret, json.dumps(ret, indent=0))
jtxt = '{"u":%s,"c":%s}' % (uret, json.dumps(ret, separators=(",\n", ": ")))
zi = len(uret.split('\n"pd":')) - 1
self.log("%s #%d+%d %.2fsec" % (lm, zi, len(ret), time.time() - t0))
self.reply(jtxt.encode("utf-8", "replace"), mime="application/json")
@@ -3906,7 +4152,17 @@ class HttpCli(object):
e2d = "e2d" in vn.flags
e2t = "e2t" in vn.flags
self.html_head += vn.flags.get("html_head", "")
add_og = "og" in vn.flags
if add_og:
if "th" in self.uparam or "raw" in self.uparam:
og_ua = add_og = False
elif self.args.og_ua:
og_ua = add_og = self.args.og_ua.search(self.ua)
else:
og_ua = False
add_og = True
og_fn = ""
if "b" in self.uparam:
self.out_headers["X-Robots-Tag"] = "noindex, nofollow"
@@ -3914,13 +4170,15 @@ class HttpCli(object):
is_dk = False
fk_pass = False
icur = None
if is_dir and (e2t or e2d):
if (e2t or e2d) and (is_dir or add_og):
idx = self.conn.get_u2idx()
if idx and hasattr(idx, "p_end"):
icur = idx.get_cur(dbv.realpath)
icur = idx.get_cur(dbv)
th_fmt = self.uparam.get("th")
if self.can_read or (self.can_get and vn.flags.get("dk")):
if self.can_read or (
self.can_get and (vn.flags.get("dk") or "fk" not in vn.flags)
):
if th_fmt is not None:
nothumb = "dthumb" in dbv.flags
if is_dir:
@@ -3967,7 +4225,7 @@ class HttpCli(object):
elif self.can_write and th_fmt is not None:
return self.tx_svg("upload\nonly")
elif self.can_get and self.avn:
if not self.can_read and self.can_get and self.avn:
axs = self.avn.axs
if self.uname not in axs.uhtml:
pass
@@ -4013,6 +4271,17 @@ class HttpCli(object):
self.log(t % (correct, got, self.req, abspath), 6)
return self.tx_404()
if add_og:
if og_ua or self.host not in self.headers.get("referer", ""):
self.vpath, og_fn = vsplit(self.vpath)
vpath = self.vpath
vn, rem = self.asrv.vfs.get(self.vpath, self.uname, False, False)
abspath = vn.dcanonical(rem)
dbv, vrem = vn.get_dbv(rem)
is_dir = stat.S_ISDIR(st.st_mode)
is_dk = True
vpnodes.pop()
if (
(abspath.endswith(".md") or self.can_delete)
and "nohtml" not in vn.flags
@@ -4024,7 +4293,10 @@ class HttpCli(object):
):
return self.tx_md(vn, abspath)
return self.tx_file(abspath)
if not add_og or not og_fn:
return self.tx_file(
abspath, None if st.st_size or "nopipe" in vn.flags else vn.realpath
)
elif is_dir and not self.can_read:
if self._use_dirkey(abspath):
@@ -4071,7 +4343,11 @@ class HttpCli(object):
is_ls = "ls" in self.uparam
is_js = self.args.force_js or self.cookies.get("js") == "y"
if not is_ls and (self.ua.startswith("curl/") or self.ua.startswith("fetch")):
if (
not is_ls
and not add_og
and (self.ua.startswith("curl/") or self.ua.startswith("fetch"))
):
self.uparam["ls"] = "v"
is_ls = True
@@ -4145,6 +4421,7 @@ class HttpCli(object):
"dsort": vf["sort"],
"dcrop": vf["crop"],
"dth3x": vf["th3x"],
"dvol": self.args.au_vol,
"themes": self.args.themes,
"turbolvl": self.args.turbo,
"u2j": self.args.u2j,
@@ -4196,7 +4473,7 @@ class HttpCli(object):
for k in ["zip", "tar"]:
v = self.uparam.get(k)
if v is not None:
if v is not None and (not add_og or not og_fn):
return self.tx_zip(k, v, self.vpath, vn, rem, [])
fsroot, vfs_ls, vfs_virt = vn.ls(
@@ -4210,6 +4487,10 @@ class HttpCli(object):
ls_names = [x[0] for x in vfs_ls]
ls_names.extend(list(vfs_virt.keys()))
if add_og and og_fn and not self.can_read:
ls_names = [og_fn]
is_js = True
# check for old versions of files,
# [num-backups, most-recent, hist-path]
hist: dict[str, tuple[int, float, str]] = {}
@@ -4271,12 +4552,14 @@ class HttpCli(object):
margin = "DIR"
elif add_dk:
zs = absreal(fspath)
margin = '<a href="%s?k=%s&zip" rel="nofollow">zip</a>' % (
margin = '<a href="%s?k=%s&zip=crc" rel="nofollow">zip</a>' % (
quotep(href),
self.gen_fk(2, self.args.dk_salt, zs, 0, 0)[:add_dk],
)
else:
margin = '<a href="%s?zip" rel="nofollow">zip</a>' % (quotep(href),)
margin = '<a href="%s?zip=crc" rel="nofollow">zip</a>' % (
quotep(href),
)
elif fn in hist:
margin = '<a href="%s.hist/%s">#%s</a>' % (
base,
@@ -4420,6 +4703,9 @@ class HttpCli(object):
else:
taglist = list(tagset)
if not files and not dirs and not readme and not logues[0] and not logues[1]:
logues[1] = "this folder is empty"
if is_ls:
ls_ret["dirs"] = dirs
ls_ret["files"] = files
@@ -4471,6 +4757,146 @@ class HttpCli(object):
if "mth" in vn.flags:
j2a["def_hcols"] = list(vn.flags["mth"])
if add_og and "raw" not in self.uparam:
j2a["this"] = self
cgv["og_fn"] = og_fn
if og_fn and vn.flags.get("og_tpl"):
tpl = vn.flags["og_tpl"]
if "EXT" in tpl:
zs = og_fn.split(".")[-1].lower()
tpl2 = tpl.replace("EXT", zs)
if os.path.exists(tpl2):
tpl = tpl2
with self.conn.hsrv.mutex:
if tpl not in self.conn.hsrv.j2:
tdir, tname = os.path.split(tpl)
j2env = jinja2.Environment()
j2env.loader = jinja2.FileSystemLoader(tdir)
self.conn.hsrv.j2[tpl] = j2env.get_template(tname)
thumb = ""
is_pic = is_vid = is_au = False
covernames = self.args.th_coversd
for fn in ls_names:
if fn.lower() in covernames:
thumb = fn
break
if og_fn:
ext = og_fn.split(".")[-1].lower()
if ext in self.thumbcli.thumbable:
is_pic = (
ext in self.thumbcli.fmt_pil
or ext in self.thumbcli.fmt_vips
or ext in self.thumbcli.fmt_ffi
)
is_vid = ext in self.thumbcli.fmt_ffv
is_au = ext in self.thumbcli.fmt_ffa
if not thumb or not is_au:
thumb = og_fn
file = next((x for x in files if x["name"] == og_fn), None)
else:
file = None
url_base = "%s://%s/%s" % (
"https" if self.is_https else "http",
self.host,
self.args.RS + quotep(vpath),
)
j2a["og_is_pic"] = is_pic
j2a["og_is_vid"] = is_vid
j2a["og_is_au"] = is_au
if thumb:
fmt = vn.flags.get("og_th", "j")
th_base = ujoin(url_base, quotep(thumb))
query = "th=%s&cache" % (fmt,)
query = ub64enc(query.encode("utf-8")).decode("utf-8")
# discord looks at file extension, not content-type...
query += "/th.jpg" if "j" in fmt else "/th.webp"
j2a["og_thumb"] = "%s/.uqe/%s" % (th_base, query)
j2a["og_fn"] = og_fn
j2a["og_file"] = file
if og_fn:
og_fn_q = quotep(og_fn)
query = ub64enc(b"raw").decode("utf-8")
query += "/%s" % (og_fn_q,)
j2a["og_url"] = ujoin(url_base, og_fn_q)
j2a["og_raw"] = j2a["og_url"] + "/.uqe/" + query
else:
j2a["og_url"] = j2a["og_raw"] = url_base
if not vn.flags.get("og_no_head"):
ogh = {"twitter:card": "summary"}
title = str(vn.flags.get("og_title") or "")
if thumb:
ogh["og:image"] = j2a["og_thumb"]
zso = vn.flags.get("og_desc") or ""
if zso != "-":
ogh["og:description"] = str(zso)
zs = vn.flags.get("og_site") or self.args.name
if zs not in ("", "-"):
ogh["og:site_name"] = zs
tagmap = {}
if is_au:
title = str(vn.flags.get("og_title_a") or "")
ogh["og:type"] = "music.song"
ogh["og:audio"] = j2a["og_raw"]
tagmap = {
"artist": "og:music:musician",
"album": "og:music:album",
".dur": "og:music:duration",
}
elif is_vid:
title = str(vn.flags.get("og_title_v") or "")
ogh["og:type"] = "video.other"
ogh["og:video"] = j2a["og_raw"]
tagmap = {
"title": "og:title",
".dur": "og:video:duration",
}
elif is_pic:
title = str(vn.flags.get("og_title_i") or "")
ogh["twitter:card"] = "summary_large_image"
ogh["twitter:image"] = ogh["og:image"] = j2a["og_raw"]
try:
for k, v in file["tags"].items():
zs = "{{ %s }}" % (k,)
title = title.replace(zs, str(v))
except:
pass
title = re.sub(r"\{\{ [^}]+ \}\}", "", title)
while title.startswith(" - "):
title = title[3:]
while title.endswith(" - "):
title = title[:3]
if vn.flags.get("og_s_title") or not title:
title = str(vn.flags.get("og_title") or "")
for tag, hname in tagmap.items():
try:
v = file["tags"][tag]
if not v:
continue
ogh[hname] = int(v) if tag == ".dur" else v
except:
pass
ogh["og:title"] = title
oghs = [
'\t<meta property="%s" content="%s">'
% (k, html_escape(str(v), True, True))
for k, v in ogh.items()
]
zs = self.html_head + "\n%s\n" % ("\n".join(oghs),)
self.html_head = zs.replace("\n\n", "\n")
html = self.j2s(tpl, **j2a)
self.reply(html.encode("utf-8", "replace"))
return True

View File

@@ -55,6 +55,7 @@ class HttpConn(object):
self.E: EnvParams = self.args.E
self.asrv: AuthSrv = hsrv.asrv # mypy404
self.u2fh: Util.FHC = hsrv.u2fh # mypy404
self.pipes: Util.CachedDict = hsrv.pipes # mypy404
self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
self.xff_nm: Optional[NetMap] = hsrv.xff_nm
self.xff_lan: NetMap = hsrv.xff_lan # type: ignore

View File

@@ -61,6 +61,7 @@ from .u2idx import U2idx
from .util import (
E_SCK,
FHC,
CachedDict,
Daemon,
Garda,
Magician,
@@ -130,6 +131,7 @@ class HttpSrv(object):
self.t_periodic: Optional[threading.Thread] = None
self.u2fh = FHC()
self.pipes = CachedDict(0.2)
self.metrics = Metrics(self)
self.nreq = 0
self.nsus = 0
@@ -264,10 +266,7 @@ class HttpSrv(object):
msg = "subscribed @ {}:{} f{} p{}".format(hip, port, fno, os.getpid())
self.log(self.name, msg)
def fun() -> None:
self.broker.say("cb_httpsrv_up")
threading.Thread(target=fun, name="sig-hsrv-up1").start()
Daemon(self.broker.say, "sig-hsrv-up1", ("cb_httpsrv_up",))
while not self.stopping:
if self.args.log_conn:

View File

@@ -292,6 +292,22 @@ class MDNS(MCast):
def run2(self) -> None:
last_hop = time.time()
ihop = self.args.mc_hop
try:
if self.args.no_poll:
raise Exception()
fd2sck = {}
srvpoll = select.poll()
for sck in self.srv:
fd = sck.fileno()
fd2sck[fd] = sck
srvpoll.register(fd, select.POLLIN)
except Exception as ex:
srvpoll = None
if not self.args.no_poll:
t = "WARNING: failed to poll(), will use select() instead: %r"
self.log(t % (ex,), 3)
while self.running:
timeout = (
0.02 + random.random() * 0.07
@@ -300,8 +316,13 @@ class MDNS(MCast):
if self.unsolicited
else (last_hop + ihop if ihop else 180)
)
rdy = select.select(self.srv, [], [], timeout)
rx: list[socket.socket] = rdy[0] # type: ignore
if srvpoll:
pr = srvpoll.poll(timeout * 1000)
rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
else:
rdy = select.select(self.srv, [], [], timeout)
rx: list[socket.socket] = rdy[0] # type: ignore
self.rx4.cln()
self.rx6.cln()
buf = b""

View File

@@ -179,7 +179,7 @@ class Metrics(object):
tnbytes = 0
tnfiles = 0
for vpath, vol in allvols:
cur = idx.get_cur(vol.realpath)
cur = idx.get_cur(vol)
if not cur:
continue

View File

@@ -7,12 +7,15 @@ import os
import shutil
import subprocess as sp
import sys
import tempfile
from .__init__ import ANYWIN, EXE, PY2, WINDOWS, E, unicode
from .authsrv import VFS
from .bos import bos
from .util import (
FFMPEG_URL,
REKOBO_LKEY,
VF_CAREFUL,
fsenc,
min_ex,
pybin,
@@ -20,12 +23,13 @@ from .util import (
runcmd,
sfsenc,
uncyg,
wunlink,
)
if True: # pylint: disable=using-constant-test
from typing import Any, Union
from typing import Any, Optional, Union
from .util import RootLogger
from .util import NamedLogger, RootLogger
def have_ff(scmd: str) -> bool:
@@ -107,6 +111,53 @@ class MParser(object):
raise Exception()
def au_unpk(
log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None
) -> str:
ret = ""
try:
ext = abspath.split(".")[-1].lower()
au, pk = fmt_map[ext].split(".")
fd, ret = tempfile.mkstemp("." + au)
if pk == "gz":
import gzip
fi = gzip.GzipFile(abspath, mode="rb")
elif pk == "xz":
import lzma
fi = lzma.open(abspath, "rb")
elif pk == "zip":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
zil = zf.infolist()
zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
fi = zf.open(zil[0])
with os.fdopen(fd, "wb") as fo:
while True:
buf = fi.read(32768)
if not buf:
break
fo.write(buf)
return ret
except Exception as ex:
if ret:
t = "failed to decompress audio file [%s]: %r"
log(t % (abspath, ex))
wunlink(log, ret, vn.flags if vn else VF_CAREFUL)
return abspath
def ffprobe(
abspath: str, timeout: int = 60
) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]]]:
@@ -281,7 +332,7 @@ class MTag(object):
or_ffprobe = " or FFprobe"
if self.backend == "mutagen":
self.get = self.get_mutagen
self._get = self.get_mutagen
try:
from mutagen import version # noqa: F401
except:
@@ -290,7 +341,7 @@ class MTag(object):
if self.backend == "ffprobe":
self.usable = self.can_ffprobe
self.get = self.get_ffprobe
self._get = self.get_ffprobe
self.prefer_mt = True
if not HAVE_FFPROBE:
@@ -460,6 +511,17 @@ class MTag(object):
return r1
def get(self, abspath: str) -> dict[str, Union[str, float]]:
ext = abspath.split(".")[-1].lower()
if ext not in self.args.au_unpk:
return self._get(abspath)
ap = au_unpk(self.log, self.args.au_unpk, abspath)
ret = self._get(ap)
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret
def get_mutagen(self, abspath: str) -> dict[str, Union[str, float]]:
ret: dict[str, tuple[int, Any]] = {}
@@ -553,10 +615,16 @@ class MTag(object):
except:
raise # might be expected outside cpython
ext = abspath.split(".")[-1].lower()
if ext in self.args.au_unpk:
ap = au_unpk(self.log, self.args.au_unpk, abspath)
else:
ap = abspath
ret: dict[str, Any] = {}
for tagname, parser in sorted(parsers.items(), key=lambda x: (x[1].pri, x[0])):
try:
cmd = [parser.bin, abspath]
cmd = [parser.bin, ap]
if parser.bin.endswith(".py"):
cmd = [pybin] + cmd
@@ -593,4 +661,7 @@ class MTag(object):
t = "mtag error: tagname {}, parser {}, file {} => {}"
self.log(t.format(tagname, parser.bin, abspath, min_ex()))
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret

View File

@@ -206,6 +206,7 @@ class MCast(object):
except:
t = "announce failed on {} [{}]:\n{}"
self.log(t.format(netdev, ip, min_ex()), 3)
sck.close()
if self.args.zm_msub:
for s1 in self.srv.values():

View File

@@ -127,7 +127,7 @@ class SMB(object):
self.log("smb", msg, c)
def start(self) -> None:
Daemon(self.srv.start)
Daemon(self.srv.start, "smbd")
def _auth_cb(self, *a, **ka):
debug("auth-result: %s %s", a, ka)

View File

@@ -141,9 +141,29 @@ class SSDPd(MCast):
self.log("stopped", 2)
def run2(self) -> None:
try:
if self.args.no_poll:
raise Exception()
fd2sck = {}
srvpoll = select.poll()
for sck in self.srv:
fd = sck.fileno()
fd2sck[fd] = sck
srvpoll.register(fd, select.POLLIN)
except Exception as ex:
srvpoll = None
if not self.args.no_poll:
t = "WARNING: failed to poll(), will use select() instead: %r"
self.log(t % (ex,), 3)
while self.running:
rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
rx: list[socket.socket] = rdy[0] # type: ignore
if srvpoll:
pr = srvpoll.poll((self.args.z_chk or 180) * 1000)
rx = [fd2sck[x[0]] for x in pr if x[1] & select.POLLIN]
else:
rdy = select.select(self.srv, [], [], self.args.z_chk or 180)
rx: list[socket.socket] = rdy[0] # type: ignore
self.rxc.cln()
buf = b""
addr = ("0", 0)

View File

@@ -1,13 +1,13 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import re
import stat
import tarfile
from queue import Queue
from .authsrv import AuthSrv
from .bos import bos
from .sutil import StreamArc, errdesc
from .util import Daemon, fsenc, min_ex
@@ -45,12 +45,12 @@ class StreamTar(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
cmp: str = "",
**kwargs: Any
):
super(StreamTar, self).__init__(log, args, fgen)
super(StreamTar, self).__init__(log, asrv, fgen)
self.ci = 0
self.co = 0
@@ -148,7 +148,7 @@ class StreamTar(StreamArc):
errors.append((f["vp"], ex))
if errors:
self.errf, txt = errdesc(errors)
self.errf, txt = errdesc(self.asrv.vfs, errors)
self.log("\n".join(([repr(self.errf)] + txt[1:])))
self.ser(self.errf)

View File

@@ -1,15 +1,15 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import tempfile
from datetime import datetime
from .__init__ import CORES
from .authsrv import AuthSrv, VFS
from .bos import bos
from .th_cli import ThumbCli
from .util import UTC, vjoin
from .util import UTC, vjoin, vol_san
if True: # pylint: disable=using-constant-test
from typing import Any, Generator, Optional
@@ -21,12 +21,13 @@ class StreamArc(object):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
**kwargs: Any
):
self.log = log
self.args = args
self.asrv = asrv
self.args = asrv.args
self.fgen = fgen
self.stopped = False
@@ -103,15 +104,20 @@ def enthumb(
return f
def errdesc(errors: list[tuple[str, str]]) -> tuple[dict[str, Any], list[str]]:
def errdesc(
vfs: VFS, errors: list[tuple[str, str]]
) -> tuple[dict[str, Any], list[str]]:
report = ["copyparty failed to add the following files to the archive:", ""]
for fn, err in errors:
report.extend([" file: {}".format(fn), "error: {}".format(err), ""])
btxt = "\r\n".join(report).encode("utf-8", "replace")
btxt = vol_san(list(vfs.all_vols.values()), btxt)
with tempfile.NamedTemporaryFile(prefix="copyparty-", delete=False) as tf:
tf_path = tf.name
tf.write("\r\n".join(report).encode("utf-8", "replace"))
tf.write(btxt)
dt = datetime.now(UTC).strftime("%Y-%m%d-%H%M%S")

View File

@@ -240,6 +240,10 @@ class SvcHub(object):
if not HAVE_FFMPEG or not HAVE_FFPROBE:
decs.pop("ff", None)
# compressed formats; "s3z=s3m.zip, s3gz=s3m.gz, ..."
zlss = [x.strip().lower().split("=", 1) for x in args.au_unpk.split(",")]
args.au_unpk = {x[0]: x[1] for x in zlss}
self.args.th_dec = list(decs.keys())
self.thumbsrv = None
want_ff = False
@@ -280,6 +284,8 @@ class SvcHub(object):
if not re.match("^(0|[qv][0-9]|[0-9]{2,3}k)$", args.q_mp3.lower()):
t = "invalid mp3 transcoding quality [%s] specified; only supports [0] to disable, a CBR value such as [192k], or a CQ/CRF value such as [v2]"
raise Exception(t % (args.q_mp3,))
else:
args.au_unpk = {}
args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
@@ -293,13 +299,14 @@ class SvcHub(object):
from .ftpd import Ftpd
self.ftpd: Optional[Ftpd] = None
Daemon(self.start_ftpd, "start_ftpd")
zms += "f" if args.ftp else "F"
if args.tftp:
from .tftpd import Tftpd
self.tftpd: Optional[Tftpd] = None
if args.ftp or args.ftps or args.tftp:
Daemon(self.start_ftpd, "start_tftpd")
if args.smb:
@@ -388,7 +395,7 @@ class SvcHub(object):
self.sigterm()
def sigterm(self) -> None:
os.kill(os.getpid(), signal.SIGTERM)
self.signal_handler(signal.SIGTERM, None)
def cb_httpsrv_up(self) -> None:
self.httpsrv_up += 1
@@ -424,6 +431,12 @@ class SvcHub(object):
t = "WARNING: found config files in [%s]: %s\n config files are not expected here, and will NOT be loaded (unless your setup is intentionally hella funky)"
self.log("root", t % (E.cfg, ", ".join(hits)), 3)
if self.args.no_bauth:
t = "WARNING: --no-bauth disables support for the Android app; you may want to use --bauth-last instead"
self.log("root", t, 3)
if self.args.bauth_last:
self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
def _process_config(self) -> bool:
al = self.args
@@ -520,7 +533,7 @@ class SvcHub(object):
al.exp_md = odfusion(exp, al.exp_md.replace(" ", ","))
al.exp_lg = odfusion(exp, al.exp_lg.replace(" ", ","))
for k in ["no_hash", "no_idx"]:
for k in ["no_hash", "no_idx", "og_ua"]:
ptn = getattr(self.args, k)
if ptn:
setattr(self.args, k, re.compile(ptn))
@@ -544,6 +557,17 @@ class SvcHub(object):
except:
raise Exception("invalid --rm-retry [%s]" % (self.args.rm_retry,))
try:
zf1, zf2 = self.args.mv_retry.split("/")
self.args.mv_re_t = float(zf1)
self.args.mv_re_r = float(zf2)
except:
raise Exception("invalid --mv-retry [%s]" % (self.args.mv_retry,))
al.tcolor = al.tcolor.lstrip("#")
if len(al.tcolor) == 3: # fc5 => ffcc55
al.tcolor = "".join([x * 2 for x in al.tcolor])
return True
def _ipa2re(self, txt) -> Optional[re.Pattern]:

View File

@@ -1,12 +1,12 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import calendar
import stat
import time
import zlib
from .authsrv import AuthSrv
from .bos import bos
from .sutil import StreamArc, errdesc
from .util import min_ex, sanitize_fn, spack, sunpack, yieldfile
@@ -219,13 +219,13 @@ class StreamZip(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
asrv: AuthSrv,
fgen: Generator[dict[str, Any], None, None],
utf8: bool = False,
pre_crc: bool = False,
**kwargs: Any
) -> None:
super(StreamZip, self).__init__(log, args, fgen)
super(StreamZip, self).__init__(log, asrv, fgen)
self.utf8 = utf8
self.pre_crc = pre_crc
@@ -302,7 +302,7 @@ class StreamZip(StreamArc):
mbuf = b""
if errors:
errf, txt = errdesc(errors)
errf, txt = errdesc(self.asrv.vfs, errors)
self.log("\n".join(([repr(errf)] + txt[1:])))
for x in self.ser(errf):
yield x

View File

@@ -463,6 +463,12 @@ class TcpSrv(object):
sys.stderr.flush()
def _qr(self, t1: dict[str, list[int]], t2: dict[str, list[int]]) -> str:
t2c = {zs: zli for zs, zli in t2.items() if zs in ("127.0.0.1", "::1")}
t2b = {zs: zli for zs, zli in t2.items() if ":" in zs and zs not in t2c}
t2 = {zs: zli for zs, zli in t2.items() if zs not in t2b and zs not in t2c}
t2.update(t2b) # first ipv4, then ipv6...
t2.update(t2c) # ...and finally localhost
ip = None
ips = list(t1) + list(t2)
qri = self.args.qri

View File

@@ -107,6 +107,11 @@ class ThumbCli(object):
fmt = sfmt
elif fmt[:1] == "p" and not is_au:
t = "cannot thumbnail [%s]: png only allowed for waveforms"
self.log(t % (rem), 6)
return None
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))

View File

@@ -15,7 +15,7 @@ from queue import Queue
from .__init__ import ANYWIN, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, au_unpk, ffprobe
from .util import BytesIO # type: ignore
from .util import (
FFMPEG_URL,
@@ -28,6 +28,7 @@ from .util import (
runcmd,
statdir,
vsplit,
wrename,
wunlink,
)
@@ -296,6 +297,12 @@ class ThumbSrv(object):
ext = abspath.split(".")[-1].lower()
png_ok = False
funs = []
if ext in self.args.au_unpk:
ap_unpk = au_unpk(self.log, self.args.au_unpk, abspath, vn)
else:
ap_unpk = abspath
if not bos.path.exists(tpath):
for lib in self.args.th_dec:
if lib == "pil" and ext in self.fmt_pil:
@@ -315,9 +322,6 @@ class ThumbSrv(object):
else:
funs.append(self.conv_spec)
if not png_ok and tpath.endswith(".png"):
raise Pebkac(400, "png only allowed for waveforms")
tdir, tfn = os.path.split(tpath)
ttpath = os.path.join(tdir, "w", tfn)
try:
@@ -327,7 +331,10 @@ class ThumbSrv(object):
for fun in funs:
try:
fun(abspath, ttpath, fmt, vn)
if not png_ok and tpath.endswith(".png"):
raise Exception("png only allowed for waveforms")
fun(ap_unpk, ttpath, fmt, vn)
break
except Exception as ex:
msg = "{} could not create thumbnail of {}\n{}"
@@ -345,8 +352,11 @@ class ThumbSrv(object):
except:
pass
if abspath != ap_unpk:
wunlink(self.log, ap_unpk, vn.flags)
try:
bos.rename(ttpath, tpath)
wrename(self.log, ttpath, tpath, vn.flags)
except:
pass
@@ -583,6 +593,25 @@ class ThumbSrv(object):
cmd += [fsenc(tpath)]
self._run_ff(cmd, vn)
if "pngquant" in vn.flags:
wtpath = tpath + ".png"
cmd = [
b"pngquant",
b"--strip",
b"--nofs",
b"--output",
fsenc(wtpath),
fsenc(tpath),
]
ret = runcmd(cmd, timeout=vn.flags["convt"], nice=True, oom=400)[0]
if ret:
try:
wunlink(self.log, wtpath, vn.flags)
except:
pass
else:
wrename(self.log, wtpath, tpath, vn.flags)
def conv_spec(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
@@ -645,8 +674,8 @@ class ThumbSrv(object):
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in tags:
raise Exception("not audio")
if quality.endswith("k"):
@@ -667,7 +696,7 @@ class ThumbSrv(object):
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
b"-map_metadata", b"-1",
] + self.big_tags(rawtags) + [
b"-map", b"0:a:0",
b"-ar", b"44100",
b"-ac", b"2",
@@ -683,16 +712,16 @@ class ThumbSrv(object):
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
tags, rawtags = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in tags:
raise Exception("not audio")
try:
dur = ret[".dur"][1]
dur = tags[".dur"][1]
except:
dur = 0
src_opus = abspath.lower().endswith(".opus") or ret["ac"][1] == "opus"
src_opus = abspath.lower().endswith(".opus") or tags["ac"][1] == "opus"
want_caf = tpath.endswith(".caf")
tmp_opus = tpath
if want_caf:
@@ -713,7 +742,7 @@ class ThumbSrv(object):
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
b"-map_metadata", b"-1",
] + self.big_tags(rawtags) + [
b"-map", b"0:a:0",
b"-c:a", b"libopus",
b"-b:a", bq,
@@ -770,6 +799,16 @@ class ThumbSrv(object):
except:
pass
def big_tags(self, raw_tags: dict[str, list[str]]) -> list[bytes]:
ret = []
for k, vs in raw_tags.items():
for v in vs:
if len(str(v)) >= 1024:
bv = k.encode("utf-8", "replace")
ret += [b"-metadata", bv + b"="]
break
return ret
def poke(self, tdir: str) -> None:
if not self.poke_cd.poke(tdir):
return

View File

@@ -62,6 +62,17 @@ class U2idx(object):
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("u2idx", msg, c)
def shutdown(self) -> None:
for cur in self.cur.values():
db = cur.connection
try:
db.interrupt()
except:
pass
cur.close()
db.close()
def fsearch(
self, uname: str, vols: list[VFS], body: dict[str, Any]
) -> list[dict[str, Any]]:
@@ -81,14 +92,18 @@ class U2idx(object):
except:
raise Pebkac(500, min_ex())
def get_cur(self, ptop: str) -> Optional["sqlite3.Cursor"]:
def get_cur(self, vn: VFS) -> Optional["sqlite3.Cursor"]:
if not HAVE_SQLITE3:
return None
cur = self.cur.get(ptop)
cur = self.cur.get(vn.realpath)
if cur:
return cur
if "e2d" not in vn.flags:
return None
ptop = vn.realpath
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
@@ -317,7 +332,7 @@ class U2idx(object):
ptop = vol.realpath
flags = vol.flags
cur = self.get_cur(ptop)
cur = self.get_cur(vol)
if not cur:
continue

View File

@@ -10,7 +10,6 @@ import math
import os
import re
import shutil
import signal
import stat
import subprocess as sp
import tempfile
@@ -29,6 +28,7 @@ from .fsutil import Fstab
from .mtag import MParser, MTag
from .util import (
HAVE_SQLITE3,
VF_CAREFUL,
SYMTIME,
Daemon,
MTHash,
@@ -136,6 +136,7 @@ class Up2k(object):
self.need_rescan: set[str] = set()
self.db_act = 0.0
self.reg_mutex = threading.Lock()
self.registry: dict[str, dict[str, dict[str, Any]]] = {}
self.flags: dict[str, dict[str, Any]] = {}
self.droppable: dict[str, list[str]] = {}
@@ -143,7 +144,7 @@ class Up2k(object):
self.volsize: dict["sqlite3.Cursor", int] = {}
self.volstate: dict[str, str] = {}
self.vol_act: dict[str, float] = {}
self.busy_aps: set[str] = set()
self.busy_aps: dict[str, int] = {}
self.dupesched: dict[str, list[tuple[str, str, float]]] = {}
self.snap_prev: dict[str, Optional[tuple[int, float]]] = {}
@@ -182,7 +183,7 @@ class Up2k(object):
t = "could not initialize sqlite3, will use in-memory registry only"
self.log(t, 3)
self.fstab = Fstab(self.log_func)
self.fstab = Fstab(self.log_func, self.args)
self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
if self.args.hash_mt < 2:
@@ -200,11 +201,15 @@ class Up2k(object):
Daemon(self.deferred_init, "up2k-deferred-init")
def reload(self, rescan_all_vols: bool) -> None:
"""mutex me"""
"""mutex(main) me"""
self.log("reload #{} scheduled".format(self.gid + 1))
all_vols = self.asrv.vfs.all_vols
scan_vols = [k for k, v in all_vols.items() if v.realpath not in self.registry]
with self.reg_mutex:
scan_vols = [
k for k, v in all_vols.items() if v.realpath not in self.registry
]
if rescan_all_vols:
scan_vols = list(all_vols.keys())
@@ -217,7 +222,7 @@ class Up2k(object):
if self.stop:
# up-mt consistency not guaranteed if init is interrupted;
# drop caches for a full scan on next boot
with self.mutex:
with self.mutex, self.reg_mutex:
self._drop_caches()
if self.pp:
@@ -283,10 +288,27 @@ class Up2k(object):
min(1000 * 24 * 60 * 60 - 1, time.time() - self.db_act)
),
}
return json.dumps(ret, indent=4)
return json.dumps(ret, separators=(",\n", ": "))
def find_job_by_ap(self, ptop: str, ap: str) -> str:
try:
if ANYWIN:
ap = ap.replace("\\", "/")
vp = ap[len(ptop) :].strip("/")
dn, fn = vsplit(vp)
with self.reg_mutex:
tab2 = self.registry[ptop]
for job in tab2.values():
if job["prel"] == dn and job["name"] == fn:
return json.dumps(job, separators=(",\n", ": "))
except:
pass
return "{}"
def get_unfinished_by_user(self, uname, ip) -> str:
if PY2 or not self.mutex.acquire(timeout=2):
if PY2 or not self.reg_mutex.acquire(timeout=2):
return '[{"timeout":1}]'
ret: list[tuple[int, str, int, int, int]] = []
@@ -315,17 +337,25 @@ class Up2k(object):
)
ret.append(zt5)
finally:
self.mutex.release()
self.reg_mutex.release()
if ANYWIN:
ret = [(x[0], x[1].replace("\\", "/"), x[2], x[3], x[4]) for x in ret]
ret.sort(reverse=True)
ret2 = [
{"at": at, "vp": "/" + vp, "pd": 100 - ((nn * 100) // (nh or 1)), "sz": sz}
{
"at": at,
"vp": "/" + quotep(vp),
"pd": 100 - ((nn * 100) // (nh or 1)),
"sz": sz,
}
for (at, vp, sz, nn, nh) in ret
]
return json.dumps(ret2, indent=0)
return json.dumps(ret2, separators=(",\n", ": "))
def get_unfinished(self) -> str:
if PY2 or not self.mutex.acquire(timeout=0.5):
if PY2 or not self.reg_mutex.acquire(timeout=0.5):
return ""
ret: dict[str, tuple[int, int]] = {}
@@ -347,17 +377,17 @@ class Up2k(object):
ret[ptop] = (nbytes, nfiles)
finally:
self.mutex.release()
self.reg_mutex.release()
return json.dumps(ret, indent=4)
return json.dumps(ret, separators=(",\n", ": "))
def get_volsize(self, ptop: str) -> tuple[int, int]:
with self.mutex:
with self.reg_mutex:
return self._get_volsize(ptop)
def get_volsizes(self, ptops: list[str]) -> list[tuple[int, int]]:
ret = []
with self.mutex:
with self.reg_mutex:
for ptop in ptops:
ret.append(self._get_volsize(ptop))
@@ -385,7 +415,7 @@ class Up2k(object):
def _rescan(
self, all_vols: dict[str, VFS], scan_vols: list[str], wait: bool, fscan: bool
) -> str:
"""mutex me"""
"""mutex(main) me"""
if not wait and self.pp:
return "cannot initiate; scan is already in progress"
@@ -428,7 +458,13 @@ class Up2k(object):
# important; not deferred by db_act
timeout = self._check_lifetimes()
timeout = min(timeout, now + self._check_xiu())
try:
timeout = min(timeout, now + self._check_xiu())
except Exception as ex:
if "closed cursor" in str(ex):
self.log("sched_rescan: lost db")
return
raise
with self.mutex:
for vp, vol in sorted(self.asrv.vfs.all_vols.items()):
@@ -667,15 +703,15 @@ class Up2k(object):
self.log(msg, c=3)
live_vols = []
with self.mutex:
with self.mutex, self.reg_mutex:
# only need to protect register_vpath but all in one go feels right
for vol in vols:
try:
bos.makedirs(vol.realpath) # gonna happen at snap anyways
dir_is_empty(self.log_func, not self.args.no_scandir, vol.realpath)
except:
except Exception as ex:
self.volstate[vol.vpath] = "OFFLINE (cannot access folder)"
self.log("cannot access " + vol.realpath, c=1)
self.log("cannot access %s: %r" % (vol.realpath, ex), c=1)
continue
if scan_vols and vol.vpath not in scan_vols:
@@ -709,7 +745,7 @@ class Up2k(object):
if self.args.re_dhash or [zv for zv in vols if "e2tsr" in zv.flags]:
self.args.re_dhash = False
with self.mutex:
with self.mutex, self.reg_mutex:
self._drop_caches()
for vol in vols:
@@ -786,7 +822,9 @@ class Up2k(object):
self.volstate[vol.vpath] = "online (mtp soon)"
for vol in need_vac:
reg = self.register_vpath(vol.realpath, vol.flags)
with self.mutex, self.reg_mutex:
reg = self.register_vpath(vol.realpath, vol.flags)
assert reg
cur, _ = reg
with self.mutex:
@@ -800,7 +838,9 @@ class Up2k(object):
if vol.flags["dbd"] == "acid":
continue
reg = self.register_vpath(vol.realpath, vol.flags)
with self.mutex, self.reg_mutex:
reg = self.register_vpath(vol.realpath, vol.flags)
try:
assert reg
cur, db_path = reg
@@ -847,6 +887,7 @@ class Up2k(object):
def register_vpath(
self, ptop: str, flags: dict[str, Any]
) -> Optional[tuple["sqlite3.Cursor", str]]:
"""mutex(main,reg) me"""
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
@@ -869,7 +910,7 @@ class Up2k(object):
ft = "\033[0;32m{}{:.0}"
ff = "\033[0;35m{}{:.0}"
fv = "\033[0;36m{}:\033[90m{}"
fx = set(("html_head", "rm_re_t", "rm_re_r"))
fx = set(("html_head", "rm_re_t", "rm_re_r", "mv_re_t", "mv_re_r"))
fd = vf_bmap()
fd.update(vf_cmap())
fd.update(vf_vmap())
@@ -1001,8 +1042,11 @@ class Up2k(object):
return None
def _verify_db_cache(self, cur: "sqlite3.Cursor", vpath: str) -> None:
# check if volume config changed since last use; drop caches if so
zsl = [vpath] + list(sorted(self.asrv.vfs.all_vols.keys()))
# check if list of intersecting volumes changed since last use; drop caches if so
prefix = (vpath + "/").lstrip("/")
zsl = [x for x in self.asrv.vfs.all_vols if x.startswith(prefix)]
zsl = [x[len(prefix) :] for x in zsl]
zsl.sort()
zb = hashlib.sha1("\n".join(zsl).encode("utf-8", "replace")).digest()
vcfg = base64.urlsafe_b64encode(zb[:18]).decode("ascii")
@@ -1030,7 +1074,9 @@ class Up2k(object):
dev = cst.st_dev if vol.flags.get("xdev") else 0
with self.mutex:
reg = self.register_vpath(top, vol.flags)
with self.reg_mutex:
reg = self.register_vpath(top, vol.flags)
assert reg and self.pp
cur, db_path = reg
@@ -1610,7 +1656,7 @@ class Up2k(object):
if e2vp and rewark:
self.hub.retcode = 1
os.kill(os.getpid(), signal.SIGTERM)
Daemon(self.hub.sigterm)
raise Exception("{} files have incorrect hashes".format(len(rewark)))
if not e2vu or not rewark:
@@ -1627,7 +1673,7 @@ class Up2k(object):
def _build_tags_index(self, vol: VFS) -> tuple[int, int, bool]:
ptop = vol.realpath
with self.mutex:
with self.mutex, self.reg_mutex:
reg = self.register_vpath(ptop, vol.flags)
assert reg and self.pp
@@ -1648,6 +1694,7 @@ class Up2k(object):
return ret
def _drop_caches(self) -> None:
"""mutex(main,reg) me"""
self.log("dropping caches for a full filesystem scan")
for vol in self.asrv.vfs.all_vols.values():
reg = self.register_vpath(vol.realpath, vol.flags)
@@ -1823,7 +1870,7 @@ class Up2k(object):
params: tuple[Any, ...],
flt: int,
) -> tuple[tempfile.SpooledTemporaryFile[bytes], int]:
"""mutex me"""
"""mutex(main) me"""
n = 0
c2 = cur.connection.cursor()
tf = tempfile.SpooledTemporaryFile(1024 * 1024 * 8, "w+b", prefix="cpp-tq-")
@@ -2157,7 +2204,7 @@ class Up2k(object):
ip: str,
at: float,
) -> int:
"""will mutex"""
"""will mutex(main)"""
assert self.mtag
try:
@@ -2189,7 +2236,7 @@ class Up2k(object):
abspath: str,
tags: dict[str, Union[str, float]],
) -> int:
"""mutex me"""
"""mutex(main) me"""
assert self.mtag
if not bos.path.isfile(abspath):
@@ -2474,28 +2521,36 @@ class Up2k(object):
cur.connection.commit()
def _job_volchk(self, cj: dict[str, Any]) -> None:
if not self.register_vpath(cj["ptop"], cj["vcfg"]):
if cj["ptop"] not in self.registry:
raise Pebkac(410, "location unavailable")
def handle_json(self, cj: dict[str, Any], busy_aps: set[str]) -> dict[str, Any]:
def handle_json(
self, cj: dict[str, Any], busy_aps: dict[str, int]
) -> dict[str, Any]:
# busy_aps is u2fh (always undefined if -j0) so this is safe
self.busy_aps = busy_aps
got_lock = False
try:
# bit expensive; 3.9=10x 3.11=2x
if self.mutex.acquire(timeout=10):
self._job_volchk(cj)
self.mutex.release()
got_lock = True
with self.reg_mutex:
return self._handle_json(cj)
else:
t = "cannot receive uploads right now;\nserver busy with {}.\nPlease wait; the client will retry..."
raise Pebkac(503, t.format(self.blocked or "[unknown]"))
except TypeError:
if not PY2:
raise
with self.mutex:
self._job_volchk(cj)
with self.mutex, self.reg_mutex:
return self._handle_json(cj)
finally:
if got_lock:
self.mutex.release()
def _handle_json(self, cj: dict[str, Any]) -> dict[str, Any]:
ptop = cj["ptop"]
if not self.register_vpath(ptop, cj["vcfg"]):
if ptop not in self.registry:
raise Pebkac(410, "location unavailable")
cj["name"] = sanitize_fn(cj["name"], "", [".prologue.html", ".epilogue.html"])
cj["poke"] = now = self.db_act = self.vol_act[ptop] = time.time()
wark = self._get_wark(cj)
@@ -2510,7 +2565,7 @@ class Up2k(object):
# refuse out-of-order / multithreaded uploading if sprs False
sprs = self.fstab.get(pdir) != "ng"
with self.mutex:
if True:
jcur = self.cur.get(ptop)
reg = self.registry[ptop]
vfs = self.asrv.vfs.all_vols[cj["vtop"]]
@@ -2948,7 +3003,7 @@ class Up2k(object):
def handle_chunk(
self, ptop: str, wark: str, chash: str
) -> tuple[int, list[int], str, float, bool]:
with self.mutex:
with self.mutex, self.reg_mutex:
self.db_act = self.vol_act[ptop] = time.time()
job = self.registry[ptop].get(wark)
if not job:
@@ -2991,7 +3046,7 @@ class Up2k(object):
return chunksize, ofs, path, job["lmod"], job["sprs"]
def release_chunk(self, ptop: str, wark: str, chash: str) -> bool:
with self.mutex:
with self.reg_mutex:
job = self.registry[ptop].get(wark)
if job:
job["busy"].pop(chash, None)
@@ -2999,7 +3054,7 @@ class Up2k(object):
return True
def confirm_chunk(self, ptop: str, wark: str, chash: str) -> tuple[int, str]:
with self.mutex:
with self.mutex, self.reg_mutex:
self.db_act = self.vol_act[ptop] = time.time()
try:
job = self.registry[ptop][wark]
@@ -3022,16 +3077,16 @@ class Up2k(object):
if self.args.nw:
self.regdrop(ptop, wark)
return ret, dst
return ret, dst
def finish_upload(self, ptop: str, wark: str, busy_aps: set[str]) -> None:
self.busy_aps = busy_aps
with self.mutex:
with self.mutex, self.reg_mutex:
self._finish_upload(ptop, wark)
def _finish_upload(self, ptop: str, wark: str) -> None:
"""mutex(main,reg) me"""
try:
job = self.registry[ptop][wark]
pdir = djoin(job["ptop"], job["prel"])
@@ -3044,12 +3099,11 @@ class Up2k(object):
t = "finish_upload {} with remaining chunks {}"
raise Pebkac(500, t.format(wark, job["need"]))
# self.log("--- " + wark + " " + dst + " finish_upload atomic " + dst, 4)
atomic_move(src, dst)
upt = job.get("at") or time.time()
vflags = self.flags[ptop]
atomic_move(self.log, src, dst, vflags)
times = (int(time.time()), int(job["lmod"]))
self.log(
"no more chunks, setting times {} ({}) on {}".format(
@@ -3105,6 +3159,7 @@ class Up2k(object):
cur.connection.commit()
def regdrop(self, ptop: str, wark: str) -> None:
"""mutex(main,reg) me"""
olds = self.droppable[ptop]
if wark:
olds.append(wark)
@@ -3199,16 +3254,23 @@ class Up2k(object):
at: float,
skip_xau: bool = False,
) -> None:
"""mutex(main) me"""
self.db_rm(db, rd, fn, sz)
if not ip:
db_ip = ""
else:
# plugins may expect this to look like an actual IP
db_ip = "1.1.1.1" if self.args.no_db_ip else ip
sql = "insert into up values (?,?,?,?,?,?,?)"
v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
v = (wark, int(ts), sz, rd, fn, db_ip, int(at or 0))
try:
db.execute(sql, v)
except:
assert self.mem_cur
rd, fn = s3enc(self.mem_cur, rd, fn)
v = (wark, int(ts), sz, rd, fn, ip or "", int(at or 0))
v = (wark, int(ts), sz, rd, fn, db_ip, int(at or 0))
db.execute(sql, v)
self.volsize[db] += sz
@@ -3312,7 +3374,7 @@ class Up2k(object):
vn, rem = self.asrv.vfs.get(vpath, uname, *permsets[0])
vn, rem = vn.get_dbv(rem)
ptop = vn.realpath
with self.mutex:
with self.mutex, self.reg_mutex:
abrt_cfg = self.flags.get(ptop, {}).get("u2abort", 1)
addr = (ip or "\n") if abrt_cfg in (1, 2) else ""
user = (uname or "\n") if abrt_cfg in (1, 3) else ""
@@ -3320,7 +3382,10 @@ class Up2k(object):
for wark, job in reg.items():
if (user and user != job["user"]) or (addr and addr != job["addr"]):
continue
if djoin(job["prel"], job["name"]) == rem:
jrem = djoin(job["prel"], job["name"])
if ANYWIN:
jrem = jrem.replace("\\", "/")
if jrem == rem:
if job["ptop"] != ptop:
t = "job.ptop [%s] != vol.ptop [%s] ??"
raise Exception(t % (job["ptop"] != ptop))
@@ -3416,7 +3481,7 @@ class Up2k(object):
continue
n_files += 1
with self.mutex:
with self.mutex, self.reg_mutex:
cur = None
try:
ptop = dbv.realpath
@@ -3534,6 +3599,7 @@ class Up2k(object):
def _mv_file(
self, uname: str, svp: str, dvp: str, curs: set["sqlite3.Cursor"]
) -> str:
"""mutex(main) me; will mutex(reg)"""
svn, srem = self.asrv.vfs.get(svp, uname, True, False, True)
svn, srem = svn.get_dbv(srem)
@@ -3614,7 +3680,9 @@ class Up2k(object):
if c2 and c2 != c1:
self._copy_tags(c1, c2, w)
has_dupes = self._forget_file(svn.realpath, srem, c1, w, is_xvol, fsize)
with self.reg_mutex:
has_dupes = self._forget_file(svn.realpath, srem, c1, w, is_xvol, fsize)
if not is_xvol:
has_dupes = self._relink(w, svn.realpath, srem, dabs)
@@ -3653,7 +3721,7 @@ class Up2k(object):
self._symlink(dlink, dabs, dvn.flags, lmod=ftime)
wunlink(self.log, sabs, svn.flags)
else:
atomic_move(sabs, dabs)
atomic_move(self.log, sabs, dabs, svn.flags)
except OSError as ex:
if ex.errno != errno.EXDEV:
@@ -3744,7 +3812,10 @@ class Up2k(object):
drop_tags: bool,
sz: int,
) -> bool:
"""forgets file in db, fixes symlinks, does not delete"""
"""
mutex(main,reg) me
forgets file in db, fixes symlinks, does not delete
"""
srd, sfn = vsplit(vrem)
has_dupes = False
self.log("forgetting {}".format(vrem))
@@ -3830,8 +3901,7 @@ class Up2k(object):
self.log("linkswap [{}] and [{}]".format(sabs, slabs))
mt = bos.path.getmtime(slabs, False)
flags = self.flags.get(ptop) or {}
wunlink(self.log, slabs, flags)
bos.rename(sabs, slabs)
atomic_move(self.log, sabs, slabs, flags)
bos.utime(slabs, (int(time.time()), int(mt)), False)
self._symlink(slabs, sabs, flags, False)
full[slabs] = (ptop, rem)
@@ -4070,7 +4140,7 @@ class Up2k(object):
self.do_snapshot()
def do_snapshot(self) -> None:
with self.mutex:
with self.mutex, self.reg_mutex:
for k, reg in self.registry.items():
self._snap_reg(k, reg)
@@ -4138,11 +4208,11 @@ class Up2k(object):
path2 = "{}.{}".format(path, os.getpid())
body = {"droppable": self.droppable[ptop], "registry": reg}
j = json.dumps(body, indent=2, sort_keys=True).encode("utf-8")
j = json.dumps(body, sort_keys=True, separators=(",\n", ": ")).encode("utf-8")
with gzip.GzipFile(path2, "wb") as f:
f.write(j)
atomic_move(path2, path)
atomic_move(self.log, path2, path, VF_CAREFUL)
self.log("snap: {} |{}|".format(path, len(reg.keys())))
self.snap_prev[ptop] = etag
@@ -4211,7 +4281,7 @@ class Up2k(object):
raise Exception("invalid hash task")
try:
if not self._hash_t(task):
if not self._hash_t(task) and self.stop:
return
except Exception as ex:
self.log("failed to hash %s: %s" % (task, ex), 1)
@@ -4221,7 +4291,7 @@ class Up2k(object):
) -> bool:
ptop, vtop, flags, rd, fn, ip, at, usr, skip_xau = task
# self.log("hashq {} pop {}/{}/{}".format(self.n_hashq, ptop, rd, fn))
with self.mutex:
with self.mutex, self.reg_mutex:
if not self.register_vpath(ptop, flags):
return True
@@ -4239,7 +4309,7 @@ class Up2k(object):
wark = up2k_wark_from_hashlist(self.salt, inf.st_size, hashes)
with self.mutex:
with self.mutex, self.reg_mutex:
self.idx_wark(
self.flags[ptop],
rd,
@@ -4315,6 +4385,18 @@ class Up2k(object):
for x in list(self.spools):
self._unspool(x)
for cur in self.cur.values():
db = cur.connection
try:
db.interrupt()
except:
pass
cur.close()
db.close()
self.registry = {}
def up2k_chunksize(filesize: int) -> int:
chunksize = 1024 * 1024

View File

@@ -35,6 +35,9 @@ from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, VT100, WINDOWS
from .__version__ import S_BUILD_DT, S_VERSION
from .stolen import surrogateescape
ub64dec = base64.urlsafe_b64decode
ub64enc = base64.urlsafe_b64encode
try:
from datetime import datetime, timezone
@@ -355,6 +358,21 @@ APPLESAN_TXT = r"/(__MACOS|Icon\r\r)|/\.(_|DS_Store|AppleDouble|LSOverride|Docum
APPLESAN_RE = re.compile(APPLESAN_TXT)
HUMANSIZE_UNITS = ("B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB")
UNHUMANIZE_UNITS = {
"b": 1,
"k": 1024,
"m": 1024 * 1024,
"g": 1024 * 1024 * 1024,
"t": 1024 * 1024 * 1024 * 1024,
"p": 1024 * 1024 * 1024 * 1024 * 1024,
"e": 1024 * 1024 * 1024 * 1024 * 1024 * 1024,
}
VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
pybin = sys.executable or ""
if EXE:
pybin = ""
@@ -460,13 +478,22 @@ class Daemon(threading.Thread):
r: bool = True,
ka: Optional[dict[Any, Any]] = None,
) -> None:
threading.Thread.__init__(
self, target=target, name=name, args=a or (), kwargs=ka
)
threading.Thread.__init__(self, name=name)
self.a = a or ()
self.ka = ka or {}
self.fun = target
self.daemon = True
if r:
self.start()
def run(self):
if not ANYWIN and not PY2:
signal.pthread_sigmask(
signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]
)
self.fun(*self.a, **self.ka)
class Netdev(object):
def __init__(self, ip: str, idx: int, name: str, desc: str):
@@ -759,15 +786,46 @@ class CachedSet(object):
self.oldest = now
class CachedDict(object):
def __init__(self, maxage: float) -> None:
self.c: dict[str, tuple[float, Any]] = {}
self.maxage = maxage
self.oldest = 0.0
def set(self, k: str, v: Any) -> None:
now = time.time()
self.c[k] = (now, v)
if now - self.oldest < self.maxage:
return
c = self.c = {k: v for k, v in self.c.items() if now - v[0] < self.maxage}
try:
self.oldest = min([x[0] for x in c.values()])
except:
self.oldest = now
def get(self, k: str) -> Optional[tuple[str, Any]]:
try:
ts, ret = self.c[k]
now = time.time()
if now - ts > self.maxage:
del self.c[k]
return None
return ret
except:
return None
class FHC(object):
class CE(object):
def __init__(self, fh: typing.BinaryIO) -> None:
self.ts: float = 0
self.fhs = [fh]
self.all_fhs = set([fh])
def __init__(self) -> None:
self.cache: dict[str, FHC.CE] = {}
self.aps: set[str] = set()
self.aps: dict[str, int] = {}
def close(self, path: str) -> None:
try:
@@ -779,7 +837,7 @@ class FHC(object):
fh.close()
del self.cache[path]
self.aps.remove(path)
del self.aps[path]
def clean(self) -> None:
if not self.cache:
@@ -800,9 +858,12 @@ class FHC(object):
return self.cache[path].fhs.pop()
def put(self, path: str, fh: typing.BinaryIO) -> None:
self.aps.add(path)
if path not in self.aps:
self.aps[path] = 0
try:
ce = self.cache[path]
ce.all_fhs.add(fh)
ce.fhs.append(fh)
except:
ce = self.CE(fh)
@@ -827,6 +888,7 @@ class ProgressPrinter(threading.Thread):
self.start()
def run(self) -> None:
sigblock()
tp = 0
msg = None
no_stdout = self.args.q
@@ -1271,6 +1333,15 @@ def log_thrs(log: Callable[[str, str, int], None], ival: float, name: str) -> No
log(name, "\033[0m \033[33m".join(tv), 3)
def sigblock():
if ANYWIN or PY2:
return
signal.pthread_sigmask(
signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]
)
def vol_san(vols: list["VFS"], txt: bytes) -> bytes:
txt0 = txt
for vol in vols:
@@ -1292,10 +1363,11 @@ def vol_san(vols: list["VFS"], txt: bytes) -> bytes:
def min_ex(max_lines: int = 8, reverse: bool = False) -> str:
et, ev, tb = sys.exc_info()
stb = traceback.extract_tb(tb)
stb = traceback.extract_tb(tb) if tb else traceback.extract_stack()[:-1]
fmt = "%s @ %d <%s>: %s"
ex = [fmt % (fp.split(os.sep)[-1], ln, fun, txt) for fp, ln, fun, txt in stb]
ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
if et or ev or tb:
ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
return "\n".join(ex[-max_lines:][:: -1 if reverse else 1])
@@ -1748,7 +1820,7 @@ def gencookie(k: str, v: str, r: str, tls: bool, dur: int = 0, txt: str = "") ->
def humansize(sz: float, terse: bool = False) -> str:
for unit in ["B", "KiB", "MiB", "GiB", "TiB"]:
for unit in HUMANSIZE_UNITS:
if sz < 1024:
break
@@ -1769,12 +1841,7 @@ def unhumanize(sz: str) -> int:
pass
mc = sz[-1:].lower()
mi = {
"k": 1024,
"m": 1024 * 1024,
"g": 1024 * 1024 * 1024,
"t": 1024 * 1024 * 1024 * 1024,
}.get(mc, 1)
mi = UNHUMANIZE_UNITS.get(mc, 1)
return int(float(sz[:-1]) * mi)
@@ -1997,6 +2064,7 @@ def vsplit(vpath: str) -> tuple[str, str]:
return vpath.rsplit("/", 1) # type: ignore
# vpath-join
def vjoin(rd: str, fn: str) -> str:
if rd and fn:
return rd + "/" + fn
@@ -2004,6 +2072,14 @@ def vjoin(rd: str, fn: str) -> str:
return rd or fn
# url-join
def ujoin(rd: str, fn: str) -> str:
if rd and fn:
return rd.rstrip("/") + "/" + fn.lstrip("/")
else:
return rd or fn
def _w8dec2(txt: bytes) -> str:
"""decodes filesystem-bytes to wtf8"""
return surrogateescape.decodefilename(txt)
@@ -2125,26 +2201,29 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
log("lsof failed; " + min_ex(), 3)
def atomic_move(usrc: str, udst: str) -> None:
src = fsenc(usrc)
dst = fsenc(udst)
if not PY2:
os.replace(src, dst)
def _fs_mvrm(
log: "NamedLogger", src: str, dst: str, atomic: bool, flags: dict[str, Any]
) -> bool:
bsrc = fsenc(src)
bdst = fsenc(dst)
if atomic:
k = "mv_re_"
act = "atomic-rename"
osfun = os.replace
args = [bsrc, bdst]
elif dst:
k = "mv_re_"
act = "rename"
osfun = os.rename
args = [bsrc, bdst]
else:
if os.path.exists(dst):
os.unlink(dst)
k = "rm_re_"
act = "delete"
osfun = os.unlink
args = [bsrc]
os.rename(src, dst)
def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
maxtime = flags.get("rm_re_t", 0.0)
bpath = fsenc(abspath)
if not maxtime:
os.unlink(bpath)
return True
chill = flags.get("rm_re_r", 0.0)
maxtime = flags.get(k + "t", 0.0)
chill = flags.get(k + "r", 0.0)
if chill < 0.001:
chill = 0.1
@@ -2152,14 +2231,19 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
t0 = now = time.time()
for attempt in range(90210):
try:
if ino and os.stat(bpath).st_ino != ino:
log("inode changed; aborting delete")
if ino and os.stat(bsrc).st_ino != ino:
t = "src inode changed; aborting %s %s"
log(t % (act, src), 1)
return False
os.unlink(bpath)
if (dst and not atomic) and os.path.exists(bdst):
t = "something appeared at dst; aborting rename [%s] ==> [%s]"
log(t % (src, dst), 1)
return False
osfun(*args)
if attempt:
now = time.time()
t = "deleted in %.2f sec, attempt %d"
log(t % (now - t0, attempt + 1))
t = "%sd in %.2f sec, attempt %d: %s"
log(t % (act, now - t0, attempt + 1, src))
return True
except OSError as ex:
now = time.time()
@@ -2169,15 +2253,45 @@ def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
raise
if not attempt:
if not PY2:
ino = os.stat(bpath).st_ino
t = "delete failed (err.%d); retrying for %d sec: %s"
log(t % (ex.errno, maxtime + 0.99, abspath))
ino = os.stat(bsrc).st_ino
t = "%s failed (err.%d); retrying for %d sec: [%s]"
log(t % (act, ex.errno, maxtime + 0.99, src))
time.sleep(chill)
return False # makes pylance happy
def atomic_move(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> None:
bsrc = fsenc(src)
bdst = fsenc(dst)
if PY2:
if os.path.exists(bdst):
_fs_mvrm(log, dst, "", False, flags) # unlink
_fs_mvrm(log, src, dst, False, flags) # rename
elif flags.get("mv_re_t"):
_fs_mvrm(log, src, dst, True, flags)
else:
os.replace(bsrc, bdst)
def wrename(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> bool:
if not flags.get("mv_re_t"):
os.rename(fsenc(src), fsenc(dst))
return True
return _fs_mvrm(log, src, dst, False, flags)
def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
if not flags.get("rm_re_t"):
os.unlink(fsenc(abspath))
return True
return _fs_mvrm(log, abspath, "", False, flags)
def get_df(abspath: str) -> tuple[Optional[int], Optional[int]]:
try:
# some fuses misbehave
@@ -2410,6 +2524,7 @@ def sendfile_py(
s: socket.socket,
bufsz: int,
slp: int,
use_poll: bool,
) -> int:
remains = upper - lower
f.seek(lower)
@@ -2438,22 +2553,31 @@ def sendfile_kern(
s: socket.socket,
bufsz: int,
slp: int,
use_poll: bool,
) -> int:
out_fd = s.fileno()
in_fd = f.fileno()
ofs = lower
stuck = 0.0
if use_poll:
poll = select.poll()
poll.register(out_fd, select.POLLOUT)
while ofs < upper:
stuck = stuck or time.time()
try:
req = min(2 ** 30, upper - ofs)
select.select([], [out_fd], [], 10)
if use_poll:
poll.poll(10000)
else:
select.select([], [out_fd], [], 10)
n = os.sendfile(out_fd, in_fd, ofs, req)
stuck = 0
except OSError as ex:
# client stopped reading; do another select
d = time.time() - stuck
if d < 3600 and ex.errno == errno.EWOULDBLOCK:
time.sleep(0.02)
continue
n = 0
@@ -2597,7 +2721,7 @@ def unescape_cookie(orig: str) -> str:
def guess_mime(url: str, fallback: str = "application/octet-stream") -> str:
try:
_, ext = url.rsplit(".", 1)
ext = url.rsplit(".", 1)[1].lower()
except:
return fallback

View File

@@ -29,6 +29,7 @@ window.baguetteBox = (function () {
isOverlayVisible = false,
touch = {}, // start-pos
touchFlag = false, // busy
scrollTimer = 0,
re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4)(\?|$)/i,
anims = ['slideIn', 'fadeIn', 'none'],
@@ -91,6 +92,30 @@ window.baguetteBox = (function () {
touchendHandler();
};
var overlayWheelHandler = function (e) {
if (!options.noScrollbars || anymod(e))
return;
ev(e);
var x = e.deltaX,
y = e.deltaY,
d = Math.abs(x) > Math.abs(y) ? x : y;
if (e.deltaMode)
d *= 10;
if (Date.now() - scrollTimer < (Math.abs(d) > 20 ? 100 : 300))
return;
scrollTimer = Date.now();
if (d > 0)
showNextImage();
else
showPreviousImage();
};
var trapFocusInsideOverlay = function (e) {
if (overlay.style.display === 'block' && (overlay.contains && !overlay.contains(e.target))) {
e.stopPropagation();
@@ -451,6 +476,7 @@ window.baguetteBox = (function () {
bind(document, 'keyup', keyUpHandler);
bind(document, 'fullscreenchange', onFSC);
bind(overlay, 'click', overlayClickHandler);
bind(overlay, 'wheel', overlayWheelHandler);
bind(btnPrev, 'click', showPreviousImage);
bind(btnNext, 'click', showNextImage);
bind(btnClose, 'click', hideOverlay);
@@ -473,6 +499,7 @@ window.baguetteBox = (function () {
unbind(document, 'keyup', keyUpHandler);
unbind(document, 'fullscreenchange', onFSC);
unbind(overlay, 'click', overlayClickHandler);
unbind(overlay, 'wheel', overlayWheelHandler);
unbind(btnPrev, 'click', showPreviousImage);
unbind(btnNext, 'click', showNextImage);
unbind(btnClose, 'click', hideOverlay);
@@ -583,7 +610,7 @@ window.baguetteBox = (function () {
isOverlayVisible = true;
}
function hideOverlay(e) {
function hideOverlay(e, dtor) {
ev(e);
playvid(false);
removeFromCache('#files');
@@ -592,19 +619,21 @@ window.baguetteBox = (function () {
document.body.style.overflowY = 'auto';
}
try {
if (document.fullscreenElement)
document.exitFullscreen();
}
catch (ex) { }
isFullscreen = false;
if (dtor || overlay.style.display === 'none')
return;
if (options.duringHide)
options.duringHide();
if (overlay.style.display === 'none')
return;
sethash('');
unbindEvents();
try {
document.exitFullscreen();
isFullscreen = false;
}
catch (ex) { }
// Fade out and hide the overlay
overlay.className = '';
@@ -711,8 +740,11 @@ window.baguetteBox = (function () {
});
image.setAttribute('src', imageSrc);
if (is_vid) {
image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
image.setAttribute('controls', 'controls');
image.onended = vidEnd;
image.onplay = function () { show_buttons(1); };
image.onpause = function () { show_buttons(); };
}
image.alt = thumbnailElement ? thumbnailElement.alt || '' : '';
if (options.titleTag && imageCaption)
@@ -958,6 +990,12 @@ window.baguetteBox = (function () {
}
}
function show_buttons(v) {
clmod(ebi('bbox-btns'), 'off', v);
clmod(btnPrev, 'off', v);
clmod(btnNext, 'off', v);
}
function bounceAnimation(direction) {
slider.className = options.animation == 'slideIn' ? 'bounce-from-' + direction : 'eog';
setTimeout(function () {
@@ -1021,10 +1059,8 @@ window.baguetteBox = (function () {
if (fx > 0.7)
return showNextImage();
clmod(ebi('bbox-btns'), 'off', 't');
clmod(btnPrev, 'off', 't');
clmod(btnNext, 'off', 't');
show_buttons('t');
if (Date.now() - ctime <= 500 && !IPHONE)
tglfull();
@@ -1065,6 +1101,7 @@ window.baguetteBox = (function () {
}
function destroyPlugin() {
hideOverlay(undefined, true);
unbindEvents();
clearCachedData();
document.getElementsByTagName('body')[0].removeChild(ebi('bbox-overlay'));

View File

@@ -28,6 +28,8 @@
--row-alt: #282828;
--scroll: #eb0;
--sel-fg: var(--bg-d1);
--sel-bg: var(--fg);
--a: #fc5;
--a-b: #c90;
@@ -267,6 +269,7 @@ html.bz {
--btn-bg: #202231;
--btn-h-bg: #2d2f45;
--btn-1-bg: #ba2959;
--btn-1-is: #f59;
--btn-1-fg: #fff;
--btn-1h-fg: #000;
--txt-sh: a;
@@ -330,6 +333,8 @@ html.c {
}
html.cz {
--bgg: var(--bg-u2);
--sel-bg: var(--bg-u5);
--sel-fg: var(--fg);
--srv-3: #fff;
--u2-tab-b1: var(--bg-d3);
}
@@ -343,6 +348,8 @@ html.cy {
--bg-d3: #f77;
--bg-d2: #ff0;
--sel-bg: #f77;
--a: #fff;
--a-hil: #fff;
--a-h-bg: #000;
@@ -588,8 +595,8 @@ html.dy {
line-height: 1.2em;
}
::selection {
color: var(--bg-d1);
background: var(--fg);
color: var(--sel-fg);
background: var(--sel-bg);
text-shadow: none;
}
html,body,tr,th,td,#files,a {
@@ -699,12 +706,12 @@ a:hover {
.s0:after,
.s1:after {
content: '⌄';
margin-left: -.1em;
margin-left: -.15em;
}
.s0r:after,
.s1r:after {
content: '⌃';
margin-left: -.1em;
margin-left: -.15em;
}
.s0:after,
.s0r:after {
@@ -715,7 +722,7 @@ a:hover {
color: var(--sort-2);
}
#files thead th:after {
margin-right: -.7em;
margin-right: -.5em;
}
#files tbody tr:hover td,
#files tbody tr:hover td+td {
@@ -744,6 +751,15 @@ html #files.hhpick thead th {
word-wrap: break-word;
overflow: hidden;
}
#files tr.fade a {
color: #999;
color: rgba(255, 255, 255, 0.4);
font-style: italic;
}
html.y #files tr.fade a {
color: #999;
color: rgba(0, 0, 0, 0.4);
}
#files tr:nth-child(2n) td {
background: var(--row-alt);
}
@@ -1600,6 +1616,7 @@ html {
padding: .2em .4em;
font-size: 1.2em;
margin: .2em;
display: inline-block;
white-space: pre;
position: relative;
top: -.12em;
@@ -1736,6 +1753,7 @@ html.y #tree.nowrap .ntree a+a:hover {
}
#files th span {
position: relative;
white-space: nowrap;
}
#files>thead>tr>th.min,
#files td.min {
@@ -1773,9 +1791,6 @@ html.y #tree.nowrap .ntree a+a:hover {
margin: .7em 0 .7em .5em;
padding-left: .5em;
}
.opwide>div>div>a {
line-height: 2em;
}
.opwide>div>h3 {
color: var(--fg-weak);
margin: 0 .4em;
@@ -3046,6 +3061,14 @@ html.b #ggrid>a {
html.b .btn {
top: -.1em;
}
html.b .btn,
html.b #u2conf a.b,
html.b #u2conf input[type="checkbox"]:not(:checked)+label {
box-shadow: 0 .05em 0 var(--bg-d3) inset;
}
html.b .tgl.btn.on {
box-shadow: 0 .05em 0 var(--btn-1-is) inset;
}
html.b #op_up2k.srch sup {
color: #fc0;
}

View File

@@ -6,7 +6,7 @@
<title>{{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
{{ html_head }}

View File

@@ -1,6 +1,8 @@
"use strict";
var XHR = XMLHttpRequest;
var XHR = XMLHttpRequest,
img_re = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i;
var Ls = {
"eng": {
"tt": "English",
@@ -139,7 +141,7 @@ var Ls = {
"ut_rand": "randomize filenames",
"ut_u2ts": "copy the last-modified timestamp$Nfrom your filesystem to the server",
"ut_mt": "continue hashing other files while uploading$N$Nmaybe disable if your CPU or HDD is a bottleneck",
"ut_ask": "ask for confirmation before upload starts",
"ut_ask": 'ask for confirmation before upload starts">💭',
"ut_pot": "improve upload speed on slow devices$Nby making the UI less complex",
"ut_srch": "don't actually upload, instead check if the files already $N exist on the server (will scan all folders you can read)",
"ut_par": "pause uploads by setting it to 0$N$Nincrease if your connection is slow / high latency$N$Nkeep it 1 on LAN or if the server HDD is a bottleneck",
@@ -192,20 +194,21 @@ var Ls = {
"cl_hpick": "tap on column headers to hide in the table below",
"cl_hcancel": "column hiding aborted",
"ct_thumb": "in grid-view, toggle icons or thumbnails$NHotkey: T",
"ct_csel": "use CTRL and SHIFT for file selection in grid-view",
"ct_ihop": "when the image viewer is closed, scroll down to the last viewed file",
"ct_dots": "show hidden files (if server permits)",
"ct_dir1st": "sort folders before files",
"ct_readme": "show README.md in folder listings",
"ct_idxh": "show index.html instead of folder listing",
"ct_sbars": "show scrollbars",
"ct_grid": '田 the grid',
"ct_thumb": 'in grid-view, toggle icons or thumbnails$NHotkey: T">🖼️ thumbs',
"ct_csel": 'use CTRL and SHIFT for file selection in grid-view">sel',
"ct_ihop": 'when the image viewer is closed, scroll down to the last viewed file">g⮯',
"ct_dots": 'show hidden files (if server permits)">dotfiles',
"ct_dir1st": 'sort folders before files">📁 first',
"ct_readme": 'show README.md in folder listings">📜 readme',
"ct_idxh": 'show index.html instead of folder listing">htm',
"ct_sbars": 'show scrollbars">⟊',
"cut_umod": "if a file already exists on the server, update the server's last-modified timestamp to match your local file (requires write+delete permissions)",
"cut_umod": "if a file already exists on the server, update the server's last-modified timestamp to match your local file (requires write+delete permissions)\">re📅",
"cut_turbo": "the yolo button, you probably DO NOT want to enable this:$N$Nuse this if you were uploading a huge amount of files and had to restart for some reason, and want to continue the upload ASAP$N$Nthis replaces the hash-check with a simple <em>&quot;does this have the same filesize on the server?&quot;</em> so if the file contents are different it will NOT be uploaded$N$Nyou should turn this off when the upload is done, and then &quot;upload&quot; the same files again to let the client verify them",
"cut_turbo": "the yolo button, you probably DO NOT want to enable this:$N$Nuse this if you were uploading a huge amount of files and had to restart for some reason, and want to continue the upload ASAP$N$Nthis replaces the hash-check with a simple <em>&quot;does this have the same filesize on the server?&quot;</em> so if the file contents are different it will NOT be uploaded$N$Nyou should turn this off when the upload is done, and then &quot;upload&quot; the same files again to let the client verify them\">turbo",
"cut_datechk": "has no effect unless the turbo button is enabled$N$Nreduces the yolo factor by a tiny amount; checks whether the file timestamps on the server matches yours$N$Nshould <em>theoretically</em> catch most unfinished / corrupted uploads, but is not a substitute for doing a verification pass with turbo disabled afterwards",
"cut_datechk": "has no effect unless the turbo button is enabled$N$Nreduces the yolo factor by a tiny amount; checks whether the file timestamps on the server matches yours$N$Nshould <em>theoretically</em> catch most unfinished / corrupted uploads, but is not a substitute for doing a verification pass with turbo disabled afterwards\">date-chk",
"cut_flag": "ensure only one tab is uploading at a time $N -- other tabs must have this enabled too $N -- only affects tabs on the same domain",
@@ -214,7 +217,7 @@ var Ls = {
"cut_nag": "OS notification when upload completes$N(only if the browser or tab is not active)",
"cut_sfx": "audible alert when upload completes$N(only if the browser or tab is not active)",
"cut_mt": "use multithreading to accelerate file hashing$N$Nthis uses web-workers and requires$Nmore RAM (up to 512 MiB extra)$N$N30% faster https, 4.5x faster http,$Nand 5.3x faster on android phones",
"cut_mt": "use multithreading to accelerate file hashing$N$Nthis uses web-workers and requires$Nmore RAM (up to 512 MiB extra)$N$N30% faster https, 4.5x faster http,$Nand 5.3x faster on android phones\">mt",
"cft_text": "favicon text (blank and refresh to disable)",
"cft_fg": "foreground color",
@@ -283,6 +286,7 @@ var Ls = {
"im_hnf": "that image no longer exists",
"f_empty": 'this folder is empty',
"f_chide": 'this will hide the column «{0}»\n\nyou can unhide columns in the settings tab',
"f_bigtxt": "this file is {0} MiB large -- really view as text?",
"fbd_more": '<div id="blazy">showing <code>{0}</code> of <code>{1}</code> files; <a href="#" id="bd_more">show {2}</a> or <a href="#" id="bd_all">show all</a></div>',
@@ -290,6 +294,8 @@ var Ls = {
"f_dls": 'the file links in the current folder have\nbeen changed into download links',
"f_partial": "To safely download a file which is currently being uploaded, please click the file which has the same filename, but without the <code>.PARTIAL</code> file extension. Please press CANCEL or Escape to do this.\n\nPressing OK / Enter will ignore this warning and continue downloading the <code>.PARTIAL</code> scratchfile instead, which will almost definitely give you corrupted data.",
"ft_paste": "paste {0} items$NHotkey: ctrl-V",
"fr_eperm": 'cannot rename:\nyou do not have “move” permission in this folder',
"fd_eperm": 'cannot delete:\nyou do not have “delete” permission in this folder',
@@ -335,6 +341,9 @@ var Ls = {
"fp_err": "move failed:\n",
"fp_confirm": "move these {0} items here?",
"fp_etab": 'failed to read clipboard from other browser tab',
"fp_name": "uploading a file from your device. Give it a name:",
"fp_both_m": '<h6>choose what to paste</h6><code>Enter</code> = Move {0} files from «{1}»\n<code>ESC</code> = Upload {2} files from your device',
"fp_both_b": '<a href="#" id="modal-ok">Move</a><a href="#" id="modal-ng">Upload</a>',
"mk_noname": "type a name into the text field on the left before you do that :p",
@@ -420,6 +429,7 @@ var Ls = {
"un_fclr": "clear filter",
"un_derr": 'unpost-delete failed:\n',
"un_f5": 'something broke, please try a refresh or hit F5',
"un_uf5": "sorry but you have to refresh the page (for example by pressing F5 or CTRL-R) before this upload can be aborted",
"un_nou": '<b>warning:</b> server too busy to show unfinished uploads; click the "refresh" link in a bit',
"un_noc": '<b>warning:</b> unpost of fully uploaded files is not enabled/permitted in server config',
"un_max": "showing first 2000 files (use the filter)",
@@ -641,7 +651,7 @@ var Ls = {
"ut_rand": "finn opp nye tilfeldige filnavn",
"ut_u2ts": "gi filen på serveren samme$Ntidsstempel som lokalt hos deg",
"ut_mt": "fortsett å befare køen mens opplastning foregår$N$Nskru denne av dersom du har en$Ntreg prosessor eller harddisk",
"ut_ask": "bekreft filutvalg før opplastning starter",
"ut_ask": 'bekreft filutvalg før opplastning starter">💭',
"ut_pot": "forbedre ytelsen på trege enheter ved å$Nforenkle brukergrensesnittet",
"ut_srch": "utfør søk istedenfor å laste opp --$Nleter igjennom alle mappene du har lov til å se",
"ut_par": "sett til 0 for å midlertidig stanse opplastning$N$Nhøye verdier (4 eller 8) kan gi bedre ytelse,$Nspesielt på trege internettlinjer$N$Nbør ikke være høyere enn 1 på LAN$Neller hvis serveren sin harddisk er treg",
@@ -694,20 +704,21 @@ var Ls = {
"cl_hpick": "klikk på overskriften til kolonnene du ønsker å skjule i tabellen nedenfor",
"cl_hcancel": "kolonne-skjuling avbrutt",
"ct_thumb": "vis miniatyrbilder istedenfor ikoner$NSnarvei: T",
"ct_csel": "bruk tastene CTRL og SHIFT for markering av filer i ikonvisning",
"ct_ihop": "bla ned til sist viste bilde når bildeviseren lukkes",
"ct_dots": "vis skjulte filer (gitt at serveren tillater det)",
"ct_dir1st": "sorter slik at mapper kommer foran filer",
"ct_readme": "vis README.md nedenfor filene",
"ct_idxh": "vis index.html istedenfor fil-liste",
"ct_sbars": "vis rullgardiner / skrollefelt",
"ct_grid": '田 ikoner',
"ct_thumb": 'vis miniatyrbilder istedenfor ikoner$NSnarvei: T">🖼️ bilder',
"ct_csel": 'bruk tastene CTRL og SHIFT for markering av filer i ikonvisning">merk',
"ct_ihop": 'bla ned til sist viste bilde når bildeviseren lukkes">g⮯',
"ct_dots": 'vis skjulte filer (gitt at serveren tillater det)">.synlig',
"ct_dir1st": 'sorter slik at mapper kommer foran filer">📁 først',
"ct_readme": 'vis README.md nedenfor filene">📜 readme',
"ct_idxh": 'vis index.html istedenfor fil-liste">htm',
"ct_sbars": 'vis rullgardiner / skrollefelt">⟊',
"cut_umod": "i tilfelle en fil du laster opp allerede finnes på serveren, så skal serverens tidsstempel oppdateres slik at det stemmer overens med din lokale fil (krever rettighetene write+delete)",
"cut_umod": 'i tilfelle en fil du laster opp allerede finnes på serveren, så skal serverens tidsstempel oppdateres slik at det stemmer overens med din lokale fil (krever rettighetene write+delete)">re📅',
"cut_turbo": "forenklet befaring ved opplastning; bør sannsynlig <em>ikke</em> skrus på:$N$Nnyttig dersom du var midt i en svær opplastning som måtte restartes av en eller annen grunn, og du vil komme igang igjen så raskt som overhodet mulig.$N$Nnår denne er skrudd på så forenkles befaringen kraftig; istedenfor å utføre en trygg sjekk på om filene finnes på serveren i god stand, så sjekkes kun om <em>filstørrelsen</em> stemmer. Så dersom en korrupt fil skulle befinne seg på serveren allerede, på samme sted med samme størrelse og navn, så blir det <em>ikke oppdaget</em>.$N$Ndet anbefales å kun benytte denne funksjonen for å komme seg raskt igjennom selve opplastningen, for så å skru den av, og til slutt &quot;laste opp&quot; de samme filene én gang til -- slik at integriteten kan verifiseres",
"cut_turbo": "forenklet befaring ved opplastning; bør sannsynlig <em>ikke</em> skrus på:$N$Nnyttig dersom du var midt i en svær opplastning som måtte restartes av en eller annen grunn, og du vil komme igang igjen så raskt som overhodet mulig.$N$Nnår denne er skrudd på så forenkles befaringen kraftig; istedenfor å utføre en trygg sjekk på om filene finnes på serveren i god stand, så sjekkes kun om <em>filstørrelsen</em> stemmer. Så dersom en korrupt fil skulle befinne seg på serveren allerede, på samme sted med samme størrelse og navn, så blir det <em>ikke oppdaget</em>.$N$Ndet anbefales å kun benytte denne funksjonen for å komme seg raskt igjennom selve opplastningen, for så å skru den av, og til slutt &quot;laste opp&quot; de samme filene én gang til -- slik at integriteten kan verifiseres\">turbo",
"cut_datechk": "har ingen effekt dersom turbo er avslått$N$Ngjør turbo bittelitt tryggere ved å sjekke datostemplingen på filene (i tillegg til filstørrelse)$N$N<em>burde</em> oppdage og gjenoppta de fleste ufullstendige opplastninger, men er <em>ikke</em> en fullverdig erstatning for å deaktivere turbo og gjøre en skikkelig sjekk",
"cut_datechk": "har ingen effekt dersom turbo er avslått$N$Ngjør turbo bittelitt tryggere ved å sjekke datostemplingen på filene (i tillegg til filstørrelse)$N$N<em>burde</em> oppdage og gjenoppta de fleste ufullstendige opplastninger, men er <em>ikke</em> en fullverdig erstatning for å deaktivere turbo og gjøre en skikkelig sjekk\">date-chk",
"cut_flag": "samkjører nettleserfaner slik at bare én $N kan holde på med befaring / opplastning $N -- andre faner må også ha denne skrudd på $N -- fungerer kun innenfor samme domene",
@@ -716,7 +727,7 @@ var Ls = {
"cut_nag": "meldingsvarsel når opplastning er ferdig$N(kun on nettleserfanen ikke er synlig)",
"cut_sfx": "lydvarsel når opplastning er ferdig$N(kun on nettleserfanen ikke er synlig)",
"cut_mt": "raskere befaring ved å bruke hele CPU'en$N$Ndenne funksjonen anvender web-workers$Nog krever mer RAM (opptil 512 MiB ekstra)$N$N30% raskere https, 4.5x raskere http,$Nog 5.3x raskere på android-telefoner",
"cut_mt": "raskere befaring ved å bruke hele CPU'en$N$Ndenne funksjonen anvender web-workers$Nog krever mer RAM (opptil 512 MiB ekstra)$N$N30% raskere https, 4.5x raskere http,$Nog 5.3x raskere på android-telefoner\">mt",
"cft_text": "ikontekst (blank ut og last siden på nytt for å deaktivere)",
"cft_fg": "farge",
@@ -785,6 +796,7 @@ var Ls = {
"im_hnf": "bildet finnes ikke lenger",
"f_empty": 'denne mappen er tom',
"f_chide": 'dette vil skjule kolonnen «{0}»\n\nfanen for "andre innstillinger" lar deg vise kolonnen igjen',
"f_bigtxt": "denne filen er hele {0} MiB -- vis som tekst?",
"fbd_more": '<div id="blazy">viser <code>{0}</code> av <code>{1}</code> filer; <a href="#" id="bd_more">vis {2}</a> eller <a href="#" id="bd_all">vis alle</a></div>',
@@ -792,6 +804,8 @@ var Ls = {
"f_dls": 'linkene i denne mappen er nå\nomgjort til nedlastningsknapper',
"f_partial": "For å laste ned en fil som enda ikke er ferdig opplastet, klikk på filen som har samme filnavn som denne, men uten <code>.PARTIAL</code> på slutten. Da vil serveren passe på at nedlastning går bra. Derfor anbefales det sterkt å trykke ABRYT eller Escape-tasten.\n\nHvis du virkelig ønsker å laste ned denne <code>.PARTIAL</code>-filen på en ukontrollert måte, trykk OK / Enter for å ignorere denne advarselen. Slik vil du høyst sannsynlig motta korrupt data.",
"ft_paste": "Lim inn {0} filer$NSnarvei: ctrl-V",
"fr_eperm": 'kan ikke endre navn:\ndu har ikke “move”-rettigheten i denne mappen',
"fd_eperm": 'kan ikke slette:\ndu har ikke “delete”-rettigheten i denne mappen',
@@ -837,6 +851,9 @@ var Ls = {
"fp_err": "flytting feilet:\n",
"fp_confirm": "flytt disse {0} filene hit?",
"fp_etab": 'kunne ikke lese listen med filer ifra den andre nettleserfanen',
"fp_name": "Laster opp én fil fra enheten din. Velg filnavn:",
"fp_both_m": '<h6>hva skal limes inn her?</h6><code>Enter</code> = Flytt {0} filer fra «{1}»\n<code>ESC</code> = Last opp {2} filer fra enheten din',
"fp_both_b": '<a href="#" id="modal-ok">Flytt</a><a href="#" id="modal-ng">Last opp</a>',
"mk_noname": "skriv inn et navn i tekstboksen til venstre først :p",
@@ -922,6 +939,7 @@ var Ls = {
"un_fclr": "nullstill filter",
"un_derr": 'unpost-sletting feilet:\n',
"un_f5": 'noe gikk galt, prøv å oppdatere listen eller trykk F5',
"un_uf5": "beklager, men du må laste siden på nytt (f.eks. ved å trykke F5 eller CTRL-R) før denne opplastningen kan avbrytes",
"un_nou": '<b>advarsel:</b> kan ikke vise ufullstendige opplastninger akkurat nå; klikk på oppdater-linken om litt',
"un_noc": '<b>advarsel:</b> angring av fullførte opplastninger er deaktivert i serverkonfigurasjonen',
"un_max": "viser de første 2000 filene (bruk filteret for å innsnevre)",
@@ -1006,8 +1024,13 @@ var Ls = {
"lang_set": "passer det å laste siden på nytt?",
},
};
var LANGS = ["eng", "nor"],
L = Ls[sread("cpp_lang", LANGS) || lang] || Ls.eng || Ls.nor;
var LANGS = ["eng", "nor"];
if (window.langmod)
langmod();
var L = Ls[sread("cpp_lang", LANGS) || lang] || Ls.eng || Ls.nor;
for (var a = 0; a < LANGS.length; a++) {
for (var b = a + 1; b < LANGS.length; b++) {
@@ -1194,15 +1217,15 @@ ebi('op_cfg').innerHTML = (
' <h3>' + L.cl_opts + '</h3>\n' +
' <div>\n' +
' <a id="tooltips" class="tgl btn" href="#" tt="◔ ◡ ◔"> tooltips</a>\n' +
' <a id="griden" class="tgl btn" href="#" tt="' + L.wt_grid + '">田 the grid</a>\n' +
' <a id="thumbs" class="tgl btn" href="#" tt="' + L.ct_thumb + '">🖼️ thumbs</a>\n' +
' <a id="csel" class="tgl btn" href="#" tt="' + L.ct_csel + '">sel</a>\n' +
' <a id="ihop" class="tgl btn" href="#" tt="' + L.ct_ihop + '">g⮯</a>\n' +
' <a id="dotfiles" class="tgl btn" href="#" tt="' + L.ct_dots + '">dotfiles</a>\n' +
' <a id="dir1st" class="tgl btn" href="#" tt="' + L.ct_dir1st + '">📁 first</a>\n' +
' <a id="ireadme" class="tgl btn" href="#" tt="' + L.ct_readme + '">📜 readme</a>\n' +
' <a id="idxh" class="tgl btn" href="#" tt="' + L.ct_idxh + '">htm</a>\n' +
' <a id="sbars" class="tgl btn" href="#" tt="' + L.ct_sbars + '">⟊</a>\n' +
' <a id="griden" class="tgl btn" href="#" tt="' + L.wt_grid + '">' + L.ct_grid + '</a>\n' +
' <a id="thumbs" class="tgl btn" href="#" tt="' + L.ct_thumb + '</a>\n' +
' <a id="csel" class="tgl btn" href="#" tt="' + L.ct_csel + '</a>\n' +
' <a id="ihop" class="tgl btn" href="#" tt="' + L.ct_ihop + '</a>\n' +
' <a id="dotfiles" class="tgl btn" href="#" tt="' + L.ct_dots + '</a>\n' +
' <a id="dir1st" class="tgl btn" href="#" tt="' + L.ct_dir1st + '</a>\n' +
' <a id="ireadme" class="tgl btn" href="#" tt="' + L.ct_readme + '</a>\n' +
' <a id="idxh" class="tgl btn" href="#" tt="' + L.ct_idxh + '</a>\n' +
' <a id="sbars" class="tgl btn" href="#" tt="' + L.ct_sbars + '</a>\n' +
' </div>\n' +
'</div>\n' +
'<div>\n' +
@@ -1221,11 +1244,11 @@ ebi('op_cfg').innerHTML = (
'<div>\n' +
' <h3>' + L.cl_uopts + '</h3>\n' +
' <div>\n' +
' <a id="ask_up" class="tgl btn" href="#" tt="' + L.ut_ask + '">💭</a>\n' +
' <a id="umod" class="tgl btn" href="#" tt="' + L.cut_umod + '">re📅</a>\n' +
' <a id="hashw" class="tgl btn" href="#" tt="' + L.cut_mt + '">mt</a>\n' +
' <a id="u2turbo" class="tgl btn ttb" href="#" tt="' + L.cut_turbo + '">turbo</a>\n' +
' <a id="u2tdate" class="tgl btn ttb" href="#" tt="' + L.cut_datechk + '">date-chk</a>\n' +
' <a id="ask_up" class="tgl btn" href="#" tt="' + L.ut_ask + '</a>\n' +
' <a id="umod" class="tgl btn" href="#" tt="' + L.cut_umod + '</a>\n' +
' <a id="hashw" class="tgl btn" href="#" tt="' + L.cut_mt + '</a>\n' +
' <a id="u2turbo" class="tgl btn ttb" href="#" tt="' + L.cut_turbo + '</a>\n' +
' <a id="u2tdate" class="tgl btn ttb" href="#" tt="' + L.cut_datechk + '</a>\n' +
' <a id="flag_en" class="tgl btn" href="#" tt="' + L.cut_flag + '">💤</a>\n' +
' <a id="u2sort" class="tgl btn" href="#" tt="' + L.cut_az + '">az</a>\n' +
' <a id="upnag" class="tgl btn" href="#" tt="' + L.cut_nag + '">🔔</a>\n' +
@@ -1407,6 +1430,12 @@ var ACtx = !IPHONE && (window.AudioContext || window.webkitAudioContext),
dk, mp;
if (window.og_fn) {
hash0 = 1;
hist_replace(vsplit(get_evpath())[0]);
}
var mpl = (function () {
var have_mctl = 'mediaSession' in navigator && window.MediaMetadata;
@@ -1539,22 +1568,24 @@ var mpl = (function () {
set_tint();
r.acode = function (url) {
var c = true;
var c = true,
cs = url.split('?')[0];
if (!have_acode)
c = false;
else if (/\.(wav|flac)$/i.exec(url))
else if (/\.(wav|flac)$/i.exec(cs))
c = r.ac_flac;
else if (/\.(aac|m4a)$/i.exec(url))
else if (/\.(aac|m4a)$/i.exec(cs))
c = r.ac_aac;
else if (/\.(ogg|opus)$/i.exec(url) && !can_ogg)
else if (/\.(ogg|opus)$/i.exec(cs) && !can_ogg)
c = true;
else if (re_au_native.exec(url))
else if (re_au_native.exec(cs))
c = false;
if (!c)
return url;
return addq(url, 'th=') + (can_ogg ? 'opus' : (IPHONE || MACOS) ? 'caf' : 'mp3');
return addq(url, 'th=' + (can_ogg ? 'opus' : (IPHONE || MACOS) ? 'caf' : 'mp3'));
};
r.pp = function () {
@@ -1669,7 +1700,7 @@ catch (ex) { }
var re_au_native = (can_ogg || have_acode) ? /\.(aac|flac|m4a|mp3|ogg|opus|wav)$/i : /\.(aac|flac|m4a|mp3|wav)$/i,
re_au_all = /\.(aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk)$/i;
re_au_all = /\.(aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|itgz|itxz|itz|m4a|mdgz|mdxz|mdz|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|s3gz|s3xz|s3z|tak|tta|ulaw|wav|wma|wv|xm|xmgz|xmxz|xmz|xpk)$/i;
// extract songs + add play column
@@ -1706,7 +1737,7 @@ function MPlayer() {
}
}
r.vol = clamp(fcfg_get('vol', IPHONE ? 1 : 0.5), 0, 1);
r.vol = clamp(fcfg_get('vol', IPHONE ? 1 : dvol / 100), 0, 1);
r.expvol = function (v) {
return 0.5 * v + 0.5 * v * v;
@@ -1804,7 +1835,7 @@ function MPlayer() {
r.preload = function (url, full) {
var t0 = Date.now(),
fname = uricom_dec(url.split('/').pop());
fname = uricom_dec(url.split('/').pop().split('?')[0]);
url = addq(mpl.acode(url), 'cache=987&_=' + ACB);
mpl.preload_url = full ? url : null;
@@ -2016,8 +2047,8 @@ var pbar = (function () {
r.buf = canvas_cfg(ebi('barbuf'));
r.pos = canvas_cfg(ebi('barpos'));
r.buf.ctx.font = '.5em sans-serif';
r.pos.ctx.font = '1em sans-serif';
r.pos.ctx.strokeStyle = 'rgba(24,56,0,0.4)';
r.pos.ctx.font = '.9em sans-serif';
r.pos.ctx.strokeStyle = 'rgba(24,56,0,0.5)';
r.drawbuf();
r.drawpos();
if (!r.pos.can.onmouseleave)
@@ -2183,7 +2214,7 @@ var pbar = (function () {
var m1 = pctx.measureText(t1),
m1b = pctx.measureText(t1 + ":88"),
m2 = pctx.measureText(t2),
yt = pc.h / 3 * 2.1,
yt = pc.h * 0.94,
xt1 = pc.w - (m1.width + 12),
xt2 = x < m1.width * 1.4 ? (x + 12) : (Math.min(pc.w - m1b.width, x - 12) - m2.width);
@@ -2534,7 +2565,7 @@ var mpui = (function () {
rem = pos > 1 ? len - pos : 999,
full = null;
if (rem < 7 || (!mpl.fullpre && (rem < 40 || pos > 10))) {
if (rem < 7 || (!mpl.fullpre && (rem < 40 || (rem < 90 && pos > 10)))) {
preloaded = fpreloaded = mp.au.rsrc;
full = false;
}
@@ -3228,7 +3259,7 @@ function autoplay_blocked(seek) {
var tid = mp.au.tid,
fn = mp.tracks[tid].split(/\//).pop();
fn = uricom_dec(fn.replace(/\+/g, ' '));
fn = uricom_dec(fn.replace(/\+/g, ' ').split('?')[0]);
modal.confirm('<h6>' + L.mm_hashplay + '</h6>\n«' + esc(fn) + '»', function () {
// chrome 91 may permanently taint on a failed play()
@@ -3275,6 +3306,21 @@ function scan_hash(v) {
function eval_hash() {
window.onpopstate = treectl.onpopfun;
if (hash0 && window.og_fn) {
var all = msel.getall(), mi;
for (var a = 0; a < all.length; a++)
if (og_fn == uricom_dec(vsplit(all[a].vp)[1].split('?')[0])) {
mi = all[a];
break;
}
if (mi && img_re.exec(og_fn))
hash0 = '#g' + mi.id;
if (ebi('a' + mi.id))
hash0 = '#a' + mi.id;
}
var v = hash0;
hash0 = null;
if (!v)
@@ -3588,6 +3634,7 @@ var fileman = (function () {
bdel = ebi('fdel'),
bcut = ebi('fcut'),
bpst = ebi('fpst'),
t_paste,
r = {};
r.clip = null;
@@ -4065,8 +4112,73 @@ var fileman = (function () {
}
};
r.paste = function (e) {
ev(e);
document.onpaste = function (e) {
var xfer = e.clipboardData || window.clipboardData;
if (!xfer || !xfer.files || !xfer.files.length)
return;
var files = [];
for (var a = 0, aa = xfer.files.length; a < aa; a++)
files.push(xfer.files[a]);
clearTimeout(t_paste);
if (!r.clip.length)
return r.clip_up(files);
var src = r.clip.length == 1 ? r.clip[0] : vsplit(r.clip[0])[0],
msg = L.fp_both_m.format(r.clip.length, src, files.length);
modal.confirm(msg, r.paste, function () { r.clip_up(files); }, null, L.fp_both_b);
};
r.clip_up = function (files) {
goto_up2k();
var good = [], nil = [], bad = [];
for (var a = 0, aa = files.length; a < aa; a++) {
var fobj = files[a], dst = good;
try {
if (fobj.size < 1)
dst = nil;
}
catch (ex) {
dst = bad;
}
dst.push([fobj, fobj.name]);
}
var doit = function (is_img) {
jwrite('fman_clip', [Date.now()]);
r.clip = [];
var x = up2k.uc.ask_up;
if (is_img)
up2k.uc.ask_up = false;
up2k.gotallfiles[0](good, nil, bad, up2k.gotallfiles.slice(1));
up2k.uc.ask_up = x;
};
if (good.length != 1)
return doit();
var fn = good[0][1],
ofs = fn.lastIndexOf('.');
// stop linux-chrome from adding the fs-path into the <input>
setTimeout(function () {
modal.prompt(L.fp_name, fn, function (v) {
good[0][1] = v;
doit(true);
}, null, null, 0, ofs > 0 ? ofs : undefined);
}, 1);
};
r.d_paste = function () {
// gets called before onpaste; defer
clearTimeout(t_paste);
t_paste = setTimeout(r.paste, 50);
};
r.paste = function () {
if (clgot(bpst, 'hide'))
return toast.err(3, L.fp_eperm);
@@ -4286,7 +4398,7 @@ var showfile = (function () {
r.show = function (url, no_push) {
var xhr = new XHR(),
m = /[?&](k=[^&]+)/.exec(url);
m = /[?&](k=[^&#]+)/.exec(url);
url = url.split('?')[0] + (m ? '?' + m[1] : '');
xhr.url = url;
@@ -4675,8 +4787,8 @@ var thegrid = (function () {
fid = oth.getAttribute('id'),
aplay = ebi('a' + fid),
atext = ebi('t' + fid),
is_txt = atext && showfile.getlang(href),
is_img = /\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp|webm|mkv|mp4)(\?|$)/i.test(href),
is_txt = atext && showfile.getlang(href) && !/\.ts$/.test(href),
is_img = img_re.test(href),
is_dir = href.endsWith('/'),
is_srch = !!ebi('unsearch'),
in_tree = is_dir && treectl.find(oth.textContent.slice(0, -1)),
@@ -4826,7 +4938,7 @@ var thegrid = (function () {
ihref = ohref;
if (r.thumbs) {
ihref = addq(ihref, 'th=') + (have_webp ? 'w' : 'j');
ihref = addq(ihref, 'th=' + (have_webp ? 'w' : 'j'));
if (!r.crop)
ihref += 'f';
if (r.x3)
@@ -5170,6 +5282,9 @@ document.onkeydown = function (e) {
if (ebi('hkhelp'))
return qsr('#hkhelp');
if (toast.visible)
return toast.hide();
if (ebi('rn_cancel'))
return ebi('rn_cancel').click();
@@ -5242,7 +5357,7 @@ document.onkeydown = function (e) {
return fileman.cut();
if (k == 'KeyV' || k == 'v')
return fileman.paste();
return fileman.d_paste();
if (k == 'KeyK' || k == 'k')
return fileman.delete();
@@ -5975,14 +6090,14 @@ var treectl = (function () {
function get_tree(top, dst, rst) {
var xhr = new XHR(),
m = /[?&](k=[^&]+)/.exec(dst),
m = /[?&](k=[^&#]+)/.exec(dst),
k = m ? '&' + m[1] : dk ? '&k=' + dk : '';
xhr.top = top;
xhr.dst = dst;
xhr.rst = rst;
xhr.ts = Date.now();
xhr.open('GET', addq(dst, 'tree=') + top + (r.dots ? '&dots' : '') + k, true);
xhr.open('GET', addq(dst, 'tree=' + top + (r.dots ? '&dots' : '') + k), true);
xhr.onload = xhr.onerror = recvtree;
xhr.send();
enspin('#tree');
@@ -6071,7 +6186,7 @@ var treectl = (function () {
cl = '';
if (dk && ehref == cevp && !/[?&]k=/.exec(qhref))
links[a].setAttribute('href', addq(qhref, 'k=') + dk);
links[a].setAttribute('href', addq(qhref, 'k=' + dk));
if (href == cdir) {
act = links[a];
@@ -6172,7 +6287,7 @@ var treectl = (function () {
return window.location = url;
var xhr = new XHR(),
m = /[?&](k=[^&]+)/.exec(url),
m = /[?&](k=[^&#]+)/.exec(url),
k = m ? '&' + m[1] : dk ? '&k=' + dk : '';
xhr.top = url.split('?')[0];
@@ -6258,7 +6373,7 @@ var treectl = (function () {
for (var a = 0; a < res.dirs.length; a++) {
var dh = res.dirs[a].href,
dn = dh.split('/')[0].split('?')[0],
m = /[?&](k=[^&]+)/.exec(dh);
m = /[?&](k=[^&#]+)/.exec(dh);
if (m)
dn += '?' + m[1];
@@ -6278,6 +6393,9 @@ var treectl = (function () {
lg1 = res.logues ? res.logues[1] || "" : "",
dirchg = get_evpath() != cdir;
if (lg1 === Ls.eng.f_empty)
lg1 = L.f_empty;
sandbox(ebi('pro'), sb_lg, '', lg0);
if (dirchg)
sandbox(ebi('epi'), sb_lg, '', lg1);
@@ -6379,8 +6497,9 @@ var treectl = (function () {
'" class="doc' + (lang ? ' bri' : '') +
'" hl="' + id + '" name="' + hname + '">-txt-</a>';
var ln = ['<tr><td>' + tn.lead + '</td><td><a href="' +
top + tn.href + '" id="' + id + '">' + hname + '</a>', tn.sz];
var cl = /\.PARTIAL$/.exec(fname) ? ' class="fade"' : '',
ln = ['<tr' + cl + '><td>' + tn.lead + '</td><td><a href="' +
top + tn.href + '" id="' + id + '">' + hname + '</a>', tn.sz];
for (var b = 0; b < res.taglist.length; b++) {
var k = res.taglist[b],
@@ -6630,7 +6749,7 @@ var treectl = (function () {
hbase = req,
cbase = location.pathname,
mdoc = /[?&]doc=/.exec('' + url),
mdk = /[?&](k=[^&]+)/.exec('' + url);
mdk = /[?&](k=[^&#]+)/.exec('' + url);
if (mdoc && hbase == cbase)
return showfile.show(hbase + showfile.sname(url.search), true);
@@ -7267,7 +7386,7 @@ var arcfmt = (function () {
if (!/^(zip|tar|pax|tgz|txz)$/.exec(txt))
continue;
var m = /(.*[?&])(tar|zip)([^&]*)(.*)$/.exec(href);
var m = /(.*[?&])(tar|zip)([^&#]*)(.*)$/.exec(href);
if (!m)
throw new Error('missing arg in url');
@@ -7343,7 +7462,7 @@ var msel = (function () {
item.vp = href.indexOf('/') !== -1 ? href : vbase + href;
if (dk) {
var m = /[?&](k=[^&]+)/.exec(qhref);
var m = /[?&](k=[^&#]+)/.exec(qhref);
item.q = m ? '?' + m[1] : '';
}
else item.q = '';
@@ -7914,6 +8033,9 @@ window.addEventListener("message", function (e) {
if (sb_lg && logues.length) {
if (logues[1] === Ls.eng.f_empty)
logues[1] = L.f_empty;
sandbox(ebi('pro'), sb_lg, '', logues[0]);
sandbox(ebi('epi'), sb_lg, '', logues[1]);
}
@@ -8090,7 +8212,17 @@ var unpost = (function () {
if (!links.length)
continue;
req.push(uricom_dec(r.files[a].vp.split('?')[0]));
var f = r.files[a];
if (f.k == 'u') {
var vp = vsplit(f.vp.split('?')[0]),
dfn = uricom_dec(vp[1]);
for (var iu = 0; iu < up2k.st.files.length; iu++) {
var uf = up2k.st.files[iu];
if (uf.name == dfn && uf.purl == vp[0])
return modal.alert(L.un_uf5);
}
}
req.push(uricom_dec(f.vp.split('?')[0]));
for (var b = 0; b < links.length; b++) {
links[b].removeAttribute('href');
links[b].innerHTML = '[busy]';
@@ -8213,6 +8345,13 @@ ebi('files').onclick = ebi('docul').onclick = function (e) {
treectl.reqls(tgt.getAttribute('href'), true);
return ev(e);
}
if (tgt && /\.PARTIAL(\?|$)/.exec('' + tgt.getAttribute('href')) && !window.partdlok) {
ev(e);
modal.confirm(L.f_partial, function () {
window.partdlok = 1;
tgt.click();
}, null);
}
tgt = e.target.closest('a[hl]');
if (tgt) {
@@ -8281,7 +8420,7 @@ function reload_browser() {
for (var a = 0; a < parts.length - 1; a++) {
link += parts[a] + '/';
var link2 = dks[link] ? addq(link, 'k=') + dks[link] : link;
var link2 = dks[link] ? addq(link, 'k=' + dks[link]) : link;
o = mknod('a');
o.setAttribute('href', link2);

View File

@@ -3,7 +3,7 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
{%- if edit %}

View File

@@ -607,10 +607,10 @@ function md_newline() {
var s = linebounds(true),
ln = s.md.substring(s.n1, s.n2),
m1 = /^( *)([0-9]+)(\. +)/.exec(ln),
m2 = /^[ \t>+-]*(\* )?/.exec(ln),
m2 = /^[ \t]*[>+*-]{0,2}[ \t]/.exec(ln),
drop = dom_src.selectionEnd - dom_src.selectionStart;
var pre = m2[0];
var pre = m2 ? m2[0] : '';
if (m1 !== null)
pre = m1[1] + (parseInt(m1[2]) + 1) + m1[3];

View File

@@ -3,7 +3,7 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/mde.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/deps/mini-fa.css?_={{ ts }}">

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
{{ html_head }}
</head>

View File

@@ -190,11 +190,21 @@ input {
padding: .5em .7em;
margin: 0 .5em 0 0;
}
input::placeholder {
font-size: 1.2em;
font-style: italic;
letter-spacing: .04em;
opacity: 0.64;
color: #930;
}
html.z input {
color: #fff;
background: #626;
border-color: #c2c;
}
html.z input::placeholder {
color: #fff;
}
html.z .num {
border-color: #777;
}

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
@@ -94,7 +94,8 @@
<div>
<form method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
<input type="hidden" name="act" value="login" />
<input type="password" name="cppwd" />
<input type="password" name="cppwd" placeholder=" password" />
<input type="hidden" name="uhash" id="uhash" value="x" />
<input type="submit" value="Login" />
{% if ahttps %}
<a id="w" href="{{ ahttps }}">switch to https</a>

View File

@@ -34,8 +34,14 @@ var Ls = {
"u2": "time since the last server write$N( upload / rename / ... )$N$N17d = 17 days$N1h23 = 1 hour 23 minutes$N4m56 = 4 minutes 56 seconds",
"v2": "use this server as a local HDD$N$NWARNING: this will show your password!",
}
},
d = Ls[sread("cpp_lang", ["eng", "nor"]) || lang] || Ls.eng || Ls.nor;
};
var LANGS = ["eng", "nor"];
if (window.langmod)
langmod();
var d = Ls[sread("cpp_lang", LANGS) || lang] || Ls.eng || Ls.nor;
for (var k in (d || {})) {
var f = k.slice(-1),
@@ -66,3 +72,5 @@ if (!ebi('c') && o.offsetTop + o.offsetHeight < window.innerHeight)
o = ebi('u');
if (o && /[0-9]+$/.exec(o.innerHTML))
o.innerHTML = shumantime(o.innerHTML);
ebi('uhash').value = '' + location.hash;

View File

@@ -6,7 +6,7 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#333">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<style>ul{padding-left:1.3em}li{margin:.4em 0}</style>

View File

@@ -148,6 +148,8 @@ try {
hook('error');
}
catch (ex) {
if (console.stdlog)
console.log = console.stdlog;
console.log('console capture failed', ex);
}
var crashed = false, ignexd = {}, evalex_fatal = false;
@@ -736,7 +738,11 @@ function vjoin(p1, p2) {
function addq(url, q) {
return url + (url.indexOf('?') < 0 ? '?' : '&') + (q === undefined ? '' : q);
var uh = url.split('#', 1),
u = uh[0],
h = uh.length == 1 ? '' : '#' + uh[1];
return u + (u.indexOf('?') < 0 ? '?' : '&') + (q === undefined ? '' : q) + h;
}
@@ -1533,6 +1539,8 @@ var modal = (function () {
cb_up = null,
cb_ok = null,
cb_ng = null,
sel_0 = 0,
sel_1 = 0,
tok, tng, prim, sec, ok_cancel;
r.load = function () {
@@ -1566,7 +1574,7 @@ var modal = (function () {
(inp || a).focus();
if (inp)
setTimeout(function () {
inp.setSelectionRange(0, inp.value.length, "forward");
inp.setSelectionRange(sel_0, sel_1, "forward");
}, 0);
document.addEventListener('focus', onfocus);
@@ -1689,16 +1697,18 @@ var modal = (function () {
r.show(html);
}
r.prompt = function (html, v, cok, cng, fun) {
r.prompt = function (html, v, cok, cng, fun, so0, so1) {
q.push(function () {
_prompt(lf2br(html), v, cok, cng, fun);
_prompt(lf2br(html), v, cok, cng, fun, so0, so1);
});
next();
}
var _prompt = function (html, v, cok, cng, fun) {
var _prompt = function (html, v, cok, cng, fun, so0, so1) {
cb_ok = cok;
cb_ng = cng === undefined ? cok : null;
cb_up = fun;
sel_0 = so0 || 0;
sel_1 = so1 === undefined ? v.length : so1;
html += '<input id="modali" type="text" ' + NOAC + ' /><div id="modalb">' + ok_cancel + '</div>';
r.show(html);

View File

@@ -1,3 +1,179 @@
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0510-1431 `v1.13.2` s3xmodit.zip
## new features
* play [compressed](https://a.ocv.me/pub/demo/music/chiptunes/compressed/#af-99f0c0e4) s3xmodit chiptunes/modules c0466279
* can now read gz/xz/zip-compressed s3m/xm/mod/it songs
* new filetypes supported: mdz, mdgz, mdxz, s3z, s3gz, s3xz, xmz, xmgz, xmxz, itz, itgz, itxz
* and if you need to fit even more tracks on the mixtape, [try mo3](https://a.ocv.me/pub/demo/music/chiptunes/compressed/#af-0bc9b877)
* option to batch-convert audio waveforms 38e4fdfe
* volflag to improve audio waveform compression with pngquant 82ce6862
* option to add or change mappings from file-extensions to mimetypes 560d7b66
* export and publish the `--help` text for online viewing 560d7b66
* now available [as html](https://ocv.me/copyparty/helptext.html) and as [plaintext](https://ocv.me/copyparty/helptext.txt), includes many features not documented in the readme
* another way to add your own UI translations 19d156ff
## bugfixes
* ensure OS signals are immediately received and processed 87c60a1e
* things like reload and shutdown signals from systemd could get lost/stuck
* fix mimetype detection for uppercase file extensions 565daee9
* when clicking a `.ts` file in the gridview, don't open it as text 925c7f0a
* ...as it's probably an mpeg transport-stream, not a typescript file
* be less aggressive in dropping volume caches e396c5c2
* very minor performance gain, only really relevant if you're doing something like burning a copyparty volume onto a CD
* previously, adding or removing any volume at all was enough to drop covers cache for all volumes; now this only happens if an intersecting volume is added/removed
## other changes
* updated dompurify to 3.1.2 566cbb65
* opengraph: add the full filename as url suffix 5c1e2390
* so discord picks a good filename when saving an image
----
# 💾 what to download?
| download link | is it good? | description |
| -- | -- | -- |
| **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** | ✅ the best 👍 | runs anywhere! only needs python |
| [a docker image](https://github.com/9001/copyparty/blob/hovudstraum/scripts/docker/README.md) | it's ok | good if you prefer docker 🐋 |
| [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) | ⚠️ [acceptable](https://github.com/9001/copyparty#copypartyexe) | for [win8](https://user-images.githubusercontent.com/241032/221445946-1e328e56-8c5b-44a9-8b9f-dee84d942535.png) or later; built-in thumbnailer |
| [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.13.0/u2c.exe) | ⚠️ acceptable | [CLI uploader](https://github.com/9001/copyparty/blob/hovudstraum/bin/u2c.py) as a win7+ exe ([video](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm)) |
| [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) | ⚠️ acceptable | similar to the regular sfx, [mostly worse](https://github.com/9001/copyparty#zipapp) |
| [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) | ⛔️ [dangerous](https://github.com/9001/copyparty#copypartyexe) | for [win7](https://user-images.githubusercontent.com/241032/221445944-ae85d1f4-d351-4837-b130-82cab57d6cca.png) -- never expose to the internet! |
| [cpp-winpe64.exe](https://github.com/9001/copyparty/releases/download/v1.10.1/copyparty-winpe64.exe) | ⛔️ dangerous | runs on [64bit WinPE](https://user-images.githubusercontent.com/241032/205454984-e6b550df-3c49-486d-9267-1614078dd0dd.png), otherwise useless |
* except for [u2c.exe](https://github.com/9001/copyparty/releases/download/v1.13.0/u2c.exe), all of the options above are mostly equivalent
* the zip and tar.gz files below are just source code
* python packages are available at [PyPI](https://pypi.org/project/copyparty/#files)
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0506-0029 `v1.13.1` ctrl-v
## new features
* upload files by `ctrl-c` from OS and `ctrl-v` into browser c5f7cfc3
* from just about any file manager (windows explorer, thunar on linux, etc.) into the copyparty web-ui
* only files, not folders, so drag-drop is still the recommended way
* empty folders show an "empty folder" banner fdda567f
* opengraph / discord embeds ea270ab9 36f2c446 48a6789d b15a4ef7
* embeds [audio with covers](https://cd.ocv.me/c/d2/d22/snowy.mp3) , [images](https://cd.ocv.me/c/d2/d22/cover.jpg) , [videos](https://cd.ocv.me/c/d2/d21/no-effect.webm) , [audio without coverart](https://cd.ocv.me/c/d2/bitconnect.mp3) (links to one of the copyparty demoservers where the feature is enabled; link those in discord to test)
* images are currently not rendering correctly once clicked on android-discord (works on ios and in browser)
* default-disabled because opengraph disables hotlinking by design
* enable with `--og` and [see readme](https://github.com/9001/copyparty#opengraph) and [the --help](https://github.com/9001/copyparty/assets/241032/2dabf21e-2470-4e20-8ef0-3821b24be1b6)
* add option to support base64-encoded url queries parceled into the url location 69517e46
* because android-specific discord bugs prevent the use of queries in opengraph tags
* improve server performance when downloading unfinished uploads, especially on slow storage 70a3cf36
* add dynamic content into `<head>` using `--html-head` which now takes files and/or jinja templates as input b6cf2d30
* `--au-vol` (default 50, same as before) sets default audio volume in percent da091aec
* add **[copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** buildscript 27485a4c
* support ie4 and the [version of winzip](https://a.ocv.me/pub/g/nerd-stuff/cpp/win311zip.png) you'd find on an average windows 3.11 pc 603d0ed7
## bugfixes
* when logging in from the 403 page, remember and apply the original url hash f8491970
* the config-reset button in the control-panel didn't clear the dotfiles preference bc2c1e42
* the search feature could discover and use stale indexes in volumes where indexing was since disabled 95d9e693
* when in doubt, periodically recheck if filesystems support sparse files f6e693f0
* reduces opportunities for confusion on servers with removable media (usb flashdrives)
----
this release introduces **[copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)**, yet another way to bring copyparty where it's needed -- very limited and with many drawbacks (see [readme](https://github.com/9001/copyparty#zipapp)) but may work when the others don't
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0420-2232 `v1.13.0` race the beam
## new features
* files can be downloaded before the upload has completed ("almost like peer-to-peer")
* watch the [release trailer](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm) 👌
* if the downloader catches up with the upload, the speed is gradually slowed down so it never runs ahead
* can be disabled with `--no-pipe`
* option `--no-db-ip` disables storing the uploader IP in the database bf585078
* u2c (cli uploader): option `--ow` to overwrite existing files on the server 439cb7f8
## bugfixes
* when running on windows, using the web-UI to abort an upload could fail 8c552f1a
* rapidly PUT-uploading and then deleting files could crash the file hasher feecb3e0
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0412-2110 `v1.12.2` ie11 fix
## new features
* new option `--bauth-last` for when you're hosting other [basic-auth](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication) services on the same domain 7b94e4ed
* makes it possible to log into copyparty as intended, but it still sees the passwords from the other service until you do
* alternatively, the other new option `--no-bauth` entirely disables basic-auth support, but that also kills [the android app](https://github.com/9001/party-up)
## bugfixes
* internet explorer isn't working?! FIX IT!!! 9e5253ef
* audio transcoding was buggy with filekeys enabled b8733653
* on windows, theoretical chance that antivirus could interrupt renaming files, so preemptively guard against that c8e3ed3a
## other changes
* add a "password" placeholder on the login page since you might think it's asking for a username da26ec36
* config buttons were jank on iOS b772a4f8
* readme: [making your homeserver accessible from the internet](https://github.com/9001/copyparty#at-home)
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0409-2334 `v1.12.1` scrolling stuff
## new features
* while viewing pictures/videos, the scrollwheel can be used to view the prev/next file 844d16b9
## bugfixes
* #81 (scrolling suddenly getting disabled) properly fixed after @icxes found another way to reproduce it (thx) 4f0cad54
* and fixed at least one javascript glitch introduced in v1.12.0 while adding dirkeys 989cc613
* directory tree sidebar could fail to render when popping browser history into the lightbox
## other changes
* music preloader is slightly less hyper f89de6b3
* u2c.exe: updated TLS-certs and deps ab18893c
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0406-2011 `v1.12.0` locksmith
## new features
* #64 dirkeys; option to auto-generate passwords for folders, so you can give someone a link to a specific folder inside a volume without sharing the rest of the volume 10bc2d92 32c912bb ef52e2c0 0ae12868
* enabled by volflag `dk` (exact folder only) and/or volflag `dks` (also subfolders); see [readme](https://github.com/9001/copyparty#dirkeys)
* audio transcoding to mp3 if browser doesn't support opus a080759a
* recursively transcode and download a folder using `?tar&mp3`
* accidentally adds support for playing just about any audio format in ie11
* audio equalizer also applies to videos 7744226b
## bugfixes
* #81 scrolling could break after viewing an image in the lightbox 9c42cbec
* on phones, audio playback could stop if network is slow/unreliable 59f815ff b88cc7b5 59a53ba9
* fixes the issue on android, but ios/safari appears to be [impossible](https://github.com/9001/copyparty/blob/hovudstraum/docs/devnotes.md#music-playback-halting-on-phones) d94b5b3f
## other changes
* updated dompurify to 3.0.11
* copyparty.exe: updated to python 3.11.9
* support for building with pyoxidizer was removed 5ab54763
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
# 2024-0323-1724 `v1.11.2` public idp volumes

View File

@@ -22,7 +22,6 @@
* [complete release](#complete-release)
* [debugging](#debugging)
* [music playback halting on phones](#music-playback-halting-on-phones) - mostly fine on android
* [todo](#todo) - roughly sorted by priority
* [discarded ideas](#discarded-ideas)
@@ -135,6 +134,9 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?zip=utf-8` | ...as a zip file |
| GET | `?zip` | ...as a WinXP-compatible zip file |
| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
| GET | `?tar&w` | pregenerate webp thumbnails |
| GET | `?tar&j` | pregenerate jpg thumbnails |
| GET | `?tar&p` | pregenerate audio waveforms |
| GET | `?ups` | show recent uploads from your IP |
| GET | `?ups&filter=f` | ...where URL contains `f` |
| GET | `?mime=foo` | specify return mimetype `foo` |
@@ -312,17 +314,11 @@ mostly fine on android, but still haven't find a way to massage iphones into be
* conditionally starting/stopping mp.fau according to mp.au.readyState <3 or <4 doesn't help
* loop=true doesn't work, and manually looping mp.fau from an onended also doesn't work (it does nothing)
* assigning fau.currentTime in a timer doesn't work, as safari merely pretends to assign it
* on ios 16.7.7, mp.fau can sometimes make everything visibly work correctly, but no audio is actually hitting the speakers
can be reproduced with `--no-sendfile --s-wr-sz 8192 --s-wr-slp 0.3 --rsp-slp 6` and then play a collection of small audio files with the screen off, `ffmpeg -i track01.cdda.flac -c:a libopus -b:a 128k -segment_time 12 -f segment smol-%02d.opus`
# todo
roughly sorted by priority
* nothing! currently
## discarded ideas
* optimization attempts which didn't improve performance

View File

@@ -46,7 +46,7 @@ open up notepad and save the following as `c:\users\you\documents\party.conf` (f
### config explained: [global]
the `[global]` section accepts any config parameters you can see when running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
the `[global]` section accepts any config parameters [listed here](https://ocv.me/copyparty/helptext.html), also viewable by running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
* `lo: ~/logs/cpp-%Y-%m%d.xz` writes compressed logs (the compression will make them delayed)
* `e2dsa` enables the upload deduplicator and file indexer, which enables searching
* `e2ts` enables music metadata indexing, making albums / titles etc. searchable too

View File

@@ -221,6 +221,11 @@ sox -DnV -r8000 -b8 -c1 /dev/shm/a.wav synth 1.1 sin 400 vol 0.02
# play icon calibration pics
for w in 150 170 190 210 230 250; do for h in 130 150 170 190 210; do /c/Program\ Files/ImageMagick-7.0.11-Q16-HDRI/magick.exe convert -size ${w}x${h} xc:brown -fill orange -draw "circle $((w/2)),$((h/2)) $((w/2)),$((h/3))" $w-$h.png; done; done
# compress chiptune modules
mkdir gz; for f in *.*; do pigz -c11 -I100 <"$f" >gz/"$f"gz; touch -r "$f" gz/"$f"gz; done
mkdir xz; for f in *.*; do xz -cz9 <"$f" >xz/"$f"xz; touch -r "$f" xz/"$f"xz; done
mkdir z; for f in *.*; do 7z a -tzip -mx=9 -mm=lzma "z/${f}z" "$f" && touch -r "$f" z/"$f"z; done
##
## vscode

View File

@@ -1,82 +1,71 @@
# recipe for building an exe with nuitka (extreme jank edition)
#
# NOTE: win7 and win10 builds both work on win10 but
# on win7 they immediately c0000005 in kernelbase.dll
#
# first install python-3.6.8-amd64.exe
# [x] add to path
#
NOTE: copyparty runs SLOWER when compiled with nuitka;
just use copyparty-sfx.py and/or pyinstaller instead
( the sfx and the pyinstaller EXEs are equally fast if you
have the latest jinja2 installed, but the older jinja that
comes bundled with the sfx is slightly faster yet )
roughly, copyparty-sfx.py is 6% faster than copyparty.exe
(win10-pyinstaller), and copyparty.exe is 10% faster than
nuitka, making copyparty-sfx.py 17% faster than nuitka
NOTE: every time a nuitka-compiled copyparty.exe is launched,
it will show the windows firewall prompt since nuitka will
pick a new unique location in %TEMP% to unpack an exe into,
unlike pyinstaller which doesn't fork itself on startup...
might be fixable by configuring nuitka differently, idk
NOTE: nuitka EXEs are larger than pyinstaller ones;
a minimal nuitka build of just the sfx (with its bundled
dependencies) was already the same size as the pyinstaller
copyparty.exe which also includes Mutagen and Pillow
NOTE: nuitka takes a lot longer to build than pyinstaller
(due to actual compilation of course, but still)
NOTE: binaries built with nuitka cannot run on windows7,
even when compiled with python 3.6 on windows 7 itself
NOTE: `--python-flags=-m` is the magic sauce to
correctly compile `from .util import Daemon`
(which otherwise only explodes at runtime)
NOTE: `--deployment` doesn't seem to affect performance
########################################################################
# copypaste the rest of this file into cmd
rem from pypi
cd \users\ed\downloads
python -m pip install --user Nuitka-0.6.14.7.tar.gz
rem https://github.com/brechtsanders/winlibs_mingw/releases/download/10.2.0-11.0.0-8.0.0-r5/winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\
copy c:\users\ed\downloads\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
rem https://github.com/ccache/ccache/releases/download/v3.7.12/ccache-3.7.12-windows-32.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\
copy c:\users\ed\downloads\ccache-3.7.12-windows-32.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\ccache-3.7.12-windows-32.zip
python -m pip install --user -U nuitka
rem https://dependencywalker.com/depends22_x64.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\
copy c:\users\ed\downloads\depends22_x64.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\depends22_x64.zip
cd %homedrive%
cd %homepath%\downloads
cd \
rd /s /q %appdata%\..\local\temp\pe-copyparty
cd \users\ed\downloads
python copyparty-sfx.py -h
cd %appdata%\..\local\temp\pe-copyparty\copyparty
rd /s /q copypuitka
mkdir copypuitka
cd copypuitka
python
import os, re
os.rename('../dep-j2/jinja2', '../jinja2')
os.rename('../dep-j2/markupsafe', '../markupsafe')
rd /s /q %temp%\pe-copyparty
python ..\copyparty-sfx.py --version
print("# nuitka dies if .__init__.stuff is imported")
with open('__init__.py','r',encoding='utf-8') as f:
t1 = f.read()
move %temp%\pe-copyparty\copyparty .\
move %temp%\pe-copyparty\partftpy .\
move %temp%\pe-copyparty\ftp\pyftpdlib .\
move %temp%\pe-copyparty\j2\jinja2 .\
move %temp%\pe-copyparty\j2\markupsafe .\
with open('util.py','r',encoding='utf-8') as f:
t2 = f.read().split('\n')[3:]
rd /s /q %temp%\pe-copyparty
t2 = [x for x in t2 if 'from .__init__' not in x]
t = t1 + '\n'.join(t2)
with open('__init__.py','w',encoding='utf-8') as f:
f.write('\n')
python -m nuitka ^
--onefile --deployment --python-flag=-m ^
--include-package=markupsafe ^
--include-package=jinja2 ^
--include-package=partftpy ^
--include-package=pyftpdlib ^
--include-data-dir=copyparty\web=copyparty\web ^
--include-data-dir=copyparty\res=copyparty\res ^
--run copyparty
with open('util.py','w',encoding='utf-8') as f:
f.write(t)
print("# local-imports fail, prefix module names")
ptn = re.compile(r'^( *from )(\.[^ ]+ import .*)')
for d, _, fs in os.walk('.'):
for f in fs:
fp = os.path.join(d, f)
if not fp.endswith('.py'):
continue
t = ''
with open(fp,'r',encoding='utf-8') as f:
for ln in [x.rstrip('\r\n') for x in f]:
m = ptn.match(ln)
if not m:
t += ln + '\n'
continue
p1, p2 = m.groups()
t += "{}copyparty{}\n".format(p1, p2).replace("__init__", "util")
with open(fp,'w',encoding='utf-8') as f:
f.write(t)
exit()
cd ..
rd /s /q bout & python -m nuitka --standalone --onefile --windows-onefile-tempdir --python-flag=no_site --assume-yes-for-downloads --include-data-dir=copyparty\web=copyparty\web --include-data-dir=copyparty\res=copyparty\res --run --output-dir=bout --mingw64 --include-package=markupsafe --include-package=jinja2 copyparty

View File

@@ -47,3 +47,25 @@ and if you want to have a monospace font in the fancy markdown editor, do this:
NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative
# `<head>`
to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
if you give it the value `@ASDF` it will try to open a file named ASDF and send the text within
if the value starts with `%` it will assume a jinja2 template and expand it; the template has access to the `HttpCli` object through a property named `this` as well as everything in `j2a` and the stuff added by `self.j2s`; see [browser.html](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.html) for inspiration or look under the hood in [httpcli.py](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/httpcli.py)
# translations
add your own translations by using the english or norwegian one from `browser.js` as a template
the easy way is to open up and modify `browser.js` in your own installation; depending on how you installed copyparty it might be named `browser.js.gz` instead, in which case just decompress it, restart copyparty, and start editing it anyways
if you're running `copyparty-sfx.py` then you'll find it at `/tmp/pe-copyparty.1000/copyparty/web` (on linux) or `%TEMP%\pe-copyparty\copyparty\web` (on windows)
* make sure to keep backups of your work religiously! since that location is volatile af
if editing `browser.js` is inconvenient in your setup then you can instead do this:
* add your translation to a separate javascript file (`tl.js`) and make it load before `browser.js` with the help of `--html-head='<script src="/tl.js"></script>'`
* as the page loads, `browser.js` will look for a function named `langmod` so define that function and make it insert your translation into the `Ls` and `LANGS` variables so it'll take effect

View File

@@ -48,6 +48,7 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [filebrowser](#filebrowser)
* [filegator](#filegator)
* [sftpgo](#sftpgo)
* [arozos](#arozos)
* [updog](#updog)
* [goshs](#goshs)
* [gimme-that](#gimme-that)
@@ -93,6 +94,7 @@ the softwares,
* `j` = [filebrowser](https://github.com/filebrowser/filebrowser)
* `k` = [filegator](https://github.com/filegator/filegator)
* `l` = [sftpgo](https://github.com/drakkan/sftpgo)
* `m` = [arozos](https://github.com/tobychui/arozos)
some softwares not in the matrixes,
* [updog](#updog)
@@ -113,22 +115,22 @@ symbol legend,
## general
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | |
| runs on iOS | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | |
| android app | | | | █ | █ | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | | |
| runs on iOS | | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ | |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ | |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | | █ |
| android app | | | | █ | █ | | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | | |
* `zero setup` = you can get a mostly working setup by just launching the app, without having to install any software or configure whatever
* `a`/copyparty remarks:
@@ -140,37 +142,39 @@ symbol legend,
* `f`/rclone must be started with the command `rclone serve webdav .` or similar
* `h`/chibisafe has undocumented windows support
* `i`/sftpgo must be launched with a command
* `m`/arozos has partial windows support
## file transfer
*the thing that copyparty is actually kinda good at*
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ |
| download folder as tar | █ | | | | | | | | | █ | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | |
| resumable uploads | █ | | | | | | | | █ | | █ | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | |
| upload acceleration | █ | | | | | | | | █ | | █ | |
| upload verification | █ | | | █ | █ | | | | █ | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | |
| keep last-modified time | █ | | | █ | █ | █ | | | | | | |
| upload rules | | | | | | | | | | | | |
| ┗ max disk usage | █ | █ | | | | | | | | | | |
| ┗ max filesize | █ | | | | | | | █ | | | █ | █ |
| ┗ max items in folder | █ | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | |
| ┗ randomize filename | █ | | | | | | | | | | | |
| ┗ mimetype reject-list | | | | | | | | | | | | |
| ┗ extension reject-list | | | | | | | | | • | | | |
| checksums provided | | | | █ | █ | | | | | | | |
| cloud storage backend | | | | █ | █ | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ | |
| download folder as tar | █ | | | | | | | | | █ | | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | | █ |
| resumable uploads | █ | | | | | | | | █ | | █ | | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | | █ |
| upload acceleration | █ | | | | | | | | █ | | █ | | |
| upload verification | █ | | | █ | █ | | | | █ | | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | | |
| race the beam ("p2p") | █ | | | | | | | | | • | | | |
| keep last-modified time | █ | | | | | | | | | | | | |
| upload rules | | | | | | | | | | | | | |
| ┗ max disk usage | █ | █ | | | | | | | █ | | | █ | █ |
| ┗ max filesize | █ | | | | | | | | | | █ | █ | |
| ┗ max items in folder | █ | | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | █ | | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | | |
| ┗ randomize filename | | | | | | | | █ | █ | | | | |
| ┗ mimetype reject-list | | | | | | | | | • | | | | • |
| ┗ extension reject-list | | | | | | | | | | | | | • |
| checksums provided | | | | █ | █ | | | | | | | | |
| cloud storage backend | | | | █ | █ | █ | | | | | █ | █ | |
* `upload segmenting` = files are sliced into chunks, making it possible to upload files larger than 100 MiB on cloudflare for example
@@ -178,6 +182,8 @@ symbol legend,
* `upload verification` = uploads are checksummed or otherwise confirmed to have been transferred correctly
* `race the beam` = files can be downloaded while they're still uploading; downloaders are slowed down such that the uploader is always ahead
* `checksums provided` = when downloading a file from the server, the file's checksum is provided for verification client-side
* `cloud storage backend` = able to serve files from (and write to) s3 or similar cloud services; `` means the software can do this with some help from `rclone mount` as a bridge
@@ -192,26 +198,27 @@ symbol legend,
* resumable/segmented uploads only over SFTP, not over HTTP
* upload rules are totals only, not over time
* can probably do extension/mimetype rejection similar to copyparty
* `m`/arozos download-as-zip is not streaming; it creates the full zipfile before download can start, and fails on big folders
## protocols and client support
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ |
| serve ftp (tcp) | █ | | | | | █ | | | | | | █ |
| serve ftps (tls) | █ | | | | | █ | | | | | | █ |
| serve tftp (udp) | █ | | | | | | | | | | | |
| serve sftp (ssh) | | | | | | █ | | | | | | █ |
| serve smb/cifs | | | | | | █ | | | | | | |
| serve dlna | | | | | | █ | | | | | | |
| listen on unix-socket | | | | █ | █ | | █ | █ | █ | | █ | █ |
| zeroconf | █ | | | | | | | | | | | |
| supports netscape 4 | | | | | | █ | | | | | • | |
| ...internet explorer 6 | | █ | | █ | | █ | | | | | • | |
| mojibake filenames | █ | | | • | • | █ | █ | • | | • | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ | █ |
| serve ftp (tcp) | █ | | | | | █ | | | | | | █ | █ |
| serve ftps (tls) | █ | | | | | █ | | | | | | █ | |
| serve tftp (udp) | █ | | | | | | | | | | | | |
| serve sftp (ssh) | | | | | | █ | | | | | | █ | █ |
| serve smb/cifs | | | | | | █ | | | | | | | |
| serve dlna | | | | | | █ | | | | | | | |
| listen on unix-socket | | | | █ | █ | | █ | █ | █ | | █ | █ | |
| zeroconf | █ | | | | | | | | | | | | █ |
| supports netscape 4 | | | | | | █ | | | | | • | | |
| ...internet explorer 6 | | █ | | █ | | █ | | | | | • | | |
| mojibake filenames | █ | | | • | • | █ | █ | • | | • | | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | | |
* `webdav` = protocol convenient for mounting a remote server as a local filesystem; see zeroconf:
* `zeroconf` = the server announces itself on the LAN, [automatically appearing](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png) on other zeroconf-capable devices
@@ -222,61 +229,66 @@ symbol legend,
* extremely minimal samba/cifs server
* netscape 4 / ie6 support is mostly listed as a joke altho some people have actually found it useful ([ie4 tho](https://user-images.githubusercontent.com/241032/118192791-fb31fe00-b446-11eb-9647-898ea8efc1f7.png))
* `l`/sftpgo translates mojibake filenames into valid utf-8 (information loss)
* `m`/arozos has readonly-support for older browsers; no uploading
## server configuration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | |
| same-port http / https | █ | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | • | • | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | | █ |
| same-port http / https | █ | | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ | |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ | |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ | |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | • | • | | • |
* `folder-rproxy` = reverse-proxying without dedicating an entire (sub)domain, using a subfolder instead
* `l`/sftpgo:
* config: users must be added through gui / api calls
* `m`/arozos:
* configuration is primarily through GUI
* reverse-proxy is not guaranteed to see the correct client IP
## server capabilities
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ |
| single-sign-on | | | | █ | █ | | | | • | | | |
| token auth | | | | █ | █ | | | █ | | | | |
| 2fa | | | | █ | █ | | | | | | | █ |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | |
| unmap subfolders | █ | | | | | | █ | | | █ | | • |
| index.html blocks list | | | | | | | █ | | | • | | |
| write-only folders | █ | | | | | | | | | | █ | █ |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | |
| file action event hooks | █ | | | | | | | | | █ | | █ |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | |
| full sync | | | | █ | █ | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ |
| dyndns updater | | █ | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | |
| curl-friendly ls | █ | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ | |
| single-sign-on | | | | █ | █ | | | | • | | | | |
| token auth | | | | █ | █ | | | █ | | | | | █ |
| 2fa | | | | █ | █ | | | | | | | █ | |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | | █ |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | | █ |
| unmap subfolders | █ | | | | | | █ | | | █ | | • | |
| index.html blocks list | | | | | | | █ | | | • | | | |
| write-only folders | █ | | | | | | | | | | █ | █ | |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ | |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | | |
| file action event hooks | █ | | | | | | | | | █ | | █ | • |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | | |
| full sync | | | | █ | █ | | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ | |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ | • |
| dyndns updater | | █ | | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | | █ |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ | • |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | | █ |
| prometheus metrics | █ | | | █ | | | | | | | | | |
| curl-friendly ls | █ | | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | █ | █ | • | | | | | |
* `unmap subfolders` = "shadowing"; mounting a local folder in the middle of an existing filesystem tree in order to disable access below that path
* `files stored as-is` = uploaded files are trivially readable from the server HDD, not sliced into chunks or in weird folder structures or anything like that
@@ -302,49 +314,51 @@ symbol legend,
* `l`/sftpgo:
* `file action event hooks` also include on-download triggers
* `upload tracking / log` in main logfile
* `m`/arozos:
* `2fa` maybe possible through LDAP/Oauth
## client features
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | |
| themes | █ | █ | | █ | | | | | █ | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | |
| multi-column sorting | █ | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | |
| ┗ audio spectrograms | █ | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | |
| ┗ gapless playback | █ | | | | | | | | • | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | |
| ┗ video transcoding | | | | | | | | | █ | | | |
| audio BPM detector | █ | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | |
| search by bpm / key | █ | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | |
| search in file contents | | | | █ | █ | | | | █ | | | |
| search by custom parser | █ | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | |
| markdown viewer | █ | | | | █ | | | | █ | | | |
| markdown editor | █ | | | | █ | | | | █ | | | |
| readme.md in listing | █ | | | █ | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | | █ |
| themes | █ | █ | | █ | | | | | █ | | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | | |
| multi-column sorting | █ | | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | | █ |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | | █ |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ audio spectrograms | █ | | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ gapless playback | █ | | | | | | | | • | | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | | █ |
| ┗ video transcoding | | | | | | | | | █ | | | | |
| audio BPM detector | █ | | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | | |
| search by bpm / key | █ | | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | | |
| search in file contents | | | | █ | █ | | | | █ | | | | |
| search by custom parser | █ | | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | | █ |
| markdown viewer | █ | | | | █ | | | | █ | | | | █ |
| markdown editor | █ | | | | █ | | | | █ | | | | █ |
| readme.md in listing | █ | | | █ | | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | | █ |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | | █ |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | | █ |
* `single-page app` = multitasking; possible to continue navigating while uploading
* `audio player » os-integration` = use the [lockscreen](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) or [media hotkeys](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) to play/pause, prev/next song
@@ -360,14 +374,14 @@ symbol legend,
## integration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | |
| flameshot | | | | | | █ | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | | |
| flameshot | | | | | | █ | | | | | | | |
* sharex `` = yes, but does not provide example sharex config
* `a`/copyparty remarks:
@@ -393,6 +407,7 @@ symbol legend,
| filebrowser | go | █ apl2 | 20 MB |
| filegator | php | █ mit | • |
| sftpgo | go | ‼ agpl | 44 MB |
| arozos | go | ░ gpl3 | 531 MB |
| updog | python | █ mit | 17 MB |
| goshs | go | █ mit | 11 MB |
| gimme-that | python | █ mit | 4.8 MB |
@@ -504,12 +519,14 @@ symbol legend,
* ✅ token auth (api keys)
## [kodbox](https://github.com/kalcaddle/kodbox)
* this thing is insane
* this thing is insane (but is getting competition from [arozos](#arozos))
* php; [docker](https://hub.docker.com/r/kodcloud/kodbox)
* 🔵 *upload segmenting, acceleration, and integrity checking!*
* ⚠️ but uploads are not resumable(?)
* ⚠️ not portable
* ⚠️ isolated on-disk file hierarchy, incompatible with other software
* ⚠️ uploading small files to copyparty is 16x faster
* ⚠️ uploading large files to copyparty is 3x faster
* ⚠️ http/webdav only; no ftp or zeroconf
* ⚠️ some parts of the GUI are in chinese
* ✅ fantastic ui/ux
@@ -569,6 +586,24 @@ symbol legend,
* ✅ on-download event hook (otherwise same as copyparty)
* ✅ more extensive permissions control
## [arozos](https://github.com/tobychui/arozos)
* big suite of applications similar to [kodbox](#kodbox), copyparty is better at downloading/uploading/music/indexing but arozos has other advantages
* go; primarily linux (limited support for windows)
* ⚠️ uploads not resumable / integrity-checked
* ⚠️ uploading small files to copyparty is 2.7x faster
* ⚠️ uploading large files to copyparty is at least 10% faster
* arozos is websocket-based, 512 KiB chunks; writes each chunk to separate files and then merges
* copyparty splices directly into the final file; faster and better for the HDD and filesystem
* ⚠️ no directory tree navpane; not as easy to navigate
* ⚠️ download-as-zip is not streaming; creates a temp.file on the server
* ⚠️ not self-contained (pulls from jsdelivr)
* ⚠️ has an audio player, but supports less filetypes
* ⚠️ limited support for configuring real-ip detection
* ✅ sftp server
* ✅ settings gui
* ✅ good-looking gui
* ✅ an IDE, msoffice viewer, rich host integration, much more
## [updog](https://github.com/sc0tfree/updog)
* python; cross-platform
* basic directory listing with upload feature

View File

@@ -3,7 +3,7 @@ WORKDIR /z
ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
ver_hashwasm=4.10.0 \
ver_marked=4.3.0 \
ver_dompf=3.0.11 \
ver_dompf=3.1.5 \
ver_mde=2.18.0 \
ver_codemirror=5.65.16 \
ver_fontawesome=5.13.0 \

61
scripts/make-pyz.sh Executable file
View File

@@ -0,0 +1,61 @@
#!/bin/bash
set -e
echo
# port install gnutar gsed coreutils
gtar=$(command -v gtar || command -v gnutar) || true
[ ! -z "$gtar" ] && command -v gsed >/dev/null && {
tar() { $gtar "$@"; }
sed() { gsed "$@"; }
command -v grealpath >/dev/null &&
realpath() { grealpath "$@"; }
}
targs=(--owner=1000 --group=1000)
[ "$OSTYPE" = msys ] &&
targs=()
[ -e copyparty/__main__.py ] || cd ..
[ -e copyparty/__main__.py ] || {
echo "run me from within the project root folder"
echo
exit 1
}
[ -e sfx/copyparty/__main__.py ] || {
echo "run ./scripts/make-sfx.py first"
echo
exit 1
}
rm -rf pyz
mkdir -p pyz
cd pyz
cp -pR ../sfx/{copyparty,partftpy} .
cp -pR ../sfx/{ftp,j2}/* .
ts=$(date -u +%s)
hts=$(date -u +%Y-%m%d-%H%M%S)
ver="$(cat ../sfx/ver)"
mkdir -p ../dist
pyz_out=../dist/copyparty.pyz
echo creating z.tar
( cd copyparty
tar -cf z.tar "${targs[@]}" --numeric-owner web res
rm -rf web res
)
echo creating loader
sed -r 's/^(VER = ).*/\1"'"$ver"'"/; s/^(STAMP = ).*/\1'$(date +%s)/ \
<../scripts/ziploader.py \
>__main__.py
echo creating pyz
rm -f $pyz_out
zip -9 -q -r $pyz_out *
echo done:
echo " $(realpath $pyz_out)"

View File

@@ -99,9 +99,6 @@ pybin=$(command -v python3 || command -v python) || {
exit 1
}
[ $CSN ] ||
CSN=sfx
langs=
use_gz=
zopf=2560
@@ -148,9 +145,9 @@ stamp=$(
done | sort | tail -n 1 | sha1sum | cut -c-16
)
rm -rf $CSN/*
mkdir -p $CSN build
cd $CSN
rm -rf sfx$CSN/*
mkdir -p sfx$CSN build
cd sfx$CSN
tmpdir="$(
printf '%s\n' "$TMPDIR" /tmp |
@@ -386,11 +383,13 @@ git describe --tags >/dev/null 2>/dev/null && {
ver="$(awk '/^VERSION *= \(/ {
gsub(/[^0-9,a-g-]/,""); gsub(/,/,"."); print; exit}' < copyparty/__version__.py)"
echo "$ver" >ver # pyz
ts=$(date -u +%s)
hts=$(date -u +%Y-%m%d-%H%M%S) # --date=@$ts (thx osx)
mkdir -p ../dist
sfx_out=../dist/copyparty-$CSN
sfx_out=../dist/copyparty-sfx$CSN
echo cleanup
find -name '*.pyc' -delete
@@ -542,7 +541,7 @@ gzres() {
}
zdir="$tmpdir/cpp-mk$CSN"
zdir="$tmpdir/cpp-mksfx$CSN"
[ -e "$zdir/$stamp" ] || rm -rf "$zdir"
mkdir -p "$zdir"
echo a > "$zdir/$stamp"

View File

@@ -118,4 +118,12 @@ base64 | head -c12 >> dist/copyparty.exe
dist/copyparty.exe --version
curl -fkT dist/copyparty.exe -b cppwd=wark https://192.168.123.1:3923/copyparty$esuf.exe
csum=$(sha512sum <dist/copyparty.exe | cut -c-56)
curl -fkT dist/copyparty.exe -b cppwd=wark https://192.168.123.1:3923/copyparty$esuf.exe >uplod.log
cat uplod.log
grep -q $csum uplod.log && echo upload OK || {
echo UPLOAD FAILED
exit 1
}

View File

@@ -1,32 +1,32 @@
f117016b1e6a7d7e745db30d3e67f1acf7957c443a0dd301b6c5e10b8368f2aa4db6be9782d2d3f84beadd139bfeef4982e40f21ca5d9065cb794eeb0e473e82 altgraph-0.17.4-py2.py3-none-any.whl
eda6c38fc4d813fee897e969ff9ecc5acc613df755ae63df0392217bbd67408b5c1f6c676f2bf5497b772a3eb4e1a360e1245e1c16ee83f0af555f1ab82c3977 Git-2.39.1-32-bit.exe
e0d2e6183437af321a36944f04a501e85181243e5fa2da3254254305dd8119161f62048bc56bff8849b49f546ff175b02b4c999401f1c404f6b88e6f46a9c96e Git-2.44.0-32-bit.exe
9d2c31701a4d3fef553928c00528a48f9e1854ab5333528b50e358a214eba90029d687f039bcda5760b6fdf9f2de3bcf3784ae21a6374cf2a97a845d33b636c6 packaging-24.0-py3-none-any.whl
17ce52ba50692a9d964f57a23ac163fb74c77fdeb2ca988a6d439ae1fe91955ff43730c073af97a7b3223093ffea3479a996b9b50ee7fba0869247a56f74baa6 pefile-2023.2.7-py3-none-any.whl
126ca016c00256f4ff13c88707ead21b3b98f3c665ae57a5bcbb80c8be3004bff36d9c7f9a1cc9d20551019708f2b195154f302d80a1e5a2026d6d0fe9f3d5f4 pyinstaller_hooks_contrib-2024.3-py2.py3-none-any.whl
749a473646c6d4c7939989649733d4c7699fd1c359c27046bf5bc9c070d1a4b8b986bbc65f60d7da725baf16dbfdd75a4c2f5bb8335f2cb5685073f5fee5c2d1 pywin32_ctypes-0.2.2-py3-none-any.whl
6e0d854040baff861e1647d2bece7d090bc793b2bd9819c56105b94090df54881a6a9b43ebd82578cd7c76d47181571b671e60672afd9def389d03c9dae84fcf setuptools-68.2.2-py3-none-any.whl
3c5adf0a36516d284a2ede363051edc1bcc9df925c5a8a9fa2e03cab579dd8d847fdad42f7fd5ba35992e08234c97d2dbfec40a9d12eec61c8dc03758f2bd88e typing_extensions-4.4.0-py3-none-any.whl
# u2c (win7)
f3390290b896019b2fa169932390e4930d1c03c014e1f6db2405ca2eb1f51f5f5213f725885853805b742997b0edb369787e5c0069d217bc4e8b957f847f58b6 certifi-2023.11.17-py3-none-any.whl
904eb57b13bea80aea861de86987e618665d37fa9ea0856e0125a9ba767a53e5064de0b9c4735435a2ddf4f16f7f7d2c75a682e1de83d9f57922bdca8e29988c charset_normalizer-3.3.0-cp37-cp37m-win32.whl
ffdd45326f4e91c02714f7a944cbcc2fdd09299f709cfa8aec0892053eef0134fb80d9ba3790afd319538a86feb619037cbf533e2f5939cb56b35bb17f56c858 idna-3.4-py3-none-any.whl
7a3bd4849f95e1715fe2e99613df70a0fedd944a9bfde71a0fadb837fe62c3431c30da4f0b75c74de6f1a459f1fdf7cb62eaf404fdbe45e2d121e0b1021f1580 certifi-2024.2.2-py3-none-any.whl
9cc8acc5e269e6421bc32bb89261101da29d6ca337d39d60b9106de9ed7904e188716e4a48d78a2c4329026443fcab7acab013d2fe43778e30d6c4e4506a1b91 charset_normalizer-3.3.2-cp37-cp37m-win32.whl
0ec1ae5c928b4a0001a254c8598b746049406e1eed720bfafa94d4474078eff76bf6e032124e2d4df4619052836523af36162443c6d746487b387d2e3476e691 idna-3.6-py3-none-any.whl
b795abb26ba2f04f1afcfb196f21f638014b26c8186f8f488f1c2d91e8e0220962fbd259dbc9c3875222eb47fc95c73fc0606aaa6602b9ebc524809c9ba3501f requests-2.31.0-py3-none-any.whl
5a25cb9b79bb6107f9055dc3e9f62ebc6d4d9ca2c730d824985c93cd82406b723c200d6300c5064e42ee9fc7a2853d6ec6661394f3ed7bac03750e1f2a6840d1 urllib3-1.26.17-py2.py3-none-any.whl
61ed4500b6361632030f05229705c5c5a52cb47e31c0e6b55151c8f3beed631cd752ca6c3d6393d56a2acf6a453cfcf801e877116123c550922249c3a976e0f4 urllib3-1.26.18-py2.py3-none-any.whl
# win7
91c025f7d94bcdf93df838fab67053165a414fc84e8496f92ecbb910dd55f6b6af5e360bbd051444066880c5a6877e75157bd95e150ead46e5c605930dfc50f2 future-0.18.2.tar.gz
c06b3295d1d0b0f0a6f9a6cd0be861b9b643b4a5ea37857f0bd41c45deaf27bb927b71922dab74e633e43d75d04a9bd0d1c4ad875569740b0f2a98dd2bfa5113 importlib_metadata-5.0.0-py3-none-any.whl
016a8cbd09384f1a9a44cb0e8274df75a8bcb2f3966bb5d708c62145289efaa5db98f75256c97e4f8046735ce2e529fbb076f284a46cdb716e89a75660200ad9 pip-23.2.1-py3-none-any.whl
d130bfa136bd171b9972b5c281c578545f2a84a909fdf18a6d2d71dd12fb3d512a7a1fa5cf7300433adece1d306eb2f22d7278f4c90e744e04dc67ba627a82c0 future-1.0.0-py3-none-any.whl
0b4d07434bf8d314f42893d90bce005545b44a509e7353a73cad26dc9360b44e2824218a1a74f8174d02eba87fba91baffa82c8901279a32ebc6b8386b1b4275 importlib_metadata-6.7.0-py3-none-any.whl
5d7462a584105bccaa9cf376f5a8c5827ead099c813c8af7392d478a4398f373d9e8cac7bbad2db51b335411ab966b21e119b1b1234c9a7ab70c6ddfc9306da6 pip-24.0-py3-none-any.whl
f298e34356b5590dde7477d7b3a88ad39c622a2bcf3fcd7c53870ce8384dd510f690af81b8f42e121a22d3968a767d2e07595036b2ed7049c8ef4d112bcf3a61 pyinstaller-5.13.2-py3-none-win32.whl
6bb73cc2db795c59c92f2115727f5c173cacc9465af7710db9ff2f2aec2d73130d0992d0f16dcb3fac222dc15c0916562d0813b2337401022020673a4461df3d python-3.7.9-amd64.exe
500747651c87f59f2436c5ab91207b5b657856e43d10083f3ce27efb196a2580fadd199a4209519b409920c562aaaa7dcbdfb83ed2072a43eaccae6e2d056f31 python-3.7.9.exe
03e50aecc85914567c114e38a1777e32628ee098756f37177bc23220eab33ac7d3ff591fd162db3b4d4e34d55cee93ef0dc67af68a69c38bb1435e0768dee57e typing_extensions-4.7.1-py3-none-any.whl
2e04acff170ca3bbceeeb18489c687126c951ec0bfd53cccfb389ba8d29a4576c1a9e8f2e5ea26c84dd21bfa2912f4e71fa72c1e2653b71e34afc0e65f1722d4 upx-4.2.2-win32.zip
68e1b618d988be56aaae4e2eb92bc0093627a00441c1074ebe680c41aa98a6161e52733ad0c59888c643a33fe56884e4f935178b2557fbbdd105e92e0d993df6 windows6.1-kb2533623-x64.msu
479a63e14586ab2f2228208116fc149ed8ee7b1e4ff360754f5bda4bf765c61af2e04b5ef123976623d04df4976b7886e0445647269da81436bd0a7b5671d361 windows6.1-kb2533623-x86.msu
ba91ab0518c61eff13e5612d9e6b532940813f6b56e6ed81ea6c7c4d45acee4d98136a383a25067512b8f75538c67c987cf3944bfa0229e3cb677e2fb81e763e zipp-3.10.0-py3-none-any.whl
ac96786e5d35882e0c5b724794329c9125c2b86ae7847f17acfc49f0d294312c6afc1c3f248655de3f0ccb4ca426d7957d02ba702f4a15e9fcd7e2c314e72c19 zipp-3.15.0-py3-none-any.whl
# win10
e3e2e6bd511dec484dd0292f4c46c55c88a885eabf15413d53edea2dd4a4dbae1571735b9424f78c0cd7f1082476a8259f31fd3f63990f726175470f636df2b3 Jinja2-3.1.3-py3-none-any.whl
d1420c8417fad7888766dd26b9706a87c63e8f33dceeb8e26d0056d5127b0b3ed9272e44b4b761132d4b3320327252eab1696520488ca64c25958896b41f547b jinja2-3.1.4-py3-none-any.whl
e21495f1d473d855103fb4a243095b498ec90eb68776b0f9b48e994990534f7286c0292448e129c507e5d70409f8a05cca58b98d59ce2a815993d0a873dfc480 MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
8a6e2b13a2ec4ef914a5d62aad3db6464d45e525a82e07f6051ed10474eae959069e165dba011aefb8207cdfd55391d73d6f06362c7eb247b08763106709526e mutagen-1.47.0-py3-none-any.whl
656015f5cc2c04aa0653ee5609c39a7e5f0b6a58c84fe26b20bd070c52d20b4effb810132f7fb771168483e9fd975cc3302837dd7a1a687ee058b0460c857cc4 packaging-23.2-py3-none-any.whl
1dfe6f66bef5c9d62c9028a964196b902772ec9e19db215f3f41acb8d2d563586988d81b455fa6b895b434e9e1e9d57e4d271d1b1214483bdb3eadffcbba6a33 pillow-10.3.0-cp311-cp311-win_amd64.whl
8760eab271e79256ae3bfb4af8ccc59010cb5d2eccdd74b325d1a533ae25eb127d51c2ec28ff90d449afed32dd7d6af62934fe9caaf1ae1f4d4831e948e912da pyinstaller-6.5.0-py3-none-win_amd64.whl
897a14d5ee5cbc6781a0f48beffc27807a4f789d58c4329d899233f615d168a5dcceddf7f8f2d5bb52212ddcf3eba4664590d9f1fdb25bb5201f44899e03b2f7 python-3.11.9-amd64.exe

View File

@@ -8,7 +8,7 @@ run ./build.sh in git-bash to build + upload the exe
to obtain the files referenced below, see ./deps.txt
download + install git (32bit OK on 64):
http://192.168.123.1:3923/ro/pyi/Git-2.39.1-32-bit.exe
http://192.168.123.1:3923/ro/pyi/Git-2.44.0-32-bit.exe
===[ copy-paste into git-bash ]================================
uname -s | grep NT-10 && w10=1 || {
@@ -16,6 +16,7 @@ uname -s | grep NT-10 && w10=1 || {
}
fns=(
altgraph-0.17.4-py2.py3-none-any.whl
packaging-24.0-py3-none-any.whl
pefile-2023.2.7-py3-none-any.whl
pyinstaller_hooks_contrib-2024.3-py2.py3-none-any.whl
pywin32_ctypes-0.2.2-py3-none-any.whl
@@ -23,29 +24,28 @@ fns=(
)
[ $w10 ] && fns+=(
pyinstaller-6.5.0-py3-none-win_amd64.whl
Jinja2-3.1.3-py3-none-any.whl
Jinja2-3.1.4-py3-none-any.whl
MarkupSafe-2.1.5-cp311-cp311-win_amd64.whl
mutagen-1.47.0-py3-none-any.whl
packaging-23.2-py3-none-any.whl
pillow-10.3.0-cp311-cp311-win_amd64.whl
python-3.11.9-amd64.exe
upx-4.2.2-win64.zip
)
[ $w7 ] && fns+=(
pyinstaller-5.13.2-py3-none-win32.whl
certifi-2023.11.17-py3-none-any.whl
chardet-5.1.0-py3-none-any.whl
idna-3.4-py3-none-any.whl
requests-2.28.2-py3-none-any.whl
urllib3-1.26.14-py2.py3-none-any.whl
certifi-2024.2.2-py3-none-any.whl
charset_normalizer-3.3.2-cp37-cp37m-win32.whl
idna-3.6-py3-none-any.whl
requests-2.31.0-py3-none-any.whl
urllib3-1.26.18-py2.py3-none-any.whl
upx-4.2.2-win32.zip
)
[ $w7 ] && fns+=(
future-0.18.2.tar.gz
importlib_metadata-5.0.0-py3-none-any.whl
pip-23.2.1-py3-none-any.whl
typing_extensions-4.4.0-py3-none-any.whl
zipp-3.10.0-py3-none-any.whl
future-1.0.0-py3-none-any.whl
importlib_metadata-6.7.0-py3-none-any.whl
pip-24.0-py3-none-any.whl
typing_extensions-4.7.1-py3-none-any.whl
zipp-3.15.0-py3-none-any.whl
)
[ $w7x64 ] && fns+=(
windows6.1-kb2533623-x64.msu
@@ -77,9 +77,10 @@ yes | unzip upx-*-win32.zip &&
mv upx-*/upx.exe . &&
python -m ensurepip &&
{ [ $w10 ] || python -m pip install --user -U pip-*.whl; } &&
{ [ $w7 ] || python -m pip install --user -U {packaging,setuptools,mutagen,Pillow,Jinja2,MarkupSafe}-*.whl; } &&
python -m pip install --user -U packaging-*.whl &&
{ [ $w7 ] || python -m pip install --user -U {setuptools,mutagen,Pillow,Jinja2,MarkupSafe}-*.whl; } &&
{ [ $w10 ] || python -m pip install --user -U {requests,urllib3,charset_normalizer,certifi,idna}-*.whl; } &&
{ [ $w10 ] || python -m pip install --user -U future-*.tar.gz importlib_metadata-*.whl typing_extensions-*.whl zipp-*.whl; } &&
{ [ $w10 ] || python -m pip install --user -U future-*.whl importlib_metadata-*.whl typing_extensions-*.whl zipp-*.whl; } &&
python -m pip install --user -U pyinstaller-*.whl pefile-*.whl pywin32_ctypes-*.whl pyinstaller_hooks_contrib-*.whl altgraph-*.whl &&
sed -ri 's/--lzma/--best/' $appd/Python/Python$pyv/site-packages/pyinstaller/building/utils.py &&
curl -fkLO https://192.168.123.1:3923/cpp/scripts/uncomment.py &&

View File

@@ -37,6 +37,8 @@ grep -E '^from .ssl_ import' $APPDATA/python/python37/site-packages/urllib3/util
echo golfed
}
sed -ri 's/(add_argument."-t[de]",.*help=")[^"]+/\1not applicable; HTTPS is disabled in this exe/; s/for some reason/in this exe for safety reasons/' u2c.py
read a b _ < <(awk -F\" '/^S_VERSION =/{$0=$2;sub(/\./," ");print}' < u2c.py)
sed -r 's/1,2,3,0/'$a,$b,0,0'/;s/1\.2\.3/'$a.$b.0/ <up2k.rc >up2k.rc2
@@ -45,4 +47,12 @@ $APPDATA/python/python37/scripts/pyinstaller -y --clean --upx-dir=. up2k.spec
./dist/u2c.exe --version
curl -fkT dist/u2c.exe -HPW:wark https://192.168.123.1:3923/
csum=$(sha512sum <dist/u2c.exe | cut -c-56)
curl -fkT dist/u2c.exe -HPW:wark https://192.168.123.1:3923/ >uplod.log
cat uplod.log
grep -q $csum uplod.log && echo upload OK || {
echo UPLOAD FAILED
exit 1
}

View File

@@ -42,7 +42,7 @@ $f$s.py --version >/dev/null
min=99999999
for ((a=0; a<$parallel; a++)); do
while [ -e .sfx-run ]; do
CSN=sfx$a ./make-sfx.sh re "$@"
CSN=$a ./make-sfx.sh re "$@"
sz=$(wc -c <$f$a$s.py | awk '{print$1}')
[ $sz -ge $min ] && continue
mv $f$a$s.py $f$s.py.$sz

View File

@@ -1,6 +1,21 @@
#!/bin/bash
set -ex
if uname | grep -iE '^(msys|mingw)'; then
pids=()
python -m unittest discover -s tests >/dev/null &
pids+=($!)
python scripts/test/smoketest.py &
pids+=($!)
for pid in ${pids[@]}; do
wait $pid
done
exit $?
fi
# osx support
gtar=$(command -v gtar || command -v gnutar) || true
[ ! -z "$gtar" ] && command -v gfind >/dev/null && {

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# coding: latin-1
from __future__ import print_function, unicode_literals
import re, os, sys, time, shutil, signal, threading, tarfile, hashlib, platform, tempfile, traceback
import re, os, sys, time, shutil, signal, tarfile, hashlib, platform, tempfile, traceback
import subprocess as sp
@@ -368,17 +368,6 @@ def get_payload():
p = a
def utime(top):
# avoid cleaners
files = [os.path.join(dp, p) for dp, dd, df in os.walk(top) for p in dd + df]
while True:
t = int(time.time())
for f in [top] + files:
os.utime(f, (t, t))
time.sleep(78123)
def confirm(rv):
msg()
msg("retcode", rv if rv else traceback.format_exc())
@@ -398,9 +387,7 @@ def run(tmp, j2, ftp):
msg("sfxdir:", tmp)
msg()
t = threading.Thread(target=utime, args=(tmp,))
t.daemon = True
t.start()
sys.argv.append("--sfx-tpoke=" + tmp)
ld = (("", ""), (j2, "j2"), (ftp, "ftp"), (not PY2, "py2"), (PY37, "py37"))
ld = [os.path.join(tmp, b) for a, b in ld if not a]

109
scripts/ziploader.py Normal file
View File

@@ -0,0 +1,109 @@
#!/usr/bin/env python3
import atexit
import os
import platform
import sys
import tarfile
import tempfile
import time
import traceback
VER = None
STAMP = None
WINDOWS = sys.platform in ["win32", "msys"]
def msg(*a, **ka):
if a:
a = ["[ZIP]", a[0]] + list(a[1:])
ka["file"] = sys.stderr
print(*a, **ka)
def utime(top):
# avoid cleaners
files = [os.path.join(dp, p) for dp, dd, df in os.walk(top) for p in dd + df]
try:
while True:
t = int(time.time())
for f in [top] + files:
os.utime(f, (t, t))
time.sleep(78123)
except Exception as ex:
print("utime:", ex, f)
def confirm(rv):
msg()
msg("retcode", rv if rv else traceback.format_exc())
if WINDOWS:
msg("*** hit enter to exit ***")
try:
input()
except:
pass
sys.exit(rv or 1)
def run():
import copyparty
from copyparty.__main__ import main as cm
td = tempfile.TemporaryDirectory(prefix="")
atexit.register(td.cleanup)
rsrc = td.name
try:
from importlib.resources import files
f = files(copyparty).joinpath("z.tar").open("rb")
except:
from importlib.resources import open_binary
f = open_binary("copyparty", "z.tar")
with tarfile.open(fileobj=f) as tf:
try:
tf.extractall(rsrc, filter="tar")
except TypeError:
tf.extractall(rsrc) # nosec (archive is safe)
f.close()
f = None
msg(" rsrc dir:", rsrc)
msg()
sys.argv.append("--sfx-tpoke=" + rsrc)
cm(rsrc=rsrc)
def main():
sysver = str(sys.version).replace("\n", "\n" + " " * 18)
pktime = time.strftime("%Y-%m-%d, %H:%M:%S", time.gmtime(STAMP))
msg()
msg(" this is: copyparty", VER)
msg(" packed at:", pktime, "UTC,", STAMP)
msg("python bin:", sys.executable)
msg("python ver:", platform.python_implementation(), sysver)
try:
run()
except SystemExit as ex:
c = ex.code
if c not in [0, -15]:
confirm(ex.code)
except KeyboardInterrupt:
pass
except:
confirm(0)
if __name__ == "__main__":
main()

View File

@@ -3,10 +3,8 @@
from __future__ import print_function, unicode_literals
import io
import os
import time
import json
import pprint
import os
import shutil
import tarfile
import tempfile
@@ -82,7 +80,8 @@ class TestHttpCli(unittest.TestCase):
x = " ".join(sorted([x["rp"] for x in x[0]]))
self.assertEqual(x, ".f0 .t/.f2 .t/f2 a/da/f4 a/f3 f0 t/.f1 t/f1")
self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"], dotsrch=False)
u2idx.shutdown()
self.args = Cfg(v=vcfg, a=["u1:u1", "u2:u2"], dotsrch=False, e2d=True)
self.asrv = AuthSrv(self.args, self.log)
u2idx = U2idx(self)
@@ -91,6 +90,8 @@ class TestHttpCli(unittest.TestCase):
# u1 can see dotfiles in volB so they should be included
xe = "a/da/f4 a/f3 f0 t/f1"
self.assertEqual(x, xe)
u2idx.shutdown()
up2k.shutdown()
##
## dirkeys
@@ -149,4 +150,4 @@ class TestHttpCli(unittest.TestCase):
return conn.s._reply.decode("utf-8").split("\r\n\r\n", 1)
def log(self, src, msg, c=0):
print(msg)
print(msg, "\033[0m")

View File

@@ -6,6 +6,7 @@ import json
import os
import unittest
from copyparty.__init__ import ANYWIN
from copyparty.authsrv import AuthSrv
from tests.util import Cfg
@@ -51,6 +52,12 @@ class TestVFS(unittest.TestCase):
vn = self.nav(au, vp)
self.assertNodes(vn, expected)
def assertApEq(self, ap, rhs):
if ANYWIN and len(ap) > 2 and ap[1] == ":":
ap = ap[2:].replace("\\", "/")
return self.assertEqual(ap, rhs)
def prep(self):
here = os.path.abspath(os.path.dirname(__file__))
cfgdir = os.path.join(here, "res", "idp")
@@ -70,7 +77,7 @@ class TestVFS(unittest.TestCase):
au = AuthSrv(Cfg(c=[cfgdir + "/1.conf"], **xcfg), self.log)
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "/")
self.assertApEq(au.vfs.realpath, "/")
self.assertNodes(au.vfs, ["vb"])
self.assertNodes(au.vfs.nodes["vb"], [])
@@ -85,7 +92,7 @@ class TestVFS(unittest.TestCase):
au = AuthSrv(Cfg(c=[cfgdir + "/2.conf"], **xcfg), self.log)
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "/")
self.assertApEq(au.vfs.realpath, "/")
self.assertNodes(au.vfs, ["vb", "vc"])
self.assertNodes(au.vfs.nodes["vb"], [])
self.assertNodes(au.vfs.nodes["vc"], [])
@@ -103,7 +110,7 @@ class TestVFS(unittest.TestCase):
au = AuthSrv(Cfg(c=[cfgdir + "/3.conf"], **xcfg), self.log)
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "")
self.assertApEq(au.vfs.realpath, "")
self.assertNodes(au.vfs, [])
self.assertAxs(au.vfs.axs, [])
@@ -112,8 +119,8 @@ class TestVFS(unittest.TestCase):
self.assertNodesAt(au, "vu", ["iua"]) # same as:
self.assertNodes(au.vfs.nodes["vu"], ["iua"])
self.assertNodes(au.vfs.nodes["vg"], ["iga"])
self.assertEqual(au.vfs.nodes["vu"].realpath, "")
self.assertEqual(au.vfs.nodes["vg"].realpath, "")
self.assertApEq(au.vfs.nodes["vu"].realpath, "")
self.assertApEq(au.vfs.nodes["vg"].realpath, "")
self.assertAxs(au.vfs.axs, [])
self.assertAxsAt(au, "vu/iua", [["iua"]]) # same as:
self.assertAxs(self.nav(au, "vu/iua").axs, [["iua"]])
@@ -127,7 +134,7 @@ class TestVFS(unittest.TestCase):
au = AuthSrv(Cfg(c=[cfgdir + "/4.conf"], **xcfg), self.log)
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "")
self.assertApEq(au.vfs.realpath, "")
self.assertNodes(au.vfs, ["vu"])
self.assertNodesAt(au, "vu", ["ua", "ub"])
self.assertAxs(au.vfs.axs, [])
@@ -147,10 +154,10 @@ class TestVFS(unittest.TestCase):
self.assertAxsAt(au, "vg", [])
self.assertAxsAt(au, "vg/iga1", [["iua"]])
self.assertAxsAt(au, "vg/iga2", [["iua", "ua"]])
self.assertEqual(self.nav(au, "vu/ua").realpath, "/u-ua")
self.assertEqual(self.nav(au, "vu/iua").realpath, "/u-iua")
self.assertEqual(self.nav(au, "vg/iga1").realpath, "/g1-iga")
self.assertEqual(self.nav(au, "vg/iga2").realpath, "/g2-iga")
self.assertApEq(self.nav(au, "vu/ua").realpath, "/u-ua")
self.assertApEq(self.nav(au, "vu/iua").realpath, "/u-iua")
self.assertApEq(self.nav(au, "vg/iga1").realpath, "/g1-iga")
self.assertApEq(self.nav(au, "vg/iga2").realpath, "/g2-iga")
au.idp_checkin(None, "iub", "iga")
self.assertAxsAt(au, "vu/iua", [["iua"]])
@@ -165,7 +172,7 @@ class TestVFS(unittest.TestCase):
au = AuthSrv(Cfg(c=[cfgdir + "/5.conf"], **xcfg), self.log)
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "")
self.assertApEq(au.vfs.realpath, "")
self.assertNodes(au.vfs, ["g", "ga", "gb"])
self.assertAxs(au.vfs.axs, [])
@@ -196,7 +203,7 @@ class TestVFS(unittest.TestCase):
self.assertAxs(au.vfs.axs, [])
self.assertEqual(au.vfs.vpath, "")
self.assertEqual(au.vfs.realpath, "")
self.assertApEq(au.vfs.realpath, "")
self.assertNodes(au.vfs, [])
au.idp_checkin(None, "iua", "")

View File

@@ -44,7 +44,7 @@ if MACOS:
from copyparty.__init__ import E
from copyparty.__main__ import init_E
from copyparty.u2idx import U2idx
from copyparty.util import FHC, Garda, Unrecv
from copyparty.util import FHC, CachedDict, Garda, Unrecv
init_E(E)
@@ -110,25 +110,25 @@ class Cfg(Namespace):
def __init__(self, a=None, v=None, c=None, **ka0):
ka = {}
ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw q rand smb srch_dbg stats vague_403 vc ver xdev xlink xvol"
ex = "daw dav_auth dav_inf dav_mac dav_rt e2d e2ds e2dsa e2t e2ts e2tsr e2v e2vu e2vp early_ban ed emp exp force_js getmod grid hardlink ih ihead magic never_symlink nid nih no_acode no_athumb no_dav no_dedup no_del no_dupe no_lifetime no_logues no_mv no_pipe no_poll no_readme no_robots no_sb_md no_sb_lg no_scandir no_tarcmp no_thumb no_vthumb no_zip nrand nw og og_no_head og_s_title q rand smb srch_dbg stats uqe vague_403 vc ver xdev xlink xvol"
ka.update(**{k: False for k in ex.split()})
ex = "dotpart dotsrch no_dhash no_fastboot no_rescan no_sendfile no_voldump re_dhash plain_ip"
ex = "dotpart dotsrch no_dhash no_fastboot no_rescan no_sendfile no_snap no_voldump re_dhash plain_ip"
ka.update(**{k: True for k in ex.split()})
ex = "ah_cli ah_gen css_browser hist js_browser no_forget no_hash no_idx nonsus_urls"
ex = "ah_cli ah_gen css_browser hist js_browser mime mimes no_forget no_hash no_idx nonsus_urls og_tpl og_ua"
ka.update(**{k: None for k in ex.split()})
ex = "hash_mt srch_time u2abort u2j"
ka.update(**{k: 1 for k in ex.split()})
ex = "reg_cap s_thead s_tbody th_convt"
ex = "au_vol mtab_age reg_cap s_thead s_tbody th_convt"
ka.update(**{k: 9 for k in ex.split()})
ex = "db_act df k304 loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
ex = "db_act k304 loris re_maxage rproxy rsp_jtr rsp_slp s_wr_slp snap_wri theme themes turbo"
ka.update(**{k: 0 for k in ex.split()})
ex = "ah_alg bname doctitle exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name textfiles unlist vname R RS SR"
ex = "ah_alg bname doctitle df exit favico idp_h_usr html_head lg_sbf log_fk md_sbf name og_desc og_site og_th og_title og_title_a og_title_v og_title_i tcolor textfiles unlist vname R RS SR"
ka.update(**{k: "" for k in ex.split()})
ex = "grp on403 on404 xad xar xau xban xbd xbr xbu xiu xm"
@@ -155,6 +155,7 @@ class Cfg(Namespace):
mte={"a": True},
mth={},
mtp=[],
mv_retry="0/0",
rm_retry="0/0",
s_rd_sz=256 * 1024,
s_wr_sz=256 * 1024,
@@ -250,6 +251,7 @@ class VHttpConn(object):
self.log_func = log
self.log_src = "a"
self.mutex = threading.Lock()
self.pipes = CachedDict(1)
self.u2mutex = threading.Lock()
self.nbyte = 0
self.nid = None
@@ -258,3 +260,7 @@ class VHttpConn(object):
self.u2fh = FHC()
self.get_u2idx = self.hsrv.get_u2idx
if WINDOWS:
os.system("rem")