Compare commits

..

1 Commits

Author SHA1 Message Date
ed
6987692021 filter apple metadata files when uploading with up2k.js:
abandoned because it is ultimately pointless; browsers running on
macos already do a decent job filtering these out, and there is
only limited benefit of doing this for all other users, probably
more harm than good in case of false positives
2024-01-11 23:42:31 +01:00
123 changed files with 1622 additions and 8083 deletions

1
.gitignore vendored
View File

@@ -12,7 +12,6 @@ copyparty.egg-info/
/dist/
/py2/
/sfx*
/pyz/
/unt/
/log/

3
.vscode/launch.json vendored
View File

@@ -19,7 +19,8 @@
"-emp",
"-e2dsa",
"-e2ts",
"-mtp=.bpm=f,bin/mtag/audio-bpm.py",
"-mtp",
".bpm=f,bin/mtag/audio-bpm.py",
"-aed:wark",
"-vsrv::r:rw,ed:c,dupe",
"-vdist:dist:r"

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2019 ed <oss@ocv.me>
Copyright (c) 2019 ed
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

275
README.md
View File

@@ -3,23 +3,20 @@
turn almost any device into a file server with resumable uploads/downloads using [*any*](#browser-support) web browser
* server only needs Python (2 or 3), all dependencies optional
* 🔌 protocols: [http](#the-browser) // [webdav](#webdav-server) // [ftp](#ftp-server) // [tftp](#tftp-server) // [smb/cifs](#smb-server)
* 🔌 protocols: [http](#the-browser) // [ftp](#ftp-server) // [webdav](#webdav-server) // [smb/cifs](#smb-server)
* 📱 [android app](#android-app) // [iPhone shortcuts](#ios-shortcuts)
👉 **[Get started](#quickstart)!** or visit the **[read-only demo server](https://a.ocv.me/pub/demo/)** 👀 running from a basement in finland
📷 **screenshots:** [browser](#the-browser) // [upload](#uploading) // [unpost](#unpost) // [thumbnails](#thumbnails) // [search](#searching) // [fsearch](#file-search) // [zip-DL](#zip-downloads) // [md-viewer](#markdown-viewer)
🎬 **videos:** [upload](https://a.ocv.me/pub/demo/pics-vids/up2k.webm) // [cli-upload](https://a.ocv.me/pub/demo/pics-vids/u2cli.webm) // [race-the-beam](https://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)
## readme toc
* top
* [quickstart](#quickstart) - just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py)** -- that's it! 🎉
* [at home](#at-home) - make it accessible over the internet
* [on servers](#on-servers) - you may also want these, especially on servers
* [features](#features) - also see [comparison to similar software](./docs/versus.md)
* [features](#features)
* [testimonials](#testimonials) - small collection of user feedback
* [motivations](#motivations) - project goals / philosophy
* [notes](#notes) - general notes
@@ -40,14 +37,12 @@ turn almost any device into a file server with resumable uploads/downloads using
* [file-search](#file-search) - dropping files into the browser also lets you see if they exist on the server
* [unpost](#unpost) - undo/delete accidental uploads
* [self-destruct](#self-destruct) - uploads can be given a lifetime
* [race the beam](#race-the-beam) - download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm))
* [file manager](#file-manager) - cut/paste, rename, and delete files/folders (if you have permission)
* [batch rename](#batch-rename) - select some files and press `F2` to bring up the rename UI
* [media player](#media-player) - plays almost every audio format there is
* [audio equalizer](#audio-equalizer) - and [dynamic range compressor](https://en.wikipedia.org/wiki/Dynamic_range_compression)
* [fix unreliable playback on android](#fix-unreliable-playback-on-android) - due to phone / app settings
* [markdown viewer](#markdown-viewer) - and there are *two* editors
* [markdown vars](#markdown-vars) - dynamic docs with serverside variable expansion
* [other tricks](#other-tricks)
* [searching](#searching) - search by size, date, path/name, mp3-tags, ...
* [server config](#server-config) - using arguments or config files, or a mix of both
@@ -58,10 +53,8 @@ turn almost any device into a file server with resumable uploads/downloads using
* [ftp server](#ftp-server) - an FTP server can be started using `--ftp 3921`
* [webdav server](#webdav-server) - with read-write support
* [connecting to webdav from windows](#connecting-to-webdav-from-windows) - using the GUI
* [tftp server](#tftp-server) - a TFTP server (read/write) can be started using `--tftp 3969`
* [smb server](#smb-server) - unsafe, slow, not recommended for wan
* [browser ux](#browser-ux) - tweaking the ui
* [opengraph](#opengraph) - discord and social-media embeds
* [file indexing](#file-indexing) - enables dedup and music search ++
* [exclude-patterns](#exclude-patterns) - to save some time
* [filesystem guards](#filesystem-guards) - avoid traversing into other filesystems
@@ -76,16 +69,14 @@ turn almost any device into a file server with resumable uploads/downloads using
* [upload events](#upload-events) - the older, more powerful approach ([examples](./bin/mtag/))
* [handlers](#handlers) - redefine behavior with plugins ([examples](./bin/handlers/))
* [identity providers](#identity-providers) - replace copyparty passwords with oauth and such
* [using the cloud as storage](#using-the-cloud-as-storage) - connecting to an aws s3 bucket and similar
* [hiding from google](#hiding-from-google) - tell search engines you dont wanna be indexed
* [themes](#themes)
* [complete examples](#complete-examples)
* [reverse-proxy](#reverse-proxy) - running copyparty next to other websites
* [real-ip](#real-ip) - teaching copyparty how to see client IPs
* [prometheus](#prometheus) - metrics/stats can be enabled
* [packages](#packages) - the party might be closer than you think
* [arch package](#arch-package) - now [available on aur](https://aur.archlinux.org/packages/copyparty) maintained by [@icxes](https://github.com/icxes)
* [fedora package](#fedora-package) - does not exist yet
* [fedora package](#fedora-package) - currently **NOT** available on [copr-pypi](https://copr.fedorainfracloud.org/coprs/g/copr/PyPI/)
* [nix package](#nix-package) - `nix profile install github:9001/copyparty`
* [nixos module](#nixos-module)
* [browser support](#browser-support) - TLDR: yes
@@ -100,7 +91,6 @@ turn almost any device into a file server with resumable uploads/downloads using
* [gotchas](#gotchas) - behavior that might be unexpected
* [cors](#cors) - cross-site request config
* [filekeys](#filekeys) - prevent filename bruteforcing
* [dirkeys](#dirkeys) - share specific folders in a volume
* [password hashing](#password-hashing) - you can hash passwords
* [https](#https) - both HTTP and HTTPS are accepted
* [recovering from crashes](#recovering-from-crashes)
@@ -110,11 +100,10 @@ turn almost any device into a file server with resumable uploads/downloads using
* [dependencies](#dependencies) - mandatory deps
* [optional dependencies](#optional-dependencies) - install these to enable bonus features
* [optional gpl stuff](#optional-gpl-stuff)
* [sfx](#sfx) - the self-contained "binary" (recommended!)
* [sfx](#sfx) - the self-contained "binary"
* [copyparty.exe](#copypartyexe) - download [copyparty.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty.exe) (win8+) or [copyparty32.exe](https://github.com/9001/copyparty/releases/latest/download/copyparty32.exe) (win7+)
* [zipapp](#zipapp) - another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz)
* [install on android](#install-on-android)
* [reporting bugs](#reporting-bugs) - ideas for context to include, and where to submit them
* [reporting bugs](#reporting-bugs) - ideas for context to include in bug reports
* [devnotes](#devnotes) - for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
@@ -126,7 +115,6 @@ just run **[copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/
* or if you cannot install python, you can use [copyparty.exe](#copypartyexe) instead
* or install [on arch](#arch-package) [on NixOS](#nixos-module) [through nix](#nix-package)
* or if you are on android, [install copyparty in termux](#install-on-android)
* or if your computer is messed up and nothing else works, [try the pyz](#zipapp)
* or if you prefer to [use docker](./scripts/docker/) 🐋 you can do that too
* docker has all deps built-in, so skip this step:
@@ -134,7 +122,7 @@ enable thumbnails (images/audio/video), media indexing, and audio transcoding by
* **Alpine:** `apk add py3-pillow ffmpeg`
* **Debian:** `apt install --no-install-recommends python3-pil ffmpeg`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg --allowerasing`
* **Fedora:** rpmfusion + `dnf install python3-pillow ffmpeg`
* **FreeBSD:** `pkg install py39-sqlite3 py39-pillow ffmpeg`
* **MacOS:** `port install py-Pillow ffmpeg`
* **MacOS** (alternative): `brew install pillow ffmpeg`
@@ -155,17 +143,6 @@ some recommended options:
* see [accounts and volumes](#accounts-and-volumes) (or `--help-accounts`) for the syntax and other permissions
### at home
make it accessible over the internet by starting a [cloudflare quicktunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/trycloudflare/) like so:
first download [cloudflared](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) and then start the tunnel with `cloudflared tunnel --url http://127.0.0.1:3923`
as the tunnel starts, it will show a URL which you can share to let anyone browse your stash or upload files to you
since people will be connecting through cloudflare, run copyparty with `--xff-hdr cf-connecting-ip` to detect client IPs correctly
### on servers
you may also want these, especially on servers:
@@ -180,24 +157,21 @@ you may also want these, especially on servers:
and remember to open the ports you want; here's a complete example including every feature copyparty has to offer:
```
firewall-cmd --permanent --add-port={80,443,3921,3923,3945,3990}/tcp # --zone=libvirt
firewall-cmd --permanent --add-port=12000-12099/tcp # --zone=libvirt
firewall-cmd --permanent --add-port={69,1900,3969,5353}/udp # --zone=libvirt
firewall-cmd --permanent --add-port=12000-12099/tcp --permanent # --zone=libvirt
firewall-cmd --permanent --add-port={1900,5353}/udp # --zone=libvirt
firewall-cmd --reload
```
(69:tftp, 1900:ssdp, 3921:ftp, 3923:http/https, 3945:smb, 3969:tftp, 3990:ftps, 5353:mdns, 12000:passive-ftp)
(1900:ssdp, 3921:ftp, 3923:http/https, 3945:smb, 3990:ftps, 5353:mdns, 12000:passive-ftp)
## features
also see [comparison to similar software](./docs/versus.md)
* backend stuff
* ☑ IPv6
* ☑ [multiprocessing](#performance) (actual multithreading)
* ☑ volumes (mountpoints)
* ☑ [accounts](#accounts-and-volumes)
* ☑ [ftp server](#ftp-server)
* ☑ [tftp server](#tftp-server)
* ☑ [webdav server](#webdav-server)
* ☑ [smb/cifs server](#smb-server)
* ☑ [qr-code](#qr-code) for quick access
@@ -213,7 +187,6 @@ also see [comparison to similar software](./docs/versus.md)
* ☑ write-only folders
* ☑ [unpost](#unpost): undo/delete accidental uploads
* ☑ [self-destruct](#self-destruct) (specified server-side or client-side)
* ☑ [race the beam](#race-the-beam) (almost like peer-to-peer)
* ☑ symlink/discard duplicates (content-matching)
* download
* ☑ single files in browser
@@ -222,7 +195,7 @@ also see [comparison to similar software](./docs/versus.md)
* browser
* ☑ [navpane](#navpane) (directory tree sidebar)
* ☑ file manager (cut/paste, delete, [batch-rename](#batch-rename))
* ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus/mp3 transcoding)
* ☑ audio player (with [OS media controls](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) and opus transcoding)
* ☑ image gallery with webm player
* ☑ textfile browser with syntax hilighting
* ☑ [thumbnails](#thumbnails)
@@ -230,7 +203,6 @@ also see [comparison to similar software](./docs/versus.md)
* ☑ ...of videos using FFmpeg
* ☑ ...of audio (spectrograms) using FFmpeg
* ☑ cache eviction (max-age; maybe max-size eventually)
* ☑ multilingual UI (english, norwegian, [add your own](./docs/rice/#translations)))
* ☑ SPA (browse while uploading)
* server indexing
* ☑ [locate files by contents](#file-search)
@@ -239,11 +211,9 @@ also see [comparison to similar software](./docs/versus.md)
* client support
* ☑ [folder sync](#folder-sync)
* ☑ [curl-friendly](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png)
* ☑ [opengraph](#opengraph) (discord embeds)
* markdown
* ☑ [viewer](#markdown-viewer)
* ☑ editor (sure why not)
* ☑ [variables](#markdown-vars)
PS: something missing? post any crazy ideas you've got as a [feature request](https://github.com/9001/copyparty/issues/new?assignees=9001&labels=enhancement&template=feature_request.md) or [discussion](https://github.com/9001/copyparty/discussions/new?category=ideas) 🤙
@@ -314,9 +284,6 @@ roughly sorted by chance of encounter
* cannot index non-ascii filenames with `-e2d`
* cannot handle filenames with mojibake
if you have a new exciting bug to share, see [reporting bugs](#reporting-bugs)
## not my bugs
same order here too
@@ -372,24 +339,9 @@ upgrade notes
* yes, using the [`g` permission](#accounts-and-volumes), see the examples there
* you can also do this with linux filesystem permissions; `chmod 111 music` will make it possible to access files and folders inside the `music` folder but not list the immediate contents -- also works with other software, not just copyparty
* can I link someone to a password-protected volume/file by including the password in the URL?
* yes, by adding `?pw=hunter2` to the end; replace `?` with `&` if there are parameters in the URL already, meaning it contains a `?` near the end
* how do I stop `.hist` folders from appearing everywhere on my HDD?
* by default, a `.hist` folder is created inside each volume for the filesystem index, thumbnails, audio transcodes, and markdown document history. Use the `--hist` global-option or the `hist` volflag to move it somewhere else; see [database location](#database-location)
* can I make copyparty download a file to my server if I give it a URL?
* yes, using [hooks](https://github.com/9001/copyparty/blob/hovudstraum/bin/hooks/wget.py)
* firefox refuses to connect over https, saying "Secure Connection Failed" or "SEC_ERROR_BAD_SIGNATURE", but the usual button to "Accept the Risk and Continue" is not shown
* firefox has corrupted its certstore; fix this by exiting firefox, then find and delete the file named `cert9.db` somewhere in your firefox profile folder
* the server keeps saying `thank you for playing` when I try to access the website
* you've gotten banned for malicious traffic! if this happens by mistake, and you're running a reverse-proxy and/or something like cloudflare, see [real-ip](#real-ip) on how to fix this
* copyparty seems to think I am using http, even though the URL is https
* your reverse-proxy is not sending the `X-Forwarded-Proto: https` header; this could be because your reverse-proxy itself is confused. Ensure that none of the intermediates (such as cloudflare) are terminating https before the traffic hits your entrypoint
* i want to learn python and/or programming and am considering looking at the copyparty source code in that occasion
* ```bash
_| _ __ _ _|_
@@ -413,7 +365,7 @@ configuring accounts/volumes with arguments:
`-v .::r,usr1,usr2:rw,usr3,usr4` = usr1/2 read-only, 3/4 read-write
permissions:
* `r` (read): browse folder contents, download files, download as zip/tar, see filekeys/dirkeys
* `r` (read): browse folder contents, download files, download as zip/tar
* `w` (write): upload files, move files *into* this folder
* `m` (move): move files/folders *from* this folder
* `d` (delete): delete files/folders
@@ -613,27 +565,21 @@ you can also zip a selection of files or folders by clicking them in the browser
![copyparty-zipsel-fs8](https://user-images.githubusercontent.com/241032/129635374-e5136e01-470a-49b1-a762-848e8a4c9cdc.png)
cool trick: download a folder by appending url-params `?tar&opus` or `?tar&mp3` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus/mp3 before they're added to the archive
cool trick: download a folder by appending url-params `?tar&opus` to transcode all audio files (except aac|m4a|mp3|ogg|opus|wma) to opus before they're added to the archive
* super useful if you're 5 minutes away from takeoff and realize you don't have any music on your phone but your server only has flac files and downloading those will burn through all your data + there wouldn't be enough time anyways
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images (`&p` for audio waveforms)
* and url-params `&j` / `&w` produce jpeg/webm thumbnails/spectrograms instead of the original audio/video/images
* can also be used to pregenerate thumbnails; combine with `--th-maxage=9999999` or `--th-clean=0`
## uploading
drag files/folders into the web-browser to upload
drag files/folders into the web-browser to upload (or use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy))
dragdrop is the recommended way, but you may also:
* select some files (not folders) in your file explorer and press CTRL-V inside the browser window
* use the [command-line uploader](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy)
* upload using [curl or sharex](#client-examples)
when uploading files through dragdrop or CTRL-V, this initiates an upload using `up2k`; there are two browser-based uploaders available:
this initiates an upload using `up2k`; there are two uploaders available:
* `[🎈] bup`, the basic uploader, supports almost every browser since netscape 4.0
* `[🚀] up2k`, the good / fancy one
NB: you can undo/delete your own uploads with `[🧯]` [unpost](#unpost) (and this is also where you abort unfinished uploads, but you have to refresh the page first)
NB: you can undo/delete your own uploads with `[🧯]` [unpost](#unpost)
up2k has several advantages:
* you can drop folders into the browser (files are added recursively)
@@ -648,7 +594,7 @@ up2k has several advantages:
> it is perfectly safe to restart / upgrade copyparty while someone is uploading to it!
> all known up2k clients will resume just fine 💪
see [up2k](./docs/devnotes.md#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
see [up2k](#up2k) for details on how it works, or watch a [demo video](https://a.ocv.me/pub/demo/pics-vids/#gf-0f6f5c0d)
![copyparty-upload-fs8](https://user-images.githubusercontent.com/241032/129635371-48fc54ca-fa91-48e3-9b1d-ba413e4b68cb.png)
@@ -714,13 +660,6 @@ clients can specify a shorter expiration time using the [up2k ui](#uploading) --
specifying a custom expiration time client-side will affect the timespan in which unposts are permitted, so keep an eye on the estimates in the up2k ui
### race the beam
download files while they're still uploading ([demo video](http://a.ocv.me/pub/g/nerd-stuff/cpp/2024-0418-race-the-beam.webm)) -- it's almost like peer-to-peer
requires the file to be uploaded using up2k (which is the default drag-and-drop uploader), alternatively the command-line program
## file manager
cut/paste, rename, and delete files/folders (if you have permission)
@@ -800,8 +739,7 @@ some hilights:
click the `play` link next to an audio file, or copy the link target to [share it](https://a.ocv.me/pub/demo/music/Ubiktune%20-%20SOUNDSHOCK%202%20-%20FM%20FUNK%20TERRROR!!/#af-1fbfba61&t=18) (optionally with a timestamp to start playing from, like that example does)
open the `[🎺]` media-player-settings tab to configure it,
* "switches":
* `[🔀]` shuffles the files inside each folder
* switches:
* `[preload]` starts loading the next track when it's about to end, reduces the silence between songs
* `[full]` does a full preload by downloading the entire next file; good for unreliable connections, bad for slow connections
* `[~s]` toggles the seekbar waveform display
@@ -811,15 +749,13 @@ open the `[🎺]` media-player-settings tab to configure it,
* `[art]` shows album art on the lockscreen
* `[🎯]` keeps the playing song scrolled into view (good when using the player as a taskbar dock)
* `[⟎]` shrinks the playback controls
* "buttons":
* `[uncache]` may fix songs that won't play correctly due to bad files in browser cache
* "at end of folder":
* playback mode:
* `[loop]` keeps looping the folder
* `[next]` plays into the next folder
* "transcode":
* `[flac]` converts `flac` and `wav` files into opus (if supported by browser) or mp3
* `[aac]` converts `aac` and `m4a` files into opus (if supported by browser) or mp3
* `[oth]` converts all other known formats into opus (if supported by browser) or mp3
* transcode:
* `[flac]` converts `flac` and `wav` files into opus
* `[aac]` converts `aac` and `m4a` files into opus
* `[oth]` converts all other known formats into opus
* `aac|ac3|aif|aiff|alac|alaw|amr|ape|au|dfpwm|dts|flac|gsm|it|m4a|mo3|mod|mp2|mp3|mpc|mptm|mt2|mulaw|ogg|okt|opus|ra|s3m|tak|tta|ulaw|wav|wma|wv|xm|xpk`
* "tint" reduces the contrast of the playback bar
@@ -856,13 +792,6 @@ other notes,
* the document preview has a max-width which is the same as an A4 paper when printed
### markdown vars
dynamic docs with serverside variable expansion to replace stuff like `{{self.ip}}` with the client's IP, or `{{srv.htime}}` with the current time on the server
see [./srv/expand/](./srv/expand/) for usage and examples
## other tricks
* you can link a particular timestamp in an audio file by adding it to the URL, such as `&20` / `&20s` / `&1m20` / `&t=1:20` after the `.../#af-c8960dab`
@@ -911,8 +840,6 @@ using arguments or config files, or a mix of both:
**NB:** as humongous as this readme is, there is also a lot of undocumented features. Run copyparty with `--help` to see all available global options; all of those can be used in the `[global]` section of config files, and everything listed in `--help-flags` can be used in volumes as volflags.
* if running in docker/podman, try this: `docker run --rm -it copyparty/ac --help`
* or see this (probably outdated): https://ocv.me/copyparty/helptext.html
* or if you prefer plaintext, https://ocv.me/copyparty/helptext.txt
## zeroconf
@@ -1013,35 +940,6 @@ known client bugs:
* latin-1 is fine, hiragana is not (not even as shift-jis on japanese xp)
## tftp server
a TFTP server (read/write) can be started using `--tftp 3969` (you probably want [ftp](#ftp-server) instead unless you are *actually* communicating with hardware from the 90s (in which case we should definitely hang some time))
> that makes this the first RTX DECT Base that has been updated using copyparty 🎉
* based on [partftpy](https://github.com/9001/partftpy)
* no accounts; read from world-readable folders, write to world-writable, overwrite in world-deletable
* needs a dedicated port (cannot share with the HTTP/HTTPS API)
* run as root (or see below) to use the spec-recommended port `69` (nice)
* can reply from a predefined portrange (good for firewalls)
* only supports the binary/octet/image transfer mode (no netascii)
* [RFC 7440](https://datatracker.ietf.org/doc/html/rfc7440) is **not** supported, so will be extremely slow over WAN
* assuming default blksize (512), expect 1100 KiB/s over 100BASE-T, 400-500 KiB/s over wifi, 200 on bad wifi
most clients expect to find TFTP on port 69, but on linux and macos you need to be root to listen on that. Alternatively, listen on 3969 and use NAT on the server to forward 69 to that port;
* on linux: `iptables -t nat -A PREROUTING -i eth0 -p udp --dport 69 -j REDIRECT --to-port 3969`
some recommended TFTP clients:
* curl (cross-platform, read/write)
* get: `curl --tftp-blksize 1428 tftp://127.0.0.1:3969/firmware.bin`
* put: `curl --tftp-blksize 1428 -T firmware.bin tftp://127.0.0.1:3969/`
* windows: `tftp.exe` (you probably already have it)
* `tftp -i 127.0.0.1 put firmware.bin`
* linux: `tftp-hpa`, `atftp`
* `atftp --option "blksize 1428" 127.0.0.1 3969 -p -l firmware.bin -r firmware.bin`
* `tftp -v -m binary 127.0.0.1 3969 -c put firmware.bin`
## smb server
unsafe, slow, not recommended for wan, enable with `--smb` for read-only or `--smbw` for read-write
@@ -1072,7 +970,7 @@ known client bugs:
* however smb1 is buggy and is not enabled by default on win10 onwards
* windows cannot access folders which contain filenames with invalid unicode or forbidden characters (`<>:"/\|?*`), or names ending with `.`
the smb protocol listens on TCP port 445, which is a privileged port on linux and macos, which would require running copyparty as root. However, this can be avoided by listening on another port using `--smb-port 3945` and then using NAT on the server to forward the traffic from 445 to there;
the smb protocol listens on TCP port 445, which is a privileged port on linux and macos, which would require running copyparty as root. However, this can be avoided by listening on another port using `--smb-port 3945` and then using NAT to forward the traffic from 445 to there;
* on linux: `iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 445 -j REDIRECT --to-port 3945`
authenticate with one of the following:
@@ -1089,23 +987,6 @@ tweaking the ui
* to sort in music order (album, track, artist, title) with filename as fallback, you could `--sort tags/Cirle,tags/.tn,tags/Artist,tags/Title,href`
* to sort by upload date, first enable showing the upload date in the listing with `-e2d -mte +.up_at` and then `--sort tags/.up_at`
see [./docs/rice](./docs/rice) for more, including how to add stuff (css/`<meta>`/...) to the html `<head>` tag, or to add your own translation
## opengraph
discord and social-media embeds
can be enabled globally with `--og` or per-volume with volflag `og`
note that this disables hotlinking because the opengraph spec demands it; to sneak past this intentional limitation, you can enable opengraph selectively by user-agent, for example `--og-ua '(Discord|Twitter|Slack)bot'` (or volflag `og_ua`)
you can also hotlink files regardless by appending `?raw` to the url
NOTE: because discord (and maybe others) strip query args such as `?raw` in opengraph tags, any links which require a filekey or dirkey will not work
if you want to entirely replace the copyparty response with your own jinja2 template, give the template filepath to `--og-tpl` or volflag `og_tpl` (all members of `HttpCli` are available through the `this` object)
## file indexing
@@ -1147,8 +1028,6 @@ to save some time, you can provide a regex pattern for filepaths to only index
similarly, you can fully ignore files/folders using `--no-idx [...]` and `:c,noidx=\.iso$`
* when running on macos, all the usual apple metadata files are excluded by default
if you set `--no-hash [...]` globally, you can enable hashing for specific volumes using flag `:c,nohash=`
### filesystem guards
@@ -1337,26 +1216,11 @@ replace 404 and 403 errors with something completely different (that's it for no
replace copyparty passwords with oauth and such
you can disable the built-in password-based login sysem, and instead replace it with a separate piece of software (an identity provider) which will then handle authenticating / authorizing of users; this makes it possible to login with passkeys / fido2 / webauthn / yubikey / ldap / active directory / oauth / many other single-sign-on contraptions
work is [ongoing](https://github.com/9001/copyparty/issues/62) to support authenticating / authorizing users based on a separate authentication proxy, which makes it possible to support oauth, single-sign-on, etc.
a popular choice is [Authelia](https://www.authelia.com/) (config-file based), another one is [authentik](https://goauthentik.io/) (GUI-based, more complex)
it is currently possible to specify `--idp-h-usr x-username`; copyparty will then skip password validation and blindly trust the username specified in the `X-Username` request header
there is a [docker-compose example](./docs/examples/docker/idp-authelia-traefik) which is hopefully a good starting point (alternatively see [./docs/idp.md](./docs/idp.md) if you're the DIY type)
a more complete example of the copyparty configuration options [look like this](./docs/examples/docker/idp/copyparty.conf)
## using the cloud as storage
connecting to an aws s3 bucket and similar
there is no built-in support for this, but you can use FUSE-software such as [rclone](https://rclone.org/) / [geesefs](https://github.com/yandex-cloud/geesefs) / [JuiceFS](https://juicefs.com/en/) to first mount your cloud storage as a local disk, and then let copyparty use (a folder in) that disk as a volume
you may experience poor upload performance this way, but that can sometimes be fixed by specifying the volflag `sparse` to force the use of sparse files; this has improved the upload speeds from `1.5 MiB/s` to over `80 MiB/s` in one case, but note that you are also more likely to discover funny bugs in your FUSE software this way, so buckle up
someone has also tested geesefs in combination with [gocryptfs](https://nuetzlich.net/gocryptfs/) with surprisingly good results, getting 60 MiB/s upload speeds on a gbit line, but JuiceFS won with 80 MiB/s using its built-in encryption
you may improve performance by specifying larger values for `--iobuf` / `--s-rd-sz` / `--s-wr-sz`
the remaining stuff (accepting user groups through another header, creating volumes on the fly) are still to-do; configuration will probably [look like this](./docs/examples/docker/idp/copyparty.conf)
## hiding from google
@@ -1392,8 +1256,6 @@ the classname of the HTML tag is set according to the selected theme, which is u
see the top of [./copyparty/web/browser.css](./copyparty/web/browser.css) where the color variables are set, and there's layout-specific stuff near the bottom
if you want to change the fonts, see [./docs/rice/](./docs/rice/)
## complete examples
@@ -1454,15 +1316,6 @@ example webserver configs:
* [apache2 config](contrib/apache/copyparty.conf) -- location-based
### real-ip
teaching copyparty how to see client IPs when running behind a reverse-proxy, or a WAF, or another protection service such as cloudflare
if you (and maybe everybody else) keep getting a message that says `thank you for playing`, then you've gotten banned for malicious traffic. This ban applies to the IP address that copyparty *thinks* identifies the shady client -- so, depending on your setup, you might have to tell copyparty where to find the correct IP
for most common setups, there should be a helpful message in the server-log explaining what to do, but see [docs/xff.md](docs/xff.md) if you want to learn more, including a quick hack to **just make it work** (which is **not** recommended, but hey...)
## prometheus
metrics/stats can be enabled at URL `/.cpr/metrics` for grafana / prometheus / etc (openmetrics 1.0.0)
@@ -1542,7 +1395,17 @@ it comes with a [systemd service](./contrib/package/arch/copyparty.service) and
## fedora package
does not exist yet; using the [copr-pypi](https://copr.fedorainfracloud.org/coprs/g/copr/PyPI/) builds is **NOT recommended** because updates can be delayed by [several months](https://github.com/fedora-copr/copr/issues/3056)
currently **NOT** available on [copr-pypi](https://copr.fedorainfracloud.org/coprs/g/copr/PyPI/) , fedora is having issues with their build servers and won't be fixed for several months
if you previously installed copyparty from copr, you may run one of the following commands to upgrade to a more recent version:
```bash
dnf install https://ocv.me/copyparty/fedora/37/python3-copyparty.fc37.noarch.rpm
dnf install https://ocv.me/copyparty/fedora/38/python3-copyparty.fc38.noarch.rpm
dnf install https://ocv.me/copyparty/fedora/39/python3-copyparty.fc39.noarch.rpm
```
to run copyparty as a service, use the [systemd service scripts](https://github.com/9001/copyparty/tree/hovudstraum/contrib/systemd), just replace `/usr/bin/python3 /usr/local/bin/copyparty-sfx.py` with `/usr/bin/copyparty`
## nix package
@@ -1672,8 +1535,8 @@ TLDR: yes
| navpane | - | yep | yep | yep | yep | yep | yep | yep |
| image viewer | - | yep | yep | yep | yep | yep | yep | yep |
| video player | - | yep | yep | yep | yep | yep | yep | yep |
| markdown editor | - | - | `*2` | `*2` | yep | yep | yep | yep |
| markdown viewer | - | `*2` | `*2` | `*2` | yep | yep | yep | yep |
| markdown editor | - | - | yep | yep | yep | yep | yep | yep |
| markdown viewer | - | yep | yep | yep | yep | yep | yep | yep |
| play mp3/m4a | - | yep | yep | yep | yep | yep | yep | yep |
| play ogg/opus | - | - | - | - | yep | yep | `*3` | yep |
| **= feature =** | ie6 | ie9 | ie10 | ie11 | ff 52 | c 49 | iOS | Andr |
@@ -1681,7 +1544,6 @@ TLDR: yes
* internet explorer 6 through 8 behave the same
* firefox 52 and chrome 49 are the final winxp versions
* `*1` yes, but extremely slow (ie10: `1 MiB/s`, ie11: `270 KiB/s`)
* `*2` only able to do plaintext documents (no markdown rendering)
* `*3` iOS 11 and newer, opus only, and requires FFmpeg on the server
quick summary of more eccentric web-browsers trying to view a directory index:
@@ -1734,10 +1596,6 @@ interact with copyparty using non-browser clients
* sharex (screenshot utility): see [./contrib/sharex.sxcu](contrib/#sharexsxcu)
* contextlet (web browser integration); see [contrib contextlet](contrib/#send-to-cppcontextletjson)
* [igloo irc](https://iglooirc.com/): Method: `post` Host: `https://you.com/up/?want=url&pw=hunter2` Multipart: `yes` File parameter: `f`
copyparty returns a truncated sha512sum of your PUT/POST as base64; you can generate the same checksum locally to verify uplaods:
b512(){ printf "$((sha512sum||shasum -a512)|sed -E 's/ .*//;s/(..)/\\x\1/g')"|base64|tr '+/' '-_'|head -c44;}
@@ -1756,7 +1614,7 @@ the commandline uploader [u2c.py](https://github.com/9001/copyparty/tree/hovudst
alternatively there is [rclone](./docs/rclone.md) which allows for bidirectional sync and is *way* more flexible (stream files straight from sftp/s3/gcs to copyparty, ...), although there is no integrity check and it won't work with files over 100 MiB if copyparty is behind cloudflare
* starting from rclone v1.63, rclone is faster than u2c.py on low-latency connections
* starting from rclone v1.63 (currently [in beta](https://beta.rclone.org/?filter=latest)), rclone will also be faster than u2c.py
## mount as drive
@@ -1765,7 +1623,7 @@ a remote copyparty server as a local filesystem; go to the control-panel and cl
alternatively, some alternatives roughly sorted by speed (unreproducible benchmark), best first:
* [rclone-webdav](./docs/rclone.md) (25s), read/WRITE (rclone v1.63 or later)
* [rclone-webdav](./docs/rclone.md) (25s), read/WRITE ([v1.63-beta](https://beta.rclone.org/?filter=latest))
* [rclone-http](./docs/rclone.md) (26s), read-only
* [partyfuse.py](./bin/#partyfusepy) (35s), read-only
* [rclone-ftp](./docs/rclone.md) (47s), read/WRITE
@@ -1805,16 +1663,14 @@ below are some tweaks roughly ordered by usefulness:
* `-q` disables logging and can help a bunch, even when combined with `-lo` to redirect logs to file
* `--hist` pointing to a fast location (ssd) will make directory listings and searches faster when `-e2d` or `-e2t` is set
* and also makes thumbnails load faster, regardless of e2d/e2t
* `--no-hash .` when indexing a network-disk if you don't care about the actual filehashes and only want the names/tags searchable
* if your volumes are on a network-disk such as NFS / SMB / s3, specifying larger values for `--iobuf` and/or `--s-rd-sz` and/or `--s-wr-sz` may help; try setting all of them to `524288` or `1048576` or `4194304`
* `--no-htp --hash-mt=0 --mtag-mt=1 --th-mt=1` minimizes the number of threads; can help in some eccentric environments (like the vscode debugger)
* `-j0` enables multiprocessing (actual multithreading), can reduce latency to `20+80/numCores` percent and generally improve performance in cpu-intensive workloads, for example:
* lots of connections (many users or heavy clients)
* simultaneous downloads and uploads saturating a 20gbps connection
* if `-e2d` is enabled, `-j2` gives 4x performance for directory listings; `-j4` gives 16x
...however it also increases the server/filesystem/HDD load during uploads, and adds an overhead to internal communication, so it is usually a better idea to don't
...however it adds an overhead to internal communication so it might be a net loss, see if it works 4 u
* using [pypy](https://www.pypy.org/) instead of [cpython](https://www.python.org/) *can* be 70% faster for some workloads, but slower for many others
* and pypy can sometimes crash on startup with `-j0` (TODO make issue)
@@ -1823,7 +1679,7 @@ below are some tweaks roughly ordered by usefulness:
when uploading files,
* chrome is recommended (unfortunately), at least compared to firefox:
* chrome is recommended, at least compared to firefox:
* up to 90% faster when hashing, especially on SSDs
* up to 40% faster when uploading over extremely fast internets
* but [u2c.py](https://github.com/9001/copyparty/blob/hovudstraum/bin/u2c.py) can be 40% faster than chrome again
@@ -1904,29 +1760,12 @@ cors can be configured with `--acao` and `--acam`, or the protections entirely d
prevent filename bruteforcing
volflag `fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
volflag `c,fk` generates filekeys (per-file accesskeys) for all files; users which have full read-access (permission `r`) will then see URLs with the correct filekey `?k=...` appended to the end, and `g` users must provide that URL including the correct key to avoid a 404
by default, filekeys are generated based on salt (`--fk-salt`) + filesystem-path + file-size + inode (if not windows); add volflag `fka` to generate slightly weaker filekeys which will not be invalidated if the file is edited (only salt + path)
permissions `wG` (write + upget) lets users upload files and receive their own filekeys, still without being able to see other uploads
### dirkeys
share specific folders in a volume without giving away full read-access to the rest -- the visitor only needs the `g` (get) permission to view the link
volflag `dk` generates dirkeys (per-directory accesskeys) for all folders, granting read-access to that folder; by default only that folder itself, no subfolders
volflag `dky` disables the actual key-check, meaning anyone can see the contents of a folder where they have `g` access, but not its subdirectories
* `dk` + `dky` gives the same behavior as if all users with `g` access have full read-access, but subfolders are hidden files (their names start with a dot), so `dky` is an alternative to renaming all the folders for that purpose, maybe just for some users
volflag `dks` lets people enter subfolders as well, and also enables download-as-zip/tar
dirkeys are generated based on another salt (`--dk-salt`) + filesystem-path and have a few limitations:
* the key does not change if the contents of the folder is modified
* if you need a new dirkey, either change the salt or rename the folder
* linking to a textfile (so it opens in the textfile viewer) is not possible if recipient doesn't have read-access
## password hashing
@@ -2018,7 +1857,7 @@ these are standalone programs and will never be imported / evaluated by copypart
# sfx
the self-contained "binary" (recommended!) [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
the self-contained "binary" [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) will unpack itself and run copyparty, assuming you have python installed of course
you can reduce the sfx size by repacking it; see [./docs/devnotes.md#sfx-repack](./docs/devnotes.md#sfx-repack)
@@ -2042,17 +1881,7 @@ can be convenient on machines where installing python is problematic, however is
meanwhile [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) instead relies on your system python which gives better performance and will stay safe as long as you keep your python install up-to-date
then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) every once ina while if you can afford the size
## zipapp
another emergency alternative, [copyparty.pyz](https://github.com/9001/copyparty/releases/latest/download/copyparty.pyz) has less features, requires python 3.7 or newer, worse compression, and more importantly is unable to benefit from more recent versions of jinja2 and such (which makes it less secure)... lots of drawbacks with this one really -- but it *may* just work if the regular sfx fails to start because the computer is messed up in certain funky ways, so it's worth a shot if all else fails
run it by doubleclicking it, or try typing `python copyparty.pyz` in your terminal/console/commandline/telex if that fails
it is a python [zipapp](https://docs.python.org/3/library/zipapp.html) meaning it doesn't have to unpack its own python code anywhere to run, so if the filesystem is busted it has a better chance of getting somewhere
* but note that it currently still needs to extract the web-resources somewhere (they'll land in the default TEMP-folder of your OS)
then again, if you are already into downloading shady binaries from the internet, you may also want my [minimal builds](./scripts/pyinstaller#ffmpeg) of [ffmpeg](https://ocv.me/stuff/bin/ffmpeg.exe) and [ffprobe](https://ocv.me/stuff/bin/ffprobe.exe) which enables copyparty to extract multimedia-info, do audio-transcoding, and thumbnails/spectrograms/waveforms, however it's much better to instead grab a [recent official build](https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip) every once ina while if you can afford the size
# install on android
@@ -2072,12 +1901,7 @@ if you want thumbnails (photos+videos) and you're okay with spending another 132
# reporting bugs
ideas for context to include, and where to submit them
please get in touch using any of the following URLs:
* https://github.com/9001/copyparty/ **(primary)**
* https://gitlab.com/9001/copyparty/ *(mirror)*
* https://codeberg.org/9001/copyparty *(mirror)*
ideas for context to include in bug reports
in general, commandline arguments (and config file if any)
@@ -2092,6 +1916,3 @@ if there's a wall of base64 in the log (thread stacks) then please include that,
# devnotes
for build instructions etc, see [./docs/devnotes.md](./docs/devnotes.md)
see [./docs/TODO.md](./docs/TODO.md) for planned features / fixes / changes

View File

@@ -223,15 +223,12 @@ install_vamp() {
# use msys2 in mingw-w64 mode
# pacman -S --needed mingw-w64-x86_64-{ffmpeg,python,python-pip,vamp-plugin-sdk}
$pybin -m pip install --user vamp || {
printf '\n\033[7malright, trying something else...\033[0m\n'
$pybin -m pip install --user --no-build-isolation vamp
}
$pybin -m pip install --user vamp
cd "$td"
echo '#include <vamp-sdk/Plugin.h>' | g++ -x c++ -c -o /dev/null - || [ -e ~/pe/vamp-sdk ] || {
printf '\033[33mcould not find the vamp-sdk, building from source\033[0m\n'
(dl_files yolo https://ocv.me/mirror/vamp-plugin-sdk-2.10.0.tar.gz)
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/2691/vamp-plugin-sdk-2.10.0.tar.gz)
sha512sum -c <(
echo "153b7f2fa01b77c65ad393ca0689742d66421017fd5931d216caa0fcf6909355fff74706fabbc062a3a04588a619c9b515a1dae00f21a57afd97902a355c48ed -"
) <vamp-plugin-sdk-2.10.0.tar.gz
@@ -247,7 +244,7 @@ install_vamp() {
cd "$td"
have_beatroot || {
printf '\033[33mcould not find the vamp beatroot plugin, building from source\033[0m\n'
(dl_files yolo https://ocv.me/mirror/beatroot-vamp-v1.0.tar.gz)
(dl_files yolo https://code.soundsoftware.ac.uk/attachments/download/885/beatroot-vamp-v1.0.tar.gz)
sha512sum -c <(
echo "1f444d1d58ccf565c0adfe99f1a1aa62789e19f5071e46857e2adfbc9d453037bc1c4dcb039b02c16240e9b97f444aaff3afb625c86aa2470233e711f55b6874 -"
) <beatroot-vamp-v1.0.tar.gz

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env python3
from __future__ import print_function, unicode_literals
S_VERSION = "1.17"
S_BUILD_DT = "2024-05-09"
S_VERSION = "1.12"
S_BUILD_DT = "2023-12-08"
"""
u2c.py: upload to copyparty
@@ -29,7 +29,7 @@ import platform
import threading
import datetime
EXE = bool(getattr(sys, "frozen", False))
EXE = sys.executable.endswith("exe")
try:
import argparse
@@ -79,21 +79,12 @@ req_ses = requests.Session()
class Daemon(threading.Thread):
def __init__(self, target, name = None, a = None):
threading.Thread.__init__(self, name=name)
self.a = a or ()
self.fun = target
def __init__(self, target, name=None, a=None):
# type: (Any, Any, Any) -> None
threading.Thread.__init__(self, target=target, args=a or (), name=name)
self.daemon = True
self.start()
def run(self):
try:
signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM])
except:
pass
self.fun(*self.a)
class File(object):
"""an up2k upload task; represents a single file"""
@@ -569,11 +560,8 @@ def handshake(ar, file, search):
}
if search:
req["srch"] = 1
else:
if ar.touch:
req["umod"] = True
if ar.ow:
req["replace"] = True
elif ar.dr:
req["replace"] = True
headers = {"Content-Type": "text/plain"} # <=1.5.1 compat
if pw:
@@ -855,12 +843,12 @@ class Ctl(object):
txt = " "
if not self.up_br:
spd = self.hash_b / ((time.time() - self.t0) or 1)
eta = (self.nbytes - self.hash_b) / (spd or 1)
spd = self.hash_b / (time.time() - self.t0)
eta = (self.nbytes - self.hash_b) / (spd + 1)
else:
spd = self.up_br / ((time.time() - self.t0_up) or 1)
spd = self.up_br / (time.time() - self.t0_up)
spd = self.spd = (self.spd or spd) * 0.9 + spd * 0.1
eta = (self.nbytes - self.up_b) / (spd or 1)
eta = (self.nbytes - self.up_b) / (spd + 1)
spd = humansize(spd)
self.eta = str(datetime.timedelta(seconds=int(eta)))
@@ -886,8 +874,6 @@ class Ctl(object):
self.st_hash = [file, ofs]
def hasher(self):
ptn = re.compile(self.ar.x.encode("utf-8"), re.I) if self.ar.x else None
sep = "{0}".format(os.sep).encode("ascii")
prd = None
ls = {}
for top, rel, inf in self.filegen:
@@ -920,12 +906,7 @@ class Ctl(object):
if self.ar.drd:
dp = os.path.join(top, rd)
lnodes = set(os.listdir(dp))
if ptn:
zs = dp.replace(sep, b"/").rstrip(b"/") + b"/"
zls = [zs + x for x in lnodes]
zls = [x for x in zls if not ptn.match(x)]
lnodes = [x.split(b"/")[-1] for x in zls]
bnames = [x for x in ls if x not in lnodes and x != b".hist"]
bnames = [x for x in ls if x not in lnodes]
vpath = self.ar.url.split("://")[-1].split("/", 1)[-1]
names = [x.decode("utf-8", "replace") for x in bnames]
locs = [vpath + srd + "/" + x for x in names]
@@ -1148,8 +1129,6 @@ source file/folder selection uses rsync syntax, meaning that:
ap.add_argument("-s", action="store_true", help="file-search (disables upload)")
ap.add_argument("-x", type=unicode, metavar="REGEX", default="", help="skip file if filesystem-abspath matches REGEX, example: '.*/\\.hist/.*'")
ap.add_argument("--ok", action="store_true", help="continue even if some local files are inaccessible")
ap.add_argument("--touch", action="store_true", help="if last-modified timestamps differ, push local to server (need write+delete perms)")
ap.add_argument("--ow", action="store_true", help="overwrite existing files instead of autorenaming")
ap.add_argument("--version", action="store_true", help="show version and exit")
ap = app.add_argument_group("compatibility")
@@ -1158,7 +1137,7 @@ source file/folder selection uses rsync syntax, meaning that:
ap = app.add_argument_group("folder sync")
ap.add_argument("--dl", action="store_true", help="delete local files after uploading")
ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally (implies --ow)")
ap.add_argument("--dr", action="store_true", help="delete remote files which don't exist locally")
ap.add_argument("--drd", action="store_true", help="delete remote files during upload instead of afterwards; reduces peak disk space usage, but will reupload instead of detecting renames")
ap = app.add_argument_group("performance tweaks")
@@ -1188,9 +1167,6 @@ source file/folder selection uses rsync syntax, meaning that:
if ar.drd:
ar.dr = True
if ar.dr:
ar.ow = True
for k in "dl dr drd".split():
errs = []
if ar.safe and getattr(ar, k):

View File

@@ -16,13 +16,11 @@
* sharex config file to upload screenshots and grab the URL
* `RequestURL`: full URL to the target folder
* `pw`: password (remove the `pw` line if anon-write)
* the `act:bput` thing is optional since copyparty v1.9.29
* using an older sharex version, maybe sharex v12.1.1 for example? dw fam i got your back 👉😎👉 [`sharex12.sxcu`](sharex12.sxcu)
### [`send-to-cpp.contextlet.json`](send-to-cpp.contextlet.json)
* browser integration, kind of? custom rightclick actions and stuff
* rightclick a pic and send it to copyparty straight from your browser
* for the [contextlet](https://addons.mozilla.org/en-US/firefox/addon/contextlets/) firefox extension
however if your copyparty is behind a reverse-proxy, you may want to use [`sharex-html.sxcu`](sharex-html.sxcu) instead:
* `RequestURL`: full URL to the target folder
* `URL`: full URL to the root folder (with trailing slash) followed by `$regex:1|1$`
* `pw`: password (remove `Parameters` if anon-write)
### [`media-osd-bgone.ps1`](media-osd-bgone.ps1)
* disables the [windows OSD popup](https://user-images.githubusercontent.com/241032/122821375-0e08df80-d2dd-11eb-9fd9-184e8aacf1d0.png) (the thing on the left) which appears every time you hit media hotkeys to adjust volume or change song while playing music with the copyparty web-ui, or most other audio players really

View File

@@ -11,14 +11,6 @@
# (5'000 requests per second, or 20gbps upload/download in parallel)
#
# on fedora/rhel, remember to setsebool -P httpd_can_network_connect 1
#
# if you are behind cloudflare (or another protection service),
# remember to reject all connections which are not coming from your
# protection service -- for cloudflare in particular, you can
# generate the list of permitted IP ranges like so:
# (curl -s https://www.cloudflare.com/ips-v{4,6} | sed 's/^/allow /; s/$/;/'; echo; echo "deny all;") > /etc/nginx/cloudflare-only.conf
#
# and then enable it below by uncomenting the cloudflare-only.conf line
upstream cpp {
server 127.0.0.1:3923 fail_timeout=1s;
@@ -29,10 +21,7 @@ server {
listen [::]:443 ssl;
server_name fs.example.com;
# uncomment the following line to reject non-cloudflare connections, ensuring client IPs cannot be spoofed:
#include /etc/nginx/cloudflare-only.conf;
location / {
proxy_pass http://cpp;
proxy_redirect off;

View File

@@ -1,8 +1,8 @@
# Maintainer: icxes <dev.null@need.moe>
pkgname=copyparty
pkgver="1.13.1"
pkgver="1.9.28"
pkgrel=1
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, TFTP, zeroconf, media indexer, thumbnails++"
pkgdesc="File server with accelerated resumable uploads, dedup, WebDAV, FTP, zeroconf, media indexer, thumbnails++"
arch=("any")
url="https://github.com/9001/${pkgname}"
license=('MIT')
@@ -21,7 +21,7 @@ optdepends=("ffmpeg: thumbnails for videos, images (slower) and audio, music tag
)
source=("https://github.com/9001/${pkgname}/releases/download/v${pkgver}/${pkgname}-${pkgver}.tar.gz")
backup=("etc/${pkgname}.d/init" )
sha256sums=("f103b784c423a45fbab47c584e4cc53d887fe0616f803bffe009fbfdab3963d7")
sha256sums=("0134f2298b8793f3a8a0a9ba6e7e5a5bf3dc12da5501b100c2f17d061f167fb9")
build() {
cd "${srcdir}/${pkgname}-${pkgver}"

View File

@@ -1,5 +1,5 @@
{
"url": "https://github.com/9001/copyparty/releases/download/v1.13.1/copyparty-sfx.py",
"version": "1.13.1",
"hash": "sha256-NFfnveCrR1SbiNlibVyU3UPePLUGJMc4XZvWdksXNd8="
"url": "https://github.com/9001/copyparty/releases/download/v1.9.28/copyparty-sfx.py",
"version": "1.9.28",
"hash": "sha256-3pHlJYmEu0C/aHPN8JL9fb6tEHYpB3/gON7SJko3tUY="
}

View File

@@ -1,11 +0,0 @@
{
"code": "// https://addons.mozilla.org/en-US/firefox/addon/contextlets/\n// https://github.com/davidmhammond/contextlets\n\nvar url = 'http://partybox.local:3923/';\nvar pw = 'wark';\n\nvar xhr = new XMLHttpRequest();\nxhr.msg = this.info.linkUrl || this.info.srcUrl;\nxhr.open('POST', url, true);\nxhr.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded;charset=UTF-8');\nxhr.setRequestHeader('PW', pw);\nxhr.send('msg=' + xhr.msg);\n",
"contexts": [
"link"
],
"icons": null,
"patterns": "",
"scope": "background",
"title": "send to cpp",
"type": "normal"
}

19
contrib/sharex-html.sxcu Normal file
View File

@@ -0,0 +1,19 @@
{
"Version": "13.5.0",
"Name": "copyparty-html",
"DestinationType": "ImageUploader",
"RequestMethod": "POST",
"RequestURL": "http://127.0.0.1:3923/sharex",
"Parameters": {
"pw": "wark"
},
"Body": "MultipartFormData",
"Arguments": {
"act": "bput"
},
"FileFormName": "f",
"RegexList": [
"bytes // <a href=\"/([^\"]+)\""
],
"URL": "http://127.0.0.1:3923/$regex:1|1$"
}

View File

@@ -1,19 +1,17 @@
{
"Version": "15.0.0",
"Version": "13.5.0",
"Name": "copyparty",
"DestinationType": "ImageUploader",
"RequestMethod": "POST",
"RequestURL": "http://127.0.0.1:3923/sharex",
"Parameters": {
"pw": "wark",
"j": null
},
"Headers": {
"pw": "PUT_YOUR_PASSWORD_HERE_MY_DUDE"
},
"Body": "MultipartFormData",
"Arguments": {
"act": "bput"
},
"FileFormName": "f",
"URL": "{json:files[0].url}"
"URL": "$json:files[0].url$"
}

View File

@@ -1,13 +0,0 @@
{
"Name": "copyparty",
"DestinationType": "ImageUploader, TextUploader, FileUploader",
"RequestURL": "http://127.0.0.1:3923/sharex",
"FileFormName": "f",
"Arguments": {
"act": "bput"
},
"Headers": {
"accept": "url",
"pw": "PUT_YOUR_PASSWORD_HERE_MY_DUDE"
}
}

View File

@@ -56,6 +56,7 @@ class EnvParams(object):
self.t0 = time.time()
self.mod = ""
self.cfg = ""
self.ox = getattr(sys, "oxidized", None)
E = EnvParams()

203
copyparty/__main__.py Normal file → Executable file
View File

@@ -20,40 +20,25 @@ import time
import traceback
import uuid
from .__init__ import (
ANYWIN,
CORES,
EXE,
MACOS,
PY2,
VT100,
WINDOWS,
E,
EnvParams,
unicode,
)
from .__init__ import ANYWIN, CORES, EXE, PY2, VT100, WINDOWS, E, EnvParams, unicode
from .__version__ import CODENAME, S_BUILD_DT, S_VERSION
from .authsrv import expand_config_file, split_cfg_ln, upgrade_cfg_fmt
from .cfg import flagcats, onedash
from .svchub import SvcHub
from .util import (
APPLESAN_TXT,
DEF_EXP,
DEF_MTE,
DEF_MTH,
IMPLICATIONS,
JINJA_VER,
MIMES,
PARTFTPY_VER,
PY_DESC,
PYFTPD_VER,
SQLITE_VER,
UNPLICATIONS,
Daemon,
align_tab,
ansi_re,
dedent,
min_ex,
py_desc,
pybin,
termsize,
wrap,
@@ -159,8 +144,7 @@ def warn(msg: str) -> None:
def init_E(EE: EnvParams) -> None:
# some cpython alternatives (such as pyoxidizer) can
# __init__ several times, so do expensive stuff here
# __init__ runs 18 times when oxidized; do expensive stuff here
E = EE # pylint: disable=redefined-outer-name
@@ -193,9 +177,34 @@ def init_E(EE: EnvParams) -> None:
raise Exception("could not find a writable path for config")
E.mod = os.path.dirname(os.path.realpath(__file__))
if E.mod.endswith("__init__"):
E.mod = os.path.dirname(E.mod)
def _unpack() -> str:
import atexit
import tarfile
import tempfile
from importlib.resources import open_binary
td = tempfile.TemporaryDirectory(prefix="")
atexit.register(td.cleanup)
tdn = td.name
with open_binary("copyparty", "z.tar") as tgz:
with tarfile.open(fileobj=tgz) as tf:
try:
tf.extractall(tdn, filter="tar")
except TypeError:
tf.extractall(tdn) # nosec (archive is safe)
return tdn
try:
E.mod = os.path.dirname(os.path.realpath(__file__))
if E.mod.endswith("__init__"):
E.mod = os.path.dirname(E.mod)
except:
if not E.ox:
raise
E.mod = _unpack()
if sys.platform == "win32":
bdir = os.environ.get("APPDATA") or os.environ.get("TEMP") or "."
@@ -252,19 +261,6 @@ def get_fk_salt() -> str:
return ret.decode("utf-8")
def get_dk_salt() -> str:
fp = os.path.join(E.cfg, "dk-salt.txt")
try:
with open(fp, "rb") as f:
ret = f.read().strip()
except:
ret = base64.b64encode(os.urandom(30))
with open(fp, "wb") as f:
f.write(ret + b"\n")
return ret.decode("utf-8")
def get_ah_salt() -> str:
fp = os.path.join(E.cfg, "ah-salt.txt")
try:
@@ -386,7 +382,7 @@ def configure_ssl_ciphers(al: argparse.Namespace) -> None:
def args_from_cfg(cfg_path: str) -> list[str]:
lines: list[str] = []
expand_config_file(None, lines, cfg_path, "")
expand_config_file(lines, cfg_path, "")
lines = upgrade_cfg_fmt(None, argparse.Namespace(vc=False), lines, "")
ret: list[str] = []
@@ -472,16 +468,6 @@ def disable_quickedit() -> None:
cmode(True, mode | 4)
def sfx_tpoke(top: str):
files = [os.path.join(dp, p) for dp, dd, df in os.walk(top) for p in dd + df]
while True:
t = int(time.time())
for f in [top] + files:
os.utime(f, (t, t))
time.sleep(78123)
def showlic() -> None:
p = os.path.join(E.mod, "res", "COPYING.txt")
if not os.path.exists(p):
@@ -504,10 +490,6 @@ def get_sects():
* "\033[33mperm\033[0m" is "permissions,username1,username2,..."
* "\033[32mvolflag\033[0m" is config flags to set on this volume
--grp takes groupname:username1,username2,...
and groupnames can be used instead of usernames in -v
by prefixing the groupname with %
list of permissions:
"r" (read): list folder contents, download files
"w" (write): upload files; need "r" to see the uploads
@@ -832,7 +814,7 @@ def build_flags_desc():
v = v.replace("\n", "\n ")
ret += "\n \033[36m{}\033[35m {}".format(k, v)
return ret
return ret + "\033[0m"
# fmt: off
@@ -844,14 +826,11 @@ def add_general(ap, nc, srvname):
ap2.add_argument("-nc", metavar="NUM", type=int, default=nc, help="max num clients")
ap2.add_argument("-j", metavar="CORES", type=int, default=1, help="max num cpu cores, 0=all")
ap2.add_argument("-a", metavar="ACCT", type=u, action="append", help="add account, \033[33mUSER\033[0m:\033[33mPASS\033[0m; example [\033[32med:wark\033[0m]")
ap2.add_argument("-v", metavar="VOL", type=u, action="append", help="add volume, \033[33mSRC\033[0m:\033[33mDST\033[0m:\033[33mFLAG\033[0m; examples [\033[32m.::r\033[0m], [\033[32m/mnt/nas/music:/music:r:aed\033[0m], see --help-accounts")
ap2.add_argument("--grp", metavar="G:N,N", type=u, action="append", help="add group, \033[33mNAME\033[0m:\033[33mUSER1\033[0m,\033[33mUSER2\033[0m,\033[33m...\033[0m; example [\033[32madmins:ed,foo,bar\033[0m]")
ap2.add_argument("-v", metavar="VOL", type=u, action="append", help="add volume, \033[33mSRC\033[0m:\033[33mDST\033[0m:\033[33mFLAG\033[0m; examples [\033[32m.::r\033[0m], [\033[32m/mnt/nas/music:/music:r:aed\033[0m]")
ap2.add_argument("-ed", action="store_true", help="enable the ?dots url parameter / client option which allows clients to see dotfiles / hidden files (volflag=dots)")
ap2.add_argument("--urlform", metavar="MODE", type=u, default="print,get", help="how to handle url-form POSTs; see \033[33m--help-urlform\033[0m")
ap2.add_argument("--wintitle", metavar="TXT", type=u, default="cpp @ $pub", help="server terminal title, for example [\033[32m$ip-10.1.2.\033[0m] or [\033[32m$ip-]")
ap2.add_argument("--name", metavar="TXT", type=u, default=srvname, help="server name (displayed topleft in browser and in mDNS)")
ap2.add_argument("--mime", metavar="EXT=MIME", type=u, action="append", help="map file \033[33mEXT\033[0mension to \033[33mMIME\033[0mtype, for example [\033[32mjpg=image/jpeg\033[0m]")
ap2.add_argument("--mimes", action="store_true", help="list default mimetype mapping and exit")
ap2.add_argument("--license", action="store_true", help="show licenses and exit")
ap2.add_argument("--version", action="store_true", help="show versions and exit")
@@ -868,21 +847,11 @@ def add_qr(ap, tty):
ap2.add_argument("--qrz", metavar="N", type=int, default=0, help="[\033[32m1\033[0m]=1x, [\033[32m2\033[0m]=2x, [\033[32m0\033[0m]=auto (try [\033[32m2\033[0m] on broken fonts)")
def add_fs(ap):
ap2 = ap.add_argument_group("filesystem options")
rm_re_def = "15/0.1" if ANYWIN else "0/0"
ap2.add_argument("--rm-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be deleted because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=rm_retry)")
ap2.add_argument("--mv-retry", metavar="T/R", type=u, default=rm_re_def, help="if a file cannot be renamed because it is busy, continue trying for \033[33mT\033[0m seconds, retry every \033[33mR\033[0m seconds; disable with 0/0 (volflag=mv_retry)")
ap2.add_argument("--iobuf", metavar="BYTES", type=int, default=256*1024, help="file I/O buffer-size; if your volumes are on a network drive, try increasing to \033[32m524288\033[0m or even \033[32m4194304\033[0m (and let me know if that improves your performance)")
ap2.add_argument("--mtab-age", metavar="SEC", type=int, default=60, help="rebuild mountpoint cache every \033[33mSEC\033[0m to keep track of sparse-files support; keep low on servers with removable media")
def add_upload(ap):
ap2 = ap.add_argument_group('upload options')
ap2.add_argument("--dotpart", action="store_true", help="dotfile incomplete uploads, hiding them from clients unless \033[33m-ed\033[0m")
ap2.add_argument("--plain-ip", action="store_true", help="when avoiding filename collisions by appending the uploader's ip to the filename: append the plaintext ip instead of salting and hashing the ip")
ap2.add_argument("--unpost", metavar="SEC", type=int, default=3600*12, help="grace period where uploads can be deleted by the uploader, even without delete permissions; 0=disabled, default=12h")
ap2.add_argument("--u2abort", metavar="NUM", type=int, default=1, help="clients can abort incomplete uploads by using the unpost tab (requires \033[33m-e2d\033[0m). [\033[32m0\033[0m] = never allowed (disable feature), [\033[32m1\033[0m] = allow if client has the same IP as the upload AND is using the same account, [\033[32m2\033[0m] = just check the IP, [\033[32m3\033[0m] = just check account-name (volflag=u2abort)")
ap2.add_argument("--blank-wt", metavar="SEC", type=int, default=300, help="file write grace period (any client can write to a blank file last-modified more recently than \033[33mSEC\033[0m seconds ago)")
ap2.add_argument("--reg-cap", metavar="N", type=int, default=38400, help="max number of uploads to keep in memory when running without \033[33m-e2d\033[0m; roughly 1 MiB RAM per 600")
ap2.add_argument("--no-fpool", action="store_true", help="disable file-handle pooling -- instead, repeatedly close and reopen files during upload (bad idea to enable this on windows and/or cow filesystems)")
@@ -901,7 +870,7 @@ def add_upload(ap):
ap2.add_argument("--df", metavar="GiB", type=float, default=0, help="ensure \033[33mGiB\033[0m free disk space by rejecting upload requests")
ap2.add_argument("--sparse", metavar="MiB", type=int, default=4, help="windows-only: minimum size of incoming uploads through up2k before they are made into sparse files")
ap2.add_argument("--turbo", metavar="LVL", type=int, default=0, help="configure turbo-mode in up2k client; [\033[32m-1\033[0m] = forbidden/always-off, [\033[32m0\033[0m] = default-off and warn if enabled, [\033[32m1\033[0m] = default-off, [\033[32m2\033[0m] = on, [\033[32m3\033[0m] = on and disable datecheck")
ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16 for cross-atlantic (max=64)")
ap2.add_argument("--u2j", metavar="JOBS", type=int, default=2, help="web-client: number of file chunks to upload in parallel; 1 or 2 is good for low-latency (same-country) connections, 4-8 for android clients, 16-32 for cross-atlantic (max=64)")
ap2.add_argument("--u2sort", metavar="TXT", type=u, default="s", help="upload order; [\033[32ms\033[0m]=smallest-first, [\033[32mn\033[0m]=alphabetical, [\033[32mfs\033[0m]=force-s, [\033[32mfn\033[0m]=force-n -- alphabetical is a bit slower on fiber/LAN but makes it easier to eyeball if everything went fine")
ap2.add_argument("--write-uplog", action="store_true", help="write POST reports to textfiles in working-directory")
@@ -913,8 +882,8 @@ def add_network(ap):
ap2.add_argument("--ll", action="store_true", help="include link-local IPv4/IPv6 in mDNS replies, even if the NIC has routable IPs (breaks some mDNS clients)")
ap2.add_argument("--rproxy", metavar="DEPTH", type=int, default=1, help="which ip to associate clients with; [\033[32m0\033[0m]=tcp, [\033[32m1\033[0m]=origin (first x-fwd, unsafe), [\033[32m2\033[0m]=outermost-proxy, [\033[32m3\033[0m]=second-proxy, [\033[32m-1\033[0m]=closest-proxy")
ap2.add_argument("--xff-hdr", metavar="NAME", type=u, default="x-forwarded-for", help="if reverse-proxied, which http header to read the client's real ip from")
ap2.add_argument("--xff-src", metavar="CIDR", type=u, default="127.0.0.0/8, ::1/128", help="comma-separated list of trusted reverse-proxy CIDRs; only accept the real-ip header (\033[33m--xff-hdr\033[0m) and IdP headers if the incoming connection is from an IP within either of these subnets. Specify [\033[32mlan\033[0m] to allow all LAN / private / non-internet IPs. Can be disabled with [\033[32many\033[0m] if you are behind cloudflare (or similar) and are using \033[32m--xff-hdr=cf-connecting-ip\033[0m (or similar)")
ap2.add_argument("--ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m; examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
ap2.add_argument("--xff-src", metavar="IP", type=u, default="127., ::1", help="comma-separated list of trusted reverse-proxy IPs; only accept the real-ip header (\033[33m--xff-hdr\033[0m) if the incoming connection is from an IP starting with either of these. Can be disabled with [\033[32many\033[0m] if you are behind cloudflare (or similar) and are using \033[32m--xff-hdr=cf-connecting-ip\033[0m (or similar)")
ap2.add_argument("--ipa", metavar="PREFIX", type=u, default="", help="only accept connections from IP-addresses starting with \033[33mPREFIX\033[0m; example: [\033[32m127., 10.89., 192.168.\033[0m]")
ap2.add_argument("--rp-loc", metavar="PATH", type=u, default="", help="if reverse-proxying on a location instead of a dedicated domain/subdomain, provide the base location here; example: [\033[32m/foo/bar\033[0m]")
if ANYWIN:
ap2.add_argument("--reuseaddr", action="store_true", help="set reuseaddr on listening sockets on windows; allows rapid restart of copyparty at the expense of being able to accidentally start multiple instances")
@@ -922,7 +891,6 @@ def add_network(ap):
ap2.add_argument("--freebind", action="store_true", help="allow listening on IPs which do not yet exist, for example if the network interfaces haven't finished going up. Only makes sense for IPs other than '0.0.0.0', '127.0.0.1', '::', and '::1'. May require running as root (unless net.ipv6.ip_nonlocal_bind)")
ap2.add_argument("--s-thead", metavar="SEC", type=int, default=120, help="socket timeout (read request header)")
ap2.add_argument("--s-tbody", metavar="SEC", type=float, default=186, help="socket timeout (read/write request/response bodies). Use 60 on fast servers (default is extremely safe). Disable with 0 if reverse-proxied for a 2%% speed boost")
ap2.add_argument("--s-rd-sz", metavar="B", type=int, default=256*1024, help="socket read size in bytes (indirectly affects filesystem writes; recommendation: keep equal-to or lower-than \033[33m--iobuf\033[0m)")
ap2.add_argument("--s-wr-sz", metavar="B", type=int, default=256*1024, help="socket write size in bytes")
ap2.add_argument("--s-wr-slp", metavar="SEC", type=float, default=0, help="debug: socket write delay in seconds")
ap2.add_argument("--rsp-slp", metavar="SEC", type=float, default=0, help="debug: response delay in seconds")
@@ -962,11 +930,8 @@ def add_cert(ap, cert_path):
def add_auth(ap):
ap2 = ap.add_argument_group('IdP / identity provider / user authentication options')
ap2.add_argument("--idp-h-usr", metavar="HN", type=u, default="", help="bypass the copyparty authentication checks and assume the request-header \033[33mHN\033[0m contains the username of the requesting user (for use with authentik/oauth/...)\n\033[1;31mWARNING:\033[0m if you enable this, make sure clients are unable to specify this header themselves; must be washed away and replaced by a reverse-proxy")
return
ap2.add_argument("--idp-h-grp", metavar="HN", type=u, default="", help="assume the request-header \033[33mHN\033[0m contains the groupname of the requesting user; can be referenced in config files for group-based access control")
ap2.add_argument("--idp-h-key", metavar="HN", type=u, default="", help="optional but recommended safeguard; your reverse-proxy will insert a secret header named \033[33mHN\033[0m into all requests, and the other IdP headers will be ignored if this header is not present")
ap2.add_argument("--idp-gsep", metavar="RE", type=u, default="|:;+,", help="if there are multiple groups in \033[33m--idp-h-grp\033[0m, they are separated by one of the characters in \033[33mRE\033[0m")
ap2.add_argument("--no-bauth", action="store_true", help="disable basic-authentication support; do not accept passwords from the 'Authenticate' header at all. NOTE: This breaks support for the android app")
ap2.add_argument("--bauth-last", action="store_true", help="keeps basic-authentication enabled, but only as a last-resort; if a cookie is also provided then the cookie wins")
def add_zeroconf(ap):
@@ -1010,12 +975,12 @@ def add_zc_ssdp(ap):
def add_ftp(ap):
ap2 = ap.add_argument_group('FTP options (TCP only)')
ap2 = ap.add_argument_group('FTP options')
ap2.add_argument("--ftp", metavar="PORT", type=int, help="enable FTP server on \033[33mPORT\033[0m, for example \033[32m3921")
ap2.add_argument("--ftps", metavar="PORT", type=int, help="enable FTPS server on \033[33mPORT\033[0m, for example \033[32m3990")
ap2.add_argument("--ftpv", action="store_true", help="verbose")
ap2.add_argument("--ftp4", action="store_true", help="only listen on IPv4")
ap2.add_argument("--ftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m; specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
ap2.add_argument("--ftp-ipa", metavar="PFX", type=u, default="", help="only accept connections from IP-addresses starting with \033[33mPFX\033[0m; specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Example: [\033[32m127., 10.89., 192.168.\033[0m]")
ap2.add_argument("--ftp-wt", metavar="SEC", type=int, default=7, help="grace period for resuming interrupted uploads (any client can write to any file last-modified more recently than \033[33mSEC\033[0m seconds ago)")
ap2.add_argument("--ftp-nat", metavar="ADDR", type=u, help="the NAT address to use for passive connections")
ap2.add_argument("--ftp-pr", metavar="P-P", type=u, help="the range of TCP ports to use for passive connections, for example \033[32m12000-13000")
@@ -1030,18 +995,6 @@ def add_webdav(ap):
ap2.add_argument("--dav-auth", action="store_true", help="force auth for all folders (required by davfs2 when only some folders are world-readable) (volflag=davauth)")
def add_tftp(ap):
ap2 = ap.add_argument_group('TFTP options (UDP only)')
ap2.add_argument("--tftp", metavar="PORT", type=int, help="enable TFTP server on \033[33mPORT\033[0m, for example \033[32m69 \033[0mor \033[32m3969")
ap2.add_argument("--tftpv", action="store_true", help="verbose")
ap2.add_argument("--tftpvv", action="store_true", help="verboser")
ap2.add_argument("--tftp-no-fast", action="store_true", help="debug: disable optimizations")
ap2.add_argument("--tftp-lsf", metavar="PTN", type=u, default="\\.?(dir|ls)(\\.txt)?", help="return a directory listing if a file with this name is requested and it does not exist; defaults matches .ls, dir, .dir.txt, ls.txt, ...")
ap2.add_argument("--tftp-nols", action="store_true", help="if someone tries to download a directory, return an error instead of showing its directory listing")
ap2.add_argument("--tftp-ipa", metavar="CIDR", type=u, default="", help="only accept connections from IP-addresses inside \033[33mCIDR\033[0m; specify [\033[32many\033[0m] to disable inheriting \033[33m--ipa\033[0m. Examples: [\033[32mlan\033[0m] or [\033[32m10.89.0.0/16, 192.168.33.0/24\033[0m]")
ap2.add_argument("--tftp-pr", metavar="P-P", type=u, help="the range of UDP ports to use for data transfer, for example \033[32m12000-13000")
def add_smb(ap):
ap2 = ap.add_argument_group('SMB/CIFS options')
ap2.add_argument("--smb", action="store_true", help="enable smb (read-only) -- this requires running copyparty as root on linux and macos unless \033[33m--smb-port\033[0m is set above 1024 and your OS does port-forwarding from 445 to that.\n\033[1;31mWARNING:\033[0m this protocol is DANGEROUS and buggy! Never expose to the internet!")
@@ -1106,8 +1059,6 @@ def add_optouts(ap):
ap2.add_argument("--no-zip", action="store_true", help="disable download as zip/tar")
ap2.add_argument("--no-tarcmp", action="store_true", help="disable download as compressed tar (?tar=gz, ?tar=bz2, ?tar=xz, ?tar=gz:9, ...)")
ap2.add_argument("--no-lifetime", action="store_true", help="do not allow clients (or server config) to schedule an upload to be deleted after a given time")
ap2.add_argument("--no-pipe", action="store_true", help="disable race-the-beam (lockstep download of files which are currently being uploaded) (volflag=nopipe)")
ap2.add_argument("--no-db-ip", action="store_true", help="do not write uploader IPs into the database")
def add_safety(ap):
@@ -1133,21 +1084,19 @@ def add_safety(ap):
ap2.add_argument("--ban-url", metavar="N,W,B", type=u, default="9,2,1440", help="hitting more than \033[33mN\033[0m sus URL's in \033[33mW\033[0m minutes = ban for \033[33mB\033[0m minutes; applies only to permissions g/G/h (decent replacement for \033[33m--ban-404\033[0m if that can't be used)")
ap2.add_argument("--sus-urls", metavar="R", type=u, default=r"\.php$|(^|/)wp-(admin|content|includes)/", help="URLs which are considered sus / eligible for banning; disable with blank or [\033[32mno\033[0m]")
ap2.add_argument("--nonsus-urls", metavar="R", type=u, default=r"^(favicon\.ico|robots\.txt)$|^apple-touch-icon|^\.well-known", help="harmless URLs ignored from 404-bans; disable with blank or [\033[32mno\033[0m]")
ap2.add_argument("--early-ban", action="store_true", help="if a client is banned, reject its connection as soon as possible; not a good idea to enable when proxied behind cloudflare since it could ban your reverse-proxy")
ap2.add_argument("--aclose", metavar="MIN", type=int, default=10, help="if a client maxes out the server connection limit, downgrade it from connection:keep-alive to connection:close for \033[33mMIN\033[0m minutes (and also kill its active connections) -- disable with 0")
ap2.add_argument("--loris", metavar="B", type=int, default=60, help="if a client maxes out the server connection limit without sending headers, ban it for \033[33mB\033[0m minutes; disable with [\033[32m0\033[0m]")
ap2.add_argument("--acao", metavar="V[,V]", type=u, default="*", help="Access-Control-Allow-Origin; list of origins (domains/IPs without port) to accept requests from; [\033[32mhttps://1.2.3.4\033[0m]. Default [\033[32m*\033[0m] allows requests from all sites but removes cookies and http-auth; only ?pw=hunter2 survives")
ap2.add_argument("--acam", metavar="V[,V]", type=u, default="GET,HEAD", help="Access-Control-Allow-Methods; list of methods to accept from offsite ('*' behaves like \033[33m--acao\033[0m's description)")
def add_salt(ap, fk_salt, dk_salt, ah_salt):
def add_salt(ap, fk_salt, ah_salt):
ap2 = ap.add_argument_group('salting options')
ap2.add_argument("--ah-alg", metavar="ALG", type=u, default="none", help="account-pw hashing algorithm; one of these, best to worst: \033[32margon2 scrypt sha2 none\033[0m (each optionally followed by alg-specific comma-sep. config)")
ap2.add_argument("--ah-salt", metavar="SALT", type=u, default=ah_salt, help="account-pw salt; ignored if \033[33m--ah-alg\033[0m is none (default)")
ap2.add_argument("--ah-gen", metavar="PW", type=u, default="", help="generate hashed password for \033[33mPW\033[0m, or read passwords from STDIN if \033[33mPW\033[0m is [\033[32m-\033[0m]")
ap2.add_argument("--ah-cli", action="store_true", help="launch an interactive shell which hashes passwords without ever storing or displaying the original passwords")
ap2.add_argument("--fk-salt", metavar="SALT", type=u, default=fk_salt, help="per-file accesskey salt; used to generate unpredictable URLs for hidden files")
ap2.add_argument("--dk-salt", metavar="SALT", type=u, default=dk_salt, help="per-directory accesskey salt; used to generate unpredictable URLs to share folders with users who only have the 'get' permission")
ap2.add_argument("--warksalt", metavar="SALT", type=u, default="hunter2", help="up2k file-hash salt; serves no purpose, no reason to change this (but delete all databases if you do)")
@@ -1189,9 +1138,7 @@ def add_thumbnail(ap):
ap2.add_argument("--th-size", metavar="WxH", default="320x256", help="thumbnail res (volflag=thsize)")
ap2.add_argument("--th-mt", metavar="CORES", type=int, default=CORES, help="num cpu cores to use for generating thumbnails")
ap2.add_argument("--th-convt", metavar="SEC", type=float, default=60, help="conversion timeout in seconds (volflag=convt)")
ap2.add_argument("--th-ram-max", metavar="GB", type=float, default=6, help="max memory usage (GiB) permitted by thumbnailer; not very accurate")
ap2.add_argument("--th-crop", metavar="TXT", type=u, default="y", help="crop thumbnails to 4:3 or keep dynamic height; client can override in UI unless force. [\033[32mfy\033[0m]=crop, [\033[32mfn\033[0m]=nocrop, [\033[32mfy\033[0m]=force-y, [\033[32mfn\033[0m]=force-n (volflag=crop)")
ap2.add_argument("--th-x3", metavar="TXT", type=u, default="n", help="show thumbs at 3x resolution; client can override in UI unless force. [\033[32mfy\033[0m]=yes, [\033[32mfn\033[0m]=no, [\033[32mfy\033[0m]=force-yes, [\033[32mfn\033[0m]=force-no (volflag=th3x)")
ap2.add_argument("--th-no-crop", action="store_true", help="dynamic height; show full image by default (client can override in UI) (volflag=nocrop)")
ap2.add_argument("--th-dec", metavar="LIBS", default="vips,pil,ff", help="image decoders, in order of preference")
ap2.add_argument("--th-no-jpg", action="store_true", help="disable jpg output")
ap2.add_argument("--th-no-webp", action="store_true", help="disable webp output")
@@ -1208,21 +1155,17 @@ def add_thumbnail(ap):
ap2.add_argument("--th-r-vips", metavar="T,T", type=u, default="avif,exr,fit,fits,fts,gif,hdr,heic,jp2,jpeg,jpg,jpx,jxl,nii,pfm,pgm,png,ppm,svg,tif,tiff,webp", help="image formats to decode using pyvips")
ap2.add_argument("--th-r-ffi", metavar="T,T", type=u, default="apng,avif,avifs,bmp,dds,dib,fit,fits,fts,gif,hdr,heic,heics,heif,heifs,icns,ico,jp2,jpeg,jpg,jpx,jxl,pbm,pcx,pfm,pgm,png,pnm,ppm,psd,qoi,sgi,tga,tif,tiff,webp,xbm,xpm", help="image formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffv", metavar="T,T", type=u, default="3gp,asf,av1,avc,avi,flv,h264,h265,hevc,m4v,mjpeg,mjpg,mkv,mov,mp4,mpeg,mpeg2,mpegts,mpg,mpg2,mts,nut,ogm,ogv,rm,ts,vob,webm,wmv", help="video formats to decode using ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,itgz,itxz,itz,m4a,mdgz,mdxz,mdz,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,s3gz,s3xz,s3z,tak,tta,ulaw,wav,wma,wv,xm,xmgz,xmxz,xmz,xpk", help="audio formats to decode using ffmpeg")
ap2.add_argument("--au-unpk", metavar="E=F.C", type=u, default="mdz=mod.zip, mdgz=mod.gz, mdxz=mod.xz, s3z=s3m.zip, s3gz=s3m.gz, s3xz=s3m.xz, xmz=xm.zip, xmgz=xm.gz, xmxz=xm.xz, itz=it.zip, itgz=it.gz, itxz=it.xz", help="audio formats to decompress before passing to ffmpeg")
ap2.add_argument("--th-r-ffa", metavar="T,T", type=u, default="aac,ac3,aif,aiff,alac,alaw,amr,apac,ape,au,bonk,dfpwm,dts,flac,gsm,ilbc,it,m4a,mo3,mod,mp2,mp3,mpc,mptm,mt2,mulaw,ogg,okt,opus,ra,s3m,tak,tta,ulaw,wav,wma,wv,xm,xpk", help="audio formats to decode using ffmpeg")
def add_transcoding(ap):
ap2 = ap.add_argument_group('transcoding options')
ap2.add_argument("--q-opus", metavar="KBPS", type=int, default=128, help="target bitrate for transcoding to opus; set 0 to disable")
ap2.add_argument("--q-mp3", metavar="QUALITY", type=u, default="q2", help="target quality for transcoding to mp3, for example [\033[32m192k\033[0m] (CBR) or [\033[32mq0\033[0m] (CQ/CRF, q0=maxquality, q9=smallest); set 0 to disable")
ap2.add_argument("--no-acode", action="store_true", help="disable audio transcoding")
ap2.add_argument("--no-bacode", action="store_true", help="disable batch audio transcoding by folder download (zip/tar)")
ap2.add_argument("--ac-maxage", metavar="SEC", type=int, default=86400, help="delete cached transcode output after \033[33mSEC\033[0m seconds")
def add_db_general(ap, hcores):
noidx = APPLESAN_TXT if MACOS else ""
ap2 = ap.add_argument_group('general db options')
ap2.add_argument("-e2d", action="store_true", help="enable up2k database, making files searchable + enables upload deduplication")
ap2.add_argument("-e2ds", action="store_true", help="scan writable folders for new files on startup; sets \033[33m-e2d\033[0m")
@@ -1232,9 +1175,9 @@ def add_db_general(ap, hcores):
ap2.add_argument("-e2vp", action="store_true", help="on hash mismatch: panic and quit copyparty")
ap2.add_argument("--hist", metavar="PATH", type=u, help="where to store volume data (db, thumbs) (volflag=hist)")
ap2.add_argument("--no-hash", metavar="PTN", type=u, help="regex: disable hashing of matching absolute-filesystem-paths during e2ds folder scans (volflag=nohash)")
ap2.add_argument("--no-idx", metavar="PTN", type=u, default=noidx, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
ap2.add_argument("--no-idx", metavar="PTN", type=u, help="regex: disable indexing of matching absolute-filesystem-paths during e2ds folder scans (volflag=noidx)")
ap2.add_argument("--no-dhash", action="store_true", help="disable rescan acceleration; do full database integrity check -- makes the db ~5%% smaller and bootup/rescans 3~10x slower")
ap2.add_argument("--re-dhash", action="store_true", help="force a cache rebuild on startup; enable this once if it gets out of sync (should never be necessary)")
ap2.add_argument("--re-dhash", action="store_true", help="rebuild the cache if it gets out of sync (for example crash on startup during metadata scanning)")
ap2.add_argument("--no-forget", action="store_true", help="never forget indexed files, even when deleted from disk -- makes it impossible to ever upload the same file twice -- only useful for offloading uploads to a cloud service or something (volflag=noforget)")
ap2.add_argument("--dbd", metavar="PROFILE", default="wal", help="database durability profile; sets the tradeoff between robustness and speed, see \033[33m--help-dbd\033[0m (volflag=dbd)")
ap2.add_argument("--xlink", action="store_true", help="on upload: check all volumes for dupes, not just the target volume (volflag=xlink)")
@@ -1272,38 +1215,19 @@ def add_txt(ap):
ap2.add_argument("--exp-lg", metavar="V,V,V", type=u, default=DEF_EXP, help="comma/space-separated list of placeholders to expand in prologue/epilogue files (volflag=exp_lg)")
def add_og(ap):
ap2 = ap.add_argument_group('og / open graph / discord-embed options')
ap2.add_argument("--og", action="store_true", help="disable hotlinking and return an html document instead; this is required by open-graph, but can also be useful on its own (volflag=og)")
ap2.add_argument("--og-ua", metavar="RE", type=u, default="", help="only disable hotlinking / engage OG behavior if the useragent matches regex \033[33mRE\033[0m (volflag=og_ua)")
ap2.add_argument("--og-tpl", metavar="PATH", type=u, default="", help="do not return the regular copyparty html, but instead load the jinja2 template at \033[33mPATH\033[0m (if path contains 'EXT' then EXT will be replaced with the requested file's extension) (volflag=og_tpl)")
ap2.add_argument("--og-no-head", action="store_true", help="do not automatically add OG entries into <head> (useful if you're doing this yourself in a template or such) (volflag=og_no_head)")
ap2.add_argument("--og-th", metavar="FMT", type=u, default="jf3", help="thumbnail format; j=jpeg, jf=jpeg-uncropped, jf3=jpeg-uncropped-large, w=webm, ... (volflag=og_th)")
ap2.add_argument("--og-title", metavar="TXT", type=u, default="", help="fallback title if there is nothing in the \033[33m-e2t\033[0m database (volflag=og_title)")
ap2.add_argument("--og-title-a", metavar="T", type=u, default="🎵 {{ artist }} - {{ title }}", help="audio title format; takes any metadata key (volflag=og_title_a)")
ap2.add_argument("--og-title-v", metavar="T", type=u, default="{{ title }}", help="video title format; takes any metadata key (volflag=og_title_v)")
ap2.add_argument("--og-title-i", metavar="T", type=u, default="{{ title }}", help="image title format; takes any metadata key (volflag=og_title_i)")
ap2.add_argument("--og-s-title", action="store_true", help="force default title; do not read from tags (volflag=og_s_title)")
ap2.add_argument("--og-desc", metavar="TXT", type=u, default="", help="description text; same for all files, disable with [\033[32m-\033[0m] (volflag=og_desc)")
ap2.add_argument("--og-site", metavar="TXT", type=u, default="", help="sitename; defaults to \033[33m--name\033[0m, disable with [\033[32m-\033[0m] (volflag=og_site)")
ap2.add_argument("--tcolor", metavar="RGB", type=u, default="333", help="accent color (3 or 6 hex digits); may also affect safari and/or android-chrome (volflag=tcolor)")
ap2.add_argument("--uqe", action="store_true", help="query-string parceling; translate a request for \033[33m/foo/.uqe/BASE64\033[0m into \033[33m/foo?TEXT\033[0m, or \033[33m/foo/?TEXT\033[0m if the first character in \033[33mTEXT\033[0m is a slash. Automatically enabled for \033[33m--og\033[0m")
def add_ui(ap, retry):
ap2 = ap.add_argument_group('ui options')
ap2.add_argument("--grid", action="store_true", help="show grid/thumbnails by default (volflag=grid)")
ap2.add_argument("--lang", metavar="LANG", type=u, default="eng", help="language; one of the following: \033[32meng nor\033[0m")
ap2.add_argument("--theme", metavar="NUM", type=int, default=0, help="default theme to use (0..7)")
ap2.add_argument("--themes", metavar="NUM", type=int, default=8, help="number of themes installed")
ap2.add_argument("--au-vol", metavar="0-100", type=int, default=50, choices=range(0, 101), help="default audio/video volume percent")
ap2.add_argument("--sort", metavar="C,C,C", type=u, default="href", help="default sort order, comma-separated column IDs (see header tooltips), prefix with '-' for descending. Examples: \033[32mhref -href ext sz ts tags/Album tags/.tn\033[0m (volflag=sort)")
ap2.add_argument("--unlist", metavar="REGEX", type=u, default="", help="don't show files matching \033[33mREGEX\033[0m in file list. Purely cosmetic! Does not affect API calls, just the browser. Example: [\033[32m\\.(js|css)$\033[0m] (volflag=unlist)")
ap2.add_argument("--favico", metavar="TXT", type=u, default="c 000 none" if retry else "🎉 000 none", help="\033[33mfavicon-text\033[0m [ \033[33mforeground\033[0m [ \033[33mbackground\033[0m ] ], set blank to disable")
ap2.add_argument("--mpmc", metavar="URL", type=u, default="", help="change the mediaplayer-toggle mouse cursor; URL to a folder with {2..5}.png inside (or disable with [\033[32m.\033[0m])")
ap2.add_argument("--js-browser", metavar="L", type=u, help="URL to additional JS to include")
ap2.add_argument("--css-browser", metavar="L", type=u, help="URL to additional CSS to include")
ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages; can be @PATH to send the contents of a file at PATH, and/or begin with %% to render as jinja2 template (volflag=html_head)")
ap2.add_argument("--html-head", metavar="TXT", type=u, default="", help="text to append to the <head> of all HTML pages")
ap2.add_argument("--ih", action="store_true", help="if a folder contains index.html, show that instead of the directory listing by default (can be changed in the client settings UI, or add ?v to URL for override)")
ap2.add_argument("--textfiles", metavar="CSV", type=u, default="txt,nfo,diz,cue,readme", help="file extensions to present as plaintext")
ap2.add_argument("--txt-max", metavar="KiB", type=int, default=64, help="max size of embedded textfiles on ?doc= (anything bigger will be lazy-loaded by JS)")
@@ -1311,7 +1235,6 @@ def add_ui(ap, retry):
ap2.add_argument("--bname", metavar="TXT", type=u, default="--name", help="server name (displayed in filebrowser document title)")
ap2.add_argument("--pb-url", metavar="URL", type=u, default="https://github.com/9001/copyparty", help="powered-by link; disable with \033[33m-np\033[0m")
ap2.add_argument("--ver", action="store_true", help="show version on the control panel (incompatible with \033[33m-nb\033[0m)")
ap2.add_argument("--k304", metavar="NUM", type=int, default=0, help="configure the option to enable/disable k304 on the controlpanel (workaround for buggy reverse-proxies); [\033[32m0\033[0m] = hidden and default-off, [\033[32m1\033[0m] = visible and default-off, [\033[32m2\033[0m] = visible and default-on")
ap2.add_argument("--md-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for README.md docs (volflag=md_sbf); see https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe#attr-sandbox")
ap2.add_argument("--lg-sbf", metavar="FLAGS", type=u, default="downloads forms popups scripts top-navigation-by-user-activation", help="list of capabilities to ALLOW for prologue/epilogue docs (volflag=lg_sbf)")
ap2.add_argument("--no-sb-md", action="store_true", help="don't sandbox README.md documents (volflags: no_sb_md | sb_md)")
@@ -1351,7 +1274,6 @@ def run_argparse(
cert_path = os.path.join(E.cfg, "cert.pem")
fk_salt = get_fk_salt()
dk_salt = get_dk_salt()
ah_salt = get_ah_salt()
# alpine peaks at 5 threads for some reason,
@@ -1372,7 +1294,6 @@ def run_argparse(
add_zeroconf(ap)
add_zc_mdns(ap)
add_zc_ssdp(ap)
add_fs(ap)
add_upload(ap)
add_db_general(ap, hcores)
add_db_metadata(ap)
@@ -1380,10 +1301,9 @@ def run_argparse(
add_transcoding(ap)
add_ftp(ap)
add_webdav(ap)
add_tftp(ap)
add_smb(ap)
add_safety(ap)
add_salt(ap, fk_salt, dk_salt, ah_salt)
add_salt(ap, fk_salt, ah_salt)
add_optouts(ap)
add_shutdown(ap)
add_yolo(ap)
@@ -1391,7 +1311,6 @@ def run_argparse(
add_hooks(ap)
add_stats(ap)
add_txt(ap)
add_og(ap)
add_ui(ap, retry)
add_admin(ap)
add_logging(ap)
@@ -1420,35 +1339,30 @@ def run_argparse(
k2 = "help_" + k.replace("-", "_")
if vars(ret)[k2]:
lprint("# %s help page (%s)" % (k, h))
lprint(t.rstrip() + "\033[0m")
lprint(t + "\033[0m")
sys.exit(0)
return ret
def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
def main(argv: Optional[list[str]] = None) -> None:
time.strptime("19970815", "%Y%m%d") # python#7980
if WINDOWS:
os.system("rem") # enables colors
init_E(E)
if rsrc: # pyz
E.mod = rsrc
if argv is None:
argv = sys.argv
f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n sqlite {} | jinja {} | pyftpd {} | tftp {}\n\033[0m'
f = '\033[36mcopyparty v{} "\033[35m{}\033[36m" ({})\n{}\033[0;36m\n sqlite v{} | jinja2 v{} | pyftpd v{}\n\033[0m'
f = f.format(
S_VERSION,
CODENAME,
S_BUILD_DT,
PY_DESC.replace("[", "\033[90m["),
py_desc().replace("[", "\033[90m["),
SQLITE_VER,
JINJA_VER,
PYFTPD_VER,
PARTFTPY_VER,
)
lprint(f)
@@ -1459,19 +1373,9 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
showlic()
sys.exit(0)
if "--mimes" in argv:
print("\n".join("%8s %s" % (k, v) for k, v in sorted(MIMES.items())))
sys.exit(0)
if EXE:
print("pybin: {}\n".format(pybin), end="")
for n, zs in enumerate(argv):
if zs.startswith("--sfx-tpoke="):
Daemon(sfx_tpoke, "sfx-tpoke", (zs.split("=", 1)[1],))
argv.pop(n)
break
ensure_locale()
ensure_webdeps()
@@ -1490,8 +1394,6 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
deprecated: list[tuple[str, str]] = [
("--salt", "--warksalt"),
("--hdr-au-usr", "--idp-h-usr"),
("--idp-h-sep", "--idp-gsep"),
("--th-no-crop", "--th-crop=n"),
]
for dk, nk in deprecated:
idx = -1
@@ -1622,9 +1524,6 @@ def main(argv: Optional[list[str]] = None, rsrc: Optional[str] = None) -> None:
if sys.version_info < (3, 6):
al.no_scandir = True
if not hasattr(os, "sendfile"):
al.no_sendfile = True
# signal.signal(signal.SIGINT, sighandler)
SvcHub(al, dal, argv, "".join(printed)).run()

View File

@@ -1,8 +1,8 @@
# coding: utf-8
VERSION = (1, 13, 2)
CODENAME = "race the beam"
BUILD_DT = (2024, 5, 10)
VERSION = (1, 9, 28)
CODENAME = "prometheable"
BUILD_DT = (2023, 12, 31)
S_VERSION = ".".join(map(str, VERSION))
S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

View File

@@ -17,9 +17,8 @@ from .bos import bos
from .cfg import flagdescs, permdescs, vf_bmap, vf_cmap, vf_vmap
from .pwhash import PWHash
from .util import (
EXTS,
IMPLICATIONS,
MIMES,
META_NOBOTS,
SQLITE_VER,
UNPLICATIONS,
UTC,
@@ -35,7 +34,6 @@ from .util import (
uncyg,
undot,
unhumanize,
vsplit,
)
if True: # pylint: disable=using-constant-test
@@ -63,10 +61,6 @@ BAD_CFG = "invalid config; {}".format(SEE_LOG)
SBADCFG = " ({})".format(BAD_CFG)
class CfgEx(Exception):
pass
class AXS(object):
def __init__(
self,
@@ -199,7 +193,7 @@ class Lim(object):
self.dft = int(time.time()) + 300
self.dfv = get_df(abspath)[0] or 0
for j in list(self.reg.values()) if self.reg else []:
self.dfv -= int(j["size"] / (len(j["hash"]) or 999) * len(j["need"]))
self.dfv -= int(j["size"] / len(j["hash"]) * len(j["need"]))
if already_written:
sz = 0
@@ -387,7 +381,7 @@ class VFS(object):
def add(self, src: str, dst: str) -> "VFS":
"""get existing, or add new path to the vfs"""
assert src == "/" or not src.endswith("/") # nosec
assert not src.endswith("/") # nosec
assert not dst.endswith("/") # nosec
if "/" in dst:
@@ -557,12 +551,7 @@ class VFS(object):
# no vfs nodes in the list of real inodes
real = [x for x in real if x[0] not in self.nodes]
dbv = self.dbv or self
for name, vn2 in sorted(self.nodes.items()):
if vn2.dbv == dbv and self.flags.get("dk"):
virt_vis[name] = vn2
continue
ok = False
zx = vn2.axs
axs = [zx.uread, zx.uwrite, zx.umove, zx.udel, zx.uget]
@@ -790,20 +779,7 @@ class AuthSrv(object):
self.warn_anonwrite = warn_anonwrite
self.line_ctr = 0
self.indent = ""
# fwd-decl
self.vfs = VFS(log_func, "", "", AXS(), {})
self.acct: dict[str, str] = {}
self.iacct: dict[str, str] = {}
self.grps: dict[str, list[str]] = {}
self.re_pwd: Optional[re.Pattern] = None
# all volumes observed since last restart
self.idp_vols: dict[str, str] = {} # vpath->abspath
# all users/groups observed since last restart
self.idp_accs: dict[str, list[str]] = {} # username->groupnames
self.idp_usr_gh: dict[str, str] = {} # username->group-header-value (cache)
self.desc = []
self.mutex = threading.Lock()
self.reload()
@@ -822,86 +798,6 @@ class AuthSrv(object):
yield prev, True
def idp_checkin(
self, broker: Optional["BrokerCli"], uname: str, gname: str
) -> bool:
if uname in self.acct:
return False
if self.idp_usr_gh.get(uname) == gname:
return False
gnames = [x.strip() for x in self.args.idp_gsep.split(gname)]
gnames.sort()
with self.mutex:
self.idp_usr_gh[uname] = gname
if self.idp_accs.get(uname) == gnames:
return False
self.idp_accs[uname] = gnames
t = "reinitializing due to new user from IdP: [%s:%s]"
self.log(t % (uname, gnames), 3)
if not broker:
# only true for tests
self._reload()
return True
broker.ask("_reload_blocking", False).get()
return True
def _map_volume_idp(
self,
src: str,
dst: str,
mount: dict[str, str],
daxs: dict[str, AXS],
mflags: dict[str, dict[str, Any]],
un_gns: dict[str, list[str]],
) -> list[tuple[str, str, str, str]]:
ret: list[tuple[str, str, str, str]] = []
visited = set()
src0 = src # abspath
dst0 = dst # vpath
un_gn = [(un, gn) for un, gns in un_gns.items() for gn in gns]
if not un_gn:
# ensure volume creation if there's no users
un_gn = [("", "")]
for un, gn in un_gn:
# if ap/vp has a user/group placeholder, make sure to keep
# track so the same user/gruop is mapped when setting perms;
# otherwise clear un/gn to indicate it's a regular volume
src1 = src0.replace("${u}", un or "\n")
dst1 = dst0.replace("${u}", un or "\n")
if src0 == src1 and dst0 == dst1:
un = ""
src = src1.replace("${g}", gn or "\n")
dst = dst1.replace("${g}", gn or "\n")
if src == src1 and dst == dst1:
gn = ""
if "\n" in (src + dst):
continue
label = "%s\n%s" % (src, dst)
if label in visited:
continue
visited.add(label)
src, dst = self._map_volume(src, dst, mount, daxs, mflags)
if src:
ret.append((src, dst, un, gn))
if un or gn:
self.idp_vols[dst] = src
return ret
def _map_volume(
self,
src: str,
@@ -909,11 +805,7 @@ class AuthSrv(object):
mount: dict[str, str],
daxs: dict[str, AXS],
mflags: dict[str, dict[str, Any]],
) -> tuple[str, str]:
src = os.path.expandvars(os.path.expanduser(src))
src = absreal(src)
dst = dst.strip("/")
) -> None:
if dst in mount:
t = "multiple filesystem-paths mounted at [/{}]:\n [{}]\n [{}]"
self.log(t.format(dst, mount[dst], src), c=1)
@@ -934,7 +826,6 @@ class AuthSrv(object):
mount[dst] = src
daxs[dst] = AXS()
mflags[dst] = {}
return (src, dst)
def _e(self, desc: Optional[str] = None) -> None:
if not self.args.vc or not self.line_ctr:
@@ -962,76 +853,32 @@ class AuthSrv(object):
self.log(t.format(self.line_ctr, c, self.indent, ln, desc))
def _all_un_gn(
self,
acct: dict[str, str],
grps: dict[str, list[str]],
) -> dict[str, list[str]]:
"""
generate list of all confirmed pairs of username/groupname seen since last restart;
in case of conflicting group memberships then it is selected as follows:
* any non-zero value from IdP group header
* otherwise take --grps / [groups]
"""
ret = {un: gns[:] for un, gns in self.idp_accs.items()}
ret.update({zs: [""] for zs in acct if zs not in ret})
for gn, uns in grps.items():
for un in uns:
try:
ret[un].append(gn)
except:
ret[un] = [gn]
return ret
def _parse_config_file(
self,
fp: str,
cfg_lines: list[str],
acct: dict[str, str],
grps: dict[str, list[str]],
daxs: dict[str, AXS],
mflags: dict[str, dict[str, Any]],
mount: dict[str, str],
) -> None:
self.desc = []
self.line_ctr = 0
expand_config_file(self.log, cfg_lines, fp, "")
expand_config_file(cfg_lines, fp, "")
if self.args.vc:
lns = ["{:4}: {}".format(n, s) for n, s in enumerate(cfg_lines, 1)]
self.log("expanded config file (unprocessed):\n" + "\n".join(lns))
cfg_lines = upgrade_cfg_fmt(self.log, self.args, cfg_lines, fp)
# due to IdP, volumes must be parsed after users and groups;
# do volumes in a 2nd pass to allow arbitrary order in config files
for npass in range(1, 3):
if self.args.vc:
self.log("parsing config files; pass %d/%d" % (npass, 2))
self._parse_config_file_2(cfg_lines, acct, grps, daxs, mflags, mount, npass)
def _parse_config_file_2(
self,
cfg_lines: list[str],
acct: dict[str, str],
grps: dict[str, list[str]],
daxs: dict[str, AXS],
mflags: dict[str, dict[str, Any]],
mount: dict[str, str],
npass: int,
) -> None:
self.line_ctr = 0
all_un_gn = self._all_un_gn(acct, grps)
cat = ""
catg = "[global]"
cata = "[accounts]"
catgrp = "[groups]"
catx = "accs:"
catf = "flags:"
ap: Optional[str] = None
vp: Optional[str] = None
vols: list[tuple[str, str, str, str]] = []
for ln in cfg_lines:
self.line_ctr += 1
ln = ln.split(" #")[0].strip()
@@ -1044,7 +891,7 @@ class AuthSrv(object):
subsection = ln in (catx, catf)
if ln.startswith("[") or subsection:
self._e()
if npass > 1 and ap is None and vp is not None:
if ap is None and vp is not None:
t = "the first line after [/{}] must be a filesystem path to share on that volume"
raise Exception(t.format(vp))
@@ -1060,8 +907,6 @@ class AuthSrv(object):
self._l(ln, 6, t)
elif ln == cata:
self._l(ln, 5, "begin user-accounts section")
elif ln == catgrp:
self._l(ln, 5, "begin user-groups section")
elif ln.startswith("[/"):
vp = ln[1:-1].strip("/")
self._l(ln, 2, "define volume at URL [/{}]".format(vp))
@@ -1098,39 +943,15 @@ class AuthSrv(object):
raise Exception(t + SBADCFG)
continue
if cat == catgrp:
try:
gn, zs1 = [zs.strip() for zs in ln.split(":", 1)]
uns = [zs.strip() for zs in zs1.split(",")]
t = "group [%s] = " % (gn,)
t += ", ".join("user [%s]" % (x,) for x in uns)
self._l(ln, 5, t)
grps[gn] = uns
except:
t = 'lines inside the [groups] section must be "groupname: user1, user2, user..."'
raise Exception(t + SBADCFG)
continue
if vp is not None and ap is None:
if npass != 2:
continue
ap = ln
ap = os.path.expandvars(os.path.expanduser(ap))
ap = absreal(ap)
self._l(ln, 2, "bound to filesystem-path [{}]".format(ap))
vols = self._map_volume_idp(ap, vp, mount, daxs, mflags, all_un_gn)
if not vols:
ap = vp = None
self._l(ln, 2, "└─no users/groups known; was not mapped")
elif len(vols) > 1:
for vol in vols:
self._l(ln, 2, "└─mapping: [%s] => [%s]" % (vol[1], vol[0]))
self._map_volume(ap, vp, mount, daxs, mflags)
continue
if cat == catx:
if npass != 2 or not ap:
# not stage2, or unmapped ${u}/${g}
continue
err = ""
try:
self._l(ln, 5, "volume access config:")
@@ -1141,20 +962,14 @@ class AuthSrv(object):
if " " in re.sub(", *", "", sv).strip():
err = "list of users is not comma-separated; "
raise Exception(err)
sv = sv.replace(" ", "")
self._read_vol_str_idp(sk, sv, vols, all_un_gn, daxs, mflags)
assert vp is not None
self._read_vol_str(sk, sv.replace(" ", ""), daxs[vp], mflags[vp])
continue
except CfgEx:
raise
except:
err += "accs entries must be 'rwmdgGhaA.: user1, user2, ...'"
raise CfgEx(err + SBADCFG)
raise Exception(err + SBADCFG)
if cat == catf:
if npass != 2 or not ap:
# not stage2, or unmapped ${u}/${g}
continue
err = ""
try:
self._l(ln, 6, "volume-specific config:")
@@ -1171,14 +986,11 @@ class AuthSrv(object):
else:
fstr += ",{}={}".format(sk, sv)
assert vp is not None
self._read_vol_str_idp(
"c", fstr[1:], vols, all_un_gn, daxs, mflags
)
self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
fstr = ""
if fstr:
self._read_vol_str_idp(
"c", fstr[1:], vols, all_un_gn, daxs, mflags
)
assert vp is not None
self._read_vol_str("c", fstr[1:], daxs[vp], mflags[vp])
continue
except:
err += "flags entries (volflags) must be one of the following:\n 'flag1, flag2, ...'\n 'key: value'\n 'flag1, flag2, key: value'"
@@ -1189,21 +1001,14 @@ class AuthSrv(object):
self._e()
self.line_ctr = 0
def _read_vol_str_idp(
self,
lvl: str,
uname: str,
vols: list[tuple[str, str, str, str]],
un_gns: dict[str, list[str]],
axs: dict[str, AXS],
flags: dict[str, dict[str, Any]],
def _read_vol_str(
self, lvl: str, uname: str, axs: AXS, flags: dict[str, Any]
) -> None:
if lvl.strip("crwmdgGhaA."):
t = "%s,%s" % (lvl, uname) if uname else lvl
raise CfgEx("invalid config value (volume or volflag): %s" % (t,))
raise Exception("invalid config value (volume or volflag): %s" % (t,))
if lvl == "c":
# here, 'uname' is not a username; it is a volflag name... sorry
cval: Union[bool, str] = True
try:
# volflag with arguments, possibly with a preceding list of bools
@@ -1215,62 +1020,16 @@ class AuthSrv(object):
while "," in uname:
# one or more bools before the final flag; eat them
n1, uname = uname.split(",", 1)
for _, vp, _, _ in vols:
self._read_volflag(flags[vp], n1, True, False)
for _, vp, _, _ in vols:
self._read_volflag(flags[vp], uname, cval, False)
self._read_volflag(flags, n1, True, False)
self._read_volflag(flags, uname, cval, False)
return
if uname == "":
uname = "*"
unames = []
for un in uname.replace(",", " ").strip().split():
if un.startswith("@"):
grp = un[1:]
uns = [x[0] for x in un_gns.items() if grp in x[1]]
if grp == "${g}":
unames.append(un)
elif not uns and not self.args.idp_h_grp:
t = "group [%s] must be defined with --grp argument (or in a [groups] config section)"
raise CfgEx(t % (grp,))
unames.extend(uns)
else:
unames.append(un)
# unames may still contain ${u} and ${g} so now expand those;
un_gn = [(un, gn) for un, gns in un_gns.items() for gn in gns]
for src, dst, vu, vg in vols:
unames2 = set(unames)
if "${u}" in unames:
if not vu:
t = "cannot use ${u} in accs of volume [%s] because the volume url does not contain ${u}"
raise CfgEx(t % (src,))
unames2.add(vu)
if "@${g}" in unames:
if not vg:
t = "cannot use @${g} in accs of volume [%s] because the volume url does not contain @${g}"
raise CfgEx(t % (src,))
unames2.update([un for un, gn in un_gn if gn == vg])
if "${g}" in unames:
t = 'the accs of volume [%s] contains "${g}" but the only supported way of specifying that is "@${g}"'
raise CfgEx(t % (src,))
unames2.discard("${u}")
unames2.discard("@${g}")
self._read_vol_str(lvl, list(unames2), axs[dst])
def _read_vol_str(self, lvl: str, unames: list[str], axs: AXS) -> None:
junkset = set()
for un in unames:
for un in uname.replace(",", " ").strip().split():
for alias, mapping in [
("h", "gh"),
("G", "gG"),
@@ -1347,18 +1106,12 @@ class AuthSrv(object):
then supplementing with config files
before finally building the VFS
"""
with self.mutex:
self._reload()
def _reload(self) -> None:
acct: dict[str, str] = {} # username:password
grps: dict[str, list[str]] = {} # groupname:usernames
daxs: dict[str, AXS] = {}
mflags: dict[str, dict[str, Any]] = {} # moutpoint:flags
mount: dict[str, str] = {} # dst:src (mountpoint:realpath)
self.idp_vols = {} # yolo
if self.args.a:
# list of username:password
for x in self.args.a:
@@ -1369,22 +1122,9 @@ class AuthSrv(object):
t = '\n invalid value "{}" for argument -a, must be username:password'
raise Exception(t.format(x))
if self.args.grp:
# list of groupname:username,username,...
for x in self.args.grp:
try:
# accept both = and : as separator between groupname and usernames,
# accept both , and : as separators between usernames
zs1, zs2 = x.replace("=", ":").split(":", 1)
grps[zs1] = zs2.replace(":", ",").split(",")
except:
t = '\n invalid value "{}" for argument --grp, must be groupname:username1,username2,...'
raise Exception(t.format(x))
if self.args.v:
# list of src:dst:permset:permset:...
# permset is <rwmdgGhaA.>[,username][,username] or <c>,<flag>[=args]
all_un_gn = self._all_un_gn(acct, grps)
for v_str in self.args.v:
m = re_vol.match(v_str)
if not m:
@@ -1394,19 +1134,20 @@ class AuthSrv(object):
if WINDOWS:
src = uncyg(src)
vols = self._map_volume_idp(src, dst, mount, daxs, mflags, all_un_gn)
# print("\n".join([src, dst, perms]))
src = absreal(src)
dst = dst.strip("/")
self._map_volume(src, dst, mount, daxs, mflags)
for x in perms.split(":"):
lvl, uname = x.split(",", 1) if "," in x else [x, ""]
self._read_vol_str_idp(lvl, uname, vols, all_un_gn, daxs, mflags)
self._read_vol_str(lvl, uname, daxs[dst], mflags[dst])
if self.args.c:
for cfg_fn in self.args.c:
lns: list[str] = []
try:
self._parse_config_file(
cfg_fn, lns, acct, grps, daxs, mflags, mount
)
self._parse_config_file(cfg_fn, lns, acct, daxs, mflags, mount)
zs = "#\033[36m cfg files in "
zst = [x[len(zs) :] for x in lns if x.startswith(zs)]
@@ -1437,14 +1178,13 @@ class AuthSrv(object):
mount = cased
if not mount and not self.args.idp_h_usr:
if not mount:
# -h says our defaults are CWD at root and read/write for everyone
axs = AXS(["*"], ["*"], None, None)
vfs = VFS(self.log_func, absreal("."), "", axs, {})
elif "" not in mount:
# there's volumes but no root; make root inaccessible
vfs = VFS(self.log_func, "", "", AXS(), {})
vfs.flags["tcolor"] = self.args.tcolor
vfs.flags["d2d"] = True
maxdepth = 0
@@ -1474,13 +1214,9 @@ class AuthSrv(object):
vol.all_vps.sort(key=lambda x: len(x[0]), reverse=True)
vol.root = vfs
zss = set(acct)
zss.update(self.idp_accs)
zss.discard("*")
unames = ["*"] + list(sorted(zss))
for perm in "read write move del get pget html admin dot".split():
axs_key = "u" + perm
unames = ["*"] + list(acct.keys())
for vp, vol in vfs.all_vols.items():
zx = getattr(vol.axs, axs_key)
if "*" in zx:
@@ -1514,20 +1250,18 @@ class AuthSrv(object):
]:
for usr in d:
all_users[usr] = 1
if usr != "*" and usr not in acct and usr not in self.idp_accs:
if usr != "*" and usr not in acct:
missing_users[usr] = 1
if "*" not in d:
associated_users[usr] = 1
if missing_users:
zs = ", ".join(k for k in sorted(missing_users))
if self.args.idp_h_usr:
t = "the following users are unknown, and assumed to come from IdP: "
self.log(t + zs, c=6)
else:
t = "you must -a the following users: "
self.log(t + zs, c=1)
raise Exception(BAD_CFG)
self.log(
"you must -a the following users: "
+ ", ".join(k for k in sorted(missing_users)),
c=1,
)
raise Exception(BAD_CFG)
if LEELOO_DALLAS in all_users:
raise Exception("sorry, reserved username: " + LEELOO_DALLAS)
@@ -1668,6 +1402,13 @@ class AuthSrv(object):
if not vol.flags.get("robots"):
vol.flags["norobots"] = True
for vol in vfs.all_vols.values():
h = [vol.flags.get("html_head", self.args.html_head)]
if vol.flags.get("norobots"):
h.insert(0, META_NOBOTS)
vol.flags["html_head"] = "\n".join([x for x in h if x])
for vol in vfs.all_vols.values():
if self.args.no_vthumb:
vol.flags["dvthumb"] = True
@@ -1689,20 +1430,6 @@ class AuthSrv(object):
vol.flags["fk"] = int(fk) if fk is not True else 8
have_fk = True
dk = vol.flags.get("dk")
dks = vol.flags.get("dks")
dky = vol.flags.get("dky")
if dks is not None and dky is not None:
t = "WARNING: volume /%s has both dks and dky enabled; this is too yolo and not permitted"
raise Exception(t % (vol.vpath,))
if dks and not dk:
dk = dks
if dky and not dk:
dk = dky
if dk:
vol.flags["dk"] = int(dk) if dk is not True else 8
if have_fk and re.match(r"^[0-9\.]+$", self.args.fk_salt):
self.log("filekey salt: {}".format(self.args.fk_salt))
@@ -1730,11 +1457,7 @@ class AuthSrv(object):
if self.args.e2d or "e2ds" in vol.flags:
vol.flags["e2d"] = True
for ga, vf in [
["no_hash", "nohash"],
["no_idx", "noidx"],
["og_ua", "og_ua"],
]:
for ga, vf in [["no_hash", "nohash"], ["no_idx", "noidx"]]:
if vf in vol.flags:
ptn = re.compile(vol.flags.pop(vf))
else:
@@ -1763,7 +1486,7 @@ class AuthSrv(object):
if k not in vol.flags:
vol.flags[k] = getattr(self.args, k)
for k in ("nrand", "u2abort"):
for k in ("nrand",):
if k in vol.flags:
vol.flags[k] = int(vol.flags[k])
@@ -1771,22 +1494,6 @@ class AuthSrv(object):
if k in vol.flags:
vol.flags[k] = float(vol.flags[k])
for k in ("mv_re", "rm_re"):
try:
zs1, zs2 = vol.flags[k + "try"].split("/")
vol.flags[k + "_t"] = float(zs1)
vol.flags[k + "_r"] = float(zs2)
except:
t = 'volume "/%s" has invalid %stry [%s]'
raise Exception(t % (vol.vpath, k, vol.flags.get(k + "try")))
if vol.flags.get("og"):
self.args.uqe = True
zs = str(vol.flags.get("tcolor", "")).lstrip("#")
if len(zs) == 3: # fc5 => ffcc55
vol.flags["tcolor"] = "".join([x * 2 for x in zs])
for k1, k2 in IMPLICATIONS:
if k1 in vol.flags:
vol.flags[k2] = True
@@ -1798,8 +1505,8 @@ class AuthSrv(object):
dbds = "acid|swal|wal|yolo"
vol.flags["dbd"] = dbd = vol.flags.get("dbd") or self.args.dbd
if dbd not in dbds.split("|"):
t = 'volume "/%s" has invalid dbd [%s]; must be one of [%s]'
raise Exception(t % (vol.vpath, dbd, dbds))
t = "invalid dbd [{}]; must be one of [{}]"
raise Exception(t.format(dbd, dbds))
# default tag cfgs if unset
for k in ("mte", "mth", "exp_md", "exp_lg"):
@@ -2035,44 +1742,20 @@ class AuthSrv(object):
except Pebkac:
self.warn_anonwrite = True
idp_err = "WARNING! The following IdP volumes are mounted directly below another volume where anonymous users can read and/or write files. This is a SECURITY HAZARD!! When copyparty is restarted, it will not know about these IdP volumes yet. These volumes will then be accessible by anonymous users UNTIL one of the users associated with their volume sends a request to the server. RECOMMENDATION: You should create a restricted volume where nobody can read/write files, and make sure that all IdP volumes are configured to appear somewhere below that volume."
for idp_vp in self.idp_vols:
parent_vp = vsplit(idp_vp)[0]
vn, _ = vfs.get(parent_vp, "*", False, False)
zs = (
"READABLE"
if "*" in vn.axs.uread
else "WRITABLE"
if "*" in vn.axs.uwrite
else ""
)
if zs:
t = '\nWARNING: Volume "/%s" appears below "/%s" and would be WORLD-%s'
idp_err += t % (idp_vp, vn.vpath, zs)
if "\n" in idp_err:
self.log(idp_err, 1)
with self.mutex:
self.vfs = vfs
self.acct = acct
self.iacct = {v: k for k, v in acct.items()}
self.vfs = vfs
self.acct = acct
self.grps = grps
self.iacct = {v: k for k, v in acct.items()}
self.re_pwd = None
pwds = [re.escape(x) for x in self.iacct.keys()]
if pwds:
if self.ah.on:
zs = r"(\[H\] pw:.*|[?&]pw=)([^&]+)"
else:
zs = r"(\[H\] pw:.*|=)(" + "|".join(pwds) + r")([]&; ]|$)"
self.re_pwd = None
pwds = [re.escape(x) for x in self.iacct.keys()]
if pwds:
if self.ah.on:
zs = r"(\[H\] pw:.*|[?&]pw=)([^&]+)"
else:
zs = r"(\[H\] pw:.*|=)(" + "|".join(pwds) + r")([]&; ]|$)"
self.re_pwd = re.compile(zs)
# to ensure it propagates into tcpsrv with mp on
if self.args.mime:
for zs in self.args.mime:
ext, mime = zs.split("=", 1)
MIMES[ext] = mime
EXTS.update({v: k for k, v in MIMES.items()})
self.re_pwd = re.compile(zs)
def setup_pwhash(self, acct: dict[str, str]) -> None:
self.ah = PWHash(self.args)
@@ -2314,12 +1997,6 @@ class AuthSrv(object):
ret.append(" {}: {}".format(u, p))
ret.append("")
if self.grps:
ret.append("[groups]")
for gn, uns in self.grps.items():
ret.append(" %s: %s" % (gn, ", ".join(uns)))
ret.append("")
for vol in self.vfs.all_vols.values():
ret.append("[/{}]".format(vol.vpath))
ret.append(" " + vol.realpath)
@@ -2417,50 +2094,27 @@ def split_cfg_ln(ln: str) -> dict[str, Any]:
return ret
def expand_config_file(
log: Optional["NamedLogger"], ret: list[str], fp: str, ipath: str
) -> None:
def expand_config_file(ret: list[str], fp: str, ipath: str) -> None:
"""expand all % file includes"""
fp = absreal(fp)
if len(ipath.split(" -> ")) > 64:
raise Exception("hit max depth of 64 includes")
if os.path.isdir(fp):
names = list(sorted(os.listdir(fp)))
cnames = [x for x in names if x.lower().endswith(".conf")]
if not cnames:
t = "warning: tried to read config-files from folder '%s' but it does not contain any "
if names:
t += ".conf files; the following files/subfolders were ignored: %s"
t = t % (fp, ", ".join(names[:8]))
else:
t += "files at all"
t = t % (fp,)
if log:
log(t, 3)
ret.append("#\033[33m %s\033[0m" % (t,))
else:
zs = "#\033[36m cfg files in %s => %s\033[0m" % (fp, cnames)
ret.append(zs)
for fn in cnames:
names = os.listdir(fp)
crumb = "#\033[36m cfg files in {} => {}\033[0m".format(fp, names)
ret.append(crumb)
for fn in sorted(names):
fp2 = os.path.join(fp, fn)
if fp2 in ipath:
if not fp2.endswith(".conf") or fp2 in ipath:
continue
expand_config_file(log, ret, fp2, ipath)
expand_config_file(ret, fp2, ipath)
return
if ret[-1] == crumb:
# no config files below; remove breadcrumb
ret.pop()
if not os.path.exists(fp):
t = "warning: tried to read config from '%s' but the file/folder does not exist"
t = t % (fp,)
if log:
log(t, 3)
ret.append("#\033[31m %s\033[0m" % (t,))
return
ipath += " -> " + fp
@@ -2474,7 +2128,7 @@ def expand_config_file(
fp2 = ln[1:].strip()
fp2 = os.path.join(os.path.dirname(fp), fp2)
ofs = len(ret)
expand_config_file(log, ret, fp2, ipath)
expand_config_file(ret, fp2, ipath)
for n in range(ofs, len(ret)):
ret[n] = pad + ret[n]
continue

View File

@@ -57,8 +57,11 @@ class BrokerMp(object):
def shutdown(self) -> None:
self.log("broker", "shutting down")
for n, proc in enumerate(self.procs):
name = "mp-shut-%d-%d" % (n, len(self.procs))
Daemon(proc.q_pend.put, name, ((0, "shutdown", []),))
thr = threading.Thread(
target=proc.q_pend.put((0, "shutdown", [])),
name="mp-shutdown-{}-{}".format(n, len(self.procs)),
)
thr.start()
with self.mutex:
procs = self.procs

View File

@@ -6,8 +6,7 @@ import os
import shutil
import time
from .__init__ import ANYWIN
from .util import Netdev, runcmd, wrename, wunlink
from .util import Netdev, runcmd
HAVE_CFSSL = True
@@ -15,12 +14,6 @@ if True: # pylint: disable=using-constant-test
from .util import RootLogger
if ANYWIN:
VF = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
else:
VF = {"mv_re_t": 0, "rm_re_t": 0}
def ensure_cert(log: "RootLogger", args) -> None:
"""
the default cert (and the entire TLS support) is only here to enable the
@@ -112,12 +105,8 @@ def _gen_ca(log: "RootLogger", args):
raise Exception("failed to translate ca-cert: {}, {}".format(rc, se), 3)
bname = os.path.join(args.crt_dir, "ca")
try:
wunlink(log, bname + ".key", VF)
except:
pass
wrename(log, bname + "-key.pem", bname + ".key", VF)
wunlink(log, bname + ".csr", VF)
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
log("cert", "new ca OK", 2)
@@ -196,11 +185,11 @@ def _gen_srv(log: "RootLogger", args, netdevs: dict[str, Netdev]):
bname = os.path.join(args.crt_dir, "srv")
try:
wunlink(log, bname + ".key", VF)
os.unlink(bname + ".key")
except:
pass
wrename(log, bname + "-key.pem", bname + ".key", VF)
wunlink(log, bname + ".csr", VF)
os.rename(bname + "-key.pem", bname + ".key")
os.unlink(bname + ".csr")
with open(os.path.join(args.crt_dir, "ca.pem"), "rb") as f:
ca = f.read()

View File

@@ -16,11 +16,11 @@ def vf_bmap() -> dict[str, str]:
"no_dedup": "copydupes",
"no_dupe": "nodupe",
"no_forget": "noforget",
"no_pipe": "nopipe",
"no_robots": "norobots",
"no_thumb": "dthumb",
"no_vthumb": "dvthumb",
"no_athumb": "dathumb",
"th_no_crop": "nocrop",
}
for k in (
"dotsrch",
@@ -39,9 +39,6 @@ def vf_bmap() -> dict[str, str]:
"magic",
"no_sb_md",
"no_sb_lg",
"og",
"og_no_head",
"og_s_title",
"rand",
"xdev",
"xlink",
@@ -59,30 +56,14 @@ def vf_vmap() -> dict[str, str]:
"re_maxage": "scan",
"th_convt": "convt",
"th_size": "thsize",
"th_crop": "crop",
"th_x3": "th3x",
}
for k in (
"dbd",
"html_head",
"lg_sbf",
"md_sbf",
"nrand",
"og_desc",
"og_site",
"og_th",
"og_title",
"og_title_a",
"og_title_v",
"og_title_i",
"og_tpl",
"og_ua",
"mv_retry",
"rm_retry",
"sort",
"tcolor",
"unlist",
"u2abort",
"u2ts",
):
ret[k] = k
@@ -95,6 +76,7 @@ def vf_cmap() -> dict[str, str]:
for k in (
"exp_lg",
"exp_md",
"html_head",
"mte",
"mth",
"mtp",
@@ -132,7 +114,6 @@ flagcats = {
"hardlink": "does dedup with hardlinks instead of symlinks",
"neversymlink": "disables symlink fallback; full copy instead",
"copydupes": "disables dedup, always saves full copies of dupes",
"sparse": "force use of sparse files, mainly for s3-backed storage",
"daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",
"nosub": "forces all uploads into the top folder of the vfs",
"magic": "enables filetype detection for nameless uploads",
@@ -147,7 +128,6 @@ flagcats = {
"rand": "force randomized filenames, 9 chars long by default",
"nrand=N": "randomized filenames are N chars long",
"u2ts=fc": "[f]orce [c]lient-last-modified or [u]pload-time",
"u2abort=1": "allow aborting unfinished uploads? 0=no 1=strict 2=ip-chk 3=acct-chk",
"sz=1k-3m": "allow filesizes between 1 KiB and 3MiB",
"df=1g": "ensure 1 GiB free disk space",
},
@@ -190,10 +170,8 @@ flagcats = {
"dvthumb": "disables video thumbnails",
"dathumb": "disables audio thumbnails (spectrograms)",
"dithumb": "disables image thumbnails",
"pngquant": "compress audio waveforms 33% better",
"thsize": "thumbnail res; WxH",
"crop": "center-cropping (y/n/fy/fn)",
"th3x": "3x resolution (y/n/fy/fn)",
"nocrop": "disable center-cropping by default",
"convt": "conversion timeout in seconds",
},
"handlers\n(better explained in --help-handlers)": {
@@ -215,7 +193,7 @@ flagcats = {
"grid": "show grid/thumbnails by default",
"sort": "default sort order",
"unlist": "dont list files matching REGEX",
"html_head=TXT": "includes TXT in the <head>, or @PATH for file at PATH",
"html_head=TXT": "includes TXT in the <head>",
"robots": "allows indexing by search engines (default)",
"norobots": "kindly asks search engines to leave",
"no_sb_md": "disable js sandbox for markdown files",
@@ -230,8 +208,6 @@ flagcats = {
"dots": "allow all users with read-access to\nenable the option to show dotfiles in listings",
"fk=8": 'generates per-file accesskeys,\nwhich are then required at the "g" permission;\nkeys are invalidated if filesize or inode changes',
"fka=8": 'generates slightly weaker per-file accesskeys,\nwhich are then required at the "g" permission;\nnot affected by filesize or inode numbers',
"mv_retry": "ms-windows: timeout for renaming busy files",
"rm_retry": "ms-windows: timeout for deleting busy files",
"davauth": "ask webdav clients to login for all folders",
"davrt": "show lastmod time of symlink destination, not the link itself\n(note: this option is always enabled for recursive listings)",
},

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import re
import time
@@ -18,26 +17,20 @@ if True: # pylint: disable=using-constant-test
class Fstab(object):
def __init__(self, log: "RootLogger", args: argparse.Namespace):
def __init__(self, log: "RootLogger"):
self.log_func = log
self.warned = False
self.trusted = False
self.tab: Optional[VFS] = None
self.oldtab: Optional[VFS] = None
self.srctab = "a"
self.cache: dict[str, str] = {}
self.age = 0.0
self.maxage = args.mtab_age
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("fstab", msg, c)
def get(self, path: str) -> str:
now = time.time()
if now - self.age > self.maxage or len(self.cache) > 9000:
self.age = now
self.oldtab = self.tab or self.oldtab
if len(self.cache) > 9000:
self.age = time.time()
self.tab = None
self.cache = {}
@@ -82,7 +75,7 @@ class Fstab(object):
self.trusted = False
def build_tab(self) -> None:
self.log("inspecting mtab for changes")
self.log("building tab")
sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
if MACOS:
@@ -91,7 +84,6 @@ class Fstab(object):
ptn = re.compile(sptn)
so, _ = chkcmd(["mount"])
tab1: list[tuple[str, str]] = []
atab = []
for ln in so.split("\n"):
m = ptn.match(ln)
if not m:
@@ -99,15 +91,6 @@ class Fstab(object):
zs1, zs2 = m.groups()
tab1.append((str(zs1), str(zs2)))
atab.append(ln)
# keep empirically-correct values if mounttab unchanged
srctab = "\n".join(sorted(atab))
if srctab == self.srctab:
self.tab = self.oldtab
return
self.log("mtab has changed; reevaluating support for sparse files")
tab1.sort(key=lambda x: (len(x[0]), x[0]))
path1, fs1 = tab1[0]
@@ -116,7 +99,6 @@ class Fstab(object):
tab.add(fs, path.lstrip("/"))
self.tab = tab
self.srctab = srctab
def relabel(self, path: str, nval: str) -> None:
assert self.tab
@@ -151,9 +133,7 @@ class Fstab(object):
self.trusted = True
except:
# prisonparty or other restrictive environment
if not self.warned:
self.warned = True
self.log("failed to build tab:\n{}".format(min_ex()), 3)
self.log("failed to build tab:\n{}".format(min_ex()), 3)
self.build_fallback()
assert self.tab

View File

@@ -20,7 +20,6 @@ from .authsrv import VFS
from .bos import bos
from .util import (
Daemon,
ODict,
Pebkac,
exclude_dotfiles,
fsenc,
@@ -218,7 +217,7 @@ class FtpFs(AbstractedFS):
raise FSE("Cannot open existing file for writing")
self.validpath(ap)
return open(fsenc(ap), mode, self.args.iobuf)
return open(fsenc(ap), mode)
def chdir(self, path: str) -> None:
nwd = join(self.cwd, path)
@@ -300,7 +299,7 @@ class FtpFs(AbstractedFS):
vp = join(self.cwd, path).lstrip("/")
try:
self.hub.up2k.handle_rm(self.uname, self.h.cli_ip, [vp], [], False, False)
self.hub.up2k.handle_rm(self.uname, self.h.cli_ip, [vp], [], False)
except Exception as ex:
raise FSE(str(ex))
@@ -410,7 +409,7 @@ class FtpHandler(FTPHandler):
if cip.startswith("::ffff:"):
cip = cip[7:]
if self.args.ftp_ipa_nm and not self.args.ftp_ipa_nm.map(cip):
if self.args.ftp_ipa_re and not self.args.ftp_ipa_re.match(cip):
logging.warning("client rejected (--ftp-ipa): %s", cip)
self.connected = False
conn.close()
@@ -546,8 +545,6 @@ class Ftpd(object):
if self.args.ftp4:
ips = [x for x in ips if ":" not in x]
ips = list(ODict.fromkeys(ips)) # dedup
ioloop = IOLoop()
for ip in ips:
for h, lp in hs:

File diff suppressed because it is too large Load Diff

View File

@@ -23,7 +23,7 @@ from .mtag import HAVE_FFMPEG
from .th_cli import ThumbCli
from .th_srv import HAVE_PIL, HAVE_VIPS
from .u2idx import U2idx
from .util import HMaccas, NetMap, shut_socket
from .util import HMaccas, shut_socket
if True: # pylint: disable=using-constant-test
from typing import Optional, Pattern, Union
@@ -50,15 +50,11 @@ class HttpConn(object):
self.addr = addr
self.hsrv = hsrv
self.u2mutex: threading.Lock = hsrv.u2mutex # mypy404
self.mutex: threading.Lock = hsrv.mutex # mypy404
self.args: argparse.Namespace = hsrv.args # mypy404
self.E: EnvParams = self.args.E
self.asrv: AuthSrv = hsrv.asrv # mypy404
self.u2fh: Util.FHC = hsrv.u2fh # mypy404
self.pipes: Util.CachedDict = hsrv.pipes # mypy404
self.ipa_nm: Optional[NetMap] = hsrv.ipa_nm
self.xff_nm: Optional[NetMap] = hsrv.xff_nm
self.xff_lan: NetMap = hsrv.xff_lan # type: ignore
self.iphash: HMaccas = hsrv.broker.iphash
self.bans: dict[str, int] = hsrv.bans
self.aclose: dict[str, int] = hsrv.aclose

View File

@@ -61,14 +61,12 @@ from .u2idx import U2idx
from .util import (
E_SCK,
FHC,
CachedDict,
Daemon,
Garda,
Magician,
Netdev,
NetMap,
absreal,
build_netmap,
ipnorm,
min_ex,
shut_socket,
@@ -105,7 +103,7 @@ class HttpSrv(object):
self.t0 = time.time()
nsuf = "-n{}-i{:x}".format(nid, os.getpid()) if nid else ""
self.magician = Magician()
self.nm = NetMap([], [])
self.nm = NetMap([], {})
self.ssdp: Optional["SSDPr"] = None
self.gpwd = Garda(self.args.ban_pw)
self.g404 = Garda(self.args.ban_404)
@@ -119,7 +117,6 @@ class HttpSrv(object):
self.bound: set[tuple[str, int]] = set()
self.name = "hsrv" + nsuf
self.mutex = threading.Lock()
self.u2mutex = threading.Lock()
self.stopping = False
self.tp_nthr = 0 # actual
@@ -131,7 +128,6 @@ class HttpSrv(object):
self.t_periodic: Optional[threading.Thread] = None
self.u2fh = FHC()
self.pipes = CachedDict(0.2)
self.metrics = Metrics(self)
self.nreq = 0
self.nsus = 0
@@ -153,10 +149,6 @@ class HttpSrv(object):
zs = os.path.join(self.E.mod, "web", "deps", "prism.js.gz")
self.prism = os.path.exists(zs)
self.ipa_nm = build_netmap(self.args.ipa)
self.xff_nm = build_netmap(self.args.xff_src)
self.xff_lan = build_netmap("lan")
self.statics: set[str] = set()
self._build_statics()
@@ -198,7 +190,7 @@ class HttpSrv(object):
for fn in df:
ap = absreal(os.path.join(dp, fn))
self.statics.add(ap)
if ap.endswith(".gz"):
if ap.endswith(".gz") or ap.endswith(".br"):
self.statics.add(ap[:-3])
def set_netdevs(self, netdevs: dict[str, Netdev]) -> None:
@@ -206,7 +198,7 @@ class HttpSrv(object):
for ip, _ in self.bound:
ips.add(ip)
self.nm = NetMap(list(ips), list(netdevs))
self.nm = NetMap(list(ips), netdevs)
def start_threads(self, n: int) -> None:
self.tp_nthr += n
@@ -228,7 +220,7 @@ class HttpSrv(object):
def periodic(self) -> None:
while True:
time.sleep(2 if self.tp_ncli or self.ncli else 10)
with self.u2mutex, self.mutex:
with self.mutex:
self.u2fh.clean()
if self.tp_q:
self.tp_ncli = max(self.ncli, self.tp_ncli - 2)
@@ -266,7 +258,10 @@ class HttpSrv(object):
msg = "subscribed @ {}:{} f{} p{}".format(hip, port, fno, os.getpid())
self.log(self.name, msg)
Daemon(self.broker.say, "sig-hsrv-up1", ("cb_httpsrv_up",))
def fun() -> None:
self.broker.say("cb_httpsrv_up")
threading.Thread(target=fun, name="sig-hsrv-up1").start()
while not self.stopping:
if self.args.log_conn:

View File

@@ -8,7 +8,7 @@ import re
from .__init__ import PY2
from .th_srv import HAVE_PIL, HAVE_PILF
from .util import BytesIO, html_escape # type: ignore
from .util import BytesIO # type: ignore
class Ico(object):
@@ -31,9 +31,10 @@ class Ico(object):
w = 100
h = 30
if as_thumb:
if not self.args.th_no_crop and as_thumb:
sw, sh = self.args.th_size.split("x")
h = int(100.0 / (float(sw) / float(sh)))
w = 100
if chrome:
# cannot handle more than ~2000 unique SVGs
@@ -98,6 +99,6 @@ class Ico(object):
fill="#{}" font-family="monospace" font-size="14px" style="letter-spacing:.5px">{}</text>
</g></svg>
"""
svg = svg.format(h, c[:6], c[6:], html_escape(ext, True))
svg = svg.format(h, c[:6], c[6:], ext)
return "image/svg+xml", svg.encode("utf-8")

View File

@@ -179,7 +179,7 @@ class Metrics(object):
tnbytes = 0
tnfiles = 0
for vpath, vol in allvols:
cur = idx.get_cur(vol)
cur = idx.get_cur(vol.realpath)
if not cur:
continue
@@ -206,9 +206,6 @@ class Metrics(object):
try:
x = self.hsrv.broker.ask("up2k.get_unfinished")
xs = x.get()
if not xs:
raise Exception("up2k mutex acquisition timed out")
xj = json.loads(xs)
for ptop, (nbytes, nfiles) in xj.items():
tnbytes += nbytes

View File

@@ -7,15 +7,12 @@ import os
import shutil
import subprocess as sp
import sys
import tempfile
from .__init__ import ANYWIN, EXE, PY2, WINDOWS, E, unicode
from .authsrv import VFS
from .bos import bos
from .util import (
FFMPEG_URL,
REKOBO_LKEY,
VF_CAREFUL,
fsenc,
min_ex,
pybin,
@@ -23,13 +20,12 @@ from .util import (
runcmd,
sfsenc,
uncyg,
wunlink,
)
if True: # pylint: disable=using-constant-test
from typing import Any, Optional, Union
from typing import Any, Union
from .util import NamedLogger, RootLogger
from .util import RootLogger
def have_ff(scmd: str) -> bool:
@@ -111,51 +107,6 @@ class MParser(object):
raise Exception()
def au_unpk(log: "NamedLogger", fmt_map: dict[str, str], abspath: str, vn: Optional[VFS] = None) -> str:
ret = ""
try:
ext = abspath.split(".")[-1].lower()
au, pk = fmt_map[ext].split(".")
fd, ret = tempfile.mkstemp("." + au)
if pk == "gz":
import gzip
fi = gzip.GzipFile(abspath, mode="rb")
elif pk == "xz":
import lzma
fi = lzma.open(abspath, "rb")
elif pk == "zip":
import zipfile
zf = zipfile.ZipFile(abspath, "r")
zil = zf.infolist()
zil = [x for x in zil if x.filename.lower().split(".")[-1] == au]
fi = zf.open(zil[0])
with os.fdopen(fd, "wb") as fo:
while True:
buf = fi.read(32768)
if not buf:
break
fo.write(buf)
return ret
except Exception as ex:
if ret:
t = "failed to decompress audio file [%s]: %r"
log(t % (abspath, ex))
wunlink(log, ret, vn.flags if vn else VF_CAREFUL)
return abspath
def ffprobe(
abspath: str, timeout: int = 60
) -> tuple[dict[str, tuple[int, Any]], dict[str, list[Any]]]:
@@ -330,7 +281,7 @@ class MTag(object):
or_ffprobe = " or FFprobe"
if self.backend == "mutagen":
self._get = self.get_mutagen
self.get = self.get_mutagen
try:
from mutagen import version # noqa: F401
except:
@@ -339,7 +290,7 @@ class MTag(object):
if self.backend == "ffprobe":
self.usable = self.can_ffprobe
self._get = self.get_ffprobe
self.get = self.get_ffprobe
self.prefer_mt = True
if not HAVE_FFPROBE:
@@ -509,17 +460,6 @@ class MTag(object):
return r1
def get(self, abspath: str) -> dict[str, Union[str, float]]:
ext = abspath.split(".")[-1].lower()
if ext not in self.args.au_unpk:
return self._get(abspath)
ap = au_unpk(self.log, self.args.au_unpk, abspath)
ret = self._get(ap)
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret
def get_mutagen(self, abspath: str) -> dict[str, Union[str, float]]:
ret: dict[str, tuple[int, Any]] = {}
@@ -611,18 +551,13 @@ class MTag(object):
pypath = str(os.pathsep.join(zsl))
env["PYTHONPATH"] = pypath
except:
raise # might be expected outside cpython
ext = abspath.split(".")[-1].lower()
if ext in self.args.au_unpk:
ap = au_unpk(self.log, self.args.au_unpk, abspath)
else:
ap = abspath
if not E.ox and not EXE:
raise
ret: dict[str, Any] = {}
for tagname, parser in sorted(parsers.items(), key=lambda x: (x[1].pri, x[0])):
try:
cmd = [parser.bin, ap]
cmd = [parser.bin, abspath]
if parser.bin.endswith(".py"):
cmd = [pybin] + cmd
@@ -659,7 +594,4 @@ class MTag(object):
t = "mtag error: tagname {}, parser {}, file {} => {}"
self.log(t.format(tagname, parser.bin, abspath, min_ex()))
if ap != abspath:
wunlink(self.log, ap, VF_CAREFUL)
return ret

View File

@@ -110,7 +110,7 @@ class MCast(object):
)
ips = [x for x in ips if x not in ("::1", "127.0.0.1")]
ips = find_prefix(ips, list(netdevs))
ips = find_prefix(ips, netdevs)
on = self.on[:]
off = self.off[:]
@@ -206,7 +206,6 @@ class MCast(object):
except:
t = "announce failed on {} [{}]:\n{}"
self.log(t.format(netdev, ip, min_ex()), 3)
sck.close()
if self.args.zm_msub:
for s1 in self.srv.values():

View File

@@ -127,7 +127,7 @@ class SMB(object):
self.log("smb", msg, c)
def start(self) -> None:
Daemon(self.srv.start, "smbd")
Daemon(self.srv.start)
def _auth_cb(self, *a, **ka):
debug("auth-result: %s %s", a, ka)
@@ -340,7 +340,7 @@ class SMB(object):
yeet("blocked delete (no-del-acc): " + vpath)
vpath = vpath.replace("\\", "/").lstrip("/")
self.hub.up2k.handle_rm(uname, "1.7.6.2", [vpath], [], False, False)
self.hub.up2k.handle_rm(uname, "1.7.6.2", [vpath], [], False)
def _utime(self, vpath: str, times: tuple[float, float]) -> None:
if not self.args.smbw:

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import re
import stat
import tarfile
@@ -45,12 +44,11 @@ class StreamTar(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
fgen: Generator[dict[str, Any], None, None],
cmp: str = "",
**kwargs: Any
):
super(StreamTar, self).__init__(log, args, fgen)
super(StreamTar, self).__init__(log, fgen)
self.ci = 0
self.co = 0
@@ -128,7 +126,7 @@ class StreamTar(StreamArc):
inf.gid = 0
self.ci += inf.size
with open(fsenc(src), "rb", self.args.iobuf) as fo:
with open(fsenc(src), "rb", 512 * 1024) as fo:
self.tar.addfile(inf, fo)
def _gen(self) -> None:

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import os
import tempfile
from datetime import datetime
@@ -21,12 +20,10 @@ class StreamArc(object):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
fgen: Generator[dict[str, Any], None, None],
**kwargs: Any
):
self.log = log
self.args = args
self.fgen = fgen
self.stopped = False
@@ -81,9 +78,7 @@ def enthumb(
) -> dict[str, Any]:
rem = f["vp"]
ext = rem.rsplit(".", 1)[-1].lower()
if (fmt == "mp3" and ext == "mp3") or (
fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|")
):
if fmt == "opus" and ext in "aac|m4a|mp3|ogg|opus|wma".split("|"):
raise Exception()
vp = vjoin(vtop, rem.split("/", 1)[1])

View File

@@ -28,7 +28,7 @@ if True: # pylint: disable=using-constant-test
import typing
from typing import Any, Optional, Union
from .__init__ import ANYWIN, EXE, MACOS, TYPE_CHECKING, E, EnvParams, unicode
from .__init__ import ANYWIN, EXE, MACOS, TYPE_CHECKING, EnvParams, unicode
from .authsrv import BAD_CFG, AuthSrv
from .cert import ensure_cert
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE
@@ -49,7 +49,6 @@ from .util import (
ODict,
alltrace,
ansi_re,
build_netmap,
min_ex,
mp,
odfusion,
@@ -95,7 +94,7 @@ class SvcHub(object):
self.stopping = False
self.stopped = False
self.reload_req = False
self.reloading = 0
self.reloading = False
self.stop_cond = threading.Condition()
self.nsigs = 3
self.retcode = 0
@@ -134,7 +133,7 @@ class SvcHub(object):
if not self._process_config():
raise Exception(BAD_CFG)
# for non-http clients (ftp, tftp)
# for non-http clients (ftp)
self.bans: dict[str, int] = {}
self.gpwd = Garda(self.args.ban_pw)
self.g404 = Garda(self.args.ban_404)
@@ -155,8 +154,6 @@ class SvcHub(object):
lg.handlers = [lh]
lg.setLevel(logging.DEBUG)
self._check_env()
if args.stackmon:
start_stackmon(args.stackmon, 0)
@@ -173,26 +170,6 @@ class SvcHub(object):
self.log("root", t.format(args.j), c=3)
args.no_fpool = True
for name, arg in (
("iobuf", "iobuf"),
("s-rd-sz", "s_rd_sz"),
("s-wr-sz", "s_wr_sz"),
):
zi = getattr(args, arg)
if zi < 32768:
t = "WARNING: expect very poor performance because you specified a very low value (%d) for --%s"
self.log("root", t % (zi, name), 3)
zi = 2
zi2 = 2 ** (zi - 1).bit_length()
if zi != zi2:
zi3 = 2 ** ((zi - 1).bit_length() - 1)
t = "WARNING: expect poor performance because --%s is not a power-of-two; consider using %d or %d instead of %d"
self.log("root", t % (name, zi2, zi3, zi), 3)
if args.s_rd_sz > args.iobuf:
t = "WARNING: --s-rd-sz (%d) is larger than --iobuf (%d); this may lead to reduced performance"
self.log("root", t % (args.s_rd_sz, args.iobuf), 3)
bri = "zy"[args.theme % 2 :][:1]
ch = "abcdefghijklmnopqrstuvwx"[int(args.theme / 2)]
args.theme = "{0}{1} {0} {1}".format(ch, bri)
@@ -240,10 +217,6 @@ class SvcHub(object):
if not HAVE_FFMPEG or not HAVE_FFPROBE:
decs.pop("ff", None)
# compressed formats; "s3z=s3m.zip, s3gz=s3m.gz, ..."
zlss = [x.strip().lower().split("=", 1) for x in args.au_unpk.split(",")]
args.au_unpk = {x[0]: x[1] for x in zlss}
self.args.th_dec = list(decs.keys())
self.thumbsrv = None
want_ff = False
@@ -280,13 +253,6 @@ class SvcHub(object):
if want_ff and ANYWIN:
self.log("thumb", "download FFmpeg to fix it:\033[0m " + FFMPEG_URL, 3)
if not args.no_acode:
if not re.match("^(0|[qv][0-9]|[0-9]{2,3}k)$", args.q_mp3.lower()):
t = "invalid mp3 transcoding quality [%s] specified; only supports [0] to disable, a CBR value such as [192k], or a CQ/CRF value such as [v2]"
raise Exception(t % (args.q_mp3,))
else:
args.au_unpk = {}
args.th_poke = min(args.th_poke, args.th_maxage, args.ac_maxage)
zms = ""
@@ -299,16 +265,9 @@ class SvcHub(object):
from .ftpd import Ftpd
self.ftpd: Optional[Ftpd] = None
Daemon(self.start_ftpd, "start_ftpd")
zms += "f" if args.ftp else "F"
if args.tftp:
from .tftpd import Tftpd
self.tftpd: Optional[Tftpd] = None
if args.ftp or args.ftps or args.tftp:
Daemon(self.start_ftpd, "start_tftpd")
if args.smb:
# impacket.dcerpc is noisy about listen timeouts
sto = socket.getdefaulttimeout()
@@ -338,12 +297,10 @@ class SvcHub(object):
def start_ftpd(self) -> None:
time.sleep(30)
if self.ftpd:
return
if hasattr(self, "ftpd") and not self.ftpd:
self.restart_ftpd()
if hasattr(self, "tftpd") and not self.tftpd:
self.restart_tftpd()
self.restart_ftpd()
def restart_ftpd(self) -> None:
if not hasattr(self, "ftpd"):
@@ -360,17 +317,6 @@ class SvcHub(object):
self.ftpd = Ftpd(self)
self.log("root", "started FTPd")
def restart_tftpd(self) -> None:
if not hasattr(self, "tftpd"):
return
from .tftpd import Tftpd
if self.tftpd:
return # todo
self.tftpd = Tftpd(self)
def thr_httpsrv_up(self) -> None:
time.sleep(1 if self.args.ign_ebind_all else 5)
expected = self.broker.num_workers * self.tcpsrv.nsrv
@@ -395,7 +341,7 @@ class SvcHub(object):
self.sigterm()
def sigterm(self) -> None:
self.signal_handler(signal.SIGTERM, None)
os.kill(os.getpid(), signal.SIGTERM)
def cb_httpsrv_up(self) -> None:
self.httpsrv_up += 1
@@ -420,23 +366,6 @@ class SvcHub(object):
Daemon(self.sd_notify, "sd-notify")
def _check_env(self) -> None:
try:
files = os.listdir(E.cfg)
except:
files = []
hits = [x for x in files if x.lower().endswith(".conf")]
if hits:
t = "WARNING: found config files in [%s]: %s\n config files are not expected here, and will NOT be loaded (unless your setup is intentionally hella funky)"
self.log("root", t % (E.cfg, ", ".join(hits)), 3)
if self.args.no_bauth:
t = "WARNING: --no-bauth disables support for the Android app; you may want to use --bauth-last instead"
self.log("root", t, 3)
if self.args.bauth_last:
self.log("root", "WARNING: ignoring --bauth-last due to --no-bauth", 3)
def _process_config(self) -> bool:
al = self.args
@@ -503,13 +432,6 @@ class SvcHub(object):
else:
setattr(al, k, re.compile(vs))
for k in "tftp_lsf".split(" "):
vs = getattr(al, k)
if not vs or vs == "no":
setattr(al, k, None)
else:
setattr(al, k, re.compile("^" + vs + "$"))
if not al.sus_urls:
al.ban_url = "no"
elif al.ban_url == "no":
@@ -517,11 +439,11 @@ class SvcHub(object):
al.xff_hdr = al.xff_hdr.lower()
al.idp_h_usr = al.idp_h_usr.lower()
al.idp_h_grp = al.idp_h_grp.lower()
al.idp_h_key = al.idp_h_key.lower()
# al.idp_h_grp = al.idp_h_grp.lower()
al.ftp_ipa_nm = build_netmap(al.ftp_ipa or al.ipa)
al.tftp_ipa_nm = build_netmap(al.tftp_ipa or al.ipa)
al.xff_re = self._ipa2re(al.xff_src)
al.ipa_re = self._ipa2re(al.ipa)
al.ftp_ipa_re = self._ipa2re(al.ftp_ipa or al.ipa)
mte = ODict.fromkeys(DEF_MTE.split(","), True)
al.mte = odfusion(mte, al.mte)
@@ -533,41 +455,11 @@ class SvcHub(object):
al.exp_md = odfusion(exp, al.exp_md.replace(" ", ","))
al.exp_lg = odfusion(exp, al.exp_lg.replace(" ", ","))
for k in ["no_hash", "no_idx", "og_ua"]:
for k in ["no_hash", "no_idx"]:
ptn = getattr(self.args, k)
if ptn:
setattr(self.args, k, re.compile(ptn))
for k in ["idp_gsep"]:
ptn = getattr(self.args, k)
if "]" in ptn:
ptn = "]" + ptn.replace("]", "")
if "[" in ptn:
ptn = ptn.replace("[", "") + "["
if "-" in ptn:
ptn = ptn.replace("-", "") + "-"
ptn = ptn.replace("\\", "\\\\").replace("^", "\\^")
setattr(self.args, k, re.compile("[%s]" % (ptn,)))
try:
zf1, zf2 = self.args.rm_retry.split("/")
self.args.rm_re_t = float(zf1)
self.args.rm_re_r = float(zf2)
except:
raise Exception("invalid --rm-retry [%s]" % (self.args.rm_retry,))
try:
zf1, zf2 = self.args.mv_retry.split("/")
self.args.mv_re_t = float(zf1)
self.args.mv_re_r = float(zf2)
except:
raise Exception("invalid --mv-retry [%s]" % (self.args.mv_retry,))
al.tcolor = al.tcolor.lstrip("#")
if len(al.tcolor) == 3: # fc5 => ffcc55
al.tcolor = "".join([x * 2 for x in al.tcolor])
return True
def _ipa2re(self, txt) -> Optional[re.Pattern]:
@@ -736,37 +628,21 @@ class SvcHub(object):
self.log("root", "ssdp startup failed;\n" + min_ex(), 3)
def reload(self) -> str:
with self.up2k.mutex:
if self.reloading:
return "cannot reload; already in progress"
self.reloading = 1
if self.reloading:
return "cannot reload; already in progress"
self.reloading = True
Daemon(self._reload, "reloading")
return "reload initiated"
def _reload(self, rescan_all_vols: bool = True) -> None:
def _reload(self) -> None:
self.log("root", "reload scheduled")
with self.up2k.mutex:
if self.reloading != 1:
return
self.reloading = 2
self.log("root", "reloading config")
self.asrv.reload()
self.up2k.reload(rescan_all_vols)
self.up2k.reload()
self.broker.reload()
self.reloading = 0
def _reload_blocking(self, rescan_all_vols: bool = True) -> None:
while True:
with self.up2k.mutex:
if self.reloading < 2:
self.reloading = 1
break
time.sleep(0.05)
# try to handle multiple pending IdP reloads at once:
time.sleep(0.2)
self._reload(rescan_all_vols=rescan_all_vols)
self.reloading = False
def stop_thr(self) -> None:
while not self.stop_req:

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import calendar
import stat
import time
@@ -219,13 +218,12 @@ class StreamZip(StreamArc):
def __init__(
self,
log: "NamedLogger",
args: argparse.Namespace,
fgen: Generator[dict[str, Any], None, None],
utf8: bool = False,
pre_crc: bool = False,
**kwargs: Any
) -> None:
super(StreamZip, self).__init__(log, args, fgen)
super(StreamZip, self).__init__(log, fgen)
self.utf8 = utf8
self.pre_crc = pre_crc
@@ -250,7 +248,7 @@ class StreamZip(StreamArc):
crc = 0
if self.pre_crc:
for buf in yieldfile(src, self.args.iobuf):
for buf in yieldfile(src):
crc = zlib.crc32(buf, crc)
crc &= 0xFFFFFFFF
@@ -259,7 +257,7 @@ class StreamZip(StreamArc):
buf = gen_hdr(None, name, sz, ts, self.utf8, crc, self.pre_crc)
yield self._ct(buf)
for buf in yieldfile(src, self.args.iobuf):
for buf in yieldfile(src):
if not self.pre_crc:
crc = zlib.crc32(buf, crc)

View File

@@ -309,7 +309,6 @@ class TcpSrv(object):
self.hub.start_zeroconf()
gencert(self.log, self.args, self.netdevs)
self.hub.restart_ftpd()
self.hub.restart_tftpd()
def shutdown(self) -> None:
self.stopping = True
@@ -463,12 +462,6 @@ class TcpSrv(object):
sys.stderr.flush()
def _qr(self, t1: dict[str, list[int]], t2: dict[str, list[int]]) -> str:
t2c = {zs: zli for zs, zli in t2.items() if zs in ("127.0.0.1", "::1")}
t2b = {zs: zli for zs, zli in t2.items() if ":" in zs and zs not in t2c}
t2 = {zs: zli for zs, zli in t2.items() if zs not in t2b and zs not in t2c}
t2.update(t2b) # first ipv4, then ipv6...
t2.update(t2c) # ...and finally localhost
ip = None
ips = list(t1) + list(t2)
qri = self.args.qri

View File

@@ -1,434 +0,0 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
try:
from types import SimpleNamespace
except:
class SimpleNamespace(object):
def __init__(self, **attr):
self.__dict__.update(attr)
import logging
import os
import re
import socket
import stat
import threading
import time
from datetime import datetime
try:
import inspect
except:
pass
from partftpy import (
TftpContexts,
TftpPacketFactory,
TftpPacketTypes,
TftpServer,
TftpStates,
)
from partftpy.TftpShared import TftpException
from .__init__ import EXE, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .util import BytesIO, Daemon, ODict, exclude_dotfiles, min_ex, runhook, undot
if True: # pylint: disable=using-constant-test
from typing import Any, Union
if TYPE_CHECKING:
from .svchub import SvcHub
lg = logging.getLogger("tftp")
debug, info, warning, error = (lg.debug, lg.info, lg.warning, lg.error)
def noop(*a, **ka) -> None:
pass
def _serverInitial(self, pkt: Any, raddress: str, rport: int) -> bool:
info("connection from %s:%s", raddress, rport)
ret = _sinitial[0](self, pkt, raddress, rport)
nm = _hub[0].args.tftp_ipa_nm
if nm and not nm.map(raddress):
yeet("client rejected (--tftp-ipa): %s" % (raddress,))
return ret
# patch ipa-check into partftpd (part 1/2)
_hub: list["SvcHub"] = []
_sinitial: list[Any] = []
class Tftpd(object):
def __init__(self, hub: "SvcHub") -> None:
self.hub = hub
self.args = hub.args
self.asrv = hub.asrv
self.log = hub.log
self.mutex = threading.Lock()
_hub[:] = []
_hub.append(hub)
lg.setLevel(logging.DEBUG if self.args.tftpv else logging.INFO)
for x in ["partftpy", "partftpy.TftpStates", "partftpy.TftpServer"]:
lgr = logging.getLogger(x)
lgr.setLevel(logging.DEBUG if self.args.tftpv else logging.INFO)
if not self.args.tftpv and not self.args.tftpvv:
# contexts -> states -> packettypes -> shared
# contexts -> packetfactory
# packetfactory -> packettypes
Cs = [
TftpPacketTypes,
TftpPacketFactory,
TftpStates,
TftpContexts,
TftpServer,
]
cbak = []
if not self.args.tftp_no_fast and not EXE:
try:
ptn = re.compile(r"(^\s*)log\.debug\(.*\)$")
for C in Cs:
cbak.append(C.__dict__)
src1 = inspect.getsource(C).split("\n")
src2 = "\n".join([ptn.sub("\\1pass", ln) for ln in src1])
cfn = C.__spec__.origin
exec (compile(src2, filename=cfn, mode="exec"), C.__dict__)
except Exception:
t = "failed to optimize tftp code; run with --tftp-noopt if there are issues:\n"
self.log("tftp", t + min_ex(), 3)
for n, zd in enumerate(cbak):
Cs[n].__dict__ = zd
for C in Cs:
C.log.debug = noop
# patch ipa-check into partftpd (part 2/2)
_sinitial[:] = []
_sinitial.append(TftpStates.TftpServerState.serverInitial)
TftpStates.TftpServerState.serverInitial = _serverInitial
# patch vfs into partftpy
TftpContexts.open = self._open
TftpStates.open = self._open
fos = SimpleNamespace()
for k in os.__dict__:
try:
setattr(fos, k, getattr(os, k))
except:
pass
fos.access = self._access
fos.mkdir = self._mkdir
fos.unlink = self._unlink
fos.sep = "/"
TftpContexts.os = fos
TftpServer.os = fos
TftpStates.os = fos
fop = SimpleNamespace()
for k in os.path.__dict__:
try:
setattr(fop, k, getattr(os.path, k))
except:
pass
fop.abspath = self._p_abspath
fop.exists = self._p_exists
fop.isdir = self._p_isdir
fop.normpath = self._p_normpath
fos.path = fop
self._disarm(fos)
ip = next((x for x in self.args.i if ":" not in x), None)
if not ip:
self.log("tftp", "IPv6 not supported for tftp; listening on 0.0.0.0", 3)
ip = "0.0.0.0"
self.port = int(self.args.tftp)
self.srv = []
self.ips = []
ports = []
if self.args.tftp_pr:
p1, p2 = [int(x) for x in self.args.tftp_pr.split("-")]
ports = list(range(p1, p2 + 1))
ips = self.args.i
if "::" in ips:
ips.append("0.0.0.0")
if self.args.ftp4:
ips = [x for x in ips if ":" not in x]
ips = list(ODict.fromkeys(ips)) # dedup
for ip in ips:
name = "tftp_%s" % (ip,)
Daemon(self._start, name, [ip, ports])
time.sleep(0.2) # give dualstack a chance
def nlog(self, msg: str, c: Union[int, str] = 0) -> None:
self.log("tftp", msg, c)
def _start(self, ip, ports):
fam = socket.AF_INET6 if ":" in ip else socket.AF_INET
have_been_alive = False
while True:
srv = TftpServer.TftpServer("/", self._ls)
with self.mutex:
self.srv.append(srv)
self.ips.append(ip)
try:
# this is the listen loop; it should block forever
srv.listen(ip, self.port, af_family=fam, ports=ports)
except:
with self.mutex:
self.srv.remove(srv)
self.ips.remove(ip)
try:
srv.sock.close()
except:
pass
try:
bound = bool(srv.listenport)
except:
bound = False
if bound:
# this instance has managed to bind at least once
have_been_alive = True
if have_been_alive:
t = "tftp server [%s]:%d crashed; restarting in 3 sec:\n%s"
error(t, ip, self.port, min_ex())
time.sleep(3)
continue
# server failed to start; could be due to dualstack (ipv6 managed to bind and this is ipv4)
if ip != "0.0.0.0" or "::" not in self.ips:
# nope, it's fatal
t = "tftp server [%s]:%d failed to start:\n%s"
error(t, ip, self.port, min_ex())
# yep; ignore
# (TODO: move the "listening @ ..." infolog in partftpy to
# after the bind attempt so it doesn't print twice)
return
info("tftp server [%s]:%d terminated", ip, self.port)
break
def stop(self):
with self.mutex:
srvs = self.srv[:]
for srv in srvs:
srv.stop()
def _v2a(self, caller: str, vpath: str, perms: list, *a: Any) -> tuple[VFS, str]:
vpath = vpath.replace("\\", "/").lstrip("/")
if not perms:
perms = [True, True]
debug('%s("%s", %s) %s\033[K\033[0m', caller, vpath, str(a), perms)
vfs, rem = self.asrv.vfs.get(vpath, "*", *perms)
return vfs, vfs.canonical(rem)
def _ls(self, vpath: str, raddress: str, rport: int, force=False) -> Any:
# generate file listing if vpath is dir.txt and return as file object
if not force:
vpath, fn = os.path.split(vpath.replace("\\", "/"))
ptn = self.args.tftp_lsf
if not ptn or not ptn.match(fn.lower()):
return None
vn, rem = self.asrv.vfs.get(vpath, "*", True, False)
fsroot, vfs_ls, vfs_virt = vn.ls(
rem,
"*",
not self.args.no_scandir,
[[True, False]],
)
dnames = set([x[0] for x in vfs_ls if stat.S_ISDIR(x[1].st_mode)])
dirs1 = [(v.st_mtime, v.st_size, k + "/") for k, v in vfs_ls if k in dnames]
fils1 = [(v.st_mtime, v.st_size, k) for k, v in vfs_ls if k not in dnames]
real1 = dirs1 + fils1
realt = [(datetime.fromtimestamp(mt), sz, fn) for mt, sz, fn in real1]
reals = [
(
"%04d-%02d-%02d %02d:%02d:%02d"
% (
zd.year,
zd.month,
zd.day,
zd.hour,
zd.minute,
zd.second,
),
sz,
fn,
)
for zd, sz, fn in realt
]
virs = [("????-??-?? ??:??:??", 0, k + "/") for k in vfs_virt.keys()]
ls = virs + reals
if "*" not in vn.axs.udot:
names = set(exclude_dotfiles([x[2] for x in ls]))
ls = [x for x in ls if x[2] in names]
try:
biggest = max([x[1] for x in ls])
except:
biggest = 0
perms = []
if "*" in vn.axs.uread:
perms.append("read")
if "*" in vn.axs.udot:
perms.append("hidden")
if "*" in vn.axs.uwrite:
if "*" in vn.axs.udel:
perms.append("overwrite")
else:
perms.append("write")
fmt = "{{}} {{:{},}} {{}}"
fmt = fmt.format(len("{:,}".format(biggest)))
retl = ["# permissions: %s" % (", ".join(perms),)]
retl += [fmt.format(*x) for x in ls]
ret = "\n".join(retl).encode("utf-8", "replace")
return BytesIO(ret + b"\n")
def _open(self, vpath: str, mode: str, *a: Any, **ka: Any) -> Any:
rd = wr = False
if mode == "rb":
rd = True
elif mode == "wb":
wr = True
else:
raise Exception("bad mode %s" % (mode,))
vfs, ap = self._v2a("open", vpath, [rd, wr])
if wr:
if "*" not in vfs.axs.uwrite:
yeet("blocked write; folder not world-writable: /%s" % (vpath,))
if bos.path.exists(ap) and "*" not in vfs.axs.udel:
yeet("blocked write; folder not world-deletable: /%s" % (vpath,))
xbu = vfs.flags.get("xbu")
if xbu and not runhook(
self.nlog, xbu, ap, vpath, "", "", 0, 0, "8.3.8.7", 0, ""
):
yeet("blocked by xbu server config: " + vpath)
if not self.args.tftp_nols and bos.path.isdir(ap):
return self._ls(vpath, "", 0, True)
if not a:
a = [self.args.iobuf]
return open(ap, mode, *a, **ka)
def _mkdir(self, vpath: str, *a) -> None:
vfs, ap = self._v2a("mkdir", vpath, [])
if "*" not in vfs.axs.uwrite:
yeet("blocked mkdir; folder not world-writable: /%s" % (vpath,))
return bos.mkdir(ap)
def _unlink(self, vpath: str) -> None:
# return bos.unlink(self._v2a("stat", vpath, *a)[1])
vfs, ap = self._v2a("delete", vpath, [True, False, False, True])
try:
inf = bos.stat(ap)
except:
return
if not stat.S_ISREG(inf.st_mode) or inf.st_size:
yeet("attempted delete of non-empty file")
vpath = vpath.replace("\\", "/").lstrip("/")
self.hub.up2k.handle_rm("*", "8.3.8.7", [vpath], [], False, False)
def _access(self, *a: Any) -> bool:
return True
def _p_abspath(self, vpath: str) -> str:
return "/" + undot(vpath)
def _p_normpath(self, *a: Any) -> str:
return ""
def _p_exists(self, vpath: str) -> bool:
try:
ap = self._v2a("p.exists", vpath, [False, False])[1]
bos.stat(ap)
return True
except:
return False
def _p_isdir(self, vpath: str) -> bool:
try:
st = bos.stat(self._v2a("p.isdir", vpath, [False, False])[1])
ret = stat.S_ISDIR(st.st_mode)
return ret
except:
return False
def _hook(self, *a: Any, **ka: Any) -> None:
src = inspect.currentframe().f_back.f_code.co_name
error("\033[31m%s:hook(%s)\033[0m", src, a)
raise Exception("nope")
def _disarm(self, fos: SimpleNamespace) -> None:
fos.chmod = self._hook
fos.chown = self._hook
fos.close = self._hook
fos.ftruncate = self._hook
fos.lchown = self._hook
fos.link = self._hook
fos.listdir = self._hook
fos.lstat = self._hook
fos.open = self._hook
fos.remove = self._hook
fos.rename = self._hook
fos.replace = self._hook
fos.scandir = self._hook
fos.stat = self._hook
fos.symlink = self._hook
fos.truncate = self._hook
fos.utime = self._hook
fos.walk = self._hook
fos.path.expanduser = self._hook
fos.path.expandvars = self._hook
fos.path.getatime = self._hook
fos.path.getctime = self._hook
fos.path.getmtime = self._hook
fos.path.getsize = self._hook
fos.path.isabs = self._hook
fos.path.isfile = self._hook
fos.path.islink = self._hook
fos.path.realpath = self._hook
def yeet(msg: str) -> None:
warning(msg)
raise TftpException(msg)

View File

@@ -57,7 +57,7 @@ class ThumbCli(object):
if is_vid and "dvthumb" in dbv.flags:
return None
want_opus = fmt in ("opus", "caf", "mp3")
want_opus = fmt in ("opus", "caf")
is_au = ext in self.fmt_ffa
if is_au:
if want_opus:
@@ -78,39 +78,16 @@ class ThumbCli(object):
if rem.startswith(".hist/th/") and rem.split(".")[-1] in ["webp", "jpg", "png"]:
return os.path.join(ptop, rem)
if fmt[:1] in "jw":
sfmt = fmt[:1]
if fmt == "j" and self.args.th_no_jpg:
fmt = "w"
if sfmt == "j" and self.args.th_no_jpg:
sfmt = "w"
if sfmt == "w":
if (
self.args.th_no_webp
or (is_img and not self.can_webp)
or (self.args.th_ff_jpg and (not is_img or preferred == "ff"))
):
sfmt = "j"
vf_crop = dbv.flags["crop"]
vf_th3x = dbv.flags["th3x"]
if "f" in vf_crop:
sfmt += "f" if "n" in vf_crop else ""
else:
sfmt += "f" if "f" in fmt else ""
if "f" in vf_th3x:
sfmt += "3" if "y" in vf_th3x else ""
else:
sfmt += "3" if "3" in fmt else ""
fmt = sfmt
elif fmt[:1] == "p" and not is_au:
t = "cannot thumbnail [%s]: png only allowed for waveforms"
self.log(t % (rem), 6)
return None
if fmt == "w":
if (
self.args.th_no_webp
or (is_img and not self.can_webp)
or (self.args.th_ff_jpg and (not is_img or preferred == "ff"))
):
fmt = "j"
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:

View File

@@ -15,10 +15,10 @@ from queue import Queue
from .__init__ import ANYWIN, TYPE_CHECKING
from .authsrv import VFS
from .bos import bos
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, au_unpk, ffprobe
from .util import BytesIO # type: ignore
from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, ffprobe
from .util import (
FFMPEG_URL,
BytesIO, # type: ignore
Cooldown,
Daemon,
Pebkac,
@@ -28,8 +28,6 @@ from .util import (
runcmd,
statdir,
vsplit,
wrename,
wunlink,
)
if True: # pylint: disable=using-constant-test
@@ -98,8 +96,8 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
# spectrograms are never cropped; strip fullsize flag
ext = rem.split(".")[-1].lower()
if ext in ffa and fmt[:2] in ("wf", "jf"):
fmt = fmt.replace("f", "")
if ext in ffa and fmt in ("wf", "jf"):
fmt = fmt[:1]
rd += "\n" + fmt
h = hashlib.sha512(afsenc(rd)).digest()
@@ -110,7 +108,7 @@ def thumb_path(histpath: str, rem: str, mtime: float, fmt: str, ffa: set[str]) -
h = hashlib.sha512(afsenc(fn)).digest()
fn = base64.urlsafe_b64encode(h).decode("ascii")[:24]
if fmt in ("opus", "caf", "mp3"):
if fmt in ("opus", "caf"):
cat = "ac"
else:
fc = fmt[:1]
@@ -131,8 +129,6 @@ class ThumbSrv(object):
self.mutex = threading.Lock()
self.busy: dict[str, list[threading.Condition]] = {}
self.ram: dict[str, float] = {}
self.memcond = threading.Condition(self.mutex)
self.stopping = False
self.nthr = max(1, self.args.th_mt)
@@ -201,10 +197,9 @@ class ThumbSrv(object):
with self.mutex:
return not self.nthr
def getres(self, vn: VFS, fmt: str) -> tuple[int, int]:
mul = 3 if "3" in fmt else 1
def getres(self, vn: VFS) -> tuple[int, int]:
w, h = vn.flags["thsize"].split("x")
return int(w) * mul, int(h) * mul
return int(w), int(h)
def get(self, ptop: str, rem: str, mtime: float, fmt: str) -> Optional[str]:
histpath = self.asrv.vfs.histtab.get(ptop)
@@ -219,7 +214,7 @@ class ThumbSrv(object):
with self.mutex:
try:
self.busy[tpath].append(cond)
self.log("joined waiting room for %s" % (tpath,))
self.log("wait {}".format(tpath))
except:
thdir = os.path.dirname(tpath)
bos.makedirs(os.path.join(thdir, "w"))
@@ -270,23 +265,6 @@ class ThumbSrv(object):
"ffa": self.fmt_ffa,
}
def wait4ram(self, need: float, ttpath: str) -> None:
ram = self.args.th_ram_max
if need > ram * 0.99:
t = "file too big; need %.2f GiB RAM, but --th-ram-max is only %.1f"
raise Exception(t % (need, ram))
while True:
with self.mutex:
used = sum([v for k, v in self.ram.items() if k != ttpath]) + need
if used < ram:
# self.log("XXX self.ram: %s" % (self.ram,), 5)
self.ram[ttpath] = need
return
with self.memcond:
# self.log("at RAM limit; used %.2f GiB, need %.2f more" % (used-need, need), 1)
self.memcond.wait(3)
def worker(self) -> None:
while not self.stopping:
task = self.q.get()
@@ -297,12 +275,6 @@ class ThumbSrv(object):
ext = abspath.split(".")[-1].lower()
png_ok = False
funs = []
if ext in self.args.au_unpk:
ap_unpk = au_unpk(self.log, self.args.au_unpk, abspath, vn)
else:
ap_unpk = abspath
if not bos.path.exists(tpath):
for lib in self.args.th_dec:
if lib == "pil" and ext in self.fmt_pil:
@@ -314,27 +286,25 @@ class ThumbSrv(object):
elif lib == "ff" and ext in self.fmt_ffa:
if tpath.endswith(".opus") or tpath.endswith(".caf"):
funs.append(self.conv_opus)
elif tpath.endswith(".mp3"):
funs.append(self.conv_mp3)
elif tpath.endswith(".png"):
funs.append(self.conv_waves)
png_ok = True
else:
funs.append(self.conv_spec)
if not png_ok and tpath.endswith(".png"):
raise Pebkac(400, "png only allowed for waveforms")
tdir, tfn = os.path.split(tpath)
ttpath = os.path.join(tdir, "w", tfn)
try:
wunlink(self.log, ttpath, vn.flags)
bos.unlink(ttpath)
except:
pass
for fun in funs:
try:
if not png_ok and tpath.endswith(".png"):
raise Exception("png only allowed for waveforms")
fun(ap_unpk, ttpath, fmt, vn)
fun(abspath, ttpath, fmt, vn)
break
except Exception as ex:
msg = "{} could not create thumbnail of {}\n{}"
@@ -348,36 +318,29 @@ class ThumbSrv(object):
else:
# ffmpeg may spawn empty files on windows
try:
wunlink(self.log, ttpath, vn.flags)
os.unlink(ttpath)
except:
pass
if abspath != ap_unpk:
wunlink(self.log, ap_unpk, vn.flags)
try:
wrename(self.log, ttpath, tpath, vn.flags)
bos.rename(ttpath, tpath)
except:
pass
with self.mutex:
subs = self.busy[tpath]
del self.busy[tpath]
self.ram.pop(ttpath, None)
for x in subs:
with x:
x.notify_all()
with self.memcond:
self.memcond.notify_all()
with self.mutex:
self.nthr -= 1
def fancy_pillow(self, im: "Image.Image", fmt: str, vn: VFS) -> "Image.Image":
# exif_transpose is expensive (loads full image + unconditional copy)
res = self.getres(vn, fmt)
res = self.getres(vn)
r = max(*res) * 2
im.thumbnail((r, r), resample=Image.LANCZOS)
try:
@@ -392,7 +355,7 @@ class ThumbSrv(object):
if rot in rots:
im = im.transpose(rots[rot])
if "f" in fmt:
if fmt.endswith("f"):
im.thumbnail(res, resample=Image.LANCZOS)
else:
iw, ih = im.size
@@ -403,13 +366,12 @@ class ThumbSrv(object):
return im
def conv_pil(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
self.wait4ram(0.2, tpath)
with Image.open(fsenc(abspath)) as im:
try:
im = self.fancy_pillow(im, fmt, vn)
except Exception as ex:
self.log("fancy_pillow {}".format(ex), "90")
im.thumbnail(self.getres(vn, fmt))
im.thumbnail(self.getres(vn))
fmts = ["RGB", "L"]
args = {"quality": 40}
@@ -433,12 +395,11 @@ class ThumbSrv(object):
im.save(tpath, **args)
def conv_vips(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
self.wait4ram(0.2, tpath)
crops = ["centre", "none"]
if "f" in fmt:
if fmt.endswith("f"):
crops = ["none"]
w, h = self.getres(vn, fmt)
w, h = self.getres(vn)
kw = {"height": h, "size": "down", "intent": "relative"}
for c in crops:
@@ -454,7 +415,6 @@ class ThumbSrv(object):
img.write_to_file(tpath, Q=40)
def conv_ffmpeg(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if not ret:
return
@@ -467,12 +427,12 @@ class ThumbSrv(object):
seek = [b"-ss", "{:.0f}".format(dur / 3).encode("utf-8")]
scale = "scale={0}:{1}:force_original_aspect_ratio="
if "f" in fmt:
if fmt.endswith("f"):
scale += "decrease,setsar=1:1"
else:
scale += "increase,crop={0}:{1},setsar=1:1"
res = self.getres(vn, fmt)
res = self.getres(vn)
bscale = scale.format(*list(res)).encode("utf-8")
# fmt: off
cmd = [
@@ -557,21 +517,8 @@ class ThumbSrv(object):
if "ac" not in ret:
raise Exception("not audio")
# jt_versi.xm: 405M/839s
dur = ret[".dur"][1] if ".dur" in ret else 300
need = 0.2 + dur / 3000
speedup = b""
if need > self.args.th_ram_max * 0.7:
self.log("waves too big (need %.2f GiB); trying to optimize" % (need,))
need = 0.2 + dur / 4200 # only helps about this much...
speedup = b"aresample=8000,"
if need > self.args.th_ram_max * 0.96:
raise Exception("file too big; cannot waves")
self.wait4ram(need, tpath)
flt = b"[0:a:0]" + speedup
flt += (
flt = (
b"[0:a:0]"
b"compand=.3|.3:1|1:-90/-60|-60/-40|-40/-30|-20/-20:6:0:-90:0.2"
b",volume=2"
b",showwavespic=s=2048x64:colors=white"
@@ -593,43 +540,12 @@ class ThumbSrv(object):
cmd += [fsenc(tpath)]
self._run_ff(cmd, vn)
if "pngquant" in vn.flags:
wtpath = tpath + ".png"
cmd = [
b"pngquant",
b"--strip",
b"--nofs",
b"--output", fsenc(wtpath),
fsenc(tpath)
]
ret = runcmd(cmd, timeout=vn.flags["convt"], nice=True, oom=400)[0]
if ret:
try:
wunlink(self.log, wtpath, vn.flags)
except:
pass
else:
wrename(self.log, wtpath, tpath, vn.flags)
def conv_spec(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
raise Exception("not audio")
# https://trac.ffmpeg.org/ticket/10797
# expect 1 GiB every 600 seconds when duration is tricky;
# simple filetypes are generally safer so let's special-case those
safe = ("flac", "wav", "aif", "aiff", "opus")
coeff = 1800 if abspath.split(".")[-1].lower() in safe else 600
dur = ret[".dur"][1] if ".dur" in ret else 300
need = 0.2 + dur / coeff
self.wait4ram(need, tpath)
fc = "[0:a:0]aresample=48000{},showspectrumpic=s="
if "3" in fmt:
fc += "1280x1024,crop=1420:1056:70:48[o]"
else:
fc += "640x512,crop=780:544:70:48[o]"
fc = "[0:a:0]aresample=48000{},showspectrumpic=s=640x512,crop=780:544:70:50[o]"
if self.args.th_ff_swr:
fco = ":filter_size=128:cutoff=0.877"
@@ -667,50 +583,10 @@ class ThumbSrv(object):
cmd += [fsenc(tpath)]
self._run_ff(cmd, vn)
def conv_mp3(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
quality = self.args.q_mp3.lower()
if self.args.no_acode or not quality:
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
raise Exception("not audio")
if quality.endswith("k"):
qk = b"-b:a"
qv = quality.encode("ascii")
else:
qk = b"-q:a"
qv = quality[1:].encode("ascii")
# extremely conservative choices for output format
# (always 2ch 44k1) because if a device is old enough
# to not support opus then it's probably also super picky
# fmt: off
cmd = [
b"ffmpeg",
b"-nostdin",
b"-v", b"error",
b"-hide_banner",
b"-i", fsenc(abspath),
b"-map_metadata", b"-1",
b"-map", b"0:a:0",
b"-ar", b"44100",
b"-ac", b"2",
b"-c:a", b"libmp3lame",
qk, qv,
fsenc(tpath)
]
# fmt: on
self._run_ff(cmd, vn, oom=300)
def conv_opus(self, abspath: str, tpath: str, fmt: str, vn: VFS) -> None:
if self.args.no_acode or not self.args.q_opus:
if self.args.no_acode:
raise Exception("disabled in server config")
self.wait4ram(0.2, tpath)
ret, _ = ffprobe(abspath, int(vn.flags["convt"] / 2))
if "ac" not in ret:
raise Exception("not audio")
@@ -726,12 +602,11 @@ class ThumbSrv(object):
if want_caf:
tmp_opus = tpath + ".opus"
try:
wunlink(self.log, tmp_opus, vn.flags)
bos.unlink(tmp_opus)
except:
pass
caf_src = abspath if src_opus else tmp_opus
bq = ("%dk" % (self.args.q_opus,)).encode("ascii")
if not want_caf or not src_opus:
# fmt: off
@@ -744,7 +619,7 @@ class ThumbSrv(object):
b"-map_metadata", b"-1",
b"-map", b"0:a:0",
b"-c:a", b"libopus",
b"-b:a", bq,
b"-b:a", b"128k",
fsenc(tmp_opus)
]
# fmt: on
@@ -767,7 +642,7 @@ class ThumbSrv(object):
b"-map_metadata", b"-1",
b"-ac", b"2",
b"-c:a", b"libopus",
b"-b:a", bq,
b"-b:a", b"128k",
b"-f", b"caf",
fsenc(tpath)
]
@@ -794,7 +669,7 @@ class ThumbSrv(object):
if tmp_opus != tpath:
try:
wunlink(self.log, tmp_opus, vn.flags)
bos.unlink(tmp_opus)
except:
pass
@@ -821,10 +696,7 @@ class ThumbSrv(object):
else:
self.log("\033[Jcln {} ({})/\033[A".format(histpath, vol))
try:
ndirs += self.clean(histpath)
except Exception as ex:
self.log("\033[Jcln err in %s: %r" % (histpath, ex), 3)
ndirs += self.clean(histpath)
self.log("\033[Jcln ok; rm {} dirs".format(ndirs))
@@ -841,7 +713,7 @@ class ThumbSrv(object):
def _clean(self, cat: str, thumbpath: str) -> int:
# self.log("cln {}".format(thumbpath))
exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf", "mp3"]
exts = ["jpg", "webp", "png"] if cat == "th" else ["opus", "caf"]
maxage = getattr(self.args, cat + "_maxage")
now = time.time()
prev_b64 = None

View File

@@ -62,17 +62,6 @@ class U2idx(object):
def log(self, msg: str, c: Union[int, str] = 0) -> None:
self.log_func("u2idx", msg, c)
def shutdown(self) -> None:
for cur in self.cur.values():
db = cur.connection
try:
db.interrupt()
except:
pass
cur.close()
db.close()
def fsearch(
self, uname: str, vols: list[VFS], body: dict[str, Any]
) -> list[dict[str, Any]]:
@@ -92,18 +81,14 @@ class U2idx(object):
except:
raise Pebkac(500, min_ex())
def get_cur(self, vn: VFS) -> Optional["sqlite3.Cursor"]:
def get_cur(self, ptop: str) -> Optional["sqlite3.Cursor"]:
if not HAVE_SQLITE3:
return None
cur = self.cur.get(vn.realpath)
cur = self.cur.get(ptop)
if cur:
return cur
if "e2d" not in vn.flags:
return None
ptop = vn.realpath
histpath = self.asrv.vfs.histtab.get(ptop)
if not histpath:
self.log("no histpath for [{}]".format(ptop))
@@ -332,7 +317,7 @@ class U2idx(object):
ptop = vol.realpath
flags = vol.flags
cur = self.get_cur(vol)
cur = self.get_cur(ptop)
if not cur:
continue

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,6 @@
# coding: utf-8
from __future__ import print_function, unicode_literals
import argparse
import base64
import contextlib
import errno
@@ -35,9 +34,6 @@ from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, VT100, WINDOWS
from .__version__ import S_BUILD_DT, S_VERSION
from .stolen import surrogateescape
ub64dec = base64.urlsafe_b64decode
ub64enc = base64.urlsafe_b64encode
try:
from datetime import datetime, timezone
@@ -171,12 +167,6 @@ except:
return struct.unpack(fmt.decode("ascii"), a)
try:
BITNESS = struct.calcsize(b"P") * 8
except:
BITNESS = struct.calcsize("P") * 8
ansi_re = re.compile("\033\\[[^mK]*[mK]")
@@ -189,7 +179,7 @@ else:
SYMTIME = sys.version_info > (3, 6) and os.utime in os.supports_follow_symlinks
META_NOBOTS = '<meta name="robots" content="noindex, nofollow">\n'
META_NOBOTS = '<meta name="robots" content="noindex, nofollow">'
FFMPEG_URL = "https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z"
@@ -353,14 +343,6 @@ CMD_EXEB = set(_exestr.encode("utf-8").split())
CMD_EXES = set(_exestr.split())
# mostly from https://github.com/github/gitignore/blob/main/Global/macOS.gitignore
APPLESAN_TXT = r"/(__MACOS|Icon\r\r)|/\.(_|DS_Store|AppleDouble|LSOverride|DocumentRevisions-|fseventsd|Spotlight-|TemporaryItems|Trashes|VolumeIcon\.icns|com\.apple\.timemachine\.donotpresent|AppleDB|AppleDesktop|apdisk)"
APPLESAN_RE = re.compile(APPLESAN_TXT)
VF_CAREFUL = {"mv_re_t": 5, "rm_re_t": 5, "mv_re_r": 0.1, "rm_re_r": 0.1}
pybin = sys.executable or ""
if EXE:
pybin = ""
@@ -384,6 +366,11 @@ def py_desc() -> str:
if ofs > 0:
py_ver = py_ver[:ofs]
try:
bitness = struct.calcsize(b"P") * 8
except:
bitness = struct.calcsize("P") * 8
host_os = platform.system()
compiler = platform.python_compiler().split("http")[0]
@@ -391,7 +378,7 @@ def py_desc() -> str:
os_ver = m.group(1) if m else ""
return "{:>9} v{} on {}{} {} [{}]".format(
interp, py_ver, host_os, BITNESS, os_ver, compiler
interp, py_ver, host_os, bitness, os_ver, compiler
)
@@ -429,32 +416,14 @@ try:
except:
PYFTPD_VER = "(None)"
try:
from partftpy.__init__ import __version__ as PARTFTPY_VER
except:
PARTFTPY_VER = "(None)"
PY_DESC = py_desc()
VERSIONS = (
"copyparty v{} ({})\n{}\n sqlite {} | jinja {} | pyftpd {} | tftp {}".format(
S_VERSION, S_BUILD_DT, PY_DESC, SQLITE_VER, JINJA_VER, PYFTPD_VER, PARTFTPY_VER
)
VERSIONS = "copyparty v{} ({})\n{}\n sqlite v{} | jinja v{} | pyftpd v{}".format(
S_VERSION, S_BUILD_DT, py_desc(), SQLITE_VER, JINJA_VER, PYFTPD_VER
)
_: Any = (mp, BytesIO, quote, unquote, SQLITE_VER, JINJA_VER, PYFTPD_VER, PARTFTPY_VER)
__all__ = [
"mp",
"BytesIO",
"quote",
"unquote",
"SQLITE_VER",
"JINJA_VER",
"PYFTPD_VER",
"PARTFTPY_VER",
]
_: Any = (mp, BytesIO, quote, unquote, SQLITE_VER, JINJA_VER, PYFTPD_VER)
__all__ = ["mp", "BytesIO", "quote", "unquote", "SQLITE_VER", "JINJA_VER", "PYFTPD_VER"]
class Daemon(threading.Thread):
@@ -466,22 +435,13 @@ class Daemon(threading.Thread):
r: bool = True,
ka: Optional[dict[Any, Any]] = None,
) -> None:
threading.Thread.__init__(self, name=name)
self.a = a or ()
self.ka = ka or {}
self.fun = target
threading.Thread.__init__(
self, target=target, name=name, args=a or (), kwargs=ka
)
self.daemon = True
if r:
self.start()
def run(self):
if not ANYWIN and not PY2:
signal.pthread_sigmask(
signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]
)
self.fun(*self.a, **self.ka)
class Netdev(object):
def __init__(self, ip: str, idx: int, name: str, desc: str):
@@ -567,33 +527,25 @@ class HLog(logging.Handler):
elif record.name.startswith("impacket"):
if self.ptn_smb_ign.match(msg):
return
elif record.name.startswith("partftpy."):
record.name = record.name[9:]
self.log_func(record.name[-21:], msg, c)
class NetMap(object):
def __init__(self, ips: list[str], cidrs: list[str], keep_lo=False) -> None:
"""
ips: list of plain ipv4/ipv6 IPs, not cidr
cidrs: list of cidr-notation IPs (ip/prefix)
"""
def __init__(self, ips: list[str], netdevs: dict[str, Netdev]) -> None:
if "::" in ips:
ips = [x for x in ips if x != "::"] + list(
[x.split("/")[0] for x in cidrs if ":" in x]
[x.split("/")[0] for x in netdevs if ":" in x]
)
ips.append("0.0.0.0")
if "0.0.0.0" in ips:
ips = [x for x in ips if x != "0.0.0.0"] + list(
[x.split("/")[0] for x in cidrs if ":" not in x]
[x.split("/")[0] for x in netdevs if ":" not in x]
)
if not keep_lo:
ips = [x for x in ips if x not in ("::1", "127.0.0.1")]
ips = find_prefix(ips, cidrs)
ips = [x for x in ips if x not in ("::1", "127.0.0.1")]
ips = find_prefix(ips, netdevs)
self.cache: dict[str, str] = {}
self.b2sip: dict[bytes, str] = {}
@@ -610,9 +562,6 @@ class NetMap(object):
self.bip.sort(reverse=True)
def map(self, ip: str) -> str:
if ip.startswith("::ffff:"):
ip = ip[7:]
try:
return self.cache[ip]
except:
@@ -774,46 +723,15 @@ class CachedSet(object):
self.oldest = now
class CachedDict(object):
def __init__(self, maxage: float) -> None:
self.c: dict[str, tuple[float, Any]] = {}
self.maxage = maxage
self.oldest = 0.0
def set(self, k: str, v: Any) -> None:
now = time.time()
self.c[k] = (now, v)
if now - self.oldest < self.maxage:
return
c = self.c = {k: v for k, v in self.c.items() if now - v[0] < self.maxage}
try:
self.oldest = min([x[0] for x in c.values()])
except:
self.oldest = now
def get(self, k: str) -> Optional[tuple[str, Any]]:
try:
ts, ret = self.c[k]
now = time.time()
if now - ts > self.maxage:
del self.c[k]
return None
return ret
except:
return None
class FHC(object):
class CE(object):
def __init__(self, fh: typing.BinaryIO) -> None:
self.ts: float = 0
self.fhs = [fh]
self.all_fhs = set([fh])
def __init__(self) -> None:
self.cache: dict[str, FHC.CE] = {}
self.aps: dict[str, int] = {}
self.aps: set[str] = set()
def close(self, path: str) -> None:
try:
@@ -825,7 +743,7 @@ class FHC(object):
fh.close()
del self.cache[path]
del self.aps[path]
self.aps.remove(path)
def clean(self) -> None:
if not self.cache:
@@ -846,12 +764,9 @@ class FHC(object):
return self.cache[path].fhs.pop()
def put(self, path: str, fh: typing.BinaryIO) -> None:
if path not in self.aps:
self.aps[path] = 0
self.aps.add(path)
try:
ce = self.cache[path]
ce.all_fhs.add(fh)
ce.fhs.append(fh)
except:
ce = self.CE(fh)
@@ -865,21 +780,16 @@ class ProgressPrinter(threading.Thread):
periodically print progress info without linefeeds
"""
def __init__(self, log: "NamedLogger", args: argparse.Namespace) -> None:
def __init__(self) -> None:
threading.Thread.__init__(self, name="pp")
self.daemon = True
self.log = log
self.args = args
self.msg = ""
self.end = False
self.n = -1
self.start()
def run(self) -> None:
sigblock()
tp = 0
msg = None
no_stdout = self.args.q
fmt = " {}\033[K\r" if VT100 else " {} $\r"
while not self.end:
time.sleep(0.1)
@@ -887,21 +797,10 @@ class ProgressPrinter(threading.Thread):
continue
msg = self.msg
now = time.time()
if msg and now - tp > 10:
tp = now
self.log("progress: %s" % (msg,), 6)
if no_stdout:
continue
uprint(fmt.format(msg))
if PY2:
sys.stdout.flush()
if no_stdout:
return
if VT100:
print("\033[K", end="")
elif msg:
@@ -1321,41 +1220,23 @@ def log_thrs(log: Callable[[str, str, int], None], ival: float, name: str) -> No
log(name, "\033[0m \033[33m".join(tv), 3)
def sigblock():
if ANYWIN or PY2:
return
signal.pthread_sigmask(
signal.SIG_BLOCK, [signal.SIGINT, signal.SIGTERM, signal.SIGUSR1]
)
def vol_san(vols: list["VFS"], txt: bytes) -> bytes:
txt0 = txt
for vol in vols:
bap = vol.realpath.encode("utf-8")
bhp = vol.histpath.encode("utf-8")
bvp = vol.vpath.encode("utf-8")
bvph = b"$hist(/" + bvp + b")"
txt = txt.replace(bap, bvp)
txt = txt.replace(bhp, bvph)
txt = txt.replace(bap.replace(b"\\", b"\\\\"), bvp)
txt = txt.replace(bhp.replace(b"\\", b"\\\\"), bvph)
if txt != txt0:
txt += b"\r\nNOTE: filepaths sanitized; see serverlog for correct values"
txt = txt.replace(vol.realpath.encode("utf-8"), vol.vpath.encode("utf-8"))
txt = txt.replace(
vol.realpath.encode("utf-8").replace(b"\\", b"\\\\"),
vol.vpath.encode("utf-8"),
)
return txt
def min_ex(max_lines: int = 8, reverse: bool = False) -> str:
et, ev, tb = sys.exc_info()
stb = traceback.extract_tb(tb) if tb else traceback.extract_stack()[:-1]
stb = traceback.extract_tb(tb)
fmt = "%s @ %d <%s>: %s"
ex = [fmt % (fp.split(os.sep)[-1], ln, fun, txt) for fp, ln, fun, txt in stb]
if et or ev or tb:
ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
ex.append("[%s] %s" % (et.__name__ if et else "(anonymous)", ev))
return "\n".join(ex[-max_lines:][:: -1 if reverse else 1])
@@ -1460,15 +1341,10 @@ def ren_open(
class MultipartParser(object):
def __init__(
self,
log_func: "NamedLogger",
args: argparse.Namespace,
sr: Unrecv,
http_headers: dict[str, str],
self, log_func: "NamedLogger", sr: Unrecv, http_headers: dict[str, str]
):
self.sr = sr
self.log = log_func
self.args = args
self.headers = http_headers
self.re_ctype = re.compile(r"^content-type: *([^; ]+)", re.IGNORECASE)
@@ -1567,7 +1443,7 @@ class MultipartParser(object):
def _read_data(self) -> Generator[bytes, None, None]:
blen = len(self.boundary)
bufsz = self.args.s_rd_sz
bufsz = 32 * 1024
while True:
try:
buf = self.sr.recv(bufsz)
@@ -1842,7 +1718,7 @@ def get_spd(nbyte: int, t0: float, t: Optional[float] = None) -> str:
if t is None:
t = time.time()
bps = nbyte / ((t - t0) or 0.001)
bps = nbyte / ((t - t0) + 0.001)
s1 = humansize(nbyte).replace(" ", "\033[33m").replace("iB", "")
s2 = humansize(bps).replace(" ", "\033[35m").replace("iB", "")
return "%s \033[0m%s/s\033[0m" % (s1, s2)
@@ -1994,10 +1870,10 @@ def ipnorm(ip: str) -> str:
return ip
def find_prefix(ips: list[str], cidrs: list[str]) -> list[str]:
def find_prefix(ips: list[str], netdevs: dict[str, Netdev]) -> list[str]:
ret = []
for ip in ips:
hit = next((x for x in cidrs if x.startswith(ip + "/") or ip == x), None)
hit = next((x for x in netdevs if x.startswith(ip + "/")), None)
if hit:
ret.append(hit)
return ret
@@ -2057,7 +1933,6 @@ def vsplit(vpath: str) -> tuple[str, str]:
return vpath.rsplit("/", 1) # type: ignore
# vpath-join
def vjoin(rd: str, fn: str) -> str:
if rd and fn:
return rd + "/" + fn
@@ -2065,14 +1940,6 @@ def vjoin(rd: str, fn: str) -> str:
return rd or fn
# url-join
def ujoin(rd: str, fn: str) -> str:
if rd and fn:
return rd.rstrip("/") + "/" + fn.lstrip("/")
else:
return rd or fn
def _w8dec2(txt: bytes) -> str:
"""decodes filesystem-bytes to wtf8"""
return surrogateescape.decodefilename(txt)
@@ -2194,95 +2061,16 @@ def lsof(log: "NamedLogger", abspath: str) -> None:
log("lsof failed; " + min_ex(), 3)
def _fs_mvrm(
log: "NamedLogger", src: str, dst: str, atomic: bool, flags: dict[str, Any]
) -> bool:
bsrc = fsenc(src)
bdst = fsenc(dst)
if atomic:
k = "mv_re_"
act = "atomic-rename"
osfun = os.replace
args = [bsrc, bdst]
elif dst:
k = "mv_re_"
act = "rename"
osfun = os.rename
args = [bsrc, bdst]
def atomic_move(usrc: str, udst: str) -> None:
src = fsenc(usrc)
dst = fsenc(udst)
if not PY2:
os.replace(src, dst)
else:
k = "rm_re_"
act = "delete"
osfun = os.unlink
args = [bsrc]
if os.path.exists(dst):
os.unlink(dst)
maxtime = flags.get(k + "t", 0.0)
chill = flags.get(k + "r", 0.0)
if chill < 0.001:
chill = 0.1
ino = 0
t0 = now = time.time()
for attempt in range(90210):
try:
if ino and os.stat(bsrc).st_ino != ino:
t = "src inode changed; aborting %s %s"
log(t % (act, src), 1)
return False
if (dst and not atomic) and os.path.exists(bdst):
t = "something appeared at dst; aborting rename [%s] ==> [%s]"
log(t % (src, dst), 1)
return False
osfun(*args)
if attempt:
now = time.time()
t = "%sd in %.2f sec, attempt %d: %s"
log(t % (act, now - t0, attempt + 1, src))
return True
except OSError as ex:
now = time.time()
if ex.errno == errno.ENOENT:
return False
if now - t0 > maxtime or attempt == 90209:
raise
if not attempt:
if not PY2:
ino = os.stat(bsrc).st_ino
t = "%s failed (err.%d); retrying for %d sec: [%s]"
log(t % (act, ex.errno, maxtime + 0.99, src))
time.sleep(chill)
return False # makes pylance happy
def atomic_move(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> None:
bsrc = fsenc(src)
bdst = fsenc(dst)
if PY2:
if os.path.exists(bdst):
_fs_mvrm(log, dst, "", False, flags) # unlink
_fs_mvrm(log, src, dst, False, flags) # rename
elif flags.get("mv_re_t"):
_fs_mvrm(log, src, dst, True, flags)
else:
os.replace(bsrc, bdst)
def wrename(log: "NamedLogger", src: str, dst: str, flags: dict[str, Any]) -> bool:
if not flags.get("mv_re_t"):
os.rename(fsenc(src), fsenc(dst))
return True
return _fs_mvrm(log, src, dst, False, flags)
def wunlink(log: "NamedLogger", abspath: str, flags: dict[str, Any]) -> bool:
if not flags.get("rm_re_t"):
os.unlink(fsenc(abspath))
return True
return _fs_mvrm(log, abspath, "", False, flags)
os.rename(src, dst)
def get_df(abspath: str) -> tuple[Optional[int], Optional[int]]:
@@ -2355,11 +2143,10 @@ def shut_socket(log: "NamedLogger", sck: socket.socket, timeout: int = 3) -> Non
sck.close()
def read_socket(
sr: Unrecv, bufsz: int, total_size: int
) -> Generator[bytes, None, None]:
def read_socket(sr: Unrecv, total_size: int) -> Generator[bytes, None, None]:
remains = total_size
while remains > 0:
bufsz = 32 * 1024
if bufsz > remains:
bufsz = remains
@@ -2373,16 +2160,16 @@ def read_socket(
yield buf
def read_socket_unbounded(sr: Unrecv, bufsz: int) -> Generator[bytes, None, None]:
def read_socket_unbounded(sr: Unrecv) -> Generator[bytes, None, None]:
try:
while True:
yield sr.recv(bufsz)
yield sr.recv(32 * 1024)
except:
return
def read_socket_chunked(
sr: Unrecv, bufsz: int, log: Optional["NamedLogger"] = None
sr: Unrecv, log: Optional["NamedLogger"] = None
) -> Generator[bytes, None, None]:
err = "upload aborted: expected chunk length, got [{}] |{}| instead"
while True:
@@ -2416,7 +2203,7 @@ def read_socket_chunked(
if log:
log("receiving %d byte chunk" % (chunklen,))
for chunk in read_socket(sr, bufsz, chunklen):
for chunk in read_socket(sr, chunklen):
yield chunk
x = sr.recv_ex(2, False)
@@ -2439,46 +2226,10 @@ def list_ips() -> list[str]:
return list(ret)
def build_netmap(csv: str):
csv = csv.lower().strip()
if csv in ("any", "all", "no", ",", ""):
return None
if csv in ("lan", "local", "private", "prvt"):
csv = "10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fd00::/8" # lan
csv += ", 169.254.0.0/16, fe80::/10" # link-local
csv += ", 127.0.0.0/8, ::1/128" # loopback
srcs = [x.strip() for x in csv.split(",") if x.strip()]
cidrs = []
for zs in srcs:
if not zs.endswith("."):
cidrs.append(zs)
continue
# translate old syntax "172.19." => "172.19.0.0/16"
words = len(zs.rstrip(".").split("."))
if words == 1:
zs += "0.0.0/8"
elif words == 2:
zs += "0.0/16"
elif words == 3:
zs += "0/24"
else:
raise Exception("invalid config value [%s]" % (zs,))
cidrs.append(zs)
ips = [x.split("/")[0] for x in cidrs]
return NetMap(ips, cidrs, True)
def yieldfile(fn: str, bufsz: int) -> Generator[bytes, None, None]:
readsz = min(bufsz, 128 * 1024)
with open(fsenc(fn), "rb", bufsz) as f:
def yieldfile(fn: str) -> Generator[bytes, None, None]:
with open(fsenc(fn), "rb", 512 * 1024) as f:
while True:
buf = f.read(readsz)
buf = f.read(128 * 1024)
if not buf:
break
@@ -2704,7 +2455,7 @@ def unescape_cookie(orig: str) -> str:
def guess_mime(url: str, fallback: str = "application/octet-stream") -> str:
try:
ext = url.rsplit(".", 1)[1].lower()
_, ext = url.rsplit(".", 1)
except:
return fallback

View File

@@ -17,10 +17,8 @@ window.baguetteBox = (function () {
titleTag: false,
async: false,
preload: 2,
refocus: true,
afterShow: null,
afterHide: null,
duringHide: null,
onChange: null,
},
overlay, slider, btnPrev, btnNext, btnHelp, btnAnim, btnRotL, btnRotR, btnSel, btnFull, btnVmode, btnClose,
@@ -29,7 +27,6 @@ window.baguetteBox = (function () {
isOverlayVisible = false,
touch = {}, // start-pos
touchFlag = false, // busy
scrollTimer = 0,
re_i = /^[^?]+\.(a?png|avif|bmp|gif|heif|jpe?g|jfif|svg|webp)(\?|$)/i,
re_v = /^[^?]+\.(webm|mkv|mp4)(\?|$)/i,
anims = ['slideIn', 'fadeIn', 'none'],
@@ -92,30 +89,6 @@ window.baguetteBox = (function () {
touchendHandler();
};
var overlayWheelHandler = function (e) {
if (!options.noScrollbars || anymod(e))
return;
ev(e);
var x = e.deltaX,
y = e.deltaY,
d = Math.abs(x) > Math.abs(y) ? x : y;
if (e.deltaMode)
d *= 10;
if (Date.now() - scrollTimer < (Math.abs(d) > 20 ? 100 : 300))
return;
scrollTimer = Date.now();
if (d > 0)
showNextImage();
else
showPreviousImage();
};
var trapFocusInsideOverlay = function (e) {
if (overlay.style.display === 'block' && (overlay.contains && !overlay.contains(e.target))) {
e.stopPropagation();
@@ -171,7 +144,7 @@ window.baguetteBox = (function () {
selectorData.galleries.push(gallery);
});
return [selectorData.galleries, options];
return selectorData.galleries;
}
function clearCachedData() {
@@ -282,19 +255,19 @@ window.baguetteBox = (function () {
if (anymod(e, true))
return;
var k = (e.code || e.key) + '', v = vid(), pos = -1;
var k = e.code + '', v = vid(), pos = -1;
if (k == "BracketLeft")
setloop(1);
else if (k == "BracketRight")
setloop(2);
else if (e.shiftKey && k != "KeyR" && k != "R")
else if (e.shiftKey && k != 'KeyR')
return;
else if (k == "ArrowLeft" || k == "KeyJ" || k == "Left" || k == "j")
else if (k == "ArrowLeft" || k == "KeyJ")
showPreviousImage();
else if (k == "ArrowRight" || k == "KeyL" || k == "Right" || k == "l")
else if (k == "ArrowRight" || k == "KeyL")
showNextImage();
else if (k == "Escape" || k == "Esc")
else if (k == "Escape")
hideOverlay();
else if (k == "Home")
showFirstImage(e);
@@ -322,9 +295,9 @@ window.baguetteBox = (function () {
}
else if (k == "KeyF")
tglfull();
else if (k == "KeyS" || k == "s")
else if (k == "KeyS")
tglsel();
else if (k == "KeyR" || k == "r" || k == "R")
else if (k == "KeyR")
rotn(e.shiftKey ? -1 : 1);
else if (k == "KeyY")
dlpic();
@@ -419,7 +392,8 @@ window.baguetteBox = (function () {
}
function dlpic() {
var url = addq(findfile()[3].href, 'cache');
var url = findfile()[3].href;
url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache';
dl_file(url);
}
@@ -476,7 +450,6 @@ window.baguetteBox = (function () {
bind(document, 'keyup', keyUpHandler);
bind(document, 'fullscreenchange', onFSC);
bind(overlay, 'click', overlayClickHandler);
bind(overlay, 'wheel', overlayWheelHandler);
bind(btnPrev, 'click', showPreviousImage);
bind(btnNext, 'click', showNextImage);
bind(btnClose, 'click', hideOverlay);
@@ -499,7 +472,6 @@ window.baguetteBox = (function () {
unbind(document, 'keyup', keyUpHandler);
unbind(document, 'fullscreenchange', onFSC);
unbind(overlay, 'click', overlayClickHandler);
unbind(overlay, 'wheel', overlayWheelHandler);
unbind(btnPrev, 'click', showPreviousImage);
unbind(btnNext, 'click', showNextImage);
unbind(btnClose, 'click', hideOverlay);
@@ -610,7 +582,7 @@ window.baguetteBox = (function () {
isOverlayVisible = true;
}
function hideOverlay(e, dtor) {
function hideOverlay(e) {
ev(e);
playvid(false);
removeFromCache('#files');
@@ -618,22 +590,16 @@ window.baguetteBox = (function () {
document.documentElement.style.overflowY = 'auto';
document.body.style.overflowY = 'auto';
}
try {
if (document.fullscreenElement)
document.exitFullscreen();
}
catch (ex) { }
isFullscreen = false;
if (dtor || overlay.style.display === 'none')
if (overlay.style.display === 'none')
return;
if (options.duringHide)
options.duringHide();
sethash('');
unbindEvents();
try {
document.exitFullscreen();
isFullscreen = false;
}
catch (ex) { }
// Fade out and hide the overlay
overlay.className = '';
@@ -647,45 +613,9 @@ window.baguetteBox = (function () {
if (options.afterHide)
options.afterHide();
options.refocus && documentLastFocus && documentLastFocus.focus();
documentLastFocus && documentLastFocus.focus();
isOverlayVisible = false;
unvid();
unfig();
}, 250);
}
function unvid(keep) {
var vids = QSA('#bbox-overlay video');
for (var a = vids.length - 1; a >= 0; a--) {
var v = vids[a];
if (v == keep)
continue;
v.src = '';
v.load();
var p = v.parentNode;
p.removeChild(v);
p.parentNode.removeChild(p);
}
}
function unfig(keep) {
var figs = QSA('#bbox-overlay figure'),
npre = options.preload || 0,
k = [];
if (keep === undefined)
keep = -9;
for (var a = keep - npre; a <= keep + npre; a++)
k.push('bbox-figure-' + a);
for (var a = figs.length - 1; a >= 0; a--) {
var f = figs[a];
if (!has(k, f.getAttribute('id')))
f.parentNode.removeChild(f);
}
}, 500);
}
function loadImage(index, callback) {
@@ -711,7 +641,7 @@ window.baguetteBox = (function () {
options.captions.call(currentGallery, imageElement) :
imageElement.getAttribute('data-caption') || imageElement.title;
imageSrc = addq(imageSrc, 'cache');
imageSrc += imageSrc.indexOf('?') < 0 ? '?cache' : '&cache';
if (is_vid && index != currentIndex)
return; // no preload
@@ -740,7 +670,6 @@ window.baguetteBox = (function () {
});
image.setAttribute('src', imageSrc);
if (is_vid) {
image.volume = clamp(fcfg_get('vol', dvol / 100), 0, 1);
image.setAttribute('controls', 'controls');
image.onended = vidEnd;
}
@@ -750,9 +679,6 @@ window.baguetteBox = (function () {
figure.appendChild(image);
if (is_vid && window.afilt)
afilt.apply(undefined, image);
if (options.async && callback)
callback();
}
@@ -782,7 +708,6 @@ window.baguetteBox = (function () {
}
function show(index, gallery) {
gallery = gallery || currentGallery;
if (!isOverlayVisible && index >= 0 && index < gallery.length) {
prepareOverlay(gallery, options);
showOverlay(index);
@@ -795,10 +720,12 @@ window.baguetteBox = (function () {
if (index >= imagesElements.length)
return bounceAnimation('right');
try {
vid().pause();
var v = vid();
if (v) {
v.src = '';
v.load();
v.parentNode.removeChild(v);
}
catch (ex) { }
currentIndex = index;
loadImage(currentIndex, function () {
@@ -807,15 +734,6 @@ window.baguetteBox = (function () {
});
updateOffset();
if (options.animation == 'none')
unvid(vid());
else
setTimeout(function () {
unvid(vid());
}, 100);
unfig(index);
if (options.onChange)
options.onChange(currentIndex, imagesElements.length);
@@ -1095,7 +1013,6 @@ window.baguetteBox = (function () {
}
function destroyPlugin() {
hideOverlay(undefined, true);
unbindEvents();
clearCachedData();
document.getElementsByTagName('body')[0].removeChild(ebi('bbox-overlay'));

View File

@@ -28,8 +28,6 @@
--row-alt: #282828;
--scroll: #eb0;
--sel-fg: var(--bg-d1);
--sel-bg: var(--fg);
--a: #fc5;
--a-b: #c90;
@@ -332,8 +330,6 @@ html.c {
}
html.cz {
--bgg: var(--bg-u2);
--sel-bg: var(--bg-u5);
--sel-fg: var(--fg);
--srv-3: #fff;
--u2-tab-b1: var(--bg-d3);
}
@@ -347,8 +343,6 @@ html.cy {
--bg-d3: #f77;
--bg-d2: #ff0;
--sel-bg: #f77;
--a: #fff;
--a-hil: #fff;
--a-h-bg: #000;
@@ -500,7 +494,6 @@ html.dz {
text-shadow: none;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
html.dy {
--fg: #000;
@@ -594,8 +587,8 @@ html.dy {
line-height: 1.2em;
}
::selection {
color: var(--sel-fg);
background: var(--sel-bg);
color: var(--bg-d1);
background: var(--fg);
text-shadow: none;
}
html,body,tr,th,td,#files,a {
@@ -610,7 +603,6 @@ html {
color: var(--fg);
background: var(--bgg);
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
text-shadow: 1px 1px 0px var(--bg-max);
}
html, body {
@@ -619,7 +611,6 @@ html, body {
}
pre, code, tt, #doc, #doc>code {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
.ayjump {
position: fixed;
@@ -705,12 +696,12 @@ a:hover {
.s0:after,
.s1:after {
content: '⌄';
margin-left: -.15em;
margin-left: -.1em;
}
.s0r:after,
.s1r:after {
content: '⌃';
margin-left: -.15em;
margin-left: -.1em;
}
.s0:after,
.s0r:after {
@@ -721,7 +712,7 @@ a:hover {
color: var(--sort-2);
}
#files thead th:after {
margin-right: -.5em;
margin-right: -.7em;
}
#files tbody tr:hover td,
#files tbody tr:hover td+td {
@@ -750,15 +741,6 @@ html #files.hhpick thead th {
word-wrap: break-word;
overflow: hidden;
}
#files tr.fade a {
color: #999;
color: rgba(255, 255, 255, 0.4);
font-style: italic;
}
html.y #files tr.fade a {
color: #999;
color: rgba(0, 0, 0, 0.4);
}
#files tr:nth-child(2n) td {
background: var(--row-alt);
}
@@ -777,7 +759,6 @@ html.y #files tr.fade a {
}
#files tbody td:nth-child(3) {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
text-align: right;
padding-right: 1em;
white-space: nowrap;
@@ -837,11 +818,6 @@ html.y #path a:hover {
.logue:empty {
display: none;
}
.logue.raw {
white-space: pre;
font-family: 'scp', 'consolas', monospace;
font-family: var(--font-mono), 'scp', 'consolas', monospace;
}
#doc>iframe,
.logue>iframe {
background: var(--bgg);
@@ -1005,10 +981,6 @@ html.y #path a:hover {
margin: 0 auto;
display: block;
}
#ggrid.nocrop>a img {
max-height: 20em;
max-height: calc(var(--grid-sz)*2);
}
#ggrid>a.dir:before {
content: '📂';
}
@@ -1175,6 +1147,9 @@ html.y #widget.open {
@keyframes spin {
100% {transform: rotate(360deg)}
}
@media (prefers-reduced-motion) {
@keyframes spin { }
}
@keyframes fadein {
0% {opacity: 0}
100% {opacity: 1}
@@ -1268,13 +1243,6 @@ html.y #widget.open {
0% {opacity:0}
100% {opacity:1}
}
#ggrid>a.glow {
animation: gexit .6s ease-out;
}
@keyframes gexit {
0% {box-shadow: 0 0 0 2em var(--a)}
100% {box-shadow: 0 0 0em 0em var(--a)}
}
#wzip a {
font-size: .4em;
margin: -.3em .1em;
@@ -1437,7 +1405,6 @@ input[type="checkbox"]:checked+label {
}
html.dz input {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
.opwide div>span>input+label {
padding: .3em 0 .3em .3em;
@@ -1615,7 +1582,6 @@ html {
padding: .2em .4em;
font-size: 1.2em;
margin: .2em;
display: inline-block;
white-space: pre;
position: relative;
top: -.12em;
@@ -1687,9 +1653,7 @@ html.cz .tgl.btn.on {
color: var(--fg-max);
}
#tree ul a.hl {
color: #fff;
color: var(--btn-1-fg);
background: #000;
background: var(--btn-1-bg);
text-shadow: none;
}
@@ -1724,7 +1688,6 @@ html.y #tree.nowrap .ntree a+a:hover {
}
.ntree a:first-child {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
font-size: 1.2em;
line-height: 0;
}
@@ -1752,7 +1715,6 @@ html.y #tree.nowrap .ntree a+a:hover {
}
#files th span {
position: relative;
white-space: nowrap;
}
#files>thead>tr>th.min,
#files td.min {
@@ -1790,6 +1752,9 @@ html.y #tree.nowrap .ntree a+a:hover {
margin: .7em 0 .7em .5em;
padding-left: .5em;
}
.opwide>div>div>a {
line-height: 2em;
}
.opwide>div>h3 {
color: var(--fg-weak);
margin: 0 .4em;
@@ -1804,7 +1769,6 @@ html.y #tree.nowrap .ntree a+a:hover {
padding: 0;
}
#thumbs,
#au_prescan,
#au_fullpre,
#au_os_seek,
#au_osd_cv,
@@ -1812,8 +1776,7 @@ html.y #tree.nowrap .ntree a+a:hover {
opacity: .3;
}
#griden.on+#thumbs,
#au_preload.on+#au_prescan,
#au_preload.on+#au_prescan+#au_fullpre,
#au_preload.on+#au_fullpre,
#au_os_ctl.on+#au_os_seek,
#au_os_ctl.on+#au_os_seek+#au_osd_cv,
#u2turbo.on+#u2tdate {
@@ -1853,10 +1816,6 @@ html.y #tree.nowrap .ntree a+a:hover {
margin: 0;
padding: 0;
}
#unpost td:nth-child(3),
#unpost td:nth-child(4) {
text-align: right;
}
#rui {
background: #fff;
background: var(--bg);
@@ -1884,7 +1843,6 @@ html.y #tree.nowrap .ntree a+a:hover {
}
#rn_vadv input {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
#rui td+td,
#rui td input[type="text"] {
@@ -1948,7 +1906,6 @@ html.y #doc {
#doc.mdo {
white-space: normal;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
}
#doc.prism * {
line-height: 1.5em;
@@ -2008,7 +1965,6 @@ a.btn,
}
#hkhelp td:first-child {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
html.noscroll,
html.noscroll .sbar {
@@ -2218,7 +2174,6 @@ html.y #bbox-overlay figcaption a {
}
#bbox-halp {
color: var(--fg-max);
background: #fff;
background: var(--bg);
position: absolute;
top: 0;
@@ -2518,7 +2473,6 @@ html.y #bbox-overlay figcaption a {
}
#op_up2k.srch td.prog {
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
font-size: 1em;
width: auto;
}
@@ -2533,7 +2487,6 @@ html.y #bbox-overlay figcaption a {
white-space: nowrap;
display: inline-block;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
#u2etas.o {
width: 20em;
@@ -2603,7 +2556,6 @@ html.y #bbox-overlay figcaption a {
#u2cards span {
color: var(--fg-max);
font-family: 'scp', monospace;
font-family: var(--font-mono), 'scp', monospace;
}
#u2cards > a:nth-child(4) > span {
display: inline-block;
@@ -2769,7 +2721,6 @@ html.b #u2conf a.b:hover {
}
.prog {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
}
#u2tab span.inf,
#u2tab span.ok,
@@ -3178,7 +3129,7 @@ html.d #treepar {
margin-top: 1.7em;
}
}
@supports (display: grid) and (gap: 1em) {
@supports (display: grid) {
#ggrid {
display: grid;
margin: 0em 0.25em;
@@ -3203,24 +3154,3 @@ html.d #treepar {
padding: 0.2em;
}
}
@media (prefers-reduced-motion) {
@keyframes spin { }
@keyframes gexit { }
@keyframes bounce { }
@keyframes bounceFromLeft { }
@keyframes bounceFromRight { }
#ggrid>a:before,
#widget.anim,
#u2tabw,
.dropdesc,
.dropdesc b,
.dropdesc>div>div {
transition: none;
}
}

View File

@@ -6,10 +6,10 @@
<title>{{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8, minimum-scale=0.6">
<meta name="theme-color" content="#{{ tcolor }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/browser.css?_={{ ts }}">
{{ html_head }}
{%- if css %}
<link rel="stylesheet" media="screen" href="{{ css }}_={{ ts }}">
{%- endif %}
@@ -148,8 +148,7 @@
logues = {{ logues|tojson if sb_lg else "[]" }},
ls0 = {{ ls0|tojson }};
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || dtheme;
document.documentElement.className = localStorage.cpp_thm || dtheme;
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/baguettebox.js?_={{ ts }}"></script>
@@ -161,4 +160,3 @@
</body>
</html>

File diff suppressed because it is too large Load Diff

View File

@@ -6,12 +6,12 @@
<title>{{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
{{ html_head }}
<style>
html{font-family:sans-serif}
td{border:1px solid #999;border-width:1px 1px 0 0;padding:0 5px}
a{display:block}
</style>
{{ html_head }}
</head>
<body>
@@ -61,4 +61,3 @@
</body>
</html>

View File

@@ -25,4 +25,3 @@
</body>
</html>

View File

@@ -2,7 +2,6 @@ html, body {
color: #333;
background: #eee;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
line-height: 1.5em;
}
html.y #helpbox a {
@@ -68,7 +67,6 @@ a {
position: relative;
display: inline-block;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
font-weight: bold;
font-size: 1.3em;
line-height: .1em;

View File

@@ -3,13 +3,13 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#{{ tcolor }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/md.css?_={{ ts }}">
{%- if edit %}
<link rel="stylesheet" href="{{ r }}/.cpr/md2.css?_={{ ts }}">
{%- endif %}
{{ html_head }}
</head>
<body>
<div id="mn"></div>
@@ -139,15 +139,16 @@ var md_opt = {
};
(function () {
var l = window.localStorage,
drk = (l && l.light) != 1,
var l = localStorage,
drk = l.light != 1,
btn = document.getElementById("lightswitch"),
f = function (e) {
if (e) { e.preventDefault(); drk = !drk; }
document.documentElement.className = drk? "z":"y";
btn.innerHTML = "go " + (drk ? "light":"dark");
try { l.light = drk? 0:1; } catch (ex) { }
l.light = drk? 0:1;
};
btn.onclick = f;
f();
})();
@@ -160,4 +161,3 @@ try { l.light = drk? 0:1; } catch (ex) { }
<script src="{{ r }}/.cpr/md2.js?_={{ ts }}"></script>
{%- endif %}
</body></html>

View File

@@ -216,11 +216,6 @@ function convert_markdown(md_text, dest_dom) {
md_html = DOMPurify.sanitize(md_html);
}
catch (ex) {
if (IE) {
dest_dom.innerHTML = 'IE cannot into markdown ;_;';
return;
}
if (ext)
md_plug_err(ex, ext[1]);
@@ -512,6 +507,13 @@ dom_navtgl.onclick = function () {
redraw();
};
if (!HTTPS && location.hostname != '127.0.0.1') try {
ebi('edit2').onclick = function (e) {
toast.err(0, "the fancy editor is only available over https");
return ev(e);
}
} catch (ex) { }
if (sread('hidenav') == 1)
dom_navtgl.onclick();

View File

@@ -9,7 +9,7 @@
width: calc(100% - 56em);
}
#mw {
left: max(0em, calc(100% - 55em));
left: calc(100% - 55em);
overflow-y: auto;
position: fixed;
bottom: 0;
@@ -56,7 +56,6 @@
padding: 0;
margin: 0;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
white-space: pre-wrap;
word-break: break-word;
overflow-wrap: break-word;

View File

@@ -163,7 +163,7 @@ redraw = (function () {
dom_sbs.onclick = setsbs;
dom_nsbs.onclick = modetoggle;
(IE ? modetoggle : onresize)();
onresize();
return onresize;
})();
@@ -368,14 +368,14 @@ function save(e) {
function save_cb() {
if (this.status !== 200)
return toast.err(0, 'Error! The file was NOT saved.\n\nError ' + this.status + ":\n" + unpre(this.responseText));
return toast.err(0, 'Error! The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
var r;
try {
r = JSON.parse(this.responseText);
}
catch (ex) {
return toast.err(0, 'Error! The file was likely NOT saved.\n\nFailed to parse reply from server:\n\n' + unpre(this.responseText));
return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
}
if (!r.ok) {
@@ -418,7 +418,7 @@ function run_savechk(lastmod, txt, btn, ntry) {
function savechk_cb() {
if (this.status !== 200)
return toast.err(0, 'Error! The file was NOT saved.\n\nError ' + this.status + ":\n" + unpre(this.responseText));
return toast.err(0, 'Error! The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
var doc1 = this.txt.replace(/\r\n/g, "\n");
var doc2 = this.responseText.replace(/\r\n/g, "\n");
@@ -933,7 +933,7 @@ var set_lno = (function () {
var keydown = function (ev) {
if (!ev && window.event) {
ev = window.event;
if (dev_fbw == 1) {
if (localStorage.dev_fbw == 1) {
toast.warn(10, 'hello from fallback code ;_;\ncheck console trace');
console.error('using window.event');
}
@@ -1009,7 +1009,7 @@ var set_lno = (function () {
md_home(ev.shiftKey);
return false;
}
if (!ev.shiftKey && ((ev.code + '').endsWith("Enter") || kc == 13)) {
if (!ev.shiftKey && (ev.code.endsWith("Enter") || kc == 13)) {
return md_newline();
}
if (!ev.shiftKey && kc == 8) {

View File

@@ -17,7 +17,6 @@ html, body {
padding: 0;
min-height: 100%;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
background: #f7f7f7;
color: #333;
}

View File

@@ -3,12 +3,12 @@
<title>📝 {{ title }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.7">
<meta name="theme-color" content="#{{ tcolor }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/mde.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/deps/mini-fa.css?_={{ ts }}">
<link rel="stylesheet" href="{{ r }}/.cpr/deps/easymde.css?_={{ ts }}">
{{ html_head }}
</head>
<body>
<div id="mw">
@@ -37,12 +37,12 @@ var md_opt = {
};
var lightswitch = (function () {
var l = window.localStorage,
drk = (l && l.light) != 1,
var l = localStorage,
drk = l.light != 1,
f = function (e) {
if (e) drk = !drk;
document.documentElement.className = drk? "z":"y";
try { l.light = drk? 0:1; } catch (ex) { }
l.light = drk? 0:1;
};
f();
return f;
@@ -54,4 +54,3 @@ try { l.light = drk? 0:1; } catch (ex) { }
<script src="{{ r }}/.cpr/deps/easymde.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/mde.js?_={{ ts }}"></script>
</body></html>

View File

@@ -134,14 +134,14 @@ function save(mde) {
function save_cb() {
if (this.status !== 200)
return toast.err(0, 'Error! The file was NOT saved.\n\nError ' + this.status + ":\n" + unpre(this.responseText));
return toast.err(0, 'Error! The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
var r;
try {
r = JSON.parse(this.responseText);
}
catch (ex) {
return toast.err(0, 'Error! The file was likely NOT saved.\n\nFailed to parse reply from server:\n\n' + unpre(this.responseText));
return toast.err(0, 'Failed to parse reply from server:\n\n' + this.responseText);
}
if (!r.ok) {
@@ -180,7 +180,7 @@ function save_cb() {
function save_chk() {
if (this.status !== 200)
return toast.err(0, 'Error! The file was NOT saved.\n\nError ' + this.status + ":\n" + unpre(this.responseText));
return toast.err(0, 'Error! The file was NOT saved.\n\n' + this.status + ": " + (this.responseText + '').replace(/^<pre>/, ""));
var doc1 = this.txt.replace(/\r\n/g, "\n");
var doc2 = this.responseText.replace(/\r\n/g, "\n");

View File

@@ -1,8 +1,3 @@
:root {
--font-main: sans-serif;
--font-serif: serif;
--font-mono: 'scp';
}
html,body,tr,th,td,#files,a {
color: inherit;
background: none;
@@ -15,7 +10,6 @@ html {
color: #ccc;
background: #333;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
text-shadow: 1px 1px 0px #000;
touch-action: manipulation;
}
@@ -29,7 +23,6 @@ html, body {
}
pre {
font-family: monospace, monospace;
font-family: var(--font-mono), monospace, monospace;
}
a {
color: #fc5;

View File

@@ -6,9 +6,9 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#{{ tcolor }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/msg.css?_={{ ts }}">
</head>
<body>
@@ -48,5 +48,4 @@
{%- endif %}
</body>
</html>
</html>

View File

@@ -2,7 +2,6 @@ html {
color: #333;
background: #f7f7f7;
font-family: sans-serif;
font-family: var(--font-main), sans-serif;
touch-action: manipulation;
}
#wrap {
@@ -128,7 +127,6 @@ pre, code {
color: #480;
background: #fff;
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
border: 1px solid rgba(128,128,128,0.3);
border-radius: .2em;
padding: .15em .2em;
@@ -190,21 +188,11 @@ input {
padding: .5em .7em;
margin: 0 .5em 0 0;
}
input::placeholder {
font-size: 1.2em;
font-style: italic;
letter-spacing: .04em;
opacity: 0.64;
color: #930;
}
html.z input {
color: #fff;
background: #626;
border-color: #c2c;
}
html.z input::placeholder {
color: #fff;
}
html.z .num {
border-color: #777;
}

View File

@@ -6,10 +6,10 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#{{ tcolor }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
{{ html_head }}
</head>
<body>
@@ -78,15 +78,13 @@
<h1 id="cc">client config:</h1>
<ul>
{% if k304 or k304vis %}
{% if k304 %}
<li><a id="h" href="{{ r }}/?k304=n">disable k304</a> (currently enabled)
{%- else %}
<li><a id="i" href="{{ r }}/?k304=y" class="r">enable k304</a> (currently disabled)
{% endif %}
<blockquote id="j">enabling this will disconnect your client on every HTTP 304, which can prevent some buggy proxies from getting stuck (suddenly not loading pages), <em>but</em> it will also make things slower in general</blockquote></li>
{% endif %}
<li><a id="k" href="{{ r }}/?reset" class="r" onclick="localStorage.clear();return true">reset client settings</a></li>
</ul>
@@ -94,8 +92,7 @@
<div>
<form method="post" enctype="multipart/form-data" action="{{ r }}/{{ qvpath }}">
<input type="hidden" name="act" value="login" />
<input type="password" name="cppwd" placeholder=" password" />
<input type="hidden" name="uhash" id="uhash" value="x" />
<input type="password" name="cppwd" />
<input type="submit" value="Login" />
{% if ahttps %}
<a id="w" href="{{ ahttps }}">switch to https</a>
@@ -113,12 +110,10 @@ var SR = {{ r|tojson }},
lang="{{ lang }}",
dfavico="{{ favico }}";
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || "{{ this.args.theme }}";
document.documentElement.className=localStorage.cpp_thm||"{{ this.args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/splash.js?_={{ ts }}"></script>
</body>
</html>

View File

@@ -6,7 +6,7 @@ var Ls = {
"d1": "tilstand",
"d2": "vis tilstanden til alle tråder",
"e1": "last innst.",
"e2": "leser inn konfigurasjonsfiler på nytt$N(kontoer, volumer, volumbrytere)$Nog kartlegger alle e2ds-volumer$N$Nmerk: endringer i globale parametere$Nkrever en full restart for å ta gjenge",
"e2": "leser inn konfigurasjonsfiler på nytt$N(kontoer, volumer, volumbrytere)$Nog kartlegger alle e2ds-volumer",
"f1": "du kan betrakte:",
"g1": "du kan laste opp til:",
"cc1": "klient-konfigurasjon",
@@ -30,18 +30,12 @@ var Ls = {
},
"eng": {
"d2": "shows the state of all active threads",
"e2": "reload config files (accounts/volumes/volflags),$Nand rescan all e2ds volumes$N$Nnote: any changes to global settings$Nrequire a full restart to take effect",
"e2": "reload config files (accounts/volumes/volflags),$Nand rescan all e2ds volumes",
"u2": "time since the last server write$N( upload / rename / ... )$N$N17d = 17 days$N1h23 = 1 hour 23 minutes$N4m56 = 4 minutes 56 seconds",
"v2": "use this server as a local HDD$N$NWARNING: this will show your password!",
}
};
var LANGS = ["eng", "nor"];
if (window.langmod)
langmod();
var d = Ls[sread("cpp_lang", LANGS) || lang] || Ls.eng || Ls.nor;
},
d = Ls[sread("cpp_lang", ["eng", "nor"]) || lang] || Ls.eng || Ls.nor;
for (var k in (d || {})) {
var f = k.slice(-1),
@@ -55,15 +49,6 @@ for (var k in (d || {})) {
o[a].setAttribute("tt", d[k]);
}
try {
if (is_idp) {
var z = ['#l+div', '#l', '#c'];
for (var a = 0; a < z.length; a++)
QS(z[a]).style.display = 'none';
}
}
catch (ex) { }
tt.init();
var o = QS('input[name="cppwd"]');
if (!ebi('c') && o.offsetTop + o.offsetHeight < window.innerHeight)
@@ -72,5 +57,3 @@ if (!ebi('c') && o.offsetTop + o.offsetHeight < window.innerHeight)
o = ebi('u');
if (o && /[0-9]+$/.exec(o.innerHTML))
o.innerHTML = shumantime(o.innerHTML);
ebi('uhash').value = '' + location.hash;

View File

@@ -6,11 +6,11 @@
<title>{{ s_doctitle }}</title>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=0.8">
<meta name="theme-color" content="#{{ tcolor }}">
<meta name="theme-color" content="#333">
{{ html_head }}
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/splash.css?_={{ ts }}">
<link rel="stylesheet" media="screen" href="{{ r }}/.cpr/ui.css?_={{ ts }}">
<style>ul{padding-left:1.3em}li{margin:.4em 0}</style>
{{ html_head }}
</head>
<body>
@@ -238,12 +238,10 @@ var SR = {{ r|tojson }},
lang="{{ lang }}",
dfavico="{{ favico }}";
var STG = window.localStorage;
document.documentElement.className = (STG && STG.cpp_thm) || "{{ args.theme }}";
document.documentElement.className=localStorage.cpp_thm||"{{ args.theme }}";
</script>
<script src="{{ r }}/.cpr/util.js?_={{ ts }}"></script>
<script src="{{ r }}/.cpr/svcs.js?_={{ ts }}"></script>
</body>
</html>

View File

@@ -1,8 +1,4 @@
:root {
--font-main: sans-serif;
--font-serif: serif;
--font-mono: 'scp';
--fg: #ccc;
--fg-max: #fff;
--bg-u2: #2b2b2b;
@@ -109,9 +105,6 @@ html {
#toast pre {
margin: 0;
}
#toast.hide {
display: none;
}
#toast.vis {
right: 1.3em;
transform: inherit;
@@ -151,10 +144,6 @@ html {
#toast.err #toastc {
background: #d06;
}
#toast code {
padding: 0 .2em;
background: rgba(0,0,0,0.2);
}
#tth {
color: #fff;
background: #111;
@@ -382,7 +371,6 @@ html.y textarea:focus {
.mdo code,
.mdo tt {
font-family: 'scp', monospace, monospace;
font-family: var(--font-mono), 'scp', monospace, monospace;
white-space: pre-wrap;
word-break: break-all;
}
@@ -452,7 +440,6 @@ html.y textarea:focus {
}
.mdo blockquote {
font-family: serif;
font-family: var(--font-serif), serif;
background: #f7f7f7;
border: .07em dashed #ccc;
padding: 0 2em;
@@ -586,11 +573,3 @@ hr {
border: .07em dashed #444;
}
}
@media (prefers-reduced-motion) {
#toast,
#toast a#toastc,
#tt {
transition: none;
}
}

View File

@@ -17,7 +17,7 @@ function goto_up2k() {
var up2k = null,
up2k_hooks = [],
hws = [],
sha_js = WebAssembly ? 'hw' : 'ac', // ff53,c57,sa11
sha_js = window.WebAssembly ? 'hw' : 'ac', // ff53,c57,sa11
m = 'will use ' + sha_js + ' instead of native sha512 due to';
try {
@@ -431,7 +431,7 @@ function U2pvis(act, btns, uc, st) {
if (sread('potato') === null) {
btn.click();
toast.inf(30, L.u_gotpot);
sdrop('potato');
localStorage.removeItem('potato');
}
u2f.appendChild(ode);
@@ -717,7 +717,7 @@ function Donut(uc, st) {
sfx();
// firefox may forget that filedrops are user-gestures so it can skip this:
if (uc.upnag && Notification && Notification.permission == 'granted')
if (uc.upnag && window.Notification && Notification.permission == 'granted')
new Notification(uc.nagtxt);
}
@@ -779,8 +779,8 @@ function up2k_init(subtle) {
};
setTimeout(function () {
if (WebAssembly && !hws.length)
fetch(SR + '/.cpr/w.hash.js?_=' + TS);
if (window.WebAssembly && !hws.length)
fetch(SR + '/.cpr/w.hash.js' + CB);
}, 1000);
function showmodal(msg) {
@@ -861,15 +861,15 @@ function up2k_init(subtle) {
bcfg_bind(uc, 'multitask', 'multitask', true, null, false);
bcfg_bind(uc, 'potato', 'potato', false, set_potato, false);
bcfg_bind(uc, 'ask_up', 'ask_up', true, null, false);
bcfg_bind(uc, 'umod', 'umod', false, null, false);
bcfg_bind(uc, 'u2ts', 'u2ts', !u2ts.endsWith('u'), set_u2ts, false);
bcfg_bind(uc, 'fsearch', 'fsearch', false, set_fsearch, false);
bcfg_bind(uc, 'flag_en', 'flag_en', false, apply_flag_cfg);
bcfg_bind(uc, 'turbo', 'u2turbo', turbolvl > 1, draw_turbo);
bcfg_bind(uc, 'datechk', 'u2tdate', turbolvl < 3, null);
bcfg_bind(uc, 'appl', 'u2appl', true, null);
bcfg_bind(uc, 'az', 'u2sort', u2sort.indexOf('n') + 1, set_u2sort);
bcfg_bind(uc, 'hashw', 'hashw', !!WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
bcfg_bind(uc, 'hashw', 'hashw', !!window.WebAssembly && (!subtle || !CHROME || MOBILE || VCHROME >= 107), set_hashw);
bcfg_bind(uc, 'upnag', 'upnag', false, set_upnag);
bcfg_bind(uc, 'upsfx', 'upsfx', false, set_upsfx);
@@ -895,7 +895,6 @@ function up2k_init(subtle) {
"bytes": {
"total": 0,
"hashed": 0,
"inflight": 0,
"uploaded": 0,
"finished": 0
},
@@ -1312,6 +1311,45 @@ function up2k_init(subtle) {
});
}
applefilter(good_files);
}
function applefilter(good_files) {
if (!uc.appl)
return gotallfiles2(good_files);
var junk = [], rmi = [];
for (var a = 0; a < good_files.length; a++) {
if (/\/(__MACOS|Icon\r\r)|\/\.(_|DS_Store|AppleDouble|LSOverride|DocumentRevisions-|fseventsd|Spotlight-|TemporaryItems|Trashes|VolumeIcon\.icns|com\.apple\.timemachine\.donotpresent|AppleDB|AppleDesktop|apdisk)/.exec(good_files[a][1])) {
junk.push(good_files[a]);
rmi.push(a);
}
}
if (!junk.length)
return gotallfiles2(good_files);
// todo: if there's hits so far, start looking for appledoubles
// (files named ._foo where foo is another file in the list)
// and get rid of those too
var msg = L.u_applef.format(junk.length, good_files.length);
for (var a = 0, aa = Math.min(20, junk.length); a < aa; a++)
msg += '-- ' + junk[a][1] + '\n';
return modal.confirm(msg, function () {
for (var a = rmi.length - 1; a >= 0; a--)
good_files.splice(rmi[a], 1);
start_actx();
gotallfiles2(good_files);
}, function () {
start_actx();
gotallfiles2(good_files);
});
}
function gotallfiles2(good_files) {
good_files.sort(function (a, b) {
a = a[1];
b = b[1];
@@ -1334,8 +1372,7 @@ function up2k_init(subtle) {
return modal.confirm(msg.join('') + '</ul>', function () {
start_actx();
up_them(good_files);
if (have_up2k_idx)
toast.inf(15, L.u_unpt, L.u_unpt);
toast.inf(15, L.u_unpt, L.u_unpt);
}, null);
up_them(good_files);
@@ -1347,9 +1384,9 @@ function up2k_init(subtle) {
var evpath = get_evpath(),
draw_each = good_files.length < 50;
if (WebAssembly && !hws.length) {
if (window.WebAssembly && !hws.length) {
for (var a = 0; a < Math.min(navigator.hardwareConcurrency || 4, 16); a++)
hws.push(new Worker(SR + '/.cpr/w.hash.js?_=' + TS));
hws.push(new Worker(SR + '/.cpr/w.hash.js' + CB));
console.log(hws.length + " hashers");
}
@@ -1394,8 +1431,6 @@ function up2k_init(subtle) {
entry.rand = true;
entry.name = 'a\n' + entry.name;
}
else if (uc.umod)
entry.umod = true;
if (biggest_file < entry.size)
biggest_file = entry.size;
@@ -1544,21 +1579,17 @@ function up2k_init(subtle) {
if (uc.fsearch)
t.push(['u2etat', st.bytes.hashed, st.bytes.hashed, st.time.hashing]);
}
var b_up = st.bytes.inflight + st.bytes.uploaded,
b_fin = st.bytes.inflight + st.bytes.finished;
if (nsend) {
st.time.uploading += td;
t.push(['u2etau', b_up, b_fin, st.time.uploading]);
t.push(['u2etau', st.bytes.uploaded, st.bytes.finished, st.time.uploading]);
}
if ((nhash || nsend) && !uc.fsearch) {
if (!b_fin) {
if (!st.bytes.finished) {
ebi('u2etat').innerHTML = L.u_etaprep;
}
else {
st.time.busy += td;
t.push(['u2etat', b_fin, b_fin, st.time.busy]);
t.push(['u2etat', st.bytes.finished, st.bytes.finished, st.time.busy]);
}
}
for (var a = 0; a < t.length; a++) {
@@ -1722,6 +1753,8 @@ function up2k_init(subtle) {
ebi('u2etas').style.textAlign = 'left';
}
etafun();
if (pvis.act == 'bz')
pvis.changecard('bz');
}
if (flag) {
@@ -1857,9 +1890,6 @@ function up2k_init(subtle) {
timer.rm(donut.do);
ebi('u2tabw').style.minHeight = '0px';
utw_minh = 0;
if (pvis.act == 'bz')
pvis.changecard('bz');
}
function chill(t) {
@@ -2257,7 +2287,6 @@ function up2k_init(subtle) {
console.log('handshake onerror, retrying', t.name, t);
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
t.keepalive = keepalive;
};
var orz = function (e) {
@@ -2265,26 +2294,16 @@ function up2k_init(subtle) {
return console.log('zombie handshake onload', t.name, t);
if (xhr.status == 200) {
try {
var response = JSON.parse(xhr.responseText);
}
catch (ex) {
apop(st.busy.handshake, t);
st.todo.handshake.unshift(t);
t.cooldown = Date.now() + 5000 + Math.floor(Math.random() * 3000);
return toast.err(0, 'Handshake error; will retry...\n\n' + L.badreply + ':\n\n' + unpre(xhr.responseText));
}
t.t_handshake = Date.now();
if (keepalive) {
apop(st.busy.handshake, t);
tasker();
return;
}
if (toast.tag === t)
toast.ok(5, L.u_fixed);
var response = JSON.parse(xhr.responseText);
if (!response.name) {
var msg = '',
smsg = '';
@@ -2488,8 +2507,6 @@ function up2k_init(subtle) {
req.srch = 1;
else if (t.rand)
req.rand = true;
else if (t.umod)
req.umod = true;
xhr.open('POST', t.purl, true);
xhr.responseType = 'text';
@@ -2556,7 +2573,6 @@ function up2k_init(subtle) {
cdr = t.size;
var orz = function (xhr) {
st.bytes.inflight -= xhr.bsent;
var txt = unpre((xhr.response && xhr.response.err) || xhr.responseText);
if (txt.indexOf('upload blocked by x') + 1) {
apop(st.busy.upload, upt);
@@ -2601,10 +2617,7 @@ function up2k_init(subtle) {
btot = Math.floor(st.bytes.total / 1024 / 1024);
xhr.upload.onprogress = function (xev) {
var nb = xev.loaded;
st.bytes.inflight += nb - xhr.bsent;
xhr.bsent = nb;
pvis.prog(t, npart, nb);
pvis.prog(t, npart, xev.loaded);
};
xhr.onload = function (xev) {
try { orz(xhr); } catch (ex) { vis_exh(ex + '', 'up2k.js', '', '', ex); }
@@ -2613,8 +2626,6 @@ function up2k_init(subtle) {
if (crashed)
return;
st.bytes.inflight -= (xhr.bsent || 0);
if (!toast.visible)
toast.warn(9.98, L.u_cuerr.format(npart, Math.ceil(t.size / chunksize), t.name), t);
@@ -2631,7 +2642,6 @@ function up2k_init(subtle) {
if (xhr.overrideMimeType)
xhr.overrideMimeType('Content-Type', 'application/octet-stream');
xhr.bsent = 0;
xhr.responseType = 'text';
xhr.send(t.fobj.slice(car, cdr));
}
@@ -2716,7 +2726,7 @@ function up2k_init(subtle) {
parallel_uploads = v;
if (v == u2j)
sdrop('nthread');
localStorage.removeItem('nthread');
else
swrite('nthread', v);
@@ -2732,9 +2742,6 @@ function up2k_init(subtle) {
if (parallel_uploads > 16)
parallel_uploads = 16;
if (parallel_uploads > 7)
toast.warn(10, L.u_maxconn);
obj.value = parallel_uploads;
bumpthread({ "target": 1 });
}
@@ -2868,8 +2875,6 @@ function up2k_init(subtle) {
new_state = false;
fixed = true;
}
if (new_state === undefined)
new_state = can_write ? false : have_up2k_idx ? true : undefined;
}
if (new_state === undefined)
@@ -2950,7 +2955,7 @@ function up2k_init(subtle) {
}
function set_hashw() {
if (!WebAssembly) {
if (!window.WebAssembly) {
bcfg_set('hashw', uc.hashw = false);
toast.err(10, L.u_nowork);
}
@@ -2967,7 +2972,7 @@ function up2k_init(subtle) {
nopenag();
}
if (!Notification || !HTTPS)
if (!window.Notification || !HTTPS)
return nopenag();
if (en && Notification.permission == 'default')
@@ -2989,7 +2994,7 @@ function up2k_init(subtle) {
};
}
if (uc.upnag && (!Notification || Notification.permission != 'granted'))
if (uc.upnag && (!window.Notification || Notification.permission != 'granted'))
bcfg_set('upnag', uc.upnag = false);
ebi('nthread_add').onclick = function (e) {

View File

@@ -12,10 +12,9 @@ if (window.CGV)
var wah = '',
STG = null,
NOAC = 'autocorrect="off" autocapitalize="off"',
L, tt, treectl, thegrid, up2k, asmCrypto, hashwasm, vbar, marked,
T0 = Date.now(),
CB = '?_=' + Date.now(),
R = SR.slice(1),
RS = R ? "/" + R : "",
HALFMAX = 8192 * 8192 * 8192 * 8192,
@@ -41,16 +40,8 @@ if (!window.FormData)
window.FormData = false;
try {
STG = window.localStorage;
STG.STG;
}
catch (ex) {
STG = null;
if ((ex + '').indexOf('sandbox') < 0)
console.log('no localStorage: ' + ex);
}
CB = '?' + document.currentScript.src.split('?').pop();
try {
if (navigator.userAgentData.mobile)
MOBILE = true;
@@ -127,7 +118,7 @@ if ((document.location + '').indexOf(',rej,') + 1)
try {
console.hist = [];
var CMAXHIST = 1000;
var CMAXHIST = 100;
var hook = function (t) {
var orig = console[t].bind(console),
cfun = function () {
@@ -154,10 +145,6 @@ catch (ex) {
}
var crashed = false, ignexd = {}, evalex_fatal = false;
function vis_exh(msg, url, lineNo, columnNo, error) {
var ekey = url + '\n' + lineNo + '\n' + msg;
if (ignexd[ekey] || crashed)
return;
msg = String(msg);
url = String(url);
@@ -173,13 +160,11 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
if (url.indexOf(' > eval') + 1 && !evalex_fatal)
return; // md timer
if (IE && url.indexOf('prism.js') + 1)
var ekey = url + '\n' + lineNo + '\n' + msg;
if (ignexd[ekey] || crashed)
return;
if (url.indexOf('easymde.js') + 1)
return; // clicking the preview pane
if (url.indexOf('deps/marked.js') + 1 && !WebAssembly)
if (url.indexOf('deps/marked.js') + 1 && !window.WebAssembly)
return; // ff<52
crashed = true;
@@ -217,24 +202,19 @@ function vis_exh(msg, url, lineNo, columnNo, error) {
}
ignexd[ekey] = true;
var ls = {},
lsk = Object.keys(localStorage),
nka = lsk.length,
nk = Math.min(200, nka);
var ls = jcp(localStorage);
if (ls.fman_clip)
ls.fman_clip = ls.fman_clip.length + ' items';
for (var a = 0; a < nk; a++) {
var k = lsk[a],
v = localStorage.getItem(k);
ls[k] = v.length > 256 ? v.slice(0, 32) + '[...' + v.length + 'b]' : v;
}
lsk = Object.keys(ls);
var lsk = Object.keys(ls);
lsk.sort();
html.push('<p class="b"><b>' + nka + ':&nbsp;</b>');
for (var a = 0; a < nk; a++)
html.push(' <b>' + esc(lsk[a]) + '</b> <code>' + esc(ls[lsk[a]]) + '</code> ');
html.push('<p class="b">');
for (var a = 0; a < lsk.length; a++) {
if (ls[lsk[a]].length > 9000)
continue;
html.push(' <b>' + esc(lsk[a]) + '</b> <code>' + esc(ls[lsk[a]]) + '</code> ');
}
html.push('</p>');
}
catch (e) { }
@@ -296,11 +276,10 @@ function anymod(e, shift_ok) {
}
var dev_fbw = sread('dev_fbw');
function ev(e) {
if (!e && window.event) {
e = window.event;
if (dev_fbw == 1) {
if (localStorage.dev_fbw == 1) {
toast.warn(10, 'hello from fallback code ;_;\ncheck console trace');
console.error('using window.event');
}
@@ -391,22 +370,6 @@ catch (ex) {
}
}
if (!window.Set)
window.Set = function () {
var r = this;
r.size = 0;
r.d = {};
r.add = function (k) {
if (!r.d[k]) {
r.d[k] = 1;
r.size++;
}
};
r.has = function (k) {
return r.d[k];
};
};
// https://stackoverflow.com/a/950146
function import_js(url, cb, ecb) {
var head = document.head || document.getElementsByTagName('head')[0];
@@ -432,25 +395,6 @@ function unsmart(txt) {
}
function namesan(txt, win, fslash) {
if (win)
txt = (txt.
replace(/</g, "").
replace(/>/g, "").
replace(/:/g, "").
replace(/"/g, "").
replace(/\\/g, "").
replace(/\|/g, "").
replace(/\?/g, "").
replace(/\*/g, ""));
if (fslash)
txt = txt.replace(/\//g, "");
return txt;
}
var crctab = (function () {
var c, tab = [];
for (var n = 0; n < 256; n++) {
@@ -737,15 +681,6 @@ function vjoin(p1, p2) {
}
function addq(url, q) {
var uh = url.split('#', 1),
u = uh[0],
h = uh.length == 1 ? '' : '#' + uh[1];
return u + (u.indexOf('?') < 0 ? '?' : '&') + (q === undefined ? '' : q) + h;
}
function uricom_enc(txt, do_fb_enc) {
try {
return encodeURIComponent(txt);
@@ -946,16 +881,9 @@ function jcp(obj) {
}
function sdrop(key) {
try {
STG.removeItem(key);
}
catch (ex) { }
}
function sread(key, al) {
try {
var ret = STG.getItem(key);
var ret = localStorage.getItem(key);
return (!al || has(al, ret)) ? ret : null;
}
catch (e) {
@@ -966,9 +894,9 @@ function sread(key, al) {
function swrite(key, val) {
try {
if (val === undefined || val === null)
STG.removeItem(key);
localStorage.removeItem(key);
else
STG.setItem(key, val);
localStorage.setItem(key, val);
}
catch (e) { }
}
@@ -1129,7 +1057,7 @@ function dl_file(url) {
function cliptxt(txt, ok) {
var fb = function () {
console.log('clip-fb');
console.log('fb');
var o = mknod('input');
o.value = txt;
document.body.appendChild(o);
@@ -1423,11 +1351,8 @@ function lf2br(txt) {
}
function hunpre(txt) {
return ('' + txt).replace(/^<pre>/, '');
}
function unpre(txt) {
return esc(hunpre(txt));
return ('' + txt).replace(/^<pre>/, '');
}
@@ -1467,23 +1392,15 @@ var toast = (function () {
}
r.hide = function (e) {
if (this === ebi('toastc'))
ev(e);
ev(e);
unscroll();
clearTimeout(te);
clmod(obj, 'vis');
r.visible = false;
r.tag = obj;
if (!WebAssembly)
te = setTimeout(function () {
obj.className = 'hide';
}, 500);
};
r.show = function (cl, sec, txt, tag) {
txt = (txt + '').slice(0, 16384);
var same = r.visible && txt == r.p_txt && r.p_sec == sec,
delta = Date.now() - r.p_t;
@@ -1539,8 +1456,6 @@ var modal = (function () {
cb_up = null,
cb_ok = null,
cb_ng = null,
sel_0 = 0,
sel_1 = 0,
tok, tng, prim, sec, ok_cancel;
r.load = function () {
@@ -1574,7 +1489,7 @@ var modal = (function () {
(inp || a).focus();
if (inp)
setTimeout(function () {
inp.setSelectionRange(sel_0, sel_1, "forward");
inp.setSelectionRange(0, inp.value.length, "forward");
}, 0);
document.addEventListener('focus', onfocus);
@@ -1643,7 +1558,7 @@ var modal = (function () {
};
var onkey = function (e) {
var k = (e.code || e.key) + '',
var k = e.code,
eok = ebi('modal-ok'),
eng = ebi('modal-ng'),
ae = document.activeElement;
@@ -1658,10 +1573,10 @@ var modal = (function () {
return ok(e);
}
if ((k == 'ArrowLeft' || k == 'ArrowRight' || k == 'Left' || k == 'Right') && eng && (ae == eok || ae == eng))
if ((k == 'ArrowLeft' || k == 'ArrowRight') && eng && (ae == eok || ae == eng))
return (ae == eok ? eng : eok).focus() || ev(e);
if (k == 'Escape' || k == 'Esc')
if (k == 'Escape')
return ng(e);
}
@@ -1697,18 +1612,16 @@ var modal = (function () {
r.show(html);
}
r.prompt = function (html, v, cok, cng, fun, so0, so1) {
r.prompt = function (html, v, cok, cng, fun) {
q.push(function () {
_prompt(lf2br(html), v, cok, cng, fun, so0, so1);
_prompt(lf2br(html), v, cok, cng, fun);
});
next();
}
var _prompt = function (html, v, cok, cng, fun, so0, so1) {
var _prompt = function (html, v, cok, cng, fun) {
cb_ok = cok;
cb_ng = cng === undefined ? cok : null;
cb_up = fun;
sel_0 = so0 || 0;
sel_1 = so1 === undefined ? v.length : so1;
html += '<input id="modali" type="text" ' + NOAC + ' /><div id="modalb">' + ok_cancel + '</div>';
r.show(html);
@@ -1895,7 +1808,7 @@ function md_thumbs(md) {
float = has(flags, 'l') ? 'left' : has(flags, 'r') ? 'right' : '';
if (!/[?&]cache/.exec(url))
url = addq(url, 'cache=i');
url += (url.indexOf('?') < 0 ? '?' : '&') + 'cache=i';
md[a] = '<a href="' + url + '" class="mdth mdth' + float.slice(0, 1) + '"><img src="' + url + '&th=w" alt="' + alt + '" /></a>' + md[a].slice(o2 + 1);
}
@@ -1941,17 +1854,21 @@ var favico = (function () {
var b64;
try {
b64 = btoa(svg ? svg_decl + svg : gx(r.txt));
//console.log('f1');
}
catch (e1) {
try {
b64 = btoa(gx(encodeURIComponent(r.txt).replace(/%([0-9A-F]{2})/g,
function x(m, v) { return String.fromCharCode('0x' + v); })));
//console.log('f2');
}
catch (e2) {
try {
b64 = btoa(gx(unescape(encodeURIComponent(r.txt))));
//console.log('f3');
}
catch (e3) {
//console.log('fe');
return;
}
}
@@ -2005,24 +1922,15 @@ function xhrchk(xhr, prefix, e404, lvl, tag) {
if (xhr.status < 400 && xhr.status >= 200)
return true;
if (tag === undefined)
tag = prefix;
var errtxt = ((xhr.response && xhr.response.err) || xhr.responseText) || '',
suf = '',
var errtxt = (xhr.response && xhr.response.err) || xhr.responseText,
fun = toast[lvl || 'err'],
is_cf = /[Cc]loud[f]lare|>Just a mo[m]ent|#cf-b[u]bbles|Chec[k]ing your br[o]wser|\/chall[e]nge-platform|"chall[e]nge-error|nable Ja[v]aScript and cook/.test(errtxt);
if (errtxt.startsWith('<pre>'))
suf = '\n\nerror-details: «' + unpre(errtxt).split('\n')[0].trim() + '»';
else
errtxt = esc(errtxt).slice(0, 32768);
if (xhr.status == 403 && !is_cf)
return toast.err(0, prefix + (L && L.xhr403 || "403: access denied\n\ntry pressing F5, maybe you got logged out") + suf, tag);
return toast.err(0, prefix + (L && L.xhr403 || "403: access denied\n\ntry pressing F5, maybe you got logged out"), tag);
if (xhr.status == 404)
return toast.err(0, prefix + e404 + suf, tag);
return toast.err(0, prefix + e404, tag);
if (is_cf && (xhr.status == 403 || xhr.status == 503)) {
var now = Date.now(), td = now - cf_cha_t;

View File

@@ -13,9 +13,6 @@
# other stuff
## [`TODO.md`](TODO.md)
* planned features / fixes / changes
## [`example.conf`](example.conf)
* example config file for `-c`

View File

@@ -1,15 +0,0 @@
a living list of upcoming features / fixes / changes, very roughly in order of priority
* download accelerator
* definitely download chunks in parallel
* maybe resumable downloads (chrome-only, jank api)
* maybe checksum validation (return sha512 of requested range in responses, and probably also warks)
* [github issue #37](https://github.com/9001/copyparty/issues/37) - upload PWA
* or [maybe not](https://arstechnica.com/tech-policy/2024/02/apple-under-fire-for-disabling-iphone-web-apps-eu-asks-developers-to-weigh-in/), or [maybe](https://arstechnica.com/gadgets/2024/03/apple-changes-course-will-keep-iphone-eu-web-apps-how-they-are-in-ios-17-4/)
* [github issue #57](https://github.com/9001/copyparty/issues/57) - config GUI
* configs given to -c can be ordered with numerical prefix
* autorevert settings if it fails to apply
* countdown until session invalidates in settings gui, with refresh-button

View File

@@ -1,96 +0,0 @@
notes from testing various buffer sizes of files and sockets
summary:
download-folder-as-tar: would be 7% faster with --iobuf 65536 (but got 20% faster in v1.11.2)
download-folder-as-zip: optimal with default --iobuf 262144
download-file-over-https: optimal with default --iobuf 262144
put-large-file: optimal with default --iobuf 262144, --s-rd-sz 262144 (and got 14% faster in v1.11.2)
post-large-file: optimal with default --iobuf 262144, --s-rd-sz 262144 (and got 18% faster in v1.11.2)
----
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/?tar
3.3 req/s 1.11.1
4.3 4.0 3.3 req/s 1.12.2
64 256 512 --iobuf 256 (prefer smaller)
32 32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/?zip
2.9 req/s 1.11.1
2.5 2.9 2.9 req/s 1.12.2
64 256 512 --iobuf 256 (prefer bigger)
32 32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/?tar
8.3 req/s 1.11.1
8.4 8.4 8.5 req/s 1.12.2
64 256 512 --iobuf 256 (prefer bigger)
32 32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/?zip
13.9 req/s 1.11.1
14.1 14.0 13.8 req/s 1.12.2
64 256 512 --iobuf 256 (prefer smaller)
32 32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/pairdupes/987a
5260 req/s 1.11.1
5246 5246 5280 5268 req/s 1.12.2
64 256 512 256 --iobuf dontcare
32 32 32 512 --s-rd-sz dontcare
oha -z10s -c1 --ipv4 --insecure https://127.0.0.1:3923/pairdupes/987a
4445 req/s 1.11.1
4462 4494 4444 req/s 1.12.2
64 256 512 --iobuf dontcare
32 32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure http://127.0.0.1:3923/bigs/gssc-02-cannonball-skydrift/track10.cdda.flac
95 req/s 1.11.1
95 97 req/s 1.12.2
64 512 --iobuf dontcare
32 32 --s-rd-sz
oha -z10s -c1 --ipv4 --insecure https://127.0.0.1:3923/bigs/gssc-02-cannonball-skydrift/track10.cdda.flac
15.4 req/s 1.11.1
15.4 15.3 14.9 15.4 req/s 1.12.2
64 256 512 512 --iobuf 256 (prefer smaller, and smaller than s-wr-sz)
32 32 32 32 --s-rd-sz
256 256 256 512 --s-wr-sz
----
python3 ~/dev/old/copyparty\ v1.11.1\ dont\ ban\ the\ pipes.py -q -i 127.0.0.1 -v .::A --daw
python3 ~/dev/copyparty/dist/copyparty-sfx.py -q -i 127.0.0.1 -v .::A --daw --iobuf $((1024*512))
oha -z10s -c1 --ipv4 --insecure -mPUT -r0 -D ~/Music/gssc-02-cannonball-skydrift/track10.cdda.flac http://127.0.0.1:3923/a.bin
10.8 req/s 1.11.1
10.8 11.5 11.8 12.1 12.2 12.3 req/s new
512 512 512 512 512 256 --iobuf 256
32 64 128 256 512 256 --s-rd-sz 256 (prefer bigger)
----
buildpost() {
b=--jeg-er-grensestaven;
printf -- "$b\r\nContent-Disposition: form-data; name=\"act\"\r\n\r\nbput\r\n$b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"a.bin\"\r\nContent-Type: audio/mpeg\r\n\r\n"
cat "$1"
printf -- "\r\n${b}--\r\n"
}
buildpost ~/Music/gssc-02-cannonball-skydrift/track10.cdda.flac >big.post
buildpost ~/Music/bottomtext.txt >smol.post
oha -z10s -c1 --ipv4 --insecure -mPOST -r0 -T 'multipart/form-data; boundary=jeg-er-grensestaven' -D big.post http://127.0.0.1:3923/?replace
9.6 11.2 11.3 11.1 10.9 req/s v1.11.2
512 512 256 128 256 --iobuf 256
32 512 256 128 128 --s-rd-sz 256
oha -z10s -c1 --ipv4 --insecure -mPOST -r0 -T 'multipart/form-data; boundary=jeg-er-grensestaven' -D smol.post http://127.0.0.1:3923/?replace
2445 2414 2401 2437
256 128 256 256 --iobuf 256
128 128 256 64 --s-rd-sz 128 (but use 256 since big posts are more important)

File diff suppressed because it is too large Load Diff

View File

@@ -20,8 +20,7 @@
* [just the sfx](#just-the-sfx)
* [build from release tarball](#build-from-release-tarball) - uses the included prebuilt webdeps
* [complete release](#complete-release)
* [debugging](#debugging)
* [music playback halting on phones](#music-playback-halting-on-phones) - mostly fine on android
* [todo](#todo) - roughly sorted by priority
* [discarded ideas](#discarded-ideas)
@@ -134,9 +133,6 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| GET | `?zip=utf-8` | ...as a zip file |
| GET | `?zip` | ...as a WinXP-compatible zip file |
| GET | `?zip=crc` | ...as an MSDOS-compatible zip file |
| GET | `?tar&w` | pregenerate webp thumbnails |
| GET | `?tar&j` | pregenerate jpg thumbnails |
| GET | `?tar&p` | pregenerate audio waveforms |
| GET | `?ups` | show recent uploads from your IP |
| GET | `?ups&filter=f` | ...where URL contains `f` |
| GET | `?mime=foo` | specify return mimetype `foo` |
@@ -168,7 +164,6 @@ authenticate using header `Cookie: cppwd=foo` or url param `&pw=foo`
| PUT | `?xz` | (binary data) | compress with xz and write into file at URL |
| mPOST | | `f=FILE` | upload `FILE` into the folder at URL |
| mPOST | `?j` | `f=FILE` | ...and reply with json |
| mPOST | `?replace` | `f=FILE` | ...and overwrite existing files |
| mPOST | | `act=mkdir`, `name=foo` | create directory `foo` at URL |
| POST | `?delete` | | delete URL recursively |
| jPOST | `?delete` | `["/foo","/bar"]` | delete `/foo` and `/bar` recursively |
@@ -223,7 +218,7 @@ if you don't need all the features, you can repack the sfx and save a bunch of s
* `269k` after `./scripts/make-sfx.sh re no-cm no-hl`
the features you can opt to drop are
* `cm`/easymde, the "fancy" markdown editor, saves ~89k
* `cm`/easymde, the "fancy" markdown editor, saves ~82k
* `hl`, prism, the syntax hilighter, saves ~41k
* `fnt`, source-code-pro, the monospace font, saves ~9k
* `dd`, the custom mouse cursor for the media player tray tab, saves ~2k
@@ -247,7 +242,6 @@ python3 -m venv .venv
pip install jinja2 strip_hints # MANDATORY
pip install mutagen # audio metadata
pip install pyftpdlib # ftp server
pip install partftpy # tftp server
pip install impacket # smb server -- disable Windows Defender if you REALLY need this on windows
pip install Pillow pyheif-pillow-opener pillow-avif-plugin # thumbnails
pip install pyvips # faster thumbnails
@@ -305,26 +299,15 @@ in the `scripts` folder:
* run `./rls.sh 1.2.3` which uploads to pypi + creates github release + sfx
# debugging
# todo
## music playback halting on phones
roughly sorted by priority
mostly fine on android, but still haven't find a way to massage iphones into behaving well
* conditionally starting/stopping mp.fau according to mp.au.readyState <3 or <4 doesn't help
* loop=true doesn't work, and manually looping mp.fau from an onended also doesn't work (it does nothing)
* assigning fau.currentTime in a timer doesn't work, as safari merely pretends to assign it
* on ios 16.7.7, mp.fau can sometimes make everything visibly work correctly, but no audio is actually hitting the speakers
can be reproduced with `--no-sendfile --s-wr-sz 8192 --s-wr-slp 0.3 --rsp-slp 6` and then play a collection of small audio files with the screen off, `ffmpeg -i track01.cdda.flac -c:a libopus -b:a 128k -segment_time 12 -f segment smol-%02d.opus`
* nothing! currently
## discarded ideas
* optimization attempts which didn't improve performance
* remove brokers / multiprocessing stuff; https://github.com/9001/copyparty/tree/no-broker
* reduce the nesting / indirections in `HttpCli` / `httpcli.py`
* nearly zero benefit from stuff like replacing all the `self.conn.hsrv` with a local `hsrv` variable
* reduce up2k roundtrips
* start from a chunk index and just go
* terminate client on bad data

View File

@@ -10,6 +10,7 @@
# q, lo: /cfg/log/%Y-%m%d.log # log to file instead of docker
# ftp: 3921 # enable ftp server on port 3921
# p: 3939 # listen on another port
# ipa: 10.89. # only allow connections from 10.89.*
# df: 16 # stop accepting uploads if less than 16 GB free disk space

View File

@@ -1,50 +0,0 @@
> [!WARNING]
> I am unable to guarantee the quality, safety, and security of anything in this folder; it is a combination of examples I found online. Please submit corrections or improvements 🙏
to try this out with minimal adjustments:
* specify what filesystem-path to share with copyparty, replacing the default/example value `/srv/pub` in `docker-compose.yml`
* add `127.0.0.1 fs.example.com traefik.example.com authelia.example.com` to your `/etc/hosts`
* `sudo docker-compose up`
* login to https://fs.example.com/ with username `authelia` password `authelia`
to use this in a safe and secure manner:
* follow a guide on setting up authelia properly (TODO:link) and use the copyparty-specific parts of this folder as inspiration for your own config; namely the `cpp` subfolder and the `copyparty` service in `docker-compose.yml`
this folder is based on:
* https://github.com/authelia/authelia/tree/39763aaed24c4abdecd884b47357a052b235942d/examples/compose/lite
incomplete list of modifications made:
* support for running with podman as root on fedora (`:z` volumes, `label:disable`)
* explicitly using authelia `v4.38.0-beta3` because config syntax changed since last stable release
* disabled automatic letsencrypt certificate signing
* reduced logging from debug to info
* added a warning that traefik is given access to the docker socket (as recommended by traefik docs) which means traefik is able to break out of the container and has full root access on the host machine
# security
there is probably/definitely room for improvement in this example setup. Some ideas taken from [github issue #62](https://github.com/9001/copyparty/issues/62):
* Add in a redis password to limit attacker lateral movement in the system
* Move redis to a private network shared with just authelia
* Pin to image hashes (or go all in on updates and add `watchtower`)
* Drop bridge networking for just exposing traefik's public ports
* Configure docker for non-root access to docker socket and then move traefik to use [non-root perms](https://docs.docker.com/engine/security/rootless/)
if you manage to improve on any of this, especially in a way that might be useful for other people, consider sending a PR :>
# performance
currently **not optimal,** at least when compared to running the python sfx outside of docker... some numbers from my laptop (ryzen4500u/fedora39):
| req/s | https D/L | http D/L | approach |
| -----:| ----------:|:--------:| -------- |
| 5200 | 1294 MiB/s | 5+ GiB/s | [copyparty-sfx.py](https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py) running on host |
| 4370 | 725 MiB/s | 4+ GiB/s | `docker run copyparty/ac` |
| 2420 | 694 MiB/s | n/a | `copyparty/ac` behind traefik |
| 75 | 694 MiB/s | n/a | traefik and authelia **(you are here)** |
authelia is behaving strangely, handling 340 requests per second for a while, but then it suddenly drops to 75 and stays there...
I'm assuming all of the performance issues is due to a misconfiguration of authelia/traefik/docker on my end, but I don't relly know where to start

View File

@@ -1,66 +0,0 @@
# based on https://github.com/authelia/authelia/blob/39763aaed24c4abdecd884b47357a052b235942d/examples/compose/lite/authelia/configuration.yml
# Authelia configuration
# This secret can also be set using the env variables AUTHELIA_JWT_SECRET_FILE
jwt_secret: a_very_important_secret
server:
address: 'tcp://:9091'
log:
level: info # debug
totp:
issuer: authelia.com
authentication_backend:
file:
path: /config/users_database.yml
access_control:
default_policy: deny
rules:
# Rules applied to everyone
- domain: traefik.example.com
policy: one_factor
- domain: fs.example.com
policy: one_factor
session:
# This secret can also be set using the env variables AUTHELIA_SESSION_SECRET_FILE
secret: unsecure_session_secret
cookies:
- name: authelia_session
domain: example.com # Should match whatever your root protected domain is
default_redirection_url: https://fs.example.com
authelia_url: https://authelia.example.com/
expiration: 3600 # 1 hour
inactivity: 300 # 5 minutes
redis:
host: redis
port: 6379
# This secret can also be set using the env variables AUTHELIA_SESSION_REDIS_PASSWORD_FILE
# password: authelia
regulation:
max_retries: 3
find_time: 120
ban_time: 300
storage:
encryption_key: you_must_generate_a_random_string_of_more_than_twenty_chars_and_configure_this
local:
path: /config/db.sqlite3
notifier:
disable_startup_check: true
smtp:
username: test
# This secret can also be set using the env variables AUTHELIA_NOTIFIER_SMTP_PASSWORD_FILE
password: password
host: mail.example.com
port: 25
sender: admin@example.com

View File

@@ -1,18 +0,0 @@
# based on https://github.com/authelia/authelia/blob/39763aaed24c4abdecd884b47357a052b235942d/examples/compose/lite/authelia/users_database.yml
# Users Database
# This file can be used if you do not have an LDAP set up.
# List of users
users:
authelia:
disabled: false
displayname: "Authelia User"
# Password is authelia
password: "$6$rounds=50000$BpLnfgDsc2WD8F2q$Zis.ixdg9s/UOJYrs56b5QEZFiZECu0qZVNsIYxBaNJ7ucIL.nlxVCT5tqh8KHG8X4tlwCFm5r6NTOZZ5qRFN/"
email: authelia@authelia.com
groups:
- admins
- dev
- su

View File

@@ -1,82 +0,0 @@
# not actually YAML but lets pretend:
# -*- mode: yaml -*-
# vim: ft=yaml:
# example config for how authelia can be used to replace
# copyparty's built-in authentication/authorization mechanism,
# providing copyparty with HTTP headers through traefik to
# signify who the user is, and what groups they belong to
#
# the filesystem-path that will be shared with copyparty is
# specified in the docker-compose in the parent folder, where
# a real filesystem-path is mapped onto this container's path `/w`,
# meaning `/w` in this config-file is actually `/srv/pub` in the
# outside world (assuming you didn't modify that value)
[global]
e2dsa # enable file indexing and filesystem scanning
e2ts # enable multimedia indexing
ansi # enable colors in log messages
#q # disable logging for more performance
# if we are confident that we got the docker-network config correct
# (meaning copyparty is only accessible through traefik, and
# traefik makes sure that all requests go through authelia),
# then accept X-Forwarded-For and IdP headers from any private IP:
xff-src: lan
# enable IdP support by expecting username/groupname in
# http-headers provided by the reverse-proxy; header "X-IdP-User"
# will contain the username, "X-IdP-Group" the groupname
idp-h-usr: remote-user
idp-h-grp: remote-groups
# DEBUG: show all incoming request headers from traefik/authelia
#ihead: *
[/] # create a volume at "/" (the webroot), which will
/w # share /w (the docker data volume, which is mapped to /srv/pub on the host in docker-compose.yml)
accs:
rw: * # everyone gets read-access, but
rwmda: @su # the group "su" gets read-write-move-delete-admin
[/u/${u}] # each user gets their own home-folder at /u/username
/w/u/${u} # which will be "u/username" in the docker data volume
accs:
r: * # read-access for anyone, and
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
[/u/${u}/priv] # each user also gets a private area at /u/username/priv
/w/u/${u}/priv # stored at DATAVOLUME/u/username/priv
accs:
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
[/lounge/${g}] # each group gets their own shared volume
/w/lounge/${g} # stored at DATAVOLUME/lounge/groupname
accs:
r: * # read-access for anyone, and
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
[/lounge/${g}/priv] # and a private area for each group too
/w/lounge/${g}/priv # stored at DATAVOLUME/lounge/groupname/priv
accs:
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
# and create some strategic volumes to prevent anyone from gaining
# unintended access to priv folders if the users/groups db is lost
[/u]
/w/u
accs:
rwmda: @su
[/lounge]
/w/lounge
accs:
rwmda: @su

View File

@@ -1,99 +0,0 @@
version: '3.3'
networks:
net:
driver: bridge
services:
copyparty:
image: copyparty/ac
container_name: idp_copyparty
user: "1000:1000" # should match the user/group of your fileshare volumes
volumes:
- ./cpp/:/cfg:z # the copyparty config folder
- /srv/pub:/w:z # this is where we declare that "/srv/pub" is the filesystem-path on the server that shall be shared online
networks:
- net
expose:
- 3923
labels:
- 'traefik.enable=true'
- 'traefik.http.routers.copyparty.rule=Host(`fs.example.com`)'
- 'traefik.http.routers.copyparty.entrypoints=https'
- 'traefik.http.routers.copyparty.tls=true'
- 'traefik.http.routers.copyparty.middlewares=authelia@docker'
stop_grace_period: 15s # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
authelia:
image: authelia/authelia:v4.38.0-beta3 # the config files in the authelia folder use the new syntax
container_name: idp_authelia
volumes:
- ./authelia:/config:z
networks:
- net
labels:
- 'traefik.enable=true'
- 'traefik.http.routers.authelia.rule=Host(`authelia.example.com`)'
- 'traefik.http.routers.authelia.entrypoints=https'
- 'traefik.http.routers.authelia.tls=true'
#- 'traefik.http.routers.authelia.tls.certresolver=letsencrypt' # uncomment this to enable automatic certificate signing (1/2)
- 'traefik.http.middlewares.authelia.forwardauth.address=http://authelia:9091/api/authz/forward-auth?authelia_url=https://authelia.example.com'
- 'traefik.http.middlewares.authelia.forwardauth.trustForwardHeader=true'
- 'traefik.http.middlewares.authelia.forwardauth.authResponseHeaders=Remote-User,Remote-Groups,Remote-Name,Remote-Email'
expose:
- 9091
restart: unless-stopped
healthcheck:
disable: true
environment:
- TZ=Etc/UTC
redis:
image: redis:7.2.4-alpine3.19
container_name: idp_redis
volumes:
- ./redis:/data:z
networks:
- net
expose:
- 6379
restart: unless-stopped
environment:
- TZ=Etc/UTC
traefik:
image: traefik:2.11.0
container_name: idp_traefik
volumes:
- ./traefik:/etc/traefik:z
- /var/run/docker.sock:/var/run/docker.sock # WARNING: this gives traefik full root-access to the host OS, but is recommended/required(?) by traefik
security_opt:
- label:disable # disable selinux because it (rightly) blocks access to docker.sock
networks:
- net
labels:
- 'traefik.enable=true'
- 'traefik.http.routers.api.rule=Host(`traefik.example.com`)'
- 'traefik.http.routers.api.entrypoints=https'
- 'traefik.http.routers.api.service=api@internal'
- 'traefik.http.routers.api.tls=true'
#- 'traefik.http.routers.api.tls.certresolver=letsencrypt' # uncomment this to enable automatic certificate signing (2/2)
- 'traefik.http.routers.api.middlewares=authelia@docker'
ports:
- '80:80'
- '443:443'
command:
- '--api'
- '--providers.docker=true'
- '--providers.docker.exposedByDefault=false'
- '--entrypoints.http=true'
- '--entrypoints.http.address=:80'
- '--entrypoints.http.http.redirections.entrypoint.to=https'
- '--entrypoints.http.http.redirections.entrypoint.scheme=https'
- '--entrypoints.https=true'
- '--entrypoints.https.address=:443'
- '--certificatesResolvers.letsencrypt.acme.email=your-email@your-domain.com'
- '--certificatesResolvers.letsencrypt.acme.storage=/etc/traefik/acme.json'
- '--certificatesResolvers.letsencrypt.acme.httpChallenge.entryPoint=http'
- '--log=true'
- '--log.level=WARNING' # DEBUG

View File

@@ -1,12 +0,0 @@
> [!WARNING]
> I am unable to guarantee the quality, safety, and security of anything in this folder; it is a combination of examples I found online. Please submit corrections or improvements 🙏
> [!WARNING]
> does not work yet... if you are able to fix this, please do!
this is based on:
* https://goauthentik.io/docker-compose.yml
* https://goauthentik.io/docs/providers/proxy/server_traefik
incomplete list of modifications made:
* support for running with podman as root on fedora (`:z` volumes, `label:disable`)

View File

@@ -1,88 +0,0 @@
# https://goauthentik.io/docker-compose.yml
---
version: "3.4"
services:
postgresql:
image: docker.io/library/postgres:12-alpine
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
start_period: 20s
interval: 30s
retries: 5
timeout: 5s
volumes:
- database:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: ${PG_PASS:?database password required}
POSTGRES_USER: ${PG_USER:-authentik}
POSTGRES_DB: ${PG_DB:-authentik}
env_file:
- .env
redis:
image: docker.io/library/redis:alpine
command: --save 60 1 --loglevel warning
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
start_period: 20s
interval: 30s
retries: 5
timeout: 3s
volumes:
- redis:/data
server:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.2.1}
restart: unless-stopped
command: server
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
volumes:
- ./media:/media
- ./custom-templates:/templates
env_file:
- .env
ports:
- "${COMPOSE_PORT_HTTP:-9000}:9000"
- "${COMPOSE_PORT_HTTPS:-9443}:9443"
depends_on:
- postgresql
- redis
worker:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.2.1}
restart: unless-stopped
command: worker
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
# `user: root` and the docker socket volume are optional.
# See more for the docker socket integration here:
# https://goauthentik.io/docs/outposts/integrations/docker
# Removing `user: root` also prevents the worker from fixing the permissions
# on the mounted folders, so when removing this make sure the folders have the correct UID/GID
# (1000:1000 by default)
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./media:/media
- ./certs:/certs
- ./custom-templates:/templates
env_file:
- .env
depends_on:
- postgresql
- redis
volumes:
database:
driver: local
redis:
driver: local

View File

@@ -1,46 +0,0 @@
# https://goauthentik.io/docs/providers/proxy/server_traefik
---
version: "3.7"
services:
traefik:
image: traefik:v2.2
container_name: traefik
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 80:80
command:
- "--api"
- "--providers.docker=true"
- "--providers.docker.exposedByDefault=false"
- "--entrypoints.web.address=:80"
authentik-proxy:
image: ghcr.io/goauthentik/proxy
ports:
- 9000:9000
- 9443:9443
environment:
AUTHENTIK_HOST: https://your-authentik.tld
AUTHENTIK_INSECURE: "false"
AUTHENTIK_TOKEN: token-generated-by-authentik
# Starting with 2021.9, you can optionally set this too
# when authentik_host for internal communication doesn't match the public URL
# AUTHENTIK_HOST_BROWSER: https://external-domain.tld
labels:
traefik.enable: true
traefik.port: 9000
traefik.http.routers.authentik.rule: Host(`app.company`) && PathPrefix(`/outpost.goauthentik.io/`)
# `authentik-proxy` refers to the service name in the compose file.
traefik.http.middlewares.authentik.forwardauth.address: http://authentik-proxy:9000/outpost.goauthentik.io/auth/traefik
traefik.http.middlewares.authentik.forwardauth.trustForwardHeader: true
traefik.http.middlewares.authentik.forwardauth.authResponseHeaders: X-authentik-username,X-authentik-groups,X-authentik-email,X-authentik-name,X-authentik-uid,X-authentik-jwt,X-authentik-meta-jwks,X-authentik-meta-outpost,X-authentik-meta-provider,X-authentik-meta-app,X-authentik-meta-version
restart: unless-stopped
whoami:
image: containous/whoami
labels:
traefik.enable: true
traefik.http.routers.whoami.rule: Host(`app.company`)
traefik.http.routers.whoami.middlewares: authentik@docker
restart: unless-stopped

View File

@@ -1,72 +0,0 @@
# not actually YAML but lets pretend:
# -*- mode: yaml -*-
# vim: ft=yaml:
# example config for how copyparty can be used with an identity
# provider, replacing the built-in authentication/authorization
# mechanism, and instead expecting the reverse-proxy to provide
# the requester's username (and possibly a group-name, for
# optional group-based access control)
#
# the filesystem-path `/w` is used as the storage location
# because that is the data-volume in the docker containers,
# because a deployment like this (with an IdP) is more commonly
# seen in containerized environments -- but this is not required
[global]
e2dsa # enable file indexing and filesystem scanning
e2ts # enable multimedia indexing
ansi # enable colors in log messages
# enable IdP support by expecting username/groupname in
# http-headers provided by the reverse-proxy; header "X-IdP-User"
# will contain the username, "X-IdP-Group" the groupname
idp-h-usr: x-idp-user
idp-h-grp: x-idp-group
[/] # create a volume at "/" (the webroot), which will
/w # share /w (the docker data volume, which is mapped to /srv/pub on the host in docker-compose.yml)
accs:
rw: * # everyone gets read-access, but
rwmda: @su # the group "su" gets read-write-move-delete-admin
[/u/${u}] # each user gets their own home-folder at /u/username
/w/u/${u} # which will be "u/username" in the docker data volume
accs:
r: * # read-access for anyone, and
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
[/u/${u}/priv] # each user also gets a private area at /u/username/priv
/w/u/${u}/priv # stored at DATAVOLUME/u/username/priv
accs:
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
[/lounge/${g}] # each group gets their own shared volume
/w/lounge/${g} # stored at DATAVOLUME/lounge/groupname
accs:
r: * # read-access for anyone, and
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
[/lounge/${g}/priv] # and a private area for each group too
/w/lounge/${g}/priv # stored at DATAVOLUME/lounge/groupname/priv
accs:
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
# and create some strategic volumes to prevent anyone from gaining
# unintended access to priv folders if the users/groups db is lost
[/u]
/w/u
accs:
rwmda: @su
[/lounge]
/w/lounge
accs:
rwmda: @su

View File

@@ -1,131 +0,0 @@
version: "3.4"
volumes:
database:
driver: local
redis:
driver: local
services:
copyparty:
image: copyparty/ac
container_name: idp_copyparty
restart: unless-stopped
user: "1000:1000" # should match the user/group of your fileshare volumes
volumes:
- ./cpp/:/cfg:z # the copyparty config folder
- /srv/pub:/w:z # this is where we declare that "/srv/pub" is the filesystem-path on the server that shall be shared online
ports:
- 3923
labels:
- 'traefik.enable=true'
- 'traefik.http.routers.fs.rule=Host(`fs.example.com`)'
- 'traefik.http.routers.fs.entrypoints=http'
#- 'traefik.http.routers.fs.middlewares=authelia@docker' # TODO: ???
healthcheck:
test: ["CMD-SHELL", "wget --spider -q 127.0.0.1:3923/?reset"]
interval: 1m
timeout: 2s
retries: 5
start_period: 15s
stop_grace_period: 15s # thumbnailer is allowed to continue finishing up for 10s after the shutdown signal
traefik:
image: traefik:v2.11
container_name: traefik
volumes:
- /var/run/docker.sock:/var/run/docker.sock # WARNING: this gives traefik full root-access to the host OS, but is recommended/required(?) by traefik
security_opt:
- label:disable # disable selinux because it (rightly) blocks access to docker.sock
ports:
- 80:80
command:
- '--api'
- '--providers.docker=true'
- '--providers.docker.exposedByDefault=false'
- '--entrypoints.web.address=:80'
postgresql:
image: docker.io/library/postgres:12-alpine
container_name: idp_postgresql
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
start_period: 20s
interval: 30s
retries: 5
timeout: 5s
volumes:
- database:/var/lib/postgresql/data:z
environment:
POSTGRES_PASSWORD: postgrass
POSTGRES_USER: authentik
POSTGRES_DB: authentik
env_file:
- .env
redis:
image: docker.io/library/redis:alpine
command: --save 60 1 --loglevel warning
container_name: idp_redis
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
start_period: 20s
interval: 30s
retries: 5
timeout: 3s
volumes:
- redis:/data:z
authentik_server:
image: ghcr.io/goauthentik/server:2024.2.1
container_name: idp_authentik_server
restart: unless-stopped
command: server
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: postgrass
volumes:
- ./media:/media:z
- ./custom-templates:/templates:z
env_file:
- .env
ports:
- 9000
- 9443
depends_on:
- postgresql
- redis
authentik_worker:
image: ghcr.io/goauthentik/server:2024.2.1
container_name: idp_authentik_worker
restart: unless-stopped
command: worker
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: postgrass
# `user: root` and the docker socket volume are optional.
# See more for the docker socket integration here:
# https://goauthentik.io/docs/outposts/integrations/docker
# Removing `user: root` also prevents the worker from fixing the permissions
# on the mounted folders, so when removing this make sure the folders have the correct UID/GID
# (1000:1000 by default)
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./media:/media:z
- ./certs:/certs:z
- ./custom-templates:/templates:z
env_file:
- .env
depends_on:
- postgresql
- redis

View File

@@ -26,56 +26,38 @@
idp-h-usr: x-idp-user
idp-h-grp: x-idp-group
# but copyparty will refuse to accept those headers unless you
# tell it the LAN IP of the reverse-proxy to expect them from,
# preventing malicious users from pretending to be the proxy;
# pay attention to the warning message in the logs and then
# adjust the following config option accordingly:
xff-src: 192.168.0.0/16
# or just allow all LAN / private IPs (probably good enough):
xff-src: lan
# an additional, optional security measure is to expect a
# secret header name from the reverse-proxy; you can enable
# this feature by setting the header-name to expect here:
#idp-h-key: shangala-bangala
# convenient debug option:
# log all incoming request headers from the proxy
#ihead: *
[/] # create a volume at "/" (the webroot), which will
/w # share /w (the docker data volume)
accs:
rw: * # everyone gets read-access, but
rwmda: @su # the group "su" gets read-write-move-delete-admin
rwmda: %su # the group "su" gets read-write-move-delete-admin
[/u/${u}] # each user gets their own home-folder at /u/username
/w/u/${u} # which will be "u/username" in the docker data volume
accs:
r: * # read-access for anyone, and
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
rwmda: ${u}, %su # read-write-move-delete-admin for that username + the "su" group
[/u/${u}/priv] # each user also gets a private area at /u/username/priv
/w/u/${u}/priv # stored at DATAVOLUME/u/username/priv
accs:
rwmda: ${u}, @su # read-write-move-delete-admin for that username + the "su" group
rwmda: ${u}, %su # read-write-move-delete-admin for that username + the "su" group
[/lounge/${g}] # each group gets their own shared volume
/w/lounge/${g} # stored at DATAVOLUME/lounge/groupname
accs:
r: * # read-access for anyone, and
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
rwmda: %${g}, %su # read-write-move-delete-admin for that group + the "su" group
[/lounge/${g}/priv] # and a private area for each group too
/w/lounge/${g}/priv # stored at DATAVOLUME/lounge/groupname/priv
accs:
rwmda: @${g}, @su # read-write-move-delete-admin for that group + the "su" group
rwmda: %${g}, %su # read-write-move-delete-admin for that group + the "su" group
# and create some strategic volumes to prevent anyone from gaining
@@ -83,8 +65,8 @@
[/u]
/w/u
accs:
rwmda: @su
rwmda: %su
[/lounge]
/w/lounge
accs:
rwmda: @su
rwmda: %su

View File

@@ -6,7 +6,7 @@ you will definitely need either [copyparty.exe](https://github.com/9001/copypart
* if you decided to grab `copyparty-sfx.py` instead of the exe you will also need to install the ["Latest Python 3 Release"](https://www.python.org/downloads/windows/)
then you probably want to download [FFmpeg](https://www.gyan.dev/ffmpeg/builds/ffmpeg-git-full.7z) and put `ffmpeg.exe` and `ffprobe.exe` in your PATH (so for example `C:\Windows\System32\`) -- this enables thumbnails, audio transcoding, and making music metadata searchable
then you probably want to download [FFmpeg](https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip) and put `ffmpeg.exe` and `ffprobe.exe` in your PATH (so for example `C:\Windows\System32\`) -- this enables thumbnails, audio transcoding, and making music metadata searchable
## the config file
@@ -46,7 +46,7 @@ open up notepad and save the following as `c:\users\you\documents\party.conf` (f
### config explained: [global]
the `[global]` section accepts any config parameters [listed here](https://ocv.me/copyparty/helptext.html), also viewable by running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
the `[global]` section accepts any config parameters you can see when running copyparty (either the exe or the sfx.py) with `--help`, so this is the same as running copyparty with arguments `--lo c:\users\you\logs\copyparty-%Y-%m%d.xz -e2dsa -e2ts --no-dedup -z -p 80,443 --theme 2 --lang nor`
* `lo: ~/logs/cpp-%Y-%m%d.xz` writes compressed logs (the compression will make them delayed)
* `e2dsa` enables the upload deduplicator and file indexer, which enables searching
* `e2ts` enables music metadata indexing, making albums / titles etc. searchable too

View File

@@ -1,22 +0,0 @@
there is a [docker-compose example](./examples/docker/idp-authelia-traefik) which is hopefully a good starting point (meaning you can skip the steps below) -- but if you want to set this up from scratch yourself (or learn about how it works), keep reading:
to configure IdP from scratch, you must place copyparty behind a reverse-proxy which sends all requests through a middleware (the IdP / identity-provider service) which will inject a set of headers into the requests, telling copyparty who the user is
in the copyparty `[global]` config, specify which headers to read client info from; username is required (`idp-h-usr: X-Authooley-User`), group(s) are optional (`idp-h-grp: X-Authooley-Groups`)
* it is also required to specify the subnet that legit requests will be coming from, for example `--xff-src=10.88.0.0/24` to allow 10.88.x.x (or `--xff-src=lan` for all private IPs), and it is recommended to configure the reverseproxy to include a secret header as proof that the other headers are also legit (and not smuggled in by a malicious client), telling copyparty the headername to expect with `idp-h-key: shangala-bangala`
# important notes
## IdP volumes are forgotten on shutdown
IdP volumes, meaning dynamically-created volumes, meaning volumes that contain `${u}` or `${g}` in their URL, will be forgotten during a server restart and then "revived" when the volume's owner sends their first request after the restart
until each IdP volume is revived, it will inherit the permissions of its parent volume (if any)
this means that, if an IdP volume is located inside a folder that is readable by anyone, then each of those IdP volumes will **also become readable by anyone** until the volume is revived
and likewise -- if the IdP volume is inside a folder that is only accessible by certain users, but the IdP volume is configured to allow access from unauthenticated users, then the contents of the volume will NOT be accessible until it is revived
until this limitation is fixed (if ever), it is recommended to place IdP volumes inside an appropriate parent volume, so they can inherit acceptable permissions until their revival; see the "strategic volumes" at the bottom of [./examples/docker/idp/copyparty.conf](./examples/docker/idp/copyparty.conf)

View File

@@ -24,10 +24,6 @@ https://github.com/giampaolo/pyftpdlib/
C: 2007 Giampaolo Rodola
L: MIT
https://github.com/9001/partftpy
C: 2010-2021 Michael P. Soulier
L: MIT
https://github.com/nayuki/QR-Code-generator/
C: Project Nayuki
L: MIT

View File

@@ -1,19 +1,3 @@
this file accidentally got committed at some point, so let's put it to use
# trivia / lore
copyparty started as [three separate php projects](https://a.ocv.me/pub/stuff/old-php-projects/); an nginx custom directory listing (which became a php script), and a php music/picture viewer, and an additional php project for resumable uploads:
* findex -- directory browser / gallery with thumbnails and a music player which sometime back in 2009 had a canvas visualizer grabbing fft data from a flash audio player
* findex.mini -- plain-listing fork of findex with streaming zip-download of folders (the js and design should look familiar)
* upper and up2k -- up2k being the star of the show and where copyparty's chunked resumable uploads came from
the first link has screenshots but if that doesn't work there's also a [tar here](https://ocv.me/dev/old-php-projects.tgz)
----
below this point is misc useless scribbles
# up2k.js
## potato detection

View File

@@ -24,27 +24,6 @@ gzip -d < .hist/up2k.snap | jq -r '.[].name' | while IFS= read -r f; do wc -c --
echo; find -type f | while IFS= read -r x; do printf '\033[A\033[36m%s\033[K\033[0m\n' "$x"; tail -c$((1024*1024)) <"$x" | xxd -a | awk 'NR==1&&/^[0: ]+.{16}$/{next} NR==2&&/^\*$/{next} NR==3&&/^[0f]+: [0 ]+65 +.{16}$/{next} {e=1} END {exit e}' || continue; printf '\033[A\033[31msus:\033[33m %s \033[0m\n\n' "$x"; done
##
## sync pics/vids from phone
## (takes all files named (IMG|PXL|PANORAMA|Screenshot)_20231224_*)
cd /storage/emulated/0/DCIM/Camera
find -mindepth 1 -maxdepth 1 | sort | cut -c3- > ls
url=https://192.168.1.3:3923/rw/pics/Camera/$d/; awk -F_ '!/^[A-Z][A-Za-z]{1,16}_[0-9]{8}[_-]/{next} {d=substr($2,1,6)} !t[d]++{print d}' ls | while read d; do grep -E "^[A-Z][A-Za-z]{1,16}_$d" ls | tr '\n' '\0' | xargs -0 python3 ~/dev/copyparty/bin/u2c.py -td $url --; done
##
## convert symlinks to hardlinks (probably safe, no guarantees)
find -type l | while IFS= read -r lnk; do [ -h "$lnk" ] || { printf 'nonlink: %s\n' "$lnk"; continue; }; dst="$(readlink -f -- "$lnk")"; [ -e "$dst" ] || { printf '???\n%s\n%s\n' "$lnk" "$dst"; continue; }; printf 'relinking:\n %s\n %s\n' "$lnk" "$dst"; rm -- "$lnk"; ln -- "$dst" "$lnk"; done
##
## convert hardlinks to symlinks (maybe not as safe? use with caution)
e=; p=; find -printf '%i %p\n' | awk '{i=$1;sub(/[^ ]+ /,"")} !n[i]++{p[i]=$0;next} {printf "real %s\nlink %s\n",p[i],$0}' | while read cls p; do [ -e "$p" ] || e=1; p="$(realpath -- "$p")" || e=1; [ -e "$p" ] || e=1; [ $cls = real ] && { real="$p"; continue; }; [ $cls = link ] || e=1; [ "$p" ] || e=1; [ $e ] && { echo "ERROR $p"; break; }; printf '\033[36m%s \033[0m -> \033[35m%s\033[0m\n' "$p" "$real"; rm "$p"; ln -s "$real" "$p" || { echo LINK FAILED; break; }; done
##
## create a test payload
@@ -221,11 +200,6 @@ sox -DnV -r8000 -b8 -c1 /dev/shm/a.wav synth 1.1 sin 400 vol 0.02
# play icon calibration pics
for w in 150 170 190 210 230 250; do for h in 130 150 170 190 210; do /c/Program\ Files/ImageMagick-7.0.11-Q16-HDRI/magick.exe convert -size ${w}x${h} xc:brown -fill orange -draw "circle $((w/2)),$((h/2)) $((w/2)),$((h/3))" $w-$h.png; done; done
# compress chiptune modules
mkdir gz; for f in *.*; do pigz -c11 -I100 <"$f" >gz/"$f"gz; touch -r "$f" gz/"$f"gz; done
mkdir xz; for f in *.*; do xz -cz9 <"$f" >xz/"$f"xz; touch -r "$f" xz/"$f"xz; done
mkdir z; for f in *.*; do 7z a -tzip -mx=9 -mm=lzma "z/${f}z" "$f" && touch -r "$f" z/"$f"z; done
##
## vscode

View File

@@ -1,71 +1,82 @@
# recipe for building an exe with nuitka (extreme jank edition)
NOTE: copyparty runs SLOWER when compiled with nuitka;
just use copyparty-sfx.py and/or pyinstaller instead
( the sfx and the pyinstaller EXEs are equally fast if you
have the latest jinja2 installed, but the older jinja that
comes bundled with the sfx is slightly faster yet )
roughly, copyparty-sfx.py is 6% faster than copyparty.exe
(win10-pyinstaller), and copyparty.exe is 10% faster than
nuitka, making copyparty-sfx.py 17% faster than nuitka
NOTE: every time a nuitka-compiled copyparty.exe is launched,
it will show the windows firewall prompt since nuitka will
pick a new unique location in %TEMP% to unpack an exe into,
unlike pyinstaller which doesn't fork itself on startup...
might be fixable by configuring nuitka differently, idk
NOTE: nuitka EXEs are larger than pyinstaller ones;
a minimal nuitka build of just the sfx (with its bundled
dependencies) was already the same size as the pyinstaller
copyparty.exe which also includes Mutagen and Pillow
NOTE: nuitka takes a lot longer to build than pyinstaller
(due to actual compilation of course, but still)
NOTE: binaries built with nuitka cannot run on windows7,
even when compiled with python 3.6 on windows 7 itself
NOTE: `--python-flags=-m` is the magic sauce to
correctly compile `from .util import Daemon`
(which otherwise only explodes at runtime)
NOTE: `--deployment` doesn't seem to affect performance
########################################################################
#
# NOTE: win7 and win10 builds both work on win10 but
# on win7 they immediately c0000005 in kernelbase.dll
#
# first install python-3.6.8-amd64.exe
# [x] add to path
#
# copypaste the rest of this file into cmd
rem from pypi
cd \users\ed\downloads
python -m pip install --user Nuitka-0.6.14.7.tar.gz
rem https://github.com/brechtsanders/winlibs_mingw/releases/download/10.2.0-11.0.0-8.0.0-r5/winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\
copy c:\users\ed\downloads\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\gcc\x86_64\10.2.0-11.0.0-8.0.0-r5\winlibs-x86_64-posix-seh-gcc-10.2.0-llvm-11.0.0-mingw-w64-8.0.0-r5.zip
python -m pip install --user -U nuitka
rem https://github.com/ccache/ccache/releases/download/v3.7.12/ccache-3.7.12-windows-32.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\
copy c:\users\ed\downloads\ccache-3.7.12-windows-32.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\ccache\v3.7.12\ccache-3.7.12-windows-32.zip
cd %homedrive%
cd %homepath%\downloads
rem https://dependencywalker.com/depends22_x64.zip
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\
mkdir C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\
copy c:\users\ed\downloads\depends22_x64.zip C:\Users\ed\AppData\Local\Nuitka\Nuitka\depends\x86_64\depends22_x64.zip
rd /s /q copypuitka
mkdir copypuitka
cd copypuitka
cd \
rd /s /q %appdata%\..\local\temp\pe-copyparty
cd \users\ed\downloads
python copyparty-sfx.py -h
cd %appdata%\..\local\temp\pe-copyparty\copyparty
rd /s /q %temp%\pe-copyparty
python ..\copyparty-sfx.py --version
python
import os, re
os.rename('../dep-j2/jinja2', '../jinja2')
os.rename('../dep-j2/markupsafe', '../markupsafe')
move %temp%\pe-copyparty\copyparty .\
move %temp%\pe-copyparty\partftpy .\
move %temp%\pe-copyparty\ftp\pyftpdlib .\
move %temp%\pe-copyparty\j2\jinja2 .\
move %temp%\pe-copyparty\j2\markupsafe .\
print("# nuitka dies if .__init__.stuff is imported")
with open('__init__.py','r',encoding='utf-8') as f:
t1 = f.read()
rd /s /q %temp%\pe-copyparty
with open('util.py','r',encoding='utf-8') as f:
t2 = f.read().split('\n')[3:]
python -m nuitka ^
--onefile --deployment --python-flag=-m ^
--include-package=markupsafe ^
--include-package=jinja2 ^
--include-package=partftpy ^
--include-package=pyftpdlib ^
--include-data-dir=copyparty\web=copyparty\web ^
--include-data-dir=copyparty\res=copyparty\res ^
--run copyparty
t2 = [x for x in t2 if 'from .__init__' not in x]
t = t1 + '\n'.join(t2)
with open('__init__.py','w',encoding='utf-8') as f:
f.write('\n')
with open('util.py','w',encoding='utf-8') as f:
f.write(t)
print("# local-imports fail, prefix module names")
ptn = re.compile(r'^( *from )(\.[^ ]+ import .*)')
for d, _, fs in os.walk('.'):
for f in fs:
fp = os.path.join(d, f)
if not fp.endswith('.py'):
continue
t = ''
with open(fp,'r',encoding='utf-8') as f:
for ln in [x.rstrip('\r\n') for x in f]:
m = ptn.match(ln)
if not m:
t += ln + '\n'
continue
p1, p2 = m.groups()
t += "{}copyparty{}\n".format(p1, p2).replace("__init__", "util")
with open(fp,'w',encoding='utf-8') as f:
f.write(t)
exit()
cd ..
rd /s /q bout & python -m nuitka --standalone --onefile --windows-onefile-tempdir --python-flag=no_site --assume-yes-for-downloads --include-data-dir=copyparty\web=copyparty\web --include-data-dir=copyparty\res=copyparty\res --run --output-dir=bout --mingw64 --include-package=markupsafe --include-package=jinja2 copyparty

52
docs/pyoxidizer.txt Normal file
View File

@@ -0,0 +1,52 @@
pyoxidizer doesn't crosscompile yet so need to build in a windows vm,
luckily possible to do mostly airgapped (https-proxy for crates)
none of this is version-specific but doing absolute links just in case
(only exception is py3.8 which is the final win7 ver)
# deps (download on linux host):
https://www.python.org/ftp/python/3.10.7/python-3.10.7-amd64.exe
https://github.com/indygreg/PyOxidizer/releases/download/pyoxidizer%2F0.22.0/pyoxidizer-0.22.0-x86_64-pc-windows-msvc.zip
https://github.com/upx/upx/releases/download/v3.96/upx-3.96-win64.zip
https://static.rust-lang.org/dist/rust-1.61.0-x86_64-pc-windows-msvc.msi
https://github.com/indygreg/python-build-standalone/releases/download/20220528/cpython-3.8.13%2B20220528-i686-pc-windows-msvc-static-noopt-full.tar.zst
# need cl.exe, prefer 2017 -- download on linux host:
https://visualstudio.microsoft.com/downloads/?q=build+tools
https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-history#release-dates-and-build-numbers
https://aka.ms/vs/15/release/vs_buildtools.exe # 2017
https://aka.ms/vs/16/release/vs_buildtools.exe # 2019
https://aka.ms/vs/17/release/vs_buildtools.exe # 2022
https://docs.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2017
# use disposable w10 vm to prep offline installer; xfer to linux host with firefox to copyparty
vs_buildtools-2017.exe --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools --add Microsoft.VisualStudio.Component.Windows10SDK.17763 --layout c:\msbt2017 --lang en-us
# need two proxies on host; s5s or ssh for msys2(socks5), and tinyproxy for rust(http)
UP=- python3 socks5server.py 192.168.123.1 4321
ssh -vND 192.168.123.1:4321 localhost
git clone https://github.com/tinyproxy/tinyproxy.git
./autogen.sh
./configure --prefix=/home/ed/pe/tinyproxy
make -j24 install
printf '%s\n' >cfg "Port 4380" "Listen 192.168.123.1"
./tinyproxy -dccfg
https://github.com/msys2/msys2-installer/releases/download/2022-09-04/msys2-x86_64-20220904.exe
export all_proxy=socks5h://192.168.123.1:4321
# if chat dies after auth (2 messages) it probably failed dns, note the h in socks5h to tunnel dns
pacman -Syuu
pacman -S git patch mingw64/mingw-w64-x86_64-zopfli
cd /c && curl -k https://192.168.123.1:3923/ro/ox/msbt2017/?tar | tar -xv
first install certs from msbt/certificates then admin-cmd `vs_buildtools.exe --noweb`,
default selection (vc++2017-v15.9-v14.16, vc++redist, vc++bt-core) += win10sdk (for io.h)
install rust without documentation, python 3.10, put upx and pyoxidizer into ~/bin,
[cmd.exe] python -m pip install --user -U wheel-0.37.1.tar.gz strip-hints-0.1.10.tar.gz
p=192.168.123.1:4380; export https_proxy=$p; export http_proxy=$p
# and with all of the one-time-setup out of the way,
mkdir /c/d; cd /c/d && curl -k https://192.168.123.1:3923/cpp/gb?pw=wark > gb && git clone gb copyparty
cd /c/d/copyparty/ && curl -k https://192.168.123.1:3923/cpp/patch?pw=wark | patch -p1
cd /c/d/copyparty/scripts && CARGO_HTTP_CHECK_REVOKE=false PATH=/c/Users/$USER/AppData/Local/Programs/Python/Python310:/c/Users/$USER/bin:"$(cygpath "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\Hostx86\x86"):$PATH" ./make-sfx.sh ox ultra

View File

@@ -1,71 +0,0 @@
# custom fonts
to change the fonts in the web-UI, first save the following text (the default font-config) to a new css file, for example named `customfonts.css` in your webroot:
```css
:root {
--font-main: sans-serif;
--font-serif: serif;
--font-mono: 'scp';
}
```
add this to your copyparty config so the css file gets loaded: `--html-head='<link rel="stylesheet" href="/customfonts.css">'`
alternatively, if you are using a config file instead of commandline args:
```yaml
[global]
html-head: <link rel="stylesheet" href="/customfonts.css">
```
restart copyparty for the config change to take effect
edit the css file you made and press `ctrl`-`shift`-`R` in the browser to see the changes as you go (no need to restart copyparty for each change)
if you are introducing a new ttf/woff font, don't forget to declare the font itself in the css file; here's one of the default fonts from `ui.css`:
```css
@font-face {
font-family: 'scp';
font-display: swap;
src: local('Source Code Pro Regular'), local('SourceCodePro-Regular'), url(deps/scp.woff2) format('woff2');
}
```
and because textboxes don't inherit fonts by default, you can force it like this:
```css
input[type=text], input[type=submit], input[type=button] { font-family: var(--font-main) }
```
and if you want to have a monospace font in the fancy markdown editor, do this:
```css
.EasyMDEContainer .CodeMirror { font-family: var(--font-mono) }
```
NB: `<textarea id="mt">` and `<div id="mtr">` in the regular markdown editor must have the same font; none of the suggestions above will cause any issues but keep it in mind if you're getting creative
# `<head>`
to add stuff to the html `<head>`, for example a css `<link>` or `<meta>` tags, use either the global-option `--html-head` or the volflag `html_head`
if you give it the value `@ASDF` it will try to open a file named ASDF and send the text within
if the value starts with `%` it will assume a jinja2 template and expand it; the template has access to the `HttpCli` object through a property named `this` as well as everything in `j2a` and the stuff added by `self.j2s`; see [browser.html](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/web/browser.html) for inspiration or look under the hood in [httpcli.py](https://github.com/9001/copyparty/blob/hovudstraum/copyparty/httpcli.py)
# translations
add your own translations by using the english or norwegian one from `browser.js` as a template
the easy way is to open up and modify `browser.js` in your own installation; depending on how you installed copyparty it might be named `browser.js.gz` instead, in which case just decompress it, restart copyparty, and start editing it anyways
if you're running `copyparty-sfx.py` then you'll find it at `/tmp/pe-copyparty.1000/copyparty/web` (on linux) or `%TEMP%\pe-copyparty\copyparty\web` (on windows)
* make sure to keep backups of your work religiously! since that location is volatile af
if editing `browser.js` is inconvenient in your setup then you can instead do this:
* add your translation to a separate javascript file (`tl.js`) and make it load before `browser.js` with the help of `--html-head='<script src="/tl.js"></script>'`
* as the page loads, `browser.js` will look for a function named `langmod` so define that function and make it insert your translation into the `Ls` and `LANGS` variables so it'll take effect

View File

@@ -48,7 +48,6 @@ currently up to date with [awesome-selfhosted](https://github.com/awesome-selfho
* [filebrowser](#filebrowser)
* [filegator](#filegator)
* [sftpgo](#sftpgo)
* [arozos](#arozos)
* [updog](#updog)
* [goshs](#goshs)
* [gimme-that](#gimme-that)
@@ -94,7 +93,6 @@ the softwares,
* `j` = [filebrowser](https://github.com/filebrowser/filebrowser)
* `k` = [filegator](https://github.com/filegator/filegator)
* `l` = [sftpgo](https://github.com/drakkan/sftpgo)
* `m` = [arozos](https://github.com/tobychui/arozos)
some softwares not in the matrixes,
* [updog](#updog)
@@ -115,22 +113,22 @@ symbol legend,
## general
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | | |
| runs on iOS | | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ | |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ | |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | | █ |
| android app | | | | █ | █ | | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| intuitive UX | | | █ | █ | █ | | █ | █ | █ | █ | █ | █ |
| config GUI | | █ | █ | █ | █ | | | █ | █ | █ | | █ |
| good documentation | | | | █ | █ | █ | █ | | | █ | █ | |
| runs on iOS | | | | | | | | | | | | |
| runs on Android | █ | | | | | █ | | | | | | |
| runs on WinXP | █ | █ | | | | █ | | | | | | |
| runs on Windows | █ | █ | █ | █ | █ | █ | █ | | █ | █ | █ | █ |
| runs on Linux | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on Macos | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| runs on FreeBSD | █ | | | • | █ | █ | █ | • | █ | █ | | █ |
| portable binary | █ | █ | █ | | | █ | █ | | | █ | | █ |
| zero setup, just go | █ | █ | █ | | | | █ | | | █ | | |
| android app | | | | █ | █ | | | | | | | |
| iOS app | | | | █ | █ | | | | | | | |
* `zero setup` = you can get a mostly working setup by just launching the app, without having to install any software or configure whatever
* `a`/copyparty remarks:
@@ -142,39 +140,37 @@ symbol legend,
* `f`/rclone must be started with the command `rclone serve webdav .` or similar
* `h`/chibisafe has undocumented windows support
* `i`/sftpgo must be launched with a command
* `m`/arozos has partial windows support
## file transfer
*the thing that copyparty is actually kinda good at*
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ | |
| download folder as tar | █ | | | | | | | | | █ | | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | | █ |
| resumable uploads | █ | | | | | | | | █ | | █ | | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | | █ |
| upload acceleration | █ | | | | | | | | █ | | █ | | |
| upload verification | █ | | | █ | █ | | | | █ | | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | | |
| race the beam ("p2p") | █ | | | | | | | | | • | | | |
| keep last-modified time | █ | | | | | | | | | | | | |
| upload rules | | | | | | | | | | | | | |
| ┗ max disk usage | █ | █ | | | | | | | █ | | | █ | █ |
| ┗ max filesize | █ | | | | | | | | | | █ | █ | |
| ┗ max items in folder | █ | | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | █ | | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | | |
| ┗ randomize filename | | | | | | | | █ | █ | | | | |
| ┗ mimetype reject-list | | | | | | | | | • | | | | • |
| ┗ extension reject-list | | | | | | | | | | | | | • |
| checksums provided | | | | █ | █ | | | | █ | | | | |
| cloud storage backend | | | | █ | █ | █ | | | | | █ | █ | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| download folder as zip | █ | █ | █ | █ | | | █ | | █ | █ | | █ |
| download folder as tar | █ | | | | | | | | | █ | | |
| upload | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| parallel uploads | █ | | | █ | █ | | • | | █ | | █ | |
| resumable uploads | █ | | | | | | | | █ | | █ | |
| upload segmenting | █ | | | | | | | █ | █ | | █ | |
| upload acceleration | █ | | | | | | | | █ | | █ | |
| upload verification | █ | | | █ | █ | | | | █ | | | |
| upload deduplication | █ | | | | █ | | | | █ | | | |
| upload a 999 TiB file | █ | | | | █ | █ | • | | █ | | █ | |
| keep last-modified time | █ | | | █ | █ | █ | | | | | | |
| upload rules | | | | | | | | | | | | |
| ┗ max disk usage | | | | | | | | | | | | |
| ┗ max filesize | █ | | | | | | | █ | | | █ | █ |
| ┗ max items in folder | █ | | | | | | | | | | | |
| ┗ max file age | █ | | | | | | | | | | | |
| ┗ max uploads over time | █ | | | | | | | | | | | |
| ┗ compress before write | █ | | | | | | | | | | | |
| ┗ randomize filename | █ | | | | | | | | | | | |
| ┗ mimetype reject-list | | | | | | | | | • | | | |
| ┗ extension reject-list | | | | | | | | | • | | | |
| checksums provided | | | | █ | █ | | | | | | | |
| cloud storage backend | | | | █ | █ | | | | | | | |
* `upload segmenting` = files are sliced into chunks, making it possible to upload files larger than 100 MiB on cloudflare for example
@@ -182,8 +178,6 @@ symbol legend,
* `upload verification` = uploads are checksummed or otherwise confirmed to have been transferred correctly
* `race the beam` = files can be downloaded while they're still uploading; downloaders are slowed down such that the uploader is always ahead
* `checksums provided` = when downloading a file from the server, the file's checksum is provided for verification client-side
* `cloud storage backend` = able to serve files from (and write to) s3 or similar cloud services; `` means the software can do this with some help from `rclone mount` as a bridge
@@ -198,27 +192,25 @@ symbol legend,
* resumable/segmented uploads only over SFTP, not over HTTP
* upload rules are totals only, not over time
* can probably do extension/mimetype rejection similar to copyparty
* `m`/arozos download-as-zip is not streaming; it creates the full zipfile before download can start, and fails on big folders
## protocols and client support
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ | █ |
| serve ftp (tcp) | █ | | | | | █ | | | | | | █ | █ |
| serve ftps (tls) | █ | | | | | █ | | | | | | █ | |
| serve tftp (udp) | █ | | | | | | | | | | | | |
| serve sftp (ssh) | | | | | | █ | | | | | | █ | █ |
| serve smb/cifs | | | | | | █ | | | | | | | |
| serve dlna | | | | | | █ | | | | | | | |
| listen on unix-socket | | | | █ | | | | | | | | | |
| zeroconf | | | | | | | | | | | | | █ |
| supports netscape 4 | | | | | | █ | | | | | • | | |
| ...internet explorer 6 | | █ | | | | █ | | | | | • | | |
| mojibake filenames | █ | | | • | • | █ | | • | | | | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| serve https | █ | | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| serve webdav | █ | | | █ | █ | █ | █ | | █ | | | █ |
| serve ftp | █ | | | | | █ | | | | | | █ |
| serve ftps | █ | | | | | █ | | | | | | █ |
| serve sftp | | | | | | | | | | | | █ |
| serve smb/cifs | | | | | | █ | | | | | | |
| serve dlna | | | | | | █ | | | | | | |
| listen on unix-socket | | | | | █ | | | | | | | |
| zeroconf | █ | | | | | | | | | | | |
| supports netscape 4 | | | | | | | | | | | | |
| ...internet explorer 6 | | | | | | █ | | | | | • | |
| mojibake filenames | █ | | | • | • | █ | | | | • | | |
| undecodable filenames | █ | | | • | • | █ | | • | | | | |
* `webdav` = protocol convenient for mounting a remote server as a local filesystem; see zeroconf:
* `zeroconf` = the server announces itself on the LAN, [automatically appearing](https://user-images.githubusercontent.com/241032/215344737-0eae8d98-9496-4256-9aa8-cd2f6971810d.png) on other zeroconf-capable devices
@@ -229,66 +221,61 @@ symbol legend,
* extremely minimal samba/cifs server
* netscape 4 / ie6 support is mostly listed as a joke altho some people have actually found it useful ([ie4 tho](https://user-images.githubusercontent.com/241032/118192791-fb31fe00-b446-11eb-9647-898ea8efc1f7.png))
* `l`/sftpgo translates mojibake filenames into valid utf-8 (information loss)
* `m`/arozos has readonly-support for older browsers; no uploading
## server configuration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | | █ |
| same-port http / https | █ | | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ | |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ | |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ | |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | • | • | | • |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| config from cmd args | █ | | | | | █ | █ | | | █ | | |
| config files | █ | █ | █ | | | █ | | █ | | █ | • | |
| runtime config reload | █ | █ | █ | | | | | █ | █ | █ | █ | |
| same-port http / https | █ | | | | | | | | | | | |
| listen multiple ports | █ | | | | | | | | | | | █ |
| virtual file system | █ | █ | █ | | | | █ | | | | | █ |
| reverse-proxy ok | █ | | █ | █ | █ | █ | █ | █ | • | • | • | █ |
| folder-rproxy ok | █ | | | | █ | █ | | • | • | • | • | |
* `folder-rproxy` = reverse-proxying without dedicating an entire (sub)domain, using a subfolder instead
* `l`/sftpgo:
* config: users must be added through gui / api calls
* `m`/arozos:
* configuration is primarily through GUI
* reverse-proxy is not guaranteed to see the correct client IP
## server capabilities
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ | |
| single-sign-on | | | | █ | █ | | | | • | | | | |
| token auth | | | | █ | █ | | | █ | | | | | █ |
| 2fa | | | | █ | █ | | | | | | | █ | |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | | █ |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | | █ |
| unmap subfolders | █ | | | | | | █ | | | █ | | • | |
| index.html blocks list | | | | | | | █ | | | • | | | |
| write-only folders | █ | | | | | | | | | | █ | █ | |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ | |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | | |
| file action event hooks | █ | | | | | | | | | █ | | █ | • |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | | |
| full sync | | | | █ | █ | | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ | |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ | • |
| dyndns updater | | █ | | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | | █ |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ | • |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | | █ |
| prometheus metrics | █ | | | █ | | | | | | | | | |
| curl-friendly ls | █ | | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | █ | █ | • | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| accounts | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ | █ |
| per-account chroot | | | | | | | | | | | | █ |
| single-sign-on | | | | █ | █ | | | | • | | | |
| token auth | | | | █ | █ | | | █ | | | | |
| 2fa | | | | █ | █ | | | | | | | █ |
| per-volume permissions | █ | █ | █ | █ | █ | █ | █ | | █ | █ | | █ |
| per-folder permissions | | | | █ | █ | | █ | | █ | █ | | █ |
| per-file permissions | | | | █ | █ | | █ | | █ | | | |
| per-file passwords | █ | | | █ | █ | | █ | | █ | | | |
| unmap subfolders | █ | | | | | | █ | | | █ | | • |
| index.html blocks list | | | | | | | █ | | | • | | |
| write-only folders | █ | | | | | | | | | | █ | █ |
| files stored as-is | █ | █ | █ | █ | | █ | █ | | | █ | █ | █ |
| file versioning | | | | █ | █ | | | | | | | |
| file encryption | | | | █ | █ | █ | | | | | | █ |
| file indexing | █ | | █ | █ | █ | | | █ | █ | █ | | |
| ┗ per-volume db | █ | | • | • | • | | | • | • | | | |
| ┗ db stored in folder | █ | | | | | | | • | • | █ | | |
| ┗ db stored out-of-tree | █ | | █ | █ | █ | | | • | • | █ | | |
| ┗ existing file tree | █ | | █ | | | | | | | █ | | |
| file action event hooks | █ | | | | | | | | | █ | | █ |
| one-way folder sync | █ | | | █ | █ | █ | | | | | | |
| full sync | | | | █ | █ | | | | | | | |
| speed throttle | | █ | █ | | | █ | | | █ | | | █ |
| anti-bruteforce | █ | █ | █ | █ | █ | | | | • | | | █ |
| dyndns updater | | █ | | | | | | | | | | |
| self-updater | | | █ | | | | | | | | | |
| log rotation | █ | | █ | █ | █ | | | • | █ | | | █ |
| upload tracking / log | █ | █ | • | █ | █ | | | █ | █ | | | |
| curl-friendly ls | █ | | | | | | | | | | | |
| curl-friendly upload | █ | | | | | | | | | | | |
* `unmap subfolders` = "shadowing"; mounting a local folder in the middle of an existing filesystem tree in order to disable access below that path
* `files stored as-is` = uploaded files are trivially readable from the server HDD, not sliced into chunks or in weird folder structures or anything like that
@@ -301,7 +288,6 @@ symbol legend,
* `curl-friendly ls` = returns a [sortable plaintext folder listing](https://user-images.githubusercontent.com/241032/215322619-ea5fd606-3654-40ad-94ee-2bc058647bb2.png) when curled
* `curl-friendly upload` = uploading with curl is just `curl -T some.bin http://.../`
* `a`/copyparty remarks:
* single-sign-on, token-auth, and 2fa is *possible* through authelia/authentik or similar, but nobody's made an example yet
* one-way folder sync from local to server can be done efficiently with [u2c.py](https://github.com/9001/copyparty/tree/hovudstraum/bin#u2cpy), or with webdav and conventional rsync
* can hot-reload config files (with just a few exceptions)
* can set per-folder permissions if that folder is made into a separate volume, so there is configuration overhead
@@ -314,51 +300,49 @@ symbol legend,
* `l`/sftpgo:
* `file action event hooks` also include on-download triggers
* `upload tracking / log` in main logfile
* `m`/arozos:
* `2fa` maybe possible through LDAP/Oauth
## client features
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | | █ |
| themes | █ | █ | | █ | | | | | █ | | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | | |
| multi-column sorting | █ | | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | | █ |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | | █ |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ audio spectrograms | █ | | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | | █ |
| ┗ gapless playback | █ | | | | | | | | • | | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | | █ |
| ┗ video transcoding | | | | | | | | | █ | | | | |
| audio BPM detector | █ | | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | | |
| search by bpm / key | █ | | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | | |
| search in file contents | | | | █ | █ | | | | █ | | | | |
| search by custom parser | █ | | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | | █ |
| markdown viewer | █ | | | | █ | | | | █ | | | | █ |
| markdown editor | █ | | | | █ | | | | █ | | | | █ |
| readme.md in listing | █ | | | █ | | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | | █ |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | | █ |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | | █ |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ---------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| single-page app | █ | | █ | █ | █ | | | █ | █ | █ | █ | |
| themes | █ | █ | | █ | | | | | █ | | | |
| directory tree nav | █ | | | | █ | | | | █ | | | |
| multi-column sorting | █ | | | | | | | | | | | |
| thumbnails | █ | | | | | | | █ | █ | | | |
| ┗ image thumbnails | █ | | | █ | █ | | | █ | █ | █ | | |
| ┗ video thumbnails | █ | | | █ | █ | | | | █ | | | |
| ┗ audio spectrograms | █ | | | | | | | | | | | |
| audio player | █ | | | █ | █ | | | | █ | | | |
| ┗ gapless playback | █ | | | | | | | | • | | | |
| ┗ audio equalizer | █ | | | | | | | | | | | |
| ┗ waveform seekbar | █ | | | | | | | | | | | |
| ┗ OS integration | █ | | | | | | | | | | | |
| ┗ transcode to lossy | █ | | | | | | | | | | | |
| video player | █ | | | █ | █ | | | | █ | █ | | |
| ┗ video transcoding | | | | | | | | | █ | | | |
| audio BPM detector | █ | | | | | | | | | | | |
| audio key detector | █ | | | | | | | | | | | |
| search by path / name | █ | █ | █ | █ | █ | | █ | | █ | █ | | |
| search by date / size | █ | | | | █ | | | █ | █ | | | |
| search by bpm / key | █ | | | | | | | | | | | |
| search by custom tags | | | | | | | | █ | █ | | | |
| search in file contents | | | | █ | █ | | | | █ | | | |
| search by custom parser | █ | | | | | | | | | | | |
| find local file | █ | | | | | | | | | | | |
| undo recent uploads | █ | | | | | | | | | | | |
| create directories | █ | | | █ | █ | | █ | █ | █ | █ | █ | █ |
| image viewer | █ | | | █ | █ | | | | █ | █ | █ | |
| markdown viewer | █ | | | | █ | | | | █ | | | |
| markdown editor | █ | | | | █ | | | | █ | | | |
| readme.md in listing | █ | | | █ | | | | | | | | |
| rename files | █ | █ | █ | █ | █ | | █ | | █ | █ | █ | █ |
| batch rename | █ | | | | | | | | █ | | | |
| cut / paste files | █ | █ | | █ | █ | | | | █ | | | |
| move files | █ | █ | | █ | █ | | █ | | █ | █ | █ | |
| delete files | █ | █ | | █ | █ | | █ | █ | █ | █ | █ | █ |
| copy files | | | | | █ | | | | █ | █ | █ | |
* `single-page app` = multitasking; possible to continue navigating while uploading
* `audio player » os-integration` = use the [lockscreen](https://user-images.githubusercontent.com/241032/142711926-0700be6c-3e31-47b3-9928-53722221f722.png) or [media hotkeys](https://user-images.githubusercontent.com/241032/215347492-b4250797-6c90-4e09-9a4c-721edf2fb15c.png) to play/pause, prev/next song
@@ -374,14 +358,14 @@ symbol legend,
## integration
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l | m |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | | |
| flameshot | | | | | | █ | | | | | | | |
| feature / software | a | b | c | d | e | f | g | h | i | j | k | l |
| ----------------------- | - | - | - | - | - | - | - | - | - | - | - | - |
| OS alert on upload | █ | | | | | | | | | | | |
| discord | █ | | | | | | | | | | | |
| ┗ announce uploads | █ | | | | | | | | | | | |
| ┗ custom embeds | | | | | | | | | | | | |
| sharex | █ | | | █ | | █ | | █ | | | | |
| flameshot | | | | | | █ | | | | | | |
* sharex `` = yes, but does not provide example sharex config
* `a`/copyparty remarks:
@@ -407,7 +391,6 @@ symbol legend,
| filebrowser | go | █ apl2 | 20 MB |
| filegator | php | █ mit | • |
| sftpgo | go | ‼ agpl | 44 MB |
| arozos | go | ░ gpl3 | 531 MB |
| updog | python | █ mit | 17 MB |
| goshs | go | █ mit | 11 MB |
| gimme-that | python | █ mit | 4.8 MB |
@@ -519,14 +502,12 @@ symbol legend,
* ✅ token auth (api keys)
## [kodbox](https://github.com/kalcaddle/kodbox)
* this thing is insane (but is getting competition from [arozos](#arozos))
* this thing is insane
* php; [docker](https://hub.docker.com/r/kodcloud/kodbox)
* 🔵 *upload segmenting, acceleration, and integrity checking!*
* ⚠️ but uploads are not resumable(?)
* ⚠️ not portable
* ⚠️ isolated on-disk file hierarchy, incompatible with other software
* ⚠️ uploading small files to copyparty is 16x faster
* ⚠️ uploading large files to copyparty is 3x faster
* ⚠️ http/webdav only; no ftp or zeroconf
* ⚠️ some parts of the GUI are in chinese
* ✅ fantastic ui/ux
@@ -586,24 +567,6 @@ symbol legend,
* ✅ on-download event hook (otherwise same as copyparty)
* ✅ more extensive permissions control
## [arozos](https://github.com/tobychui/arozos)
* big suite of applications similar to [kodbox](#kodbox), copyparty is better at downloading/uploading/music/indexing but arozos has other advantages
* go; primarily linux (limited support for windows)
* ⚠️ uploads not resumable / integrity-checked
* ⚠️ uploading small files to copyparty is 2.7x faster
* ⚠️ uploading large files to copyparty is at least 10% faster
* arozos is websocket-based, 512 KiB chunks; writes each chunk to separate files and then merges
* copyparty splices directly into the final file; faster and better for the HDD and filesystem
* ⚠️ no directory tree navpane; not as easy to navigate
* ⚠️ download-as-zip is not streaming; creates a temp.file on the server
* ⚠️ not self-contained (pulls from jsdelivr)
* ⚠️ has an audio player, but supports less filetypes
* ⚠️ limited support for configuring real-ip detection
* ✅ sftp server
* ✅ settings gui
* ✅ good-looking gui
* ✅ an IDE, msoffice viewer, rich host integration, much more
## [updog](https://github.com/sc0tfree/updog)
* python; cross-platform
* basic directory listing with upload feature

View File

@@ -1,45 +0,0 @@
when running behind a reverse-proxy, or a WAF, or another protection service such as cloudflare:
if you (and maybe everybody else) keep getting a message that says `thank you for playing`, then you've gotten banned for malicious traffic. This ban applies to the IP-address that copyparty *thinks* identifies the shady client -- so, depending on your setup, you might have to tell copyparty where to find the correct IP
knowing the correct IP is also crucial for some other features, such as the unpost feature which lets you delete your own recent uploads -- but if everybody has the same IP, well...
----
for most common setups, there should be a helpful message in the server-log explaining what to do, something like `--xff-src=10.88.0.0/16` or `--xff-src=lan` to accept the `X-Forwarded-For` header from your reverse-proxy with a LAN IP of `10.88.x.y`
if you are behind cloudflare, it is recommended to also set `--xff-hdr=cf-connecting-ip` to use a more trustworthy source of info, but then it's also very important to ensure your reverse-proxy does not accept connections from anything BUT cloudflare; you can do this by generating an ip-address allowlist and reject all other connections
* if you are using nginx as your reverse-proxy, see the [example nginx config](https://github.com/9001/copyparty/blob/hovudstraum/contrib/nginx/copyparty.conf) on how the cloudflare allowlist can be done
----
the server-log will give recommendations in the form of commandline arguments;
to do the same thing using config files, take the options that are suggested in the serverlog and put them into the `[global]` section in your `copyparty.conf` like so:
```yaml
[global]
xff-src: lan
xff-hdr: cf-connecting-ip
```
----
# but if you just want to get it working:
...and don't care about security, you can optionally disable the bot-detectors, either by specifying commandline-args `--ban-404=no --ban-403=no --ban-422=no --ban-url=no --ban-pw=no`
or by adding these lines inside the `[global]` section in your `copyparty.conf`:
```yaml
[global]
ban-404: no
ban-403: no
ban-422: no
ban-url: no
ban-pw: no
```
but remember that this will make other features insecure as well, such as unpost

48
pyoxidizer.bzl Normal file
View File

@@ -0,0 +1,48 @@
# builds win7-i386 exe on win10-ltsc-1809(17763.316)
# see docs/pyoxidizer.txt
def make_exe():
dist = default_python_distribution(flavor="standalone_static", python_version="3.8")
policy = dist.make_python_packaging_policy()
policy.allow_files = True
policy.allow_in_memory_shared_library_loading = True
#policy.bytecode_optimize_level_zero = True
#policy.include_distribution_sources = False # error instantiating embedded Python interpreter: during initializing Python main: init_fs_encoding: failed to get the Python codec of the filesystem encoding
policy.include_distribution_resources = False
policy.include_non_distribution_sources = False
policy.include_test = False
python_config = dist.make_python_interpreter_config()
#python_config.module_search_paths = ["$ORIGIN/lib"]
python_config.run_module = "copyparty"
exe = dist.to_python_executable(
name="copyparty",
config=python_config,
packaging_policy=policy,
)
exe.windows_runtime_dlls_mode = "never"
exe.windows_subsystem = "console"
exe.add_python_resources(exe.read_package_root(
path="sfx",
packages=[
"copyparty",
"jinja2",
"markupsafe",
"pyftpdlib",
"python-magic",
]
))
return exe
def make_embedded_resources(exe):
return exe.to_embedded_resources()
def make_install(exe):
files = FileManifest()
files.add_python_resource("copyparty", exe)
return files
register_target("exe", make_exe)
register_target("resources", make_embedded_resources, depends=["exe"], default_build_script=True)
register_target("install", make_install, depends=["exe"], default=True)
resolve_targets()

View File

@@ -28,7 +28,6 @@ classifiers = [
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: Jython",
"Programming Language :: Python :: Implementation :: PyPy",
"Operating System :: OS Independent",
"Environment :: Console",
"Environment :: No Input/Output (Daemon)",
"Intended Audience :: End Users/Desktop",
@@ -49,7 +48,6 @@ thumbnails2 = ["pyvips"]
audiotags = ["mutagen"]
ftpd = ["pyftpdlib"]
ftps = ["pyftpdlib", "pyopenssl"]
tftpd = ["partftpy>=0.3.1"]
pwhash = ["argon2-cffi"]
[project.scripts]

View File

@@ -1,11 +1,11 @@
FROM alpine:3.18
WORKDIR /z
ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
ver_hashwasm=4.10.0 \
ver_hashwasm=4.9.0 \
ver_marked=4.3.0 \
ver_dompf=3.1.2 \
ver_dompf=3.0.5 \
ver_mde=2.18.0 \
ver_codemirror=5.65.16 \
ver_codemirror=5.65.12 \
ver_fontawesome=5.13.0 \
ver_prism=1.29.0 \
ver_zopfli=1.0.3
@@ -24,9 +24,7 @@ ENV ver_asmcrypto=c72492f4a66e17a0e5dd8ad7874de354f3ccdaa5 \
# the scp url is regular latin from https://fonts.googleapis.com/css2?family=Source+Code+Pro&display=swap
RUN mkdir -p /z/dist/no-pk \
&& wget https://fonts.gstatic.com/s/sourcecodepro/v11/HI_SiYsKILxRpg3hIP6sJ7fM7PqlPevW.woff2 -O scp.woff2 \
&& apk add \
bash brotli cmake make g++ git gzip lame npm patch pigz \
python3 python3-dev py3-brotli sox tar unzip wget \
&& apk add cmake make g++ git bash npm patch wget tar pigz brotli gzip unzip python3 python3-dev brotli py3-brotli \
&& rm -f /usr/lib/python3*/EXTERNALLY-MANAGED \
&& wget https://github.com/openpgpjs/asmcrypto.js/archive/$ver_asmcrypto.tar.gz -O asmcrypto.tgz \
&& wget https://github.com/markedjs/marked/archive/v$ver_marked.tar.gz -O marked.tgz \
@@ -60,11 +58,6 @@ RUN mkdir -p /z/dist/no-pk \
&& tar -xf zopfli.tgz
COPY busy-mp3.sh /z/
RUN /z/busy-mp3.sh \
&& mv -v /dev/shm/busy.mp3.gz /z/dist
# build fonttools (which needs zopfli)
RUN tar -xf zopfli.tgz \
&& cd zopfli* \
@@ -87,7 +80,7 @@ RUN cd asmcrypto.js-$ver_asmcrypto \
# build hash-wasm
RUN cd hash-wasm/dist \
RUN cd hash-wasm \
&& mv sha512.umd.min.js /z/dist/sha512.hw.js
@@ -150,8 +143,9 @@ RUN ./genprism.sh $ver_prism
# compress
COPY zopfli.makefile /z/dist/
COPY brotli.makefile zopfli.makefile /z/dist/
RUN cd /z/dist \
&& make -j$(nproc) -f brotli.makefile \
&& make -j$(nproc) -f zopfli.makefile \
&& rm *.makefile \
&& mv no-pk/* . \

View File

@@ -0,0 +1,4 @@
all: $(addsuffix .br, $(wildcard easymde*))
%.br: %
brotli -jZ $<

View File

@@ -1,61 +0,0 @@
#!/bin/bash
set -e
cat >/dev/null <<EOF
a frame is 1152 samples
1sec @ 48000 = 41.66 frames
11 frames = 12672 samples = 0.264 sec
22 frames = 25344 samples = 0.528 sec
EOF
fast=1
fast=
echo
mkdir -p /dev/shm/$1
cd /dev/shm/$1
find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete
min=99999999
for freq in 425; do # {400..500}
for vol in 0; do # {10..30}
for kbps in 32; do
for fdur in 1124; do # {800..1200}
for fdu2 in 1042; do # {800..1200}
for ftyp in h; do # q h t l p
for ofs1 in 9214; do # {0..32768}
for ofs2 in 0; do # {0..4096}
for ofs3 in 0; do # {0..4096}
for nores in --nores; do # '' --nores
f=a.b$kbps$nores-f$freq-v$vol-$ftyp$fdur-$fdu2-o$ofs1-$ofs2-$ofs3
sox -r48000 -Dn -r48000 -b16 -c2 -t raw s1.pcm synth 25344s sin $freq vol 0.$vol fade $ftyp ${fdur}s 25344s ${fdu2}s
sox -r48000 -Dn -r48000 -b16 -c2 -t raw s0.pcm synth 12672s sin $freq vol 0
tail -c +$ofs1 s0.pcm >s0a.pcm
tail -c +$ofs2 s0.pcm >s0b.pcm
tail -c +$ofs3 s0.pcm >s0c.pcm
cat s{0a,1,0,0b,1,0c}.pcm > a.pcm
lame --silent -r -s 48 --bitwidth 16 --signed a.pcm -m j --resample 48 -b $kbps -q 0 $nores $f.mp3
if [ $fast ]
then gzip -c9 <$f.mp3 >$f.mp3.gz
else pigz -c11 -I1 <$f.mp3 >$f.mp3.gz
fi
sz=$(wc -c <$f.mp3.gz)
printf '\033[A%d %s\033[K\n' $sz $f
[ $sz -le $((min+10)) ] && echo
[ $sz -le $min ] && echo && min=$sz
done;done;done;done;done;done;done;done;done;done
true
f=a.b32--nores-f425-v0-h1124-1042-o9214-0-0.mp3
[ $fast ] &&
pigz -c11 -I1 <$f >busy.mp3.gz ||
mv $f.gz busy.mp3.gz
sz=$(wc -c <busy.mp3.gz)
[ "$sz" -eq 106 ] &&
echo busy.mp3 built successfully ||
echo WARNING: unexpected size of busy.mp3
find -maxdepth 1 -type f -iname 'a.*.mp3*' -delete

View File

@@ -1,6 +1,6 @@
all: $(addsuffix .gz, $(wildcard *.js *.css))
%.gz: %
pigz -11 -I 2048 $<
pigz -11 -I 573 $<
# pigz -11 -J 34 -I 100 -F < $< > $@.first

View File

@@ -1,4 +1,4 @@
FROM fedora:39
FROM fedora:38
WORKDIR /z
LABEL org.opencontainers.image.url="https://github.com/9001/copyparty" \
org.opencontainers.image.source="https://github.com/9001/copyparty/tree/hovudstraum/scripts/docker" \
@@ -21,7 +21,7 @@ RUN dnf install -y \
vips vips-jxl vips-poppler vips-magick \
python3-numpy fftw libsndfile \
gcc gcc-c++ make cmake patchelf jq \
python3-devel ffmpeg-devel fftw-devel libsndfile-devel python3-setuptools python3-wheel \
python3-devel ffmpeg-devel fftw-devel libsndfile-devel python3-setuptools \
vamp-plugin-sdk qm-vamp-plugins \
vamp-plugin-sdk-devel vamp-plugin-sdk-static \
&& rm -f /usr/lib/python3*/EXTERNALLY-MANAGED \
@@ -29,7 +29,7 @@ RUN dnf install -y \
&& bash install-deps.sh \
&& dnf erase -y \
gcc gcc-c++ make cmake patchelf jq \
python3-devel ffmpeg-devel fftw-devel libsndfile-devel python3-setuptools python3-wheel \
python3-devel ffmpeg-devel fftw-devel libsndfile-devel python3-setuptools \
vamp-plugin-sdk-devel vamp-plugin-sdk-static \
&& dnf clean all \
&& find /usr/ -name __pycache__ | xargs rm -rf \

View File

@@ -33,8 +33,6 @@ the recommended way to configure copyparty inside a container is to mount a fold
* but you can also provide arguments to the docker command if you prefer that
* config files must be named `something.conf` to get picked up
also see [docker-specific recommendations](#docker-specific-recommendations)
## editions
@@ -81,35 +79,6 @@ or using commandline arguments,
```
# faq
the following advice is best-effort and not guaranteed to be entirely correct
* q: starting a rootless container on debian 12 fails with `failed to register layer: lsetxattr user.overlay.impure /etc: operation not supported`
* a: docker's default rootless configuration on debian is to use the overlay2 storage driver; this does not work. Your options are to replace docker with podman (good choice), or to configure docker to use the `fuse-overlayfs` storage driver
# docker-specific recommendations
* copyparty will generally create a `.hist` folder at the top of each volume, which contains the filesystem index, thumbnails and such. For performance reasons, but also just to keep things tidy, it might be convenient to store these inside the config folder instead. Add the line `hist: /cfg/hists/` inside the `[global]` section of your `copyparty.conf` to do this
## enabling the ftp server
...is tricky because ftp is a weird protocol and docker is making it worse 🎉
add the following three config entries into the `[global]` section of your `copyparty.conf`:
* `ftp: 3921` to enable the service, listening for connections on port 3921
* `ftp-nat: 127.0.0.1` but replace `127.0.0.1` with the actual external IP of your server; the clients will only be able to connect to this IP, even if the server has multiple IPs
* `ftp-pr: 12000-12099` to restrict the [passive-mode](http://slacksite.com/other/ftp.html#passive) port selection range; this allows up to 100 simultaneous file transfers
then finally update your docker config so that the port-range you specified (12000-12099) is exposed to the internet
# build the images yourself
basically `./make.sh hclean pull img push` but see [devnotes.md](./devnotes.md)

Some files were not shown because too many files have changed in this diff Show More