ed
753e3cfbaf
revert 68c6794d (v1.6.2) and fix it better:
...
moving deduplicated files between volumes could drop some links
2023-10-07 22:25:44 +00:00
ed
0dc3c23b42
add alternative filekey generator; closes #52
2023-10-06 13:41:22 +00:00
ed
a62f744a18
prevent losing an out-of-volume index
...
if the server is started while an external drive is not mounted,
it would drop the database because all the files are missing
2023-09-22 23:05:07 +00:00
ed
c1efd227b7
fix inconsistent use of symlink mtimes in database;
...
on upload, dupes are by default handled by symlinking to the existing
copy on disk, writing the uploader's local mtime into the symlink mtime,
which is also what gets indexed in the db
this worked as intended, however during an -e2dsa rescan on startup the
symlink destination timestamps would be used instead, causing a reindex
and the resulting loss of uploader metadata (ip, timestamp)
will now always use the symlink's mtime;
worst-case 1% slower startup (no dhash)
this change will cause a reindex of incorrectly indexed files, however
as this has already happened at least once due to the bug being fixed,
there will be no additional loss of metadata
2023-09-01 20:29:55 +00:00
ed
5a38311481
mark offline volumes in directory tree sidebar
2023-08-26 19:00:46 +00:00
ed
cafe53c055
v1.9.0
2023-08-20 22:02:40 +00:00
ed
135ece3fbd
immediately allow uploading an interrupted and
...
deleted incomplete upload to another location
2023-08-20 19:16:35 +00:00
ed
bd3640d256
change to openmetrics
2023-08-20 18:50:14 +00:00
ed
fc0405c8f3
add prometheus metrics; closes #49
2023-08-20 17:58:06 +00:00
ed
015b87ee99
performance / cosmetic:
...
* js: use .call instead of .bind when possible
* when running without e2d, the message on startup regarding
unfinished uploads didn't show the correct filesystem path
2023-08-16 19:32:43 +00:00
ed
406e413594
hint at additional context in exceptions
2023-08-15 01:42:13 +02:00
ed
1441ccee4f
v1.8.4
2023-07-18 07:46:22 +00:00
ed
8f59afb159
fix another race (unpost):
...
unposting could collide with most other database-related activities,
causing one or the other to fail.
luckily the unprotected query performed by the unpost API happens to be
very cheap, so also the most likely to fail, and would succeed upon a
manual reattempt from the UI.
even in the worst case scenario, there would be no unrecoverable damage
as the next rescan would auto-repair any resulting inconsistencies.
2023-07-14 15:21:14 +00:00
ed
77f1e51444
fix unlikely race (e2tsr):
...
if someone with admin rights refreshes the homepage exactly as the
directory indexer decides to `_drop_caches`, the indexer thread would
die and the up2k instance would become inoperable...
luckily the probability of hitting this by chance is absolutely minimal,
and the worst case scenario is having to restart copyparty if this
happens immediately after startup; there is no risk of database damage
2023-07-14 15:20:25 +00:00
ed
b54b7213a7
more thumbnailer configs available as volflags:
...
--th-convt = convt
--th-no-crop = nocrop
--th-size = thsize
2023-07-11 22:15:37 +00:00
ed
e197895c10
support hashed passwords; closes #39
2023-06-25 21:50:33 +00:00
ed
8b0cf2c982
volflags to limit volume size / num files; closes #40
2023-06-19 00:42:45 +00:00
ed
e64b87b99b
dont hardlink symlinks (they could be relative)
2023-05-12 20:41:09 +00:00
ed
b1359f039f
linter cleanup
2023-05-07 14:38:30 +00:00
ed
83178d0836
preserve empty folders ( closes #23 ):
...
* when deleting files, do not cascade upwards through empty folders
* when moving folders, also move any empty folders inside
the only remaining action which autoremoves empty folders is
files getting deleted as they expire volume lifetimes
also prevents accidentally moving parent folders into subfolders
(even though that actually worked surprisingly well)
2023-04-29 11:30:43 +00:00
ed
cb6de0387d
a bit faster
2023-04-26 19:56:27 +00:00
ed
209b49d771
remind sqlite we have indexes
2023-03-30 21:45:58 +00:00
ed
e0c4829ec6
verify covers against db instead of fs
2023-03-15 19:48:43 +00:00
ed
b0cc396bca
v1.6.8
2023-03-12 16:10:07 +00:00
ed
2be2e9a0d8
index folder thumbs in db
2023-03-11 11:43:29 +00:00
ed
974ca773be
just to be extra sure
2023-03-09 21:49:29 +00:00
ed
c98fff1647
fix chunkpost-handshake race (affects --no-dedup only);
...
a handshake arriving in the middle of the final chunk could cause
dupes to become empty -- worst case leading to loss of data
2023-03-05 19:45:50 +00:00
ed
db95e880b2
thats not how it works
2023-02-28 22:19:06 +00:00
ed
1c2c9db8f0
retain upload time (but not ip) on file reindex
2023-02-26 19:09:24 +00:00
ed
7ea183baef
let http thread handle upload verification plugins
2023-02-26 19:07:49 +00:00
ed
ab87fac6d8
db got the wrong lastmod when linking dupes
2023-02-26 18:52:04 +00:00
ed
1e3b7eee3b
dont rmdir volume top on cleanup
2023-02-26 18:28:37 +00:00
ed
4de028fc3b
let controlpanel rescan button override lack of e2dsa
2023-02-26 18:27:10 +00:00
ed
604e5dfaaf
improve error handling / messages
2023-02-26 18:26:13 +00:00
ed
05e0c2ec9e
add xiu (batching hook; runs on idle after uploads) +
...
bunch of tweaks/fixes for hooks
2023-02-26 18:23:32 +00:00
ed
6eba9feffe
condense uploads listing on view change
2023-02-14 21:58:15 +00:00
ed
8f61e1568c
transcode chiptunes to opus;
...
* new audio/MPT formats: apac bonk dfpwm ilbc it itgz itr itz mo3 mod mptm mt2 okt s3gz s3m s3r s3z xm xmgz xmr xmz xpk
* new image/PIL formats: blp dcx emf eps fits flc fli fpx im j2k j2p psd spi wmf
2023-02-11 11:17:37 +00:00
ed
a4b56c74c7
support long filepaths on win7 + misc windows fixes
2023-02-10 18:37:37 +00:00
ed
7e2ff9825e
ensure -e2tsr takes effect by ignoring dhash
2023-02-08 22:33:02 +00:00
ed
2b644fa81b
don't alias randomized filenames
2023-02-04 13:41:43 +00:00
ed
0aa88590d0
should generalize this somehow
2023-02-02 22:35:13 +00:00
ed
e67b695b23
show filekeys in recent-uploads ui
2023-02-02 21:22:51 +00:00
ed
a918c285bf
up2k-ui: button to randomize upload filenames
2023-02-01 22:26:18 +00:00
ed
1e20eafbe0
volflag to randomize all upload filenames
2023-02-01 21:58:01 +00:00
ed
eaa6b3d0be
mute some startup noise
2023-01-29 04:33:28 +00:00
ed
68c6794d33
rewrite other symlinks after the actual move;
...
fixes volumes where symlinking is disabled
2023-01-28 01:14:29 +00:00
ed
e965b9b9e2
mkdir missing volumes on startup
2023-01-27 21:52:28 +00:00
ed
9b7b9262aa
promote dedup control to volflags
2023-01-25 21:46:15 +00:00
ed
f8e3e87a52
add event hooks
2023-01-22 23:35:31 +00:00
ed
8a7135cf41
support fat32 time precision, avoiding rescans
...
posted from warzaw airport otw to japan
2022-12-20 22:19:32 +01:00
ed
79aa40fdea
cosmetic fixes
2022-12-14 23:12:51 +00:00
ed
447ed5ab37
windows fixes
2022-12-12 21:59:50 +00:00
ed
02ad4bfab2
ensure consistency between db tables
2022-12-10 22:13:21 +00:00
ed
56b73dcc8a
up2k: add option to replace existing file
2022-12-10 19:22:16 +00:00
ed
7704b9c8a2
sqlite durability profiles
2022-12-10 10:01:33 +00:00
ed
999b7ae919
safer to merge wal on startup instead
2022-12-09 19:58:13 +00:00
ed
195eb53995
merge wal on shutdown
2022-12-07 23:09:40 +00:00
ed
06fa78f54a
windows: set .hist folder hidden
2022-12-07 22:56:30 +00:00
ed
fbed322d3b
option to skip database syncs entirely
2022-12-07 21:35:04 +00:00
ed
9b0f519e4e
switch to wal for ~2x faster uploads
2022-12-07 20:52:17 +00:00
ed
067cc23346
docs + cleanup
2022-12-03 18:58:56 +00:00
ed
89c9f45fd0
add option for cross-volume dedupe
2022-12-02 17:25:37 +00:00
ed
3a800585bc
u2cli: server is allowed to reject dupes
2022-11-29 22:09:32 +00:00
ed
0aa7d12704
add option to disable .hist/up2k.snap
2022-11-15 22:16:53 +00:00
ed
4203fc161b
misc
2022-10-30 16:31:04 +00:00
ed
54e1c8d261
remove 697 GiB upload filesize limit
2022-10-30 12:51:20 +00:00
ed
73bd2df2c6
more metadata-parser debug options
2022-10-29 21:59:59 +00:00
ed
edad3246e0
make pylance happier
2022-10-29 20:40:25 +00:00
ed
4bcd30da6b
cleaner daemon instancing
2022-10-23 12:05:44 +02:00
ed
c003dfab03
unbold ansi grays
2022-10-19 15:30:17 +02:00
ed
20c6b82bec
replace magic numbers with errno.*
2022-10-19 15:21:48 +02:00
ed
118bdcc26e
120x faster folder moves/renames
2022-10-08 19:11:03 +02:00
ed
78fa96f0f4
add unpost sanchk
2022-10-08 18:23:41 +02:00
ed
b8a93e74bf
fix default upload expiration + ux
2022-10-07 21:38:01 +02:00
ed
b299aaed93
fix some cases of deleted files not being forgotten
2022-09-26 22:19:46 +02:00
ed
4f4f106c48
add ability to forget uploads by deleting the files
2022-09-25 14:24:01 +02:00
ed
e70e926a40
support up2k uploads from old browsertabs
2022-09-24 14:35:51 +02:00
ed
0b87a4a810
allow setting lifetimes from up2k ui
2022-09-19 23:49:07 +02:00
ed
09cea66aa8
add ability to set lifetime per-file during upload
2022-09-18 13:12:38 +02:00
ed
5e9bc1127d
fix windows symlink creation
2022-09-17 13:27:54 +02:00
ed
0006f933a2
hmac uploader-ip when avoiding filename collisions
2022-09-11 08:27:45 +02:00
ed
590c763659
add unforgetti beam
2022-09-07 08:09:32 +02:00
ed
11d1267f8c
option to keep files in index when deleted
2022-09-07 01:07:21 +02:00
ed
995cd10df8
bump timeouts for zfs / bursty filesystems
2022-09-04 21:21:54 +02:00
ed
c930d8f773
add mtp debug mode
2022-09-03 19:58:10 +02:00
ed
d48a7d2398
provide tagparsers with uploader info
2022-08-15 20:23:17 +02:00
ed
7a460de3c2
windows db fix
2022-08-15 18:01:28 +02:00
ed
1c69ccc6cd
v1.3.12
2022-08-13 00:58:49 +02:00
ed
9ccd327298
add directory hashing (boots ~3x faster)
2022-08-12 23:17:18 +02:00
ed
11df36f3cf
add option to exit after scanning volumes
2022-08-12 21:20:13 +02:00
ed
ad18b6e15e
stop reindexing empty files on startup
2022-08-12 16:31:36 +02:00
ed
92ed4ba3f8
parallelize python hashing too
2022-08-10 23:12:01 +02:00
ed
4aaa111925
v1.3.9
2022-08-04 00:39:37 +02:00
ed
e31248f018
include version info on startup and in crash dumps
2022-08-04 00:11:52 +02:00
ed
4e7455268a
tag-scanner perf
2022-08-03 22:33:20 +02:00
ed
680f8ae814
add xdev/xvol indexing guards
2022-08-03 22:20:28 +02:00
ed
90555a4cea
clean-shutdown while hashing huge files
2022-08-03 21:06:10 +02:00
ed
5384c2e0f5
reentrant cleanup
2022-08-02 20:56:05 +02:00
ed
0ee574a718
forget uploads that failed to initialize
2022-08-02 06:15:18 +02:00
ed
dac2fad48e
v1.3.8
2022-07-27 16:07:26 +02:00
ed
77f624b01e
improve shumantime + use it everywhere
2022-07-27 15:07:04 +02:00
ed
e24ffebfc8
indicate write-activity on splashpage
2022-07-27 14:53:15 +02:00
ed
660705a436
defer volume reindexing on db activity
2022-07-27 11:48:47 +02:00
ed
76adac7c72
up2k-hook-ytid: add mp4/webm/mkv metadata scanner
2022-07-26 22:09:18 +02:00
ed
5dc75ebb67
async e2ts / e2v + forget deleted shadowed
2022-07-26 12:47:40 +02:00
ed
d686ce12b6
lsof db on stuck transaction
2022-07-25 02:07:59 +02:00
ed
10430b347f
fix dumb prisonparty bug
2022-07-22 20:49:35 +02:00
ed
4a76663fb2
ensure free disk space
2022-07-17 22:33:08 +02:00
ed
3fa377a580
sqlite diag
2022-07-16 20:43:26 +02:00
ed
c2a0b1b4c6
autopotato
2022-07-15 02:39:32 +02:00
ed
48b957f1d5
add -e2v (file integrity checker)
2022-07-13 00:48:39 +02:00
ed
3683984c8d
abort volume indexing on ^C
2022-07-12 21:46:07 +02:00
ed
6f75b02723
misc
2022-07-12 03:16:30 +02:00
ed
d1a03c6d17
zerobyte semantics
2022-07-11 23:17:33 +02:00
ed
c6c31702c2
cheaper file deletion
2022-07-11 01:50:18 +02:00
ed
c5b04f6fef
mtp daisychaining
2022-07-08 22:29:05 +02:00
ed
4792c2770d
fix a spin
2022-07-03 02:39:15 +02:00
ed
1c0071d9ce
perf
2022-07-03 01:40:30 +02:00
ed
5d3034c231
detect sparse support from st_blocks
2022-06-23 18:23:42 +02:00
ed
fe67c52ead
configurable list of sparse-supporting filesystems +
...
close nonsparse files after each write to force flush
2022-06-19 22:38:52 +02:00
ed
190e11f7ea
update deps + misc
2022-06-16 21:43:40 +02:00
ed
62fbb04c9d
allow moving files between filesystems
2022-06-16 18:46:50 +02:00
ed
728dc62d0b
optimize nonsparse uploads (fat32, exfat, hpfs)
2022-06-16 17:51:42 +02:00
ed
438384425a
add types, isort, errorhandling
2022-06-16 01:07:15 +02:00
ed
4e5a323c62
more cleanup
2022-06-08 01:05:35 +02:00
ed
4c4b3790c7
fix read-spin on d/c during json post + errorhandling
2022-06-07 19:02:52 +02:00
ed
bd60b464bb
fix misleading log-msg
2022-06-07 14:12:55 +02:00
ed
0143380306
help the query planner
2022-05-13 01:41:39 +02:00
ed
1d9f9f221a
louder
2022-05-12 20:55:37 +02:00
ed
2075a8b18c
skip nonregular files when indexing filesystem
2022-05-09 19:56:17 +00:00
ed
4878eb2c45
support symlinks as volume root
2022-04-30 20:26:26 +02:00
ed
0d9567575a
avoid hashing busy uploads during rescan
2022-04-29 23:16:23 +02:00
ed
e82f176289
fix deadlock on rescan during upload
2022-04-29 23:14:51 +02:00
ed
2933dce3ef
mtime blank uploads + helptext
2022-04-24 22:58:11 +02:00
ed
f35e41baf1
allow unposting with write-only access
2022-04-16 23:35:04 +02:00
ed
36c20bbe53
fix setting mtime on windows
2022-04-15 20:08:55 +02:00
ed
cba9e5b669
add hardlinks (symlink alternative) for up2k dedup
2022-04-15 19:13:53 +02:00
ed
951fdb27ca
dont scan orphaned volumes
2022-04-14 17:11:51 +02:00
ed
f0988c0e32
filter some volflags from up2k dump
2022-04-08 21:56:24 +02:00
ed
c10c70c1e5
misc
2022-03-04 21:30:31 +01:00
ed
1044aa071b
deal with consecutive dupes even without sqlite
2021-12-06 23:51:44 +01:00
ed
241ef5b99d
preserve mtimes when juggling symlinks
2021-12-04 01:58:04 +01:00
ed
c9b1c48c72
sizelimit registry + persist without e2d
2021-11-16 21:31:24 +01:00
ed
12b3a62586
fix dumb mistakes
2021-11-15 20:13:16 +01:00
ed
b206c5d64e
handle multiple simultaneous uploads of the same file
2021-11-14 15:03:11 +01:00
ed
807f272895
missed one
2021-11-06 18:33:32 +01:00
ed
f050647b43
rescan volumes on sigusr1
2021-11-06 18:20:31 +01:00
ed
f5f9e3ac97
reduce rescan/lifetime wakeups
2021-10-27 22:23:03 +02:00
ed
744727087a
better rmtree semantics
2021-10-27 09:40:20 +02:00
ed
6dade82d2c
run tag scrapers in parallel on new uploads
2021-10-27 00:47:50 +02:00
ed
6b737bf1d7
abort tagging if the file has poofed
2021-10-27 00:11:58 +02:00
ed
e4ca4260bb
support mounting entire disks on windows
2021-10-20 00:51:00 +02:00
ed
4f97d7cf8d
normalize collision suffix
2021-10-19 00:49:35 +02:00
ed
088899d59f
fix unpost in jumpvols
2021-10-18 21:08:31 +02:00
ed
2f021a0c2b
skip indexing files by regex
2021-10-12 01:40:19 +02:00
ed
d1be37a04a
nice
2021-10-09 01:33:27 +02:00
ed
b0fd8bf7d4
optimize indexer for huge filesystems
2021-10-09 01:24:19 +02:00
ed
a6935b0293
allow uploading empty files
2021-10-02 23:34:12 +02:00
ed
5886a42901
url escaping
2021-09-26 16:59:02 +02:00
ed
f10d03c238
add --no-symlink
2021-09-26 13:49:29 +02:00
ed
f9a66ffb0e
up2k: fully parallelize handshakes/uploads
2021-09-26 12:57:16 +02:00
ed
0bb9154747
catch more tagparser panics
2021-09-26 03:56:30 +02:00
ed
30c3f45072
fix deleting recently uploaded files without e2d
2021-09-26 03:45:16 +02:00
ed
0d5ca67f32
up2k-srv: add option to reuse file-handles
2021-09-26 03:44:22 +02:00
ed
076d32dee5
up2k-srv: try all dupes for matching path
2021-09-24 19:21:19 +02:00
ed
32a1c7c5d5
cosmetic
2021-09-18 02:07:29 +02:00
ed
0e884df486
keep empty folders after deleting all files
2021-09-15 23:31:49 +02:00
ed
02e3d44998
fix move/delete without -e2d (thx exci)
2021-09-07 21:20:34 +02:00
ed
21662c9f3f
error-message cleanup
2021-09-07 20:51:07 +02:00
ed
ca16c4108d
add options to disallow renaming/moving dotfiles
2021-09-06 00:17:35 +02:00
ed
4802f8cf07
better msg when unposting a deleted file
2021-08-24 00:24:50 +02:00
ed
5f42237f2c
v0.13.5
2021-08-16 08:40:26 +02:00
ed
c1d77e1041
add upload lifetimes
2021-08-09 22:17:41 +02:00
ed
8d755d41e0
per-volume rescan interval
2021-08-09 01:31:20 +02:00
ed
ee86b06676
compat + perf + ux
2021-08-08 04:02:58 +02:00
ed
9a45549b66
adding upload rules
2021-08-07 03:45:50 +02:00
ed
ae3a01038b
v0.12.12
2021-08-06 11:10:04 +02:00
ed
451d757996
fix renaming single symlinks
2021-08-03 20:12:51 +02:00
ed
a918cc67eb
only drop tags when its safe
2021-07-31 13:19:02 +02:00
ed
c164fc58a2
add unpost
2021-07-29 23:53:08 +02:00
ed
0c625a4e62
store upload ip and time
2021-07-29 00:30:10 +02:00
ed
129d33f1a0
mv/del: recursive rmdir
2021-07-27 19:15:58 +02:00
ed
1ad7a3f378
await and monitor workers on startup
2021-07-27 15:48:00 +00:00
ed
7671d791fa
rename works + more symlink fixes
2021-07-26 17:44:20 +02:00
ed
980c6fc810
add scheduled rescans + fix mv bugs
2021-07-26 02:34:56 +02:00
ed
fb40a484c5
mv(folder) works
2021-07-26 01:26:58 +02:00
ed
daa9dedcaa
rm works
2021-07-26 00:29:28 +02:00
ed
e648252479
mv works (at least in trivial cases)
2021-07-25 21:15:43 +02:00
ed
27cce086c6
fileman ui
2021-07-25 01:09:14 +02:00
ed
05a1a31cab
too soon
2021-07-24 22:20:02 +02:00
ed
d020527c6f
centralize mojibake support stuff
2021-07-24 21:56:55 +02:00
ed
4451485664
mv/rm (serverside), 100% untested
2021-07-24 20:08:31 +02:00
ed
a4e1a3738a
more deletion progress
2021-07-23 23:42:07 +02:00
ed
e3684e25f8
treat symlinks as regular files in db
2021-07-22 19:34:40 +02:00
ed
1359213196
prefer native sqlite3 backup (journal-aware)
2021-07-22 19:10:42 +02:00
ed
8eb8043a3d
fix 3rdparty namecase
2021-07-18 22:50:29 +02:00
ed
ebc9de02b0
case-insensitive tag search
2021-07-18 21:34:36 +02:00
ed
3ca41be2b4
do up2k snapshot on shutdown
2021-07-17 14:48:35 +02:00
ed
bac301ed66
get rid of iffy default-args
2021-07-12 00:15:13 +02:00
ed
8fcde2a579
move tcp accept into mp-worker
2021-07-09 15:49:36 +02:00
