| 01:44 |
deedbot |
http://thewhet.net/2017/yankee-doodle-henry-part-iv/ << The Whet - Yankee Doodle Henry, Part IV |
| 01:50 |
mircea_popescu |
!!up sdfsdfasdf |
| 01:50 |
deedbot |
sdfsdfasdf voiced for 30 minutes. |
| 02:04 |
mircea_popescu |
in a shocking development... |
| |
~ 6 hours 40 minutes ~ |
| 08:44 |
BingoBoingo |
kek |
| 08:50 |
BingoBoingo |
http://qntra.net/2017/02/automotive-insurance-premiums-surging-in-united-states-texting-blamed/#comment-88212 |
| 09:04 |
mircea_popescu |
!!up mark_00123 |
| 09:04 |
deedbot |
mark_00123 voiced for 30 minutes. |
| 09:04 |
mark_00123 |
hi mircea_popescu |
| 09:04 |
mircea_popescu |
ello |
| 09:05 |
mircea_popescu |
!!up btcreal |
| 09:05 |
deedbot |
btcreal voiced for 30 minutes. |
| |
~ 1 hours 6 minutes ~ |
| 10:11 |
deedbot |
http://trilema.com/2017/walk-hard-the-dewey-cox-story/ << Trilema - Walk Hard: The Dewey Cox Story |
| |
~ 43 minutes ~ |
| 10:54 |
mircea_popescu |
"A 17-year-old transgender boy won a Texas state girls wrestling title on Saturday". |
| 10:57 |
mircea_popescu |
also cnnleaks.com if anyone still somehow gives a shit about the fake media orgs. |
| 11:01 |
asciilifeform |
'Rules to receive the up-to-$10,000 award from Project Veritas - Project Veritas only offers awards for valuable video or other media types which was legally obtained. It is important for the submitter to follow all local, state and federal laws while obtaining video or other media for submission.' << snoar |
| |
~ 23 minutes ~ |
| 11:24 |
mircea_popescu |
myeah |
| 11:27 |
mircea_popescu |
also, apparently trump will not invite un security council to wh gagglemeets either. nor the correspondents dinner. |
| |
~ 19 minutes ~ |
| 11:47 |
mircea_popescu |
and in other holy shit 8chan, petslut.com! https://media.8ch.net/zoo/src/1414820994001.webm |
| |
~ 57 minutes ~ |
| 12:44 |
phf |
http://btcbase.org/log/2017-02-21#1616415 << http://btcbase.org/log/2016-12-03#1577072 |
| 12:44 |
a111 |
Logged on 2017-02-21 22:51 asciilifeform: phf's logger swallows without burping |
| 12:44 |
a111 |
Logged on 2016-12-03 01:35 phf: http://btcbase.org/log/2016-09-28#1549612 << so normal action is ^AACTION ... ^A, turns that when the line is too long it gets cut off (which is normal behavior) but in case of action none of client seem to do the regular split, meanwhile the irc server cutsoff terminating ^A, which breaks most parsers (including mine) |
| 12:45 |
phf |
since then fixed on btcbase |
| |
~ 16 minutes ~ |
| 13:01 |
asciilifeform |
ACHTUNG, PANZERS! |
| |
↖ |
| 13:01 |
asciilifeform |
mircea_popescu, ben_vulpes , mod6 , et al : |
| 13:01 |
asciilifeform |
http://therealbitcoin.org/ml/btc-dev/2017-February/000256.html |
| 13:01 |
asciilifeform |
[BTC-dev] (EXPERIMENTAL) Blackhole Read Timings, and the Verdict. |
| 13:11 |
asciilifeform |
in other lulz: https://arstechnica.com/security/2017/02/watershed-sha1-collision-just-broke-the-webkit-repository-others-may-follow |
| 13:11 |
asciilifeform |
'Thursday's watershed attack on the widely used SHA1 hashing function has claimed its first casualty: the version control system used by the WebKit browser engine, which became completely corrupted after someone uploaded two proof-of-concept PDF files that have identical message digests.' |
| 13:11 |
asciilifeform |
^ guess what the 'fix' was. |
| 13:11 |
asciilifeform |
^ ... hardcoded check for the published example !! |
| 13:12 |
asciilifeform |
in very other lulz, http://www.metzdowd.com/pipermail/cryptography/2017-February/031615.html ( https://archive.is/jLUGT ) << 'Bruce Schneier has recently published an impassioned plea for a United States Federal Internet Security Agency, which would likely gain control of civilian cryptography, among many other munitions.' |
| 13:22 |
asciilifeform |
lol block 454862 , 91.61kB ! >> ProcessBlock (res == 1) took : 11081ms; db write wait: 2402ms; db read wait: 597ms |
| |
~ 38 minutes ~ |
| 14:00 |
asciilifeform |
in VERY other noose, |
| 14:00 |
asciilifeform |
http://nosuchlabs.com/pub/nonblockverifying_blackhole.txt |
| 14:00 |
asciilifeform |
behold! |
| 14:01 |
asciilifeform |
finally caught this beast in the wild, in real time |
| 14:01 |
asciilifeform |
a blackhole certifiably NOT connected with block verification. |
| 14:01 |
asciilifeform |
we start with where 454862 was received, and (in record time, as it was tiny) accepted. |
| 14:02 |
asciilifeform |
grep for 'received block' -- it does not appear again in this log. because we are blackholed OUTSIDE of the verification delays. |
| 14:03 |
asciilifeform |
instead we see what appears to be a node simply pecked to death by queued-up getdata-for-block flood |
| 14:03 |
asciilifeform |
(grep for 'received getdata for: block') |
| 14:04 |
asciilifeform |
this is the mega-prize, folx, the blackhole that can carry on unabated for hours, days, weeks. |
| 14:05 |
asciilifeform |
for as long as the enemy is able to keep up the 'gimme, gimme, gimme' flood of 'ahahaha, you're giving something to allcomers! well where's mine' |
| 14:06 |
asciilifeform |
incidentally, rationing by ip is a nonstarter, notice that the requests come from a multitude of 'nodes'. |
| 14:10 |
mircea_popescu |
"ProcessBlock (res == 1) took : 167839ms; db write wait: 130117ms; db read wait: 21201ms" lelz |
| 14:14 |
asciilifeform |
the 'type 2' (non-verification) blackhole goes right back to the fundamental question of 'something to all comers', how much disk thrashing does a derp get to invoke simply by coming up with a not-yet-banned ip and a pseudonode. |
| |
↖ |
| 14:17 |
mircea_popescu |
so we're past http://btcbase.org/log/2017-02-24#1617823 ? |
| 14:17 |
a111 |
Logged on 2017-02-24 22:24 asciilifeform: there are no substantial 'other large parts', detectably |
| 14:17 |
asciilifeform |
mircea_popescu: that was in re the verification blackhole (the most common type observed on dulap) |
| 14:17 |
mircea_popescu |
ahem. |
| 14:17 |
asciilifeform |
seems to consist ~entirely of db. |
| 14:17 |
mircea_popescu |
the problem with prb is that it's run by a specific sort of retard. |
| 14:18 |
mircea_popescu |
that specific sort of retard is specified as follows : "i heard about bitcoin yesterday, and i have a solution!". |
| 14:18 |
asciilifeform |
sadly i have 0 solutions, only moar problemz |
| 14:18 |
mircea_popescu |
"i observed something on three blocks on one machine and here's the 100% conclusion ; tune in tomorrow for another one that a) fails to reference how i was wrong yesterday or address why and b) offer another 100% plus measures to be taken" is entirely undistinguishable. |
| 14:19 |
asciilifeform |
mircea_popescu: if you, using my patch or similar, determined that type1 (verification) blackhole consists of any substantial portion of something other than db wait -- i'm all ears |
| 14:20 |
mircea_popescu |
you just said this ; yourself ; above. in discussing "type 1" you are engaging in pure nytimes-ism. |
| 14:20 |
mircea_popescu |
where the FUCK was your "type 1" on the 24th ? |
| 14:20 |
asciilifeform |
mno, we had a thread where i cut'em up into classes |
| 14:20 |
mircea_popescu |
and this isn't the first time we run into the problem of ... let's call them sloppy run "experiments". but the second, this week. |
| 14:20 |
mircea_popescu |
it IS a problem. |
| 14:20 |
mircea_popescu |
link me to this thread ? |
| 14:20 |
asciilifeform |
http://btcbase.org/log/2016-12-29#1592898 |
| 14:20 |
a111 |
Logged on 2016-12-29 23:20 asciilifeform: type2 ( pete_dushenski's ) is the garden variety shitflood. which is sometimes solved by ip ban, but only in the case of 'shrapnel addressed to occupant', i.e. idiot prb nodes wildly spamming crapolade, and not in the 'bullet with your name on it' case, where somebody actually has a sybil constellation drowning your trb node in liquishit, with no SINGLE ip misbehaving in any way |
| 14:23 |
mircea_popescu |
we're discussing the discussion on the 24th. |
| 14:24 |
mircea_popescu |
which can be briefly summarized as "alf : omg all blackhole is disk wait for db ; mp : thatr's a factor, there's more" repeated a dozen times. |
| 14:24 |
asciilifeform |
and i will point out, none of my findings so far contradict mircea_popescu's original 'it sits and waits for the disk.' only question was, where. |
| 14:24 |
mircea_popescu |
and i'll point out that the problem here isn't the work, or the thought, but the fucking packaging. you get overexcited and oversignal. it detracts from very valuable stuff. |
| 14:25 |
asciilifeform |
all i got is a stopwatch. the idea is, mod6 et al can run same stopwatch, on other boxes, with other types of disk |
| 14:26 |
asciilifeform |
i also have a test going where : |
| 14:26 |
asciilifeform |
dbenv.set_cachesize(4, 0, 1); // 4GB of contiguous cache |
| |
↖ |
| 14:26 |
asciilifeform |
but it had 0 measurable effect |
| 14:26 |
asciilifeform |
and i did not bother to vpatchify it. |
| 14:26 |
mircea_popescu |
now then : a fix for the db would significantly improve a few classes of block verification delays ; and it would alleviate blackhole-like behaviour due to that, node's frozen checking a new block. there's at least 3 different dos vectors for other nodes, and a) the foregoing wouldn't help ; b) if it helped the enemy could easily upregulate the crapflood to compensate. |
| |
↖ |
| 14:26 |
asciilifeform |
(b) is where i ended up earlier. |
| 14:26 |
mircea_popescu |
yeah. |
| 14:27 |
mircea_popescu |
the other problem is that a good db fix is a very large project, because bitcoin is written insanely. and our fs db isn't moving, last i heard a month ago someone was going to try and profile an extx |
| |
↖ |
| 14:28 |
asciilifeform |
i suspect fs-as-db will have same problem as the ancient shitdb |
| 14:28 |
asciilifeform |
it doesn't aggregate accesses |
| 14:28 |
asciilifeform |
nor has any means for bolting on such a thing |
| 14:28 |
mircea_popescu |
it doesn't ? |
| 14:28 |
asciilifeform |
nope. |
| 14:28 |
mircea_popescu |
i thought modern raid controllers did |
| 14:28 |
asciilifeform |
a good raid card will aggregate BLOCK accesses |
| 14:28 |
asciilifeform |
but it has nfi re files ! |
| 14:28 |
mircea_popescu |
yes, but we do file=blocks |
| 14:28 |
mircea_popescu |
i thought that was the idea anyway |
| 14:29 |
asciilifeform |
that only takes care of block fetches |
| 14:29 |
asciilifeform |
but not tx indices |
| 14:29 |
mircea_popescu |
there is that. perhaps a better indexing scheme could be had. hence the fucking symlinks |
| 14:30 |
asciilifeform |
symlink gets you a file containing desired block, but there is no way on any unix fs that i know of, to symlink to ~an offset inside a file~ |
| 14:31 |
mircea_popescu |
but your file = block (in the fs sense) = block (in bitcoin sense) |
| 14:31 |
mircea_popescu |
we will have 1mb blocks on the filesystem. |
| 14:31 |
asciilifeform |
so this'd be a new fs. |
| 14:31 |
asciilifeform |
(a trb-i item ) |
| 14:31 |
mircea_popescu |
no, it'll be a mildly configured ext4 i guess |
| 14:32 |
mircea_popescu |
with blocks set to 1mb |
| 14:32 |
mircea_popescu |
but i have nfi whether this is even feasible, because this'd be step 2, after the "hey, what happens if you fill a disk with symlinks" EXPERIMENT returns some fucking results. |
| 14:33 |
asciilifeform |
has anyone began to do this ? |
| 14:33 |
mircea_popescu |
i dunno. coupla people sorta-promised to sorta-try. |
| 14:34 |
mircea_popescu |
but the reason it's mired in "first, experiment, profile" is because this is EXACTLY the sort of thing which should theoretically work out of the box on a modern nix, and ABSOLUTELY never does, at all. central lizard fodder. |
| 14:34 |
mircea_popescu |
you know, like linking. |
| 14:35 |
asciilifeform |
general-purpose fs is likely to be extremely wasteful of space, because it carries the assumption of rewritability |
| 14:35 |
asciilifeform |
something that bitcoin has very little use for |
| 14:36 |
mircea_popescu |
if it works on ext4 it can be implemented on fucking tape. |
| 14:36 |
asciilifeform |
how do you randomaccessfully fetch blocks from tape ? |
| 14:36 |
mircea_popescu |
there is that. i'm just saying, you can have nonrewritable media, whatevs. |
| 14:37 |
asciilifeform |
bitcoin as we have it, is almost the antithesis of 'tape problem', the ultimate in random access, to the point that it makes disk cache ~useless, every tx can depend on literally any block from 0 to current |
| 14:37 |
mircea_popescu |
yes, but blocks once written don't change. that's the non-rewritable part. |
| 14:37 |
asciilifeform |
aha |
| 14:37 |
asciilifeform |
(blocks really oughta live in antifuse rom. we had a thread..) |
| 14:38 |
mircea_popescu |
and there's actually some benefit for the network not even physically being capable to accept reorgs deeper than x. |
| 14:38 |
asciilifeform |
iirc we also had this thread |
| 14:38 |
mircea_popescu |
yea. |
| 14:39 |
asciilifeform |
http://btcbase.org/log/2016-12-19#1585884 << possibly this |
| 14:39 |
a111 |
Logged on 2016-12-19 18:18 ben_vulpes: asciilifeform: if martians produce longest chain with greatest difficulty i think by the rules of the game they own bitcoin |
| 14:40 |
mircea_popescu |
there's a difference between extension and reorg, however. |
| 14:41 |
mircea_popescu |
one's like "you were in a coma for the past thirty years, here's what happened that you don't remember" ; the other's like "you had a hallucinatory episode, your history for the past x period is bad and you'll have to rewrite it". |
| 14:41 |
asciilifeform |
extensions work entirely fine on otp rom |
| 14:42 |
asciilifeform |
mircea_popescu: what's the longest reorg you've personally observed, to date ? |
| 14:43 |
asciilifeform |
i have this notion, that there was a massive one some time before i tuned in. |
| 14:43 |
mircea_popescu |
the one during the original split with the idiots who were there baptised as power rangers |
| 14:43 |
mircea_popescu |
60ish or so deep if i recall. |
| 14:43 |
mircea_popescu |
!#s "power rangers" |
| 14:43 |
a111 |
155 results for "\"power rangers\"", http://btcbase.org/log-search?q=%22power%20rangers%22 |
| 14:44 |
asciilifeform |
was this the one where everybody used same buggy client nonsense ? |
| 14:45 |
mircea_popescu |
http://btcbase.org/log/2013-05-13#28327 << exciting times. |
| 14:45 |
a111 |
Logged on 2013-05-13 22:39 mircea_popescu: it's screwed wtf. |
| 14:45 |
mircea_popescu |
asciilifeform no, this is the one where the idiot miners "updated" and then the update failed and then usgavin gave them some of the bitcoin usg stole to compensate for the lost mining. |
| 14:45 |
asciilifeform |
aah |
| 14:47 |
mircea_popescu |
and then the usual usgtards were all over "oh, wow, you broke it and then you were "right on it and omg fixed it you're like power rangers" and so... |
| 14:49 |
mircea_popescu |
http://trilema.com/wp-content/uploads/2013/03/in-re-bitcoin-devs-are-idiots.htm << the actual event, march 11th |
| 14:52 |
asciilifeform |
the unfortunate bit that i keep coming back to in my head, is that http://btcbase.org/log/2017-02-26#1618702 + http://btcbase.org/log/2017-02-26#1618674 add up to a fundamental boojum, that is not a matter of simply 'make a faster db, buy a faster disk' |
| 14:52 |
a111 |
Logged on 2017-02-26 19:26 mircea_popescu: now then : a fix for the db would significantly improve a few classes of block verification delays ; and it would alleviate blackhole-like behaviour due to that, node's frozen checking a new block. there's at least 3 different dos vectors for other nodes, and a) the foregoing wouldn't help ; b) if it helped the enemy could easily upregulate the crapflood to compensate. |
| 14:52 |
a111 |
Logged on 2017-02-26 19:14 asciilifeform: the 'type 2' (non-verification) blackhole goes right back to the fundamental question of 'something to all comers', how much disk thrashing does a derp get to invoke simply by coming up with a not-yet-banned ip and a pseudonode. |
| 14:53 |
mircea_popescu |
we aren't actually following a purpose here, like "have a good bitcoin". we're merely proceeding from cause : db is broken and THEREFORE must be fixed. not BECAUSE it would bla bla ; but therefore. |
| 14:53 |
asciilifeform |
certainly is broken. |
| 14:54 |
asciilifeform |
however a node that can be brought to its knees by randos, regardless of how, is also likewise broken design. |
| 14:54 |
mircea_popescu |
in this sense your house is broken design, randos could tp it every hour. |
| 14:55 |
asciilifeform |
my house is a poor example, i suspect that squirrels could tip it. |
| 14:55 |
mircea_popescu |
you live in it dont you ? |
| 14:55 |
asciilifeform |
sorta. |
| 14:55 |
mircea_popescu |
well then. |
| 14:58 |
asciilifeform |
very concretely: all enemy has to do, to gum up a trb node, is to repeatedly request blocks from 0 .. current, randomly, and switch ips as needed |
| 14:58 |
mircea_popescu |
if the node allows non-ssh-tunnel access. |
| 14:58 |
asciilifeform |
correct. |
| 14:58 |
mircea_popescu |
which is something the operator can decide for self. |
| 14:58 |
asciilifeform |
'decide' |
| 14:58 |
mircea_popescu |
what ? |
| 14:59 |
asciilifeform |
i could cut off public ip access right now -- and never see a block again. |
| 14:59 |
mircea_popescu |
depends, anyone talking through your ssh pipes yet ? |
| 14:59 |
asciilifeform |
no takers. |
| 14:59 |
mircea_popescu |
so give it some time, it just got released like two days ago |
| 14:59 |
asciilifeform |
so far it's dulap <--> zoolag. |
| 14:59 |
mircea_popescu |
patience my dear alf. |
| 15:00 |
asciilifeform |
let's picture that whole deedbot l1 subscribes. the folx who run public/exposed node, so as to soak up blocks from heathen miners -- will be the ones to blackhole. |
| 15:00 |
asciilifeform |
leading to very similar picture as today's. |
| 15:05 |
asciilifeform |
if mining were not an intrinsically heathen activity, 'closed network' would solve the 'allcomers' problem. but as i currently understand, it'd simply ~move~ it. |
| 15:08 |
jurov |
asciilifeform: have you tried to merely neuter fsync? |
| 15:09 |
asciilifeform |
jurov: elaborate plox |
| 15:10 |
jurov |
fsync waits on data to be written to disk. and with all modern filesystems, this means flushing all kinds of metadata. too |
| 15:11 |
asciilifeform |
the journal? |
| 15:11 |
asciilifeform |
why would i want to turn off the journal ? |
| 15:11 |
asciilifeform |
may as well run the entire node from ramdisk |
| 15:12 |
jurov |
fsync is only remotely related to journal |
| 15:12 |
asciilifeform |
also flushes write cache , neh |
| 15:13 |
asciilifeform |
just what i need, in my node!11!!!! random corruption. |
| 15:13 |
jurov |
random corruption happens only if power is yanked |
| 15:13 |
asciilifeform |
which it WILL BE |
| 15:13 |
asciilifeform |
omfg this is elementary. |
| 15:13 |
asciilifeform |
i'm not willingly building another phuctor-1.0, nothx. |
| 15:14 |
jurov |
you can stop the daemon every day, call sync and backup the db, and start it. max 24hrs of work lost. |
| 15:15 |
asciilifeform |
FUCK that. |
| 15:15 |
asciilifeform |
the thread today where mircea_popescu sees something with even the superficial silhouette of a cheap prbistic hack, and barf mightily, taught you nothing, jurov ? |
| 15:16 |
jurov |
using spinning rust is not a cheap hack? |
| 15:16 |
asciilifeform |
spinning rust is the baseline. |
| 15:17 |
asciilifeform |
it is definitionally not 'a hack'. |
| 15:19 |
asciilifeform |
nonsense like 'let's flush write cache once in a blue moon instead of normally', 'let's let the db corrupt and SURE I PROMISE WE CAN RECOVER' is what winblowz world is built from. |
| 15:19 |
asciilifeform |
and also is how you get http://btcbase.org/log/2017-02-19#1615434 |
| 15:19 |
a111 |
Logged on 2017-02-19 03:54 asciilifeform: (iirc we had a thread where i described how corporate ameritards, if given a problem like phuctor, would happily soak up a few $mil and megawatt of iron) |
| 15:20 |
asciilifeform |
( does jurov volunteer to buy the double, triple, quadruple sets of disk, for all of my nodes, for these backups ? and what is the node to do while copying over a db ? say 'sorry , we're closed ' ? ) |
| 15:24 |
jurov |
You run some service that needs five-nines availability? I'm fine with some of my nodes being down for few minutes to have known-good state backed up. |
| 15:24 |
asciilifeform |
copying 400GB takes 'few minutes' on jurov's disk ? where can i buy one ? |
| 15:24 |
jurov |
ebay? |
| 15:24 |
asciilifeform |
link to the GB/sec disk plox ? |
| 15:26 |
jurov |
what 400 GB? who said you have to copy whole blockchain every time? |
| 15:26 |
asciilifeform |
if the entire fs is subject to random rot, as it would be without synced writes, then yes. |
| 15:26 |
asciilifeform |
entire .bitcoin dir. |
| 15:27 |
asciilifeform |
because random rot is NOT acceptable omfg why is this hard. |
| 15:27 |
jurov |
"entire fs is subject to random rot if my app won't checkpoint its files by calling fsync all the time" is NOT true. where did you get such silly notion? |
| 15:28 |
asciilifeform |
well not entire fs, but every file touched by trb |
| 15:28 |
asciilifeform |
which includes the blocks. |
| 15:28 |
asciilifeform |
and the indices. |
| 15:28 |
asciilifeform |
(and, on a hotwallet box, if you have one -- the wallet.) |
| 15:29 |
jurov |
waitwhat, it *writes* to old blk_... files? |
| 15:29 |
asciilifeform |
jurov: depends what you mean by 'old'. |
| 15:29 |
asciilifeform |
reorgs are a thing, neh. |
| 15:30 |
jurov |
if enemy can reorg half of your blk_files, some sync latency os least of your problems |
| 15:30 |
asciilifeform |
and how do you propose to guarantee detection of corruption in the tx indices ? |
| 15:30 |
jurov |
*is least |
| 15:31 |
jurov |
no detection. unclean shutdown -> restore backup |
| 15:31 |
asciilifeform |
what if block files are not in perfect sync with the backed-up index ? |
| 15:32 |
asciilifeform |
e.g. 1 block longer. |
| 15:32 |
asciilifeform |
now you need extra logic in trb per se, and this is no longer 'cheap perl hack'. |
| 15:32 |
asciilifeform |
now it is ~expensive~ shit hack. |
| 15:32 |
mircea_popescu |
there ~may~ be some optimizations that can be applied as-is to turn a jfs into something more appropriate for bitcoining than the "middle of the road" setting it ships with. |
| 15:33 |
jurov |
Suppose i shutdown bitcoind and backup .bitcoin using rsync (so that all files with recent mtime are backed up).. you say this won't work? |
| 15:33 |
mircea_popescu |
not necessarily in the sense of turning it off or altogether |
| 15:33 |
asciilifeform |
jurov: the most sane variant of your proposal i can think of would be to run the entire tx index in memory. this is just barely practical on dulap, a massive box. and would mean multi-hour regeneration of the index on warmup. but is at least theoretically an improvement, in that it does not threaten to randomly lose bits. |
| 15:34 |
mircea_popescu |
index-in-ram is actually how you run a large, infrastructural like node. |
| 15:35 |
mircea_popescu |
jurov shutdown of node can readily take > 15 minutes ; and can't even be initiated if the node is, eg, in db lock because block eating. |
| 15:36 |
asciilifeform |
mircea_popescu: plz consider publishing your patch where you made this happen; it is not a trivial change |
| 15:36 |
asciilifeform |
(re index-in-ram) |
| 15:37 |
mircea_popescu |
asciilifeform i don't really do blockchain.info style "public support". i think i have the stuff somewhere, i'll have to dig for it. basically it's, blk* live on /sda ; blkindex.dat and friends live on /sdb which happens to be a ramdisk. |
| 15:38 |
mircea_popescu |
the paths are hardcoded to .bitcoin/ in stock satoshib, not the end of the world to change them |
| 15:39 |
mircea_popescu |
i will note that this is an expensive and unrewarding activity that i personally discontinued cca 2014. |
| |
~ 18 minutes ~ |
| 15:57 |
asciilifeform |
mircea_popescu: as per yesterday's thread -- running node per se is 'expensive and unrewarding activity' aha! |
| 15:58 |
mircea_popescu |
myeah |
| 15:58 |
mircea_popescu |
no but i mean, a 30gb ramdisk node to support the general public's mistaken notions and unwarranted pretensions... meh. |
| 15:59 |
asciilifeform |
mircea_popescu: what did you do with the index if power failed (or node -- crashed) or whatever disruption ? |
| 16:00 |
mircea_popescu |
you rebuild it. |
| 16:00 |
mircea_popescu |
this is why i say expensive, it's not a case of "random vps hur durr". |
| 16:00 |
mircea_popescu |
it needs very specific complex things. |
| 16:01 |
asciilifeform |
trb does not have a knob for 'reindex and make sure blkxxxx matches the index'. |
| 16:01 |
asciilifeform |
that's what's in mircea_popescu's patch presumably. |
| 16:01 |
asciilifeform |
so it isn't simply a matter of 'make a ramdisk', no. |
| 16:01 |
asciilifeform |
(unless you're willing to live with random bitflippage, as jurov apparently is) |
| 16:05 |
asciilifeform |
i suppose one way to rebuild the index using existing mainline trb would be to nuke the .bitcoin dir entirely, and refeed the node via 'eatblock'. |
| 16:06 |
asciilifeform |
(and this is probably not measurably slower, in practice, than what mircea_popescu did) |
| 16:07 |
asciilifeform |
expect a warmup time of 2-3 days... |
| 16:07 |
jurov |
random bitfippage its purely your hallucination |
| 16:07 |
asciilifeform |
jurov: people flush write caches purely from hallucination ? maybe i don't need a disk at all then ? |
| 16:08 |
asciilifeform |
understand, if my proggy writes an x to disk, at some point, and then later i read a y, and y != x, this IS CORRUPTION |
| 16:09 |
asciilifeform |
whatever you want to call it. it is not and will not be an invited guest anywhere i am answerable for. |
| 16:09 |
asciilifeform |
if jurov wants to run his node off ramdisk -- more power to him. but don't try to spin the resulting bitrot as 'hallucination'. |
| 16:09 |
jurov |
but this will happen if, and only if, power is cut. if i sync once per day and backup the x, the backup will keep the x forever |
| 16:10 |
asciilifeform |
and if power is cut during backup ? |
| 16:11 |
jurov |
this is unsolvable problem? |
| 16:11 |
asciilifeform |
it very much is. |
| 16:11 |
jurov |
i said i do sync, then backup |
| 16:11 |
asciilifeform |
especially when it becomes known that your box fails catastrophically on power loss. |
| 16:11 |
jurov |
this means x is safely on disk |
| 16:11 |
asciilifeform |
and i'm still waiting for an answer, what is the node to do during backup and during restore of same ? |
| 16:12 |
asciilifeform |
sit like a brick ? |
| 16:12 |
jurov |
i'll use another node. or you're supposed to have a precious one and only one? |
| 16:12 |
jurov |
yes, it will sit as a brick and i'm fine with it. |
| 16:13 |
asciilifeform |
so now jurov invites me to see a node with double-digit % of down time as something acceptable ? |
| 16:14 |
jurov |
i'm still waiting for explanation how did you came to double digit downtimes. |
| 16:14 |
asciilifeform |
say i can finally buy jurov's magical disk, where backup AND restore of 30GB (that's just the tx index) takes only 1 hr. |
| 16:14 |
asciilifeform |
so now if i need 24/7 access to the network, i gotta keep... 24 nodes |
| 16:14 |
asciilifeform |
each with different scheduled hour of downtime. |
| 16:14 |
asciilifeform |
jurov: it can't run during backup. or during restore. so -- downtime. |
| 16:15 |
jurov |
you mean your disks are capable only 350kB/s read rate? |
| 16:15 |
jurov |
er. read write |
| 16:16 |
asciilifeform |
if i'm backing up across the net -- then yes, pretty close to that |
| 16:17 |
asciilifeform |
but take limit of t-->infinity. today it is 30GB, next year - 100 |
| 16:17 |
asciilifeform |
in n years - 1TB |
| 16:17 |
asciilifeform |
also daily backup then ? |
| 16:17 |
asciilifeform |
or what, martians will land by then, and give us for free 1TB/sec 1PB disks ? |
| 16:17 |
asciilifeform |
like gavin pronounced ? |
| 16:17 |
jurov |
if enemy can cause 1TB of incrememntal data daily, then we failed miserably |
| 16:17 |
asciilifeform |
yearly, jurov |
| 16:17 |
jurov |
lolwut |
| 16:18 |
asciilifeform |
the tx index ain't ever getting ~smaller~. |
| 16:18 |
asciilifeform |
eventually - and in probably not very long time -- it will exceed what your box can copy over and back in one day. |
| 16:19 |
asciilifeform |
then what will jurov say -- 'oh, no prob, just sync weekly' ??!@ |
| 16:19 |
jurov |
and can trb live with terabyte tx index at all? |
| 16:19 |
asciilifeform |
jurov: this is an open question. for all i know, bdb will reach some idiot hardcoded limit long before this. |
| 16:19 |
asciilifeform |
but i have zero interest in designing a bdb-like abortion. for any purpose. |
| 16:21 |
asciilifeform |
i already once built a system where the reward for enemy for unplugging seeped into days, weeks, then months of lost cpu time. until the unplugging became an inevitable thing. never again. |
| 16:23 |
jurov |
looky, i am just saying that right now it eats, say, a minute per block, which is 144 minute unreachability daily anyway. and am proposing short term tradeoff of having the unreachability shorted once per day. |
| 16:23 |
jurov |
i don't get why that bothers you so much |
| 16:23 |
jurov |
*shorter once per day |
| 16:24 |
asciilifeform |
because reliance on duct tape . |
| 16:24 |
asciilifeform |
it is the kind of thinking that gave us prb. as mircea_popescu described earlier today. |
| |
~ 41 minutes ~ |
| 17:06 |
mircea_popescu |
asciilifeform yes trb does reindex. |
| 17:08 |
mircea_popescu |
anyway, no the ramdisk thing doesn't work indefinitely, soon enough the block index will exceed the commodously available ram |
| |
~ 1 hours 4 minutes ~ |
| 18:12 |
jurov |
asciilifeform et al.: @ -> ' at ' mailinglist mangling fixed (it was only HTML output, nothing else was mangled) |
| 18:12 |
asciilifeform |
neato, ty jurov |
| 18:16 |
asciilifeform |
!~later tell mircea_popescu http://btcbase.org/log/2017-02-26#1618699 >> possibly not wholly useless: >> http://wotpaste.cascadianhacker.com/pastes/g97IR/?raw=true |
| 18:16 |
a111 |
Logged on 2017-02-26 19:26 asciilifeform: dbenv.set_cachesize(4, 0, 1); // 4GB of contiguous cache |
| 18:18 |
asciilifeform |
^ very puzzling result, and now finally similar to mircea_popescu's 'disk and Other Factors' |
| 18:20 |
asciilifeform |
it appeared to have 0 effect, but after cache had a chance to fill -- now this. |
| 18:24 |
mircea_popescu |
myeah. |
| 18:24 |
asciilifeform |
mircea_popescu: the 40-80 secs. verifications are still 15x slower than on zoolag (ssd box). i wonder, possibly , if the remaining delay still consists of disk -- but of block fetches, rather than tx index. |
| 18:25 |
mircea_popescu |
certainly a possibility. |
| 18:26 |
mircea_popescu |
this needs to run over many, as in thousands, of blocks. |
| 18:26 |
asciilifeform |
it is running on dulap, and will run there until further notice. |
| 18:26 |
asciilifeform |
will be interesting to see if this abolishes '30min blox' |
| 18:29 |
asciilifeform |
( i would try a larger cache, but 4GB is the idiot hardcoded max in bdb , they used a 32bit width ) |
| 18:31 |
mircea_popescu |
aha. |
| 18:31 |
mircea_popescu |
no way out of it, can't allocate more. |
| 18:31 |
asciilifeform |
64bit linuxen on x64 will happily allocate moar. it is bdb that is retarded. |
| 18:32 |
mircea_popescu |
well it's a 32 bit item |
| 18:32 |
asciilifeform |
aha |
| |
~ 23 minutes ~ |
| 18:55 |
mircea_popescu |
and in today's useless-website-of-the-week, https://www.bitrated.com/explore |
| 18:57 |
asciilifeform |
'Most reputable users' 'coblee350 Director of Engineering at Coinbase, Litecoin' |
| 18:57 |
asciilifeform |
lel |
| 18:57 |
asciilifeform |
and wtf is a 'trust agent' |
| 18:57 |
asciilifeform |
which he is also 'top' of. |
| 18:58 |
mircea_popescu |
it's been dead for about a year or so, but anyway, "oh i know, i'll make yet another fake wot website. because i'm a jew and we're fucking stupid congenitally." or some shit. |
| 18:58 |
mircea_popescu |
asciilifeform it's better because no pgp see. |
| 18:58 |
asciilifeform |
this gotta be the 'special-needs' corps of the zog |
| 18:59 |
mircea_popescu |
some "nadav ivgi" ytard |
| 18:59 |
mircea_popescu |
anyway, there's an old time bitcoin scammer that keeps pushing these |
| 18:59 |
mircea_popescu |
!#s rosenfeld |
| 18:59 |
a111 |
42 results for "rosenfeld", http://btcbase.org/log-search?q=rosenfeld |
| 19:00 |
mircea_popescu |
keeps getting idiot kids involved in stupid shit. |
| 19:00 |
asciilifeform |
what else to do, the original lure of vacuuming up coin is , i picture, ~gone by now |
| 19:00 |
asciilifeform |
so now vacuum people. |
| 19:00 |
asciilifeform |
or at the very least 'harmless heatsink' in the off chance people appear. |
| 19:01 |
mircea_popescu |
i think it's more in the vein of out and out pederasty. |
| 19:04 |
asciilifeform |
as, with actual buggery..? |
| 19:06 |
shinohai |
https://www.ethnews.com/sha-1-may-be-broken-but-ethereum-is-still-secure <<< Priceless |
| 19:14 |
mircea_popescu |
well, not really actual anything, that'd take actual existence. more in the vein of http://trilema.com/2014/why-dogecoin-is-a-scam-why-the-people-pushing-it-are-assholes-why-business-insider-is-a-contemptible-piece-of-shit-why-anyone-who-ever-worked-for-it-will-be-dancing-in-the-street-for-nickels-and-wh/#selection-265.0-265 |
| 19:16 |
shinohai |
I adore that particular Trilema article |
| 19:17 |
mircea_popescu |
ha! |
| 19:18 |
BingoBoingo |
!~ticker --market all |
| 19:18 |
jhvh1 |
BingoBoingo: Bitstamp BTCUSD last: 1179.25, vol: 3533.22164439 | BTC-E BTCUSD last: 1143.503, vol: 3586.19637 | Bitfinex BTCUSD last: 1181.9, vol: 11069.20358791 | BTCChina BTCUSD last: 1113.113456, vol: 6642.25280000 | Kraken BTCUSD last: 1166.508, vol: 1738.27238522 | Volume-weighted last average: 1158.16136332 |
| 19:18 |
BingoBoingo |
http://oglaf.com/actor-n-bishop/ |
| 19:19 |
shinohai |
And yes, they are all dancing in the street for nickels now their meme coin is basically no moar |
| 19:20 |
asciilifeform |
i'dvethunk they'dve been danced out by nao |
| 19:21 |
mircea_popescu |
speaking of, https://www.youtube.com/watch?v=uLVuLkt1JCY |
| 19:21 |
shinohai |
Nah they are knitting dogecoin socks for themselves (the homeless) on their sub now |
| 19:21 |
shinohai |
To dance in! |
| |
~ 19 minutes ~ |
| 19:40 |
asciilifeform |
!!up fromloper |
| 19:40 |
deedbot |
fromloper voiced for 30 minutes. |
| |
~ 1 hours 9 minutes ~ |
| 20:50 |
asciilifeform |
in other noose, asciilifeform saw a turkish film about the cats of istanbul |
| 20:50 |
asciilifeform |
http://www.imdb.com/title/tt4420704 |
| 20:51 |
asciilifeform |
possibly best thing i ever saw in a cinema. |
| 21:02 |
trinque |
PSA that as of now I cannot access the box hosting deedbot via SSH. |
| 21:02 |
trinque |
I'm checking in with the host |
| 21:07 |
trinque |
looks like my daily backups stopped last night (via ssh tunnel) |
| 21:10 |
asciilifeform |
ugh |
| 21:11 |
asciilifeform |
trinque: do they have remote-kvm ? |
| 21:11 |
trinque |
sure do |
| 21:11 |
trinque |
I'm obviously going to tell them *not* to reboot |
| 21:19 |
trinque |
wot.deedbot.org is also down |
| 21:19 |
trinque |
which is on the same box. |
| 21:19 |
trinque |
possible the host fucked up inbound routing |
| 21:19 |
trinque |
fail2ban would not have done anything to port 80 |
| 21:20 |
trinque |
(nor would it have nixed 22 from multiple sources, but anyway) |
| 21:22 |
trinque |
host already emailed me back; they're superb |
| 21:22 |
trinque |
"joes datacenter" |
| |
~ 27 minutes ~ |
| 21:50 |
ben_vulpes |
!!reputation trinque |
| 21:50 |
ben_vulpes |
*facepalm* |
| 21:50 |
trinque |
well the bot's gone; how would that work |
| 21:50 |
ben_vulpes |
didn't see the final part |
| 21:51 |
trinque |
I'm into the box; give me a while to actually figure out what happened |
| 21:51 |
ben_vulpes |
join, part, /me thinks cool! |
| 21:51 |
ben_vulpes |
nah, no rush |
| 21:52 |
trinque |
there's a lot of weirdo shit in the routing table right now |
| 21:52 |
ben_vulpes |
i'll just keep mashing buttons not hooked up to anything, like an ape |
| 21:57 |
trinque |
they're attached again. |
| 21:58 |
trinque |
ben_vulpes: http://p.bvulpes.com/pastes/VcIWx/?raw=true << behold the wtf |
| |
↖ |
| 22:03 |
trinque |
almost looks like a pf bug really |
| 22:03 |
trinque |
same routing table upon reboot, so I guess it's normal for their network |
| 22:03 |
trinque |
same pf.conf as before reboot, works nao |
| 22:04 |
trinque |
no weird connections or anything (not that anybody competent would leave those to find) |
| |
~ 22 minutes ~ |
| 22:27 |
trinque |
eh the test.test.local was just me leaving turds in /etc/hosts |
| 22:27 |
trinque |
so routing table's entirely normal |
| 22:27 |
trinque |
I have no idea, really, which is always the worst answer |
| |
~ 15 minutes ~ |
| 22:42 |
mod6 |
http://therealbitcoin.org/ml/btc-dev/2017-February/000256.html << interesting alf, thanks for posting. |