Show Idle (>14 d.) Chans


← 2022-02-16 | 2022-02-18 →
00:00 whaack billymg: nice, glad to hear it hasn't choked
00:06 whaack billymg: this was the cmd i used to grab the anyone-can-spend UTXOs: (took a couple of hours to run)
00:06 whaack SELECT SUM(VALUE) FROM output_input JOIN address ON output_input.address_id = address.address_id WHERE spending_tx_id IS NULL AND hex(address.address) like '00%';
00:07 billymg multi-hour query!
00:09 billymg impressive feat imo
00:12 whaack it's a blessing when it completes, haha
00:12 whaack billymg: what's the predicted completition time saying now?
00:13 whaack you also have enough blocks now to run the full test suite (not a very large one, but it covers basic use of all the commands) `python api_tests.py`
00:13 billymg the most recent estimate i just got at 365150 was 4 days, 4 hours, 24 mins
00:14 billymg ah cool
00:15 billymg i actually just ran that query for the fun of it but then killed it when i realized it's still in 2015, no segwit yet
00:16 whaack well billymg it'd be interesting to see if there were any coins in those types of addresses presegwit
00:16 asciilifeform there was at least 1 'anyone can spend puzzle' iirc
00:16 whaack bbl, 20 mins
00:16 * billymg also bbl
~ 23 minutes ~
00:40 whaack bk
~ 36 minutes ~
01:16 whaack asciilifeform: do you know if it's possible to create a multisig address without using P2SH?
01:17 whaack it looks like trb has the capability with OP_CHECKMULTSIG but i'm not sure if that's a reality
~ 40 minutes ~
01:58 shinohai Perhaps I'm a dunce here, but let's say I want to sign a message with BTC addy using bare openssl. Converted my addy to pem, sha256 hashed message to sign, then signed and converted to hex like so: `openssl dgst -sha256 -sign privkey.pem msg.sha256 |xxd -c 74 -ps` ... how does one arrive at the base64 output that bitcoind spits out when signing?
02:12 asciilifeform whaack: can verify'em but afaik not includes any way to bake'em
02:12 whaack shinohai: i believe you mean base58?
02:13 whaack trbexplorer has base58 conversion in the source, written by jfw, it's a bit of a headache, idk any command line tool that does it
02:14 whaack asciilifeform: ah ok, makes sense
02:14 shinohai whaack: For instance, if I sign the message "Hi whaack!" with address it outputs sig of HyH93YCEmfFXamlfbuYbiV0omZG6hH+i0cyd5kXs8EodCc+PmLol/tAazfAQlTtH9THF33VwX/mfLwIue+n9gac=
02:15 shinohai ("it" being standard trb )
02:16 whaack ah, i didn't know
02:17 asciilifeform shinohai: you can do it but 1) the openssl utils hash the thing, you want to sign raw turd 2) they dun enforce the 'low-S' thing
02:18 shinohai asciilifeform: I dun need to double hash input before signing (like you would if it was a BTC tx) ?
02:18 asciilifeform gory details.
~ 1 hours 9 minutes ~
03:28 crtdaydreams music
03:28 crtdaydreams morn' all.
~ 31 minutes ~
03:59 crtdaydreams http://logs.nosuchlabs.com/log/asciilifeform/2022-02-15#1079464 << wtf the story of this dude is fucked up
03:59 dulapbot Logged on 2022-02-15 14:32:08 asciilifeform: crtdaydreams: reiser
~ 6 hours 24 minutes ~
10:24 cgra http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079525 << to give a some estimate, if a block verifies in 5 sec on average, 720000*5/60/60/24 = about 42 days to verify the whole chain. the 5 secs may be a gross underestimate for the apu1 box, is merely a hunch based on my sync tests with an 9th gen usg-intel cpu
10:24 dulapbot Logged on 2022-02-16 09:26:39 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079519 << indeed will be useful for standing up nodes 'quickly' (will still be limited by $machine's bw, and by the time to verify blox, imho it'd be suicidally foolish to skip verification even for 'cemented' blox)
10:37 cgra http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079517 << possibly 'half' cuz just this, but i'll try anyway...: did you already come up with a clean algo how to parallelize verification of next and download of next+n blocks?
10:37 dulapbot Logged on 2022-02-16 00:11:56 asciilifeform: has a half-done cement-mode vpatch, not rdy for primetime justyet
10:37 cgra facing same (or a variant of) question in my attempt to formulate a 'getheaders-sync' patch
10:37 dulapbot Logged on 2022-01-28 13:27:25 cgra: this is again starting to take time beyond imagination, so i thought i'd make an attempt to loosely describe meanwhile, perhaps asciilifeform grasps what i mean:
~ 1 hours 57 minutes ~
12:35 cgra i recalled i had some actual measurements recorded, which add up to these figures: in height range 228k-288k, my avg verification time was ~760 ms, in range 289k-317k it was ~1810 ms, and in range 427k-649k it was ~5050 ms.
12:49 cgra the slowest verification i could find off-hand from own (somewhat messy) records was 9.2 seconds, block 0000000000000000000a34fb7816747e8d0e09b535e5d07f70f60c690a64c17b.
~ 36 minutes ~
13:26 cgra whaack: re podcasts in general, personally i find listening inefficient also because for a difficult/unfamiliar subject to properly sink in, i would need multiple re-reads over time. but i ack your typing pains issue
~ 1 hours 26 minutes ~
14:52 asciilifeform http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079578 << prolly won't surprise cgra that asciilifeform concluded that doing ~anything~ in parallel in trb is ~impossible w/out removing locks and potentially opening gates of hell. asciilifeform's draft 'usecement' is 100% serial, asks for 1 block at a time (and wastefully, 'getdata' to each connected peer, thinking atm how to reduce the bw guzzling)
14:52 dulapbot Logged on 2022-02-17 05:37:32 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079517 << possibly 'half' cuz just this, but i'll try anyway...: did you already come up with a clean algo how to parallelize verification of next and download of next+n blocks?
14:53 asciilifeform cgra: i suspect that the only safe way to dl them in parallel would be a ~separate proggy~ which does so and feeds to noad via eatblock
14:54 asciilifeform http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079582 << >> http://logs.nosuchlabs.com/log/trilema/2018-07-13#1834130
14:54 dulapbot Logged on 2022-02-17 07:35:19 cgra: i recalled i had some actual measurements recorded, which add up to these figures: in height range 228k-288k, my avg verification time was ~760 ms, in range 289k-317k it was ~1810 ms, and in range 427k-649k it was ~5050 ms.
14:54 dulapbot (trilema) 2018-07-13 asciilifeform: http://therealbitcoin.org/ml/btc-dev/2017-February/000256.html << for reference; the block timers.
~ 32 minutes ~
15:27 whaack !e view-height
15:27 trbexplorer block_height: 723775
15:27 trbexplorer mins_since_last_block: 2
15:30 whaack Ran another query last night, 8.83 mn coins are in p2pkh address UTXOs
15:31 whaack the query to doso was:
15:31 whaack sqlite3 db "SELECT SUM(value) FROM output_input JOIN address ON output_input.address_id = address.address_id WHERE spending_tx_id IS NULL AND hex(address.address) like '76a914%88ac'"
15:34 shinohai So p2pkh still outpacing p2sh?
15:34 shinohai (iirc was ~3mn)
15:42 whaack shinohai: i believe there are 5.2mn in p2sh https://txstats.com/dashboard/db/p2sh-statistics?orgId=1
15:43 whaack i haven't ran the query myself, that's the next one i'm going to construct
15:44 whaack the ~3mn and change i posted yesterday are in addresses that start with 0x00, which are not p2sh, they are the easiest form of 'anyonecanspend'
15:45 whaack so in the trb-addy vs. segwit war i see it as p2pkh vs. (0x00 addresses + p2sh addresses)
15:46 whaack which looks something like 8.8mn vs. 8.5mn, if trends continue it's about to cross over into gavincoin's favor
15:47 whaack cgra: i concur that podcast is a pretty shitty medium, recording is also a pita, when i say something i want to word differently i'm stuck with it unless i rerecord or start doing a collage of audio clips
15:48 shinohai http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079569 <<< tbh I had almost forgotten the 'low-S' DER thing, but shouldn't be too difficult to detect if first byte is > 0x7f.
15:48 dulapbot Logged on 2022-02-16 21:17:31 asciilifeform: shinohai: you can do it but 1) the openssl utils hash the thing, you want to sign raw turd 2) they dun enforce the 'low-S' thing
15:55 whaack http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079587 <-- My intuition firmly believes that this is the right idea. trb should come with a shit-filterer, another proggy it connects to. This other proggy exists solely to greedily grab blocks from all sources, heathen etc. It then feeds trb the info via the rpc command
15:55 dulapbot Logged on 2022-02-17 09:53:10 asciilifeform: cgra: i suspect that the only safe way to dl them in parallel would be a ~separate proggy~ which does so and feeds to noad via eatblock
15:57 whaack i don't know why you would feed via eatblock, because IMO the goal is to replicate a 'traditional sync' as much as possible
15:57 whaack also in this case the cement logic belongs in the shit-filterer, not in trb itself
~ 33 minutes ~
16:30 asciilifeform whaack: i'ma release the draft 'usecement' patch, when i'm satisfied that it worx. aint a 'final' solution, but 1) quite lightweight 2) make for sumthing close to fastest possible sync w/out major surgeries
16:30 dulapbot Logged on 2022-02-17 09:52:45 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079578 << prolly won't surprise cgra that asciilifeform concluded that doing ~anything~ in parallel in trb is ~impossible w/out removing locks and potentially opening gates of hell. asciilifeform's draft 'usecement' is 100% serial, asks for 1 block at a time (and wastefully, 'getdata' to each connected peer, thinking atm how to reduce the bw guzzling
16:33 asciilifeform note, btw, that in the earlier linked ml experiment, asciilifeform demonstrated that most of block verification cost is in the idjit slowmo db
16:34 asciilifeform plumbing in a sane single-purpose db would shave perhaps 90% of it
16:34 dulapbot Logged on 2021-11-12 12:48:05 asciilifeform: replacing the db with proper o(1) nqb+cryostat would fix this, bdb is monstrously slow
16:36 asciilifeform even the monstrous bdb thing, observe, is 'fast enuff' if noad is 'up to date'. for the most part only becomes an issue when revving up a new one. hence 'cement'.
16:38 asciilifeform http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079607 << perhaps obv., but multiple problems w/ such approach ( in what write the 'filter' ? foist python on n00bs on top of the rest of the horror? ... or anuther cpp turd ? or try in ada, where we dun even have tcp glue yet? )
16:38 dulapbot Logged on 2022-02-17 10:55:47 whaack: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079587 <-- My intuition firmly believes that this is the right idea. trb should come with a shit-filterer, another proggy it connects to. This other proggy exists solely to greedily grab blocks from all sources, heathen etc. It then feeds trb the info via the rpc command
16:41 asciilifeform the other thing is, the functionality of trb aint as cleanly separable as one would wish -- if 'filter' actually 100% knew what is a valid tx or block, you would scarcely need trb itself.
16:42 asciilifeform ( recall e.g. the 'gales wallet' people and how mp lost his shit when was explained to him that it still needs to plug into a live noad to have any notion of what inputs to use for a new tx )
16:45 asciilifeform the problem with accepting ~anything~ other than the immediately next block is fundamental, rather than incidental
16:47 asciilifeform i.e. the only kinds of block you can mechanically determine validity of are 1) one that connects immediately after one that you have 2) a cemented one
16:47 asciilifeform any other kind, if you so much as agree to keep it for 5 seconds, opens you up to drowning in liquishit.
16:50 * asciilifeform did quick napkin calculation re bw : it'd take <1h to move all blox to date through a gb/s pipe. i.e. bw aint the major concern if yer on a decent pipe.
~ 40 minutes ~
17:31 cgra http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079616 << does your cement-mode somehow bypass completely, or in part, the bdb bog?
17:31 dulapbot Logged on 2022-02-17 11:36:39 asciilifeform: even the monstrous bdb thing, observe, is 'fast enuff' if noad is 'up to date'. for the most part only becomes an issue when revving up a new one. hence 'cement'.
17:31 asciilifeform cgra: not at all
17:32 asciilifeform cgra: only bypasses the 'ask over9000 times for next block and get nuffin pertinent for n hours / d days' bog.
17:32 cgra right, as i thought
17:37 cgra on account of the 'fast enuff' + 'cement soon' points, the continous sync portion on my table (that 'getheaders-sync') prolly best without any kind of parallelization tricks, cuz less code
~ 25 minutes ~
18:03 cgra asciilifeform: re cement getdata algo, wanna point out that in theory there's also a risk of sendbuffer flooding some trb peer, especially with a lower-than-default sendbuffersize, if peer's upload is slow, and/or peer stuck at the same time doing smth else
18:04 cgra a fast node responds fast to your getdata chain and allows it going forward, while slow peers' sendbuffers build up at that pace
18:04 asciilifeform cgra: this is a headache even w/out cement neh
18:05 asciilifeform (i.e. currently)
18:06 cgra asciilifeform: normal getdata's are limited by the getblocks response size of the peer, or some other case in mind?
18:07 cgra i mean, a peer won't advertise blocks (in response to getblocks) more at a time than a half of it's sendbuffer can hold
18:08 asciilifeform cgra: in asciilifeform's current draft, asks for 1 block at a time
18:08 cgra asciilifeform: but then another round of getdata when receives the block from the fastest node?
18:08 asciilifeform correct
18:09 cgra asciilifeform: this is when the slowest node may still be wrestling with smth else, but has your first getdata queued
18:09 asciilifeform cgra: trying to think of better algo, which doesn't result in n peers attempting to send in same $block
18:09 asciilifeform aha
~ 16 minutes ~
18:25 * cgra not convinced yet, after all, that shouldn't try formulating some kinda minimalistic pipeline... cement seems to want similar too
18:25 dulapbot Logged on 2022-02-17 12:37:29 cgra: on account of the 'fast enuff' + 'cement soon' points, the continous sync portion on my table (that 'getheaders-sync') prolly best without any kind of parallelization tricks, cuz less code
18:26 asciilifeform cgra: not impossible, but would be very tricky to limit memory footprint.
18:26 cgra can't get rid of the idea that downloading a bounded buffer of N next blocks could somehow work, and without horror
18:26 asciilifeform for cemented blox certainly could work
18:27 asciilifeform tho because of the asinine pseudo-multithreadedness, only 1 peer can send/recv at a time
18:27 cgra asciilifeform: if can find it for cement, would be enough, cuz 'fast enuff' for normal operation
18:27 asciilifeform possibly even w/out this, 'fast enuff'
18:28 * asciilifeform wrote errything but the part which actually bakes the getdata's. possibly tonight will finish & testfire
18:31 * cgra keeps thinking of smth like "spread getdatas of the block range N to peers, follow peer response times and avoid overloading slow peers. buffer populates, verify block every time the lowest block in range N available, adjust buffer afterwards"
18:33 asciilifeform gnarly. 'slow' peer at a given time is usually simply 1 who is busy. 30sec later the same may be 'fastest' peer.
18:34 asciilifeform other problem is that any peer who is sending anyffin at all, ties up the whole noad.
18:34 asciilifeform whether he's sending the requested block, a duplicate, or 'latest' crapola that aint wanted at all
18:35 asciilifeform the practical pill is likely to be to use -connect and pick a reliable peer when syncing w/ cement
18:37 asciilifeform the sheer halfbakedness of the whole design is mindboggling -- e.g. wai did shitoshi do the merkle hash thing, and then decided to send blocks as monolithic turd ?
18:37 asciilifeform would've been entirely possib. to send the header, then the 1st level of merkelisms, then n+1st, etc.
18:38 asciilifeform (for n00bz: block hash is a 'merkle tree', i.e. permits reordering tx and get same hash)
18:39 * asciilifeform must bbl
~ 1 hours 1 minutes ~
19:40 * signpost tuned in, but sick. probably resurface next week or so.
19:43 signpost http://logs.nosuchlabs.com/log/asciilifeform/2022-02-04#1078280 << this is done.
19:43 dulapbot Logged on 2022-02-04 03:48:53 crtdaydreams: signpost: http://paste.deedbot.org/?id=h-Sc
19:45 signpost http://logs.nosuchlabs.com/log/asciilifeform/2022-02-12#1079042 << good old sneakernet
19:45 dulapbot Logged on 2022-02-12 21:44:32 whaack: lol maybe this is the right warez protocol (@ signpost), a mail-in your hard drive service
19:45 signpost was burned CDs "back in my day"
20:00 signpost probably best density to weight ratio for mailed items is going to be tape backup.
~ 21 minutes ~
20:21 asciilifeform signpost: afaik presently the winner of 'MB per kg for parcel' is mech. hdd. (with caveat that it takes 2nd place if both parties own a -- rather costly -- current-day tape drive)
20:22 asciilifeform a pretty tight 2nd place tho
20:24 asciilifeform this afaik is on acct of the fact that human-priced tape drives have virtually disappeared, and what remains is strictly in the 'tbtf' golden toilet market, where ancient laws concretely demand the use of tape
20:25 crtdaydreams http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079662 << Thanking you kindly.
20:25 dulapbot Logged on 2022-02-17 14:43:06 signpost: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-04#1078280 << this is done.
20:25 * asciilifeform has buncha 8 and 4mm scsi tape decks, hasn't powered'em up in some yrs
20:25 crtdaydreams http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079661 << Get well soon :)
20:25 dulapbot Logged on 2022-02-17 14:40:36 signpost: tuned in, but sick. probably resurface next week or so.
20:26 signpost asciilifeform: ah, that makes sense.
20:27 * signpost looked into these recently and yep, was taken aback at the prices for drives.
20:27 signpost ty crtdaydreams
20:30 asciilifeform e.g.: 1 vendor offers lto-9 tape, i.e. 18tb, for ~200 $; a randomly-picked shop offers 18tb hdd for 325 $.
20:31 asciilifeform the latter gives random access, can be bought in over9000 places, and doesn't need a ~5k deck to r/w it.
20:32 asciilifeform but if you need over9000 x 18tb in-house -- the tape wins.
20:32 crtdaydreams I spent some ~160 $ USD for 8TB D:
20:32 asciilifeform crtdaydreams: when?
20:32 crtdaydreams sec, Im going to get actual price
20:33 crtdaydreams ST8000DM004 @ $195 AUD apiece
20:34 crtdaydreams which is some $120 USD
20:34 crtdaydreams mebbe less
20:35 crtdaydreams the ex. rate is like .75 so $140 USD / 8TB
20:35 asciilifeform sounds moar or less ordinary
20:35 crtdaydreams Yeah, It's not like WD Blacks or anything.
20:35 asciilifeform the mega-drives always cost moar 'per mb' than the 'garden variety'
20:36 asciilifeform folx who actually need to pack 'as much as can' into $cabinet, pay the premium
20:36 crtdaydreams hehe data hoarders
20:37 asciilifeform for folx who are buying handful and not over9000, normally makes sense to simply get n moar of the thinner ones
20:37 crtdaydreams Mhm
20:38 * crtdaydreams requires morning sustenance
20:41 crtdaydreams Speaking of economy of scaled, you'd expect moar drives == moar power. There's a balance to be struck with premium disk
20:42 asciilifeform moar power, moar space, moar cooling cost
20:43 crtdaydreams well, less (physical space) per drive? The whole point is to strike up a balance between individual drive draw while increasing capacity in the same sized box, no?
~ 16 minutes ~
21:00 shinohai https://archive.is/LVgKK <<< lol DOJ has appointed a Director of Buterin's Waterfall.
21:04 * shinohai likes how USSA always appoints "Czars" but we never get any with those sweet Brezhnev eyebrows .....
~ 2 hours 20 minutes ~
23:25 verisimilitude http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079576 That's impressive in its awful nature. I recall asciilifeform calling Bitcoin Microsoft-style shitware, but that's impressive all the same.
23:25 dulapbot Logged on 2022-02-17 05:24:00 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079525 << to give a some estimate, if a block verifies in 5 sec on average, 720000*5/60/60/24 = about 42 days to verify the whole chain. the 5 secs may be a gross underestimate for the apu1 box, is merely a hunch based on my sync tests with an 9th gen usg-intel cpu
23:28 verisimilitude http://logs.nosuchlabs.com/log/asciilifeform/2022-01-18#1074264 I forgot to explicitly note this, jonsykkel, but this is how I write an ``ancient Greek program'', or more appropriately an ``ancient Roman program''.
23:28 dulapbot Logged on 2022-01-18 17:02:10 jonsykkel: http://logs.nosuchlabs.com/log/asciilifeform/2022-01-18#1074159 << looking 4wd to reading
23:29 verisimilitude I'd be elated to know what was thought about it. I'll be working on a successor article and the Ada version of the program described in this article, today, but I've been mulling over details for weeks anyway.
← 2022-02-16 | 2022-02-18 →