00:00 |
whaack |
billymg: nice, glad to hear it hasn't choked |
00:06 |
whaack |
billymg: this was the cmd i used to grab the anyone-can-spend UTXOs: (took a couple of hours to run) |
00:06 |
whaack |
SELECT SUM(VALUE) FROM output_input JOIN address ON output_input.address_id = address.address_id WHERE spending_tx_id IS NULL AND hex(address.address) like '00%'; |
00:07 |
billymg |
multi-hour query! |
00:09 |
billymg |
impressive feat imo |
00:12 |
whaack |
it's a blessing when it completes, haha |
00:12 |
whaack |
billymg: what's the predicted completition time saying now? |
00:13 |
whaack |
you also have enough blocks now to run the full test suite (not a very large one, but it covers basic use of all the commands) `python api_tests.py` |
00:13 |
billymg |
the most recent estimate i just got at 365150 was 4 days, 4 hours, 24 mins |
00:14 |
billymg |
ah cool |
00:15 |
billymg |
i actually just ran that query for the fun of it but then killed it when i realized it's still in 2015, no segwit yet |
00:16 |
whaack |
well billymg it'd be interesting to see if there were any coins in those types of addresses presegwit |
00:16 |
asciilifeform |
there was at least 1 'anyone can spend puzzle' iirc |
00:16 |
whaack |
bbl, 20 mins |
00:16 |
* |
billymg also bbl |
| |
~ 23 minutes ~ |
00:40 |
whaack |
bk |
| |
~ 36 minutes ~ |
01:16 |
whaack |
asciilifeform: do you know if it's possible to create a multisig address without using P2SH? |
01:17 |
whaack |
it looks like trb has the capability with OP_CHECKMULTSIG but i'm not sure if that's a reality |
| |
~ 40 minutes ~ |
01:58 |
shinohai |
Perhaps I'm a dunce here, but let's say I want to sign a message with BTC addy using bare openssl. Converted my addy to pem, sha256 hashed message to sign, then signed and converted to hex like so: `openssl dgst -sha256 -sign privkey.pem msg.sha256 |xxd -c 74 -ps` ... how does one arrive at the base64 output that bitcoind spits out when signing? |
02:12 |
asciilifeform |
whaack: can verify'em but afaik not includes any way to bake'em |
02:12 |
whaack |
shinohai: i believe you mean base58? |
02:13 |
whaack |
trbexplorer has base58 conversion in the source, written by jfw, it's a bit of a headache, idk any command line tool that does it |
02:14 |
whaack |
asciilifeform: ah ok, makes sense |
02:14 |
shinohai |
whaack: For instance, if I sign the message "Hi whaack!" with address it outputs sig of HyH93YCEmfFXamlfbuYbiV0omZG6hH+i0cyd5kXs8EodCc+PmLol/tAazfAQlTtH9THF33VwX/mfLwIue+n9gac= |
02:15 |
shinohai |
("it" being standard trb ) |
02:16 |
whaack |
ah, i didn't know |
02:17 |
asciilifeform |
shinohai: you can do it but 1) the openssl utils hash the thing, you want to sign raw turd 2) they dun enforce the 'low-S' thing |
| |
↖ |
02:18 |
shinohai |
asciilifeform: I dun need to double hash input before signing (like you would if it was a BTC tx) ? |
02:18 |
asciilifeform |
gory details. |
| |
~ 1 hours 9 minutes ~ |
03:28 |
crtdaydreams |
music |
03:28 |
crtdaydreams |
morn' all. |
| |
~ 31 minutes ~ |
03:59 |
crtdaydreams |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-15#1079464 << wtf the story of this dude is fucked up |
| |
↖ |
03:59 |
dulapbot |
Logged on 2022-02-15 14:32:08 asciilifeform: crtdaydreams: reiser |
| |
~ 6 hours 24 minutes ~ |
10:24 |
cgra |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079525 << to give a some estimate, if a block verifies in 5 sec on average, 720000*5/60/60/24 = about 42 days to verify the whole chain. the 5 secs may be a gross underestimate for the apu1 box, is merely a hunch based on my sync tests with an 9th gen usg-intel cpu |
| |
↖ |
10:24 |
dulapbot |
Logged on 2022-02-16 09:26:39 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079519 << indeed will be useful for standing up nodes 'quickly' (will still be limited by $machine's bw, and by the time to verify blox, imho it'd be suicidally foolish to skip verification even for 'cemented' blox) |
10:37 |
cgra |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079517 << possibly 'half' cuz just this, but i'll try anyway...: did you already come up with a clean algo how to parallelize verification of next and download of next+n blocks? |
| |
↖ ↖ |
10:37 |
dulapbot |
Logged on 2022-02-16 00:11:56 asciilifeform: has a half-done cement-mode vpatch, not rdy for primetime justyet |
10:37 |
cgra |
facing same (or a variant of) question in my attempt to formulate a 'getheaders-sync' patch |
10:37 |
dulapbot |
Logged on 2022-01-28 13:27:25 cgra: this is again starting to take time beyond imagination, so i thought i'd make an attempt to loosely describe meanwhile, perhaps asciilifeform grasps what i mean: |
| |
~ 1 hours 57 minutes ~ |
12:35 |
cgra |
i recalled i had some actual measurements recorded, which add up to these figures: in height range 228k-288k, my avg verification time was ~760 ms, in range 289k-317k it was ~1810 ms, and in range 427k-649k it was ~5050 ms. |
| |
↖ |
12:49 |
cgra |
the slowest verification i could find off-hand from own (somewhat messy) records was 9.2 seconds, block 0000000000000000000a34fb7816747e8d0e09b535e5d07f70f60c690a64c17b. |
| |
~ 36 minutes ~ |
13:26 |
cgra |
whaack: re podcasts in general, personally i find listening inefficient also because for a difficult/unfamiliar subject to properly sink in, i would need multiple re-reads over time. but i ack your typing pains issue |
| |
~ 1 hours 26 minutes ~ |
14:52 |
asciilifeform |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079578 << prolly won't surprise cgra that asciilifeform concluded that doing ~anything~ in parallel in trb is ~impossible w/out removing locks and potentially opening gates of hell. asciilifeform's draft 'usecement' is 100% serial, asks for 1 block at a time (and wastefully, 'getdata' to each connected peer, thinking atm how to reduce the bw guzzling) |
| |
↖ ↖ |
14:52 |
dulapbot |
Logged on 2022-02-17 05:37:32 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079517 << possibly 'half' cuz just this, but i'll try anyway...: did you already come up with a clean algo how to parallelize verification of next and download of next+n blocks? |
14:53 |
asciilifeform |
cgra: i suspect that the only safe way to dl them in parallel would be a ~separate proggy~ which does so and feeds to noad via eatblock |
| |
↖ |
14:54 |
asciilifeform |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079582 << >> http://logs.nosuchlabs.com/log/trilema/2018-07-13#1834130 |
14:54 |
dulapbot |
Logged on 2022-02-17 07:35:19 cgra: i recalled i had some actual measurements recorded, which add up to these figures: in height range 228k-288k, my avg verification time was ~760 ms, in range 289k-317k it was ~1810 ms, and in range 427k-649k it was ~5050 ms. |
14:54 |
dulapbot |
(trilema) 2018-07-13 asciilifeform: http://therealbitcoin.org/ml/btc-dev/2017-February/000256.html << for reference; the block timers. |
| |
~ 32 minutes ~ |
15:27 |
whaack |
!e view-height |
15:27 |
trbexplorer |
block_height: 723775 |
15:27 |
trbexplorer |
mins_since_last_block: 2 |
15:30 |
whaack |
Ran another query last night, 8.83 mn coins are in p2pkh address UTXOs |
15:31 |
whaack |
the query to doso was: |
15:31 |
whaack |
sqlite3 db "SELECT SUM(value) FROM output_input JOIN address ON output_input.address_id = address.address_id WHERE spending_tx_id IS NULL AND hex(address.address) like '76a914%88ac'" |
15:34 |
shinohai |
So p2pkh still outpacing p2sh? |
15:34 |
shinohai |
(iirc was ~3mn) |
15:42 |
whaack |
shinohai: i believe there are 5.2mn in p2sh https://txstats.com/dashboard/db/p2sh-statistics?orgId=1 |
15:43 |
whaack |
i haven't ran the query myself, that's the next one i'm going to construct |
15:44 |
whaack |
the ~3mn and change i posted yesterday are in addresses that start with 0x00, which are not p2sh, they are the easiest form of 'anyonecanspend' |
15:45 |
whaack |
so in the trb-addy vs. segwit war i see it as p2pkh vs. (0x00 addresses + p2sh addresses) |
15:46 |
whaack |
which looks something like 8.8mn vs. 8.5mn, if trends continue it's about to cross over into gavincoin's favor |
15:47 |
whaack |
cgra: i concur that podcast is a pretty shitty medium, recording is also a pita, when i say something i want to word differently i'm stuck with it unless i rerecord or start doing a collage of audio clips |
15:48 |
shinohai |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079569 <<< tbh I had almost forgotten the 'low-S' DER thing, but shouldn't be too difficult to detect if first byte is > 0x7f. |
15:48 |
dulapbot |
Logged on 2022-02-16 21:17:31 asciilifeform: shinohai: you can do it but 1) the openssl utils hash the thing, you want to sign raw turd 2) they dun enforce the 'low-S' thing |
15:55 |
whaack |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079587 <-- My intuition firmly believes that this is the right idea. trb should come with a shit-filterer, another proggy it connects to. This other proggy exists solely to greedily grab blocks from all sources, heathen etc. It then feeds trb the info via the rpc command |
| |
↖ |
15:55 |
dulapbot |
Logged on 2022-02-17 09:53:10 asciilifeform: cgra: i suspect that the only safe way to dl them in parallel would be a ~separate proggy~ which does so and feeds to noad via eatblock |
15:57 |
whaack |
i don't know why you would feed via eatblock, because IMO the goal is to replicate a 'traditional sync' as much as possible |
15:57 |
whaack |
also in this case the cement logic belongs in the shit-filterer, not in trb itself |
| |
~ 33 minutes ~ |
16:30 |
asciilifeform |
whaack: i'ma release the draft 'usecement' patch, when i'm satisfied that it worx. aint a 'final' solution, but 1) quite lightweight 2) make for sumthing close to fastest possible sync w/out major surgeries |
16:30 |
dulapbot |
Logged on 2022-02-17 09:52:45 asciilifeform: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079578 << prolly won't surprise cgra that asciilifeform concluded that doing ~anything~ in parallel in trb is ~impossible w/out removing locks and potentially opening gates of hell. asciilifeform's draft 'usecement' is 100% serial, asks for 1 block at a time (and wastefully, 'getdata' to each connected peer, thinking atm how to reduce the bw guzzling |
16:33 |
asciilifeform |
note, btw, that in the earlier linked ml experiment, asciilifeform demonstrated that most of block verification cost is in the idjit slowmo db |
| |
↖ |
16:34 |
asciilifeform |
plumbing in a sane single-purpose db would shave perhaps 90% of it |
16:34 |
dulapbot |
Logged on 2021-11-12 12:48:05 asciilifeform: replacing the db with proper o(1) nqb+cryostat would fix this, bdb is monstrously slow |
16:36 |
asciilifeform |
even the monstrous bdb thing, observe, is 'fast enuff' if noad is 'up to date'. for the most part only becomes an issue when revving up a new one. hence 'cement'. |
| |
↖ |
16:38 |
asciilifeform |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079607 << perhaps obv., but multiple problems w/ such approach ( in what write the 'filter' ? foist python on n00bs on top of the rest of the horror? ... or anuther cpp turd ? or try in ada, where we dun even have tcp glue yet? ) |
16:38 |
dulapbot |
Logged on 2022-02-17 10:55:47 whaack: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079587 <-- My intuition firmly believes that this is the right idea. trb should come with a shit-filterer, another proggy it connects to. This other proggy exists solely to greedily grab blocks from all sources, heathen etc. It then feeds trb the info via the rpc command |
16:41 |
asciilifeform |
the other thing is, the functionality of trb aint as cleanly separable as one would wish -- if 'filter' actually 100% knew what is a valid tx or block, you would scarcely need trb itself. |
16:42 |
asciilifeform |
( recall e.g. the 'gales wallet' people and how mp lost his shit when was explained to him that it still needs to plug into a live noad to have any notion of what inputs to use for a new tx ) |
16:45 |
asciilifeform |
the problem with accepting ~anything~ other than the immediately next block is fundamental, rather than incidental |
16:47 |
asciilifeform |
i.e. the only kinds of block you can mechanically determine validity of are 1) one that connects immediately after one that you have 2) a cemented one |
16:47 |
asciilifeform |
any other kind, if you so much as agree to keep it for 5 seconds, opens you up to drowning in liquishit. |
16:50 |
* |
asciilifeform did quick napkin calculation re bw : it'd take <1h to move all blox to date through a gb/s pipe. i.e. bw aint the major concern if yer on a decent pipe. |
| |
~ 40 minutes ~ |
17:31 |
cgra |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079616 << does your cement-mode somehow bypass completely, or in part, the bdb bog? |
17:31 |
dulapbot |
Logged on 2022-02-17 11:36:39 asciilifeform: even the monstrous bdb thing, observe, is 'fast enuff' if noad is 'up to date'. for the most part only becomes an issue when revving up a new one. hence 'cement'. |
17:31 |
asciilifeform |
cgra: not at all |
17:32 |
asciilifeform |
cgra: only bypasses the 'ask over9000 times for next block and get nuffin pertinent for n hours / d days' bog. |
17:32 |
cgra |
right, as i thought |
17:37 |
cgra |
on account of the 'fast enuff' + 'cement soon' points, the continous sync portion on my table (that 'getheaders-sync') prolly best without any kind of parallelization tricks, cuz less code |
| |
↖ |
| |
~ 25 minutes ~ |
18:03 |
cgra |
asciilifeform: re cement getdata algo, wanna point out that in theory there's also a risk of sendbuffer flooding some trb peer, especially with a lower-than-default sendbuffersize, if peer's upload is slow, and/or peer stuck at the same time doing smth else |
18:04 |
cgra |
a fast node responds fast to your getdata chain and allows it going forward, while slow peers' sendbuffers build up at that pace |
18:04 |
asciilifeform |
cgra: this is a headache even w/out cement neh |
18:05 |
asciilifeform |
(i.e. currently) |
18:06 |
cgra |
asciilifeform: normal getdata's are limited by the getblocks response size of the peer, or some other case in mind? |
18:07 |
cgra |
i mean, a peer won't advertise blocks (in response to getblocks) more at a time than a half of it's sendbuffer can hold |
18:08 |
asciilifeform |
cgra: in asciilifeform's current draft, asks for 1 block at a time |
18:08 |
cgra |
asciilifeform: but then another round of getdata when receives the block from the fastest node? |
18:08 |
asciilifeform |
correct |
18:09 |
cgra |
asciilifeform: this is when the slowest node may still be wrestling with smth else, but has your first getdata queued |
18:09 |
asciilifeform |
cgra: trying to think of better algo, which doesn't result in n peers attempting to send in same $block |
18:09 |
asciilifeform |
aha |
| |
~ 16 minutes ~ |
18:25 |
* |
cgra not convinced yet, after all, that shouldn't try formulating some kinda minimalistic pipeline... cement seems to want similar too |
18:25 |
dulapbot |
Logged on 2022-02-17 12:37:29 cgra: on account of the 'fast enuff' + 'cement soon' points, the continous sync portion on my table (that 'getheaders-sync') prolly best without any kind of parallelization tricks, cuz less code |
18:26 |
asciilifeform |
cgra: not impossible, but would be very tricky to limit memory footprint. |
18:26 |
cgra |
can't get rid of the idea that downloading a bounded buffer of N next blocks could somehow work, and without horror |
18:26 |
asciilifeform |
for cemented blox certainly could work |
18:27 |
asciilifeform |
tho because of the asinine pseudo-multithreadedness, only 1 peer can send/recv at a time |
18:27 |
cgra |
asciilifeform: if can find it for cement, would be enough, cuz 'fast enuff' for normal operation |
18:27 |
asciilifeform |
possibly even w/out this, 'fast enuff' |
18:28 |
* |
asciilifeform wrote errything but the part which actually bakes the getdata's. possibly tonight will finish & testfire |
18:31 |
* |
cgra keeps thinking of smth like "spread getdatas of the block range N to peers, follow peer response times and avoid overloading slow peers. buffer populates, verify block every time the lowest block in range N available, adjust buffer afterwards" |
18:33 |
asciilifeform |
gnarly. 'slow' peer at a given time is usually simply 1 who is busy. 30sec later the same may be 'fastest' peer. |
18:34 |
asciilifeform |
other problem is that any peer who is sending anyffin at all, ties up the whole noad. |
18:34 |
asciilifeform |
whether he's sending the requested block, a duplicate, or 'latest' crapola that aint wanted at all |
18:35 |
asciilifeform |
the practical pill is likely to be to use -connect and pick a reliable peer when syncing w/ cement |
18:37 |
asciilifeform |
the sheer halfbakedness of the whole design is mindboggling -- e.g. wai did shitoshi do the merkle hash thing, and then decided to send blocks as monolithic turd ? |
18:37 |
asciilifeform |
would've been entirely possib. to send the header, then the 1st level of merkelisms, then n+1st, etc. |
| |
↖ |
18:38 |
asciilifeform |
(for n00bz: block hash is a 'merkle tree', i.e. permits reordering tx and get same hash) |
| |
↖ |
18:39 |
* |
asciilifeform must bbl |
| |
~ 1 hours 1 minutes ~ |
19:40 |
* |
signpost tuned in, but sick. probably resurface next week or so. |
| |
↖ |
19:43 |
signpost |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-04#1078280 << this is done. |
| |
↖ |
19:43 |
dulapbot |
Logged on 2022-02-04 03:48:53 crtdaydreams: signpost: http://paste.deedbot.org/?id=h-Sc |
19:45 |
signpost |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-12#1079042 << good old sneakernet |
19:45 |
dulapbot |
Logged on 2022-02-12 21:44:32 whaack: lol maybe this is the right warez protocol (@ signpost), a mail-in your hard drive service |
19:45 |
signpost |
was burned CDs "back in my day" |
20:00 |
signpost |
probably best density to weight ratio for mailed items is going to be tape backup. |
| |
~ 21 minutes ~ |
20:21 |
asciilifeform |
signpost: afaik presently the winner of 'MB per kg for parcel' is mech. hdd. (with caveat that it takes 2nd place if both parties own a -- rather costly -- current-day tape drive) |
20:22 |
asciilifeform |
a pretty tight 2nd place tho |
20:24 |
asciilifeform |
this afaik is on acct of the fact that human-priced tape drives have virtually disappeared, and what remains is strictly in the 'tbtf' golden toilet market, where ancient laws concretely demand the use of tape |
20:25 |
crtdaydreams |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079662 << Thanking you kindly. |
20:25 |
dulapbot |
Logged on 2022-02-17 14:43:06 signpost: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-04#1078280 << this is done. |
20:25 |
* |
asciilifeform has buncha 8 and 4mm scsi tape decks, hasn't powered'em up in some yrs |
20:25 |
crtdaydreams |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079661 << Get well soon :) |
20:25 |
dulapbot |
Logged on 2022-02-17 14:40:36 signpost: tuned in, but sick. probably resurface next week or so. |
20:26 |
signpost |
asciilifeform: ah, that makes sense. |
20:27 |
* |
signpost looked into these recently and yep, was taken aback at the prices for drives. |
20:27 |
signpost |
ty crtdaydreams |
20:30 |
asciilifeform |
e.g.: 1 vendor offers lto-9 tape, i.e. 18tb, for ~200 $; a randomly-picked shop offers 18tb hdd for 325 $. |
20:31 |
asciilifeform |
the latter gives random access, can be bought in over9000 places, and doesn't need a ~5k deck to r/w it. |
20:32 |
asciilifeform |
but if you need over9000 x 18tb in-house -- the tape wins. |
20:32 |
crtdaydreams |
I spent some ~160 $ USD for 8TB D: |
20:32 |
asciilifeform |
crtdaydreams: when? |
20:32 |
crtdaydreams |
sec, Im going to get actual price |
20:33 |
crtdaydreams |
ST8000DM004 @ $195 AUD apiece |
20:34 |
crtdaydreams |
which is some $120 USD |
20:34 |
crtdaydreams |
mebbe less |
20:35 |
crtdaydreams |
the ex. rate is like .75 so $140 USD / 8TB |
20:35 |
asciilifeform |
sounds moar or less ordinary |
20:35 |
crtdaydreams |
Yeah, It's not like WD Blacks or anything. |
20:35 |
asciilifeform |
the mega-drives always cost moar 'per mb' than the 'garden variety' |
20:36 |
asciilifeform |
folx who actually need to pack 'as much as can' into $cabinet, pay the premium |
20:36 |
crtdaydreams |
hehe data hoarders |
20:37 |
asciilifeform |
for folx who are buying handful and not over9000, normally makes sense to simply get n moar of the thinner ones |
20:37 |
crtdaydreams |
Mhm |
20:38 |
* |
crtdaydreams requires morning sustenance |
20:41 |
crtdaydreams |
Speaking of economy of scaled, you'd expect moar drives == moar power. There's a balance to be struck with premium disk |
20:42 |
asciilifeform |
moar power, moar space, moar cooling cost |
20:43 |
crtdaydreams |
well, less (physical space) per drive? The whole point is to strike up a balance between individual drive draw while increasing capacity in the same sized box, no? |
| |
~ 16 minutes ~ |
21:00 |
shinohai |
https://archive.is/LVgKK <<< lol DOJ has appointed a Director of Buterin's Waterfall. |
21:04 |
* |
shinohai likes how USSA always appoints "Czars" but we never get any with those sweet Brezhnev eyebrows ..... |
| |
~ 2 hours 20 minutes ~ |
23:25 |
verisimilitude |
http://logs.nosuchlabs.com/log/asciilifeform/2022-02-17#1079576 That's impressive in its awful nature. I recall asciilifeform calling Bitcoin Microsoft-style shitware, but that's impressive all the same. |
23:25 |
dulapbot |
Logged on 2022-02-17 05:24:00 cgra: http://logs.nosuchlabs.com/log/asciilifeform/2022-02-16#1079525 << to give a some estimate, if a block verifies in 5 sec on average, 720000*5/60/60/24 = about 42 days to verify the whole chain. the 5 secs may be a gross underestimate for the apu1 box, is merely a hunch based on my sync tests with an 9th gen usg-intel cpu |
23:28 |
verisimilitude |
http://logs.nosuchlabs.com/log/asciilifeform/2022-01-18#1074264 I forgot to explicitly note this, jonsykkel, but this is how I write an ``ancient Greek program'', or more appropriately an ``ancient Roman program''. |
23:28 |
dulapbot |
Logged on 2022-01-18 17:02:10 jonsykkel: http://logs.nosuchlabs.com/log/asciilifeform/2022-01-18#1074159 << looking 4wd to reading |
23:29 |
verisimilitude |
I'd be elated to know what was thought about it. I'll be working on a successor article and the Ada version of the program described in this article, today, but I've been mulling over details for weeks anyway. |