Show Idle (>14 d.) Chans

← 2022-07-25 | 2022-07-27 →
01:58 * signpost has been attending to meatspace backlog but will get back into this soon.
01:59 signpost don't let me block other attempts at the problem either.
02:01 signpost using online-codes, the scheme would be to send to peers a request to transfer the item identified by $HASH. perhaps this is a message type that's part of the broadcast chain of messages.
02:02 signpost peers that care to do so reply with an offer to supply a stream of encoded "check blocks" in OC terms (encoded message blocks).
02:03 signpost requester then confirms that he wants such a stream, and the encoded chunks are supplied as direct messages to him.
02:04 signpost there are details that would be included in each step which I skimmed over, but this is a sketch.
02:04 signpost a nice thing about using OC is you don't have to have any of the peers coordinate, i.e. "A pls send M1-M3, B pls M4-6" etc
02:05 signpost they generate their own streams and they're useful together.
02:05 signpost efficiently, even.
02:06 signpost my pyencoder is up on blog
02:06 signpost I stopped working on this because I don't want to dance around the GIL to make it faster.
02:07 signpost there are tests included there which explain a lot, and which all pass.
02:07 signpost I've got a WIP lisp version on my desk that doesn't work yet, but if there's interest I'll try to get it grunted out soon.
~ 6 hours 13 minutes ~
08:21 asciilifeform signpost: makes sense. if you make it a broadcast tho, will be riotously expensive, as l2+ would have to somehow pass the replies to the requester, and the only way to do so is to broadcast right back
08:21 dulapbot (pest) 2022-07-26 asciilifeform: << imho could make sense to have a lubytron where can ask >1 peer for $frag; but not l2+ (cuz ruinously, 'geometrically' expensive, and the traffic will be pure noise to all but the 1 intended recipient)
08:26 asciilifeform ... could in principle have it so lubyism request aint rebroadcast (i.e. goes strictly to l1)
~ 5 hours 39 minutes ~
14:06 thimbronion << if this is the case does it make sense to continue work on blatta? I suspect most (including myself) will prefer to run a lisp station when one is eventually published. At the moment I can't think of a situation where one could run a python implementation that one couldn't also run a lisp implementation.
14:06 dulapbot Logged on 2022-07-26 02:06:02 signpost: I stopped working on this because I don't want to dance around the GIL to make it faster.
14:06 thimbronion I understand that it's good for the health of the network to have many implementaitons, but python ultimately seems like a dead end
14:09 thimbronion *implementations
~ 15 minutes ~
14:24 signpost it's not impossible to work around, I just didn't want to. multiprocessing with queues would probably work fine for both OC and blatta.
14:24 signpost I don't think it's time to throw anything away yet.
14:25 * signpost made some progress towards sharing a numpy array backed by shared memory between multiprocessing instances for ocpy
14:25 thimbronion aight. will keep hacking away on PROD then.
14:32 asciilifeform thimbronion: atm blatta is the flagship, fwiw
14:32 asciilifeform afaik almost all folx on pestnet, currently using it.
14:33 asciilifeform 'bird^H^H^Hblatta in the hand is worth 2 in cl' or how did it go.
14:33 thimbronion asciilifeform: hey not like I can delete it from the internet lol
14:34 asciilifeform sure but imho makes sense to carry on with it, if sumbody posts a clean cl impl can always switch then
14:34 asciilifeform supposing thimbronion has the cycles
14:37 * thimbronion currently has nothing but cycles.
14:37 * asciilifeform envious, lol
14:39 signpost also, respectfully, fuck this purism. we should be so lucky as to have so many implementations of a necessary item that we can dismiss some.
14:39 signpost :)
14:40 asciilifeform ^
14:46 signpost also my py decoder can do 10MB in 1.3sec; it's not like it's terrible on single core, just not gonna saturate a 1gbps line.
14:46 signpost *on this machine
14:46 signpost 3ghz i7 lappy
~ 35 minutes ~
15:22 asciilifeform notbad
15:23 asciilifeform signpost: is that 10MB of input chunks, or of payload (if the latter, what mass of chunks appeared in the test?)
~ 19 minutes ~
15:42 signpost 10MB of payload in 1300 byte chunks. that'd require discovery of MTU, which we can leave aside (though I'd benefit from understanding the rationale against doing MTU discovery). with 10MB payload cut into 324 byte chunks, 5.9sec to decode.
15:46 asciilifeform signpost: good q :
15:46 asciilifeform signpost: main arg against mtu discovery is the obv one
15:46 dulapbot Logged on 2022-04-01 14:28:09 asciilifeform: verisimilitude: per pest conception, you wanna be able to ensure unfraggedness even if pesting from moving truck and wardriving, which precludes 'clever' path mtu discovery etc
15:47 asciilifeform signpost: the other is, of course, that it adds moving parts / complexity, for veehhery questionable gain
15:49 signpost yeah, not the kind of thing to put on the table at this stage.
15:49 * signpost hacking on the lisp version atm, will be interesting to compare performance.
15:49 asciilifeform as it is, one can pest immed. from any working ipv4 pipe
15:50 * signpost doubtful he's anywhere near wrung all performance possible out.
15:51 asciilifeform signpost: other thought is, if you have heavier packets, verifying seal & decrypting costs you moar.
15:52 asciilifeform primary focus of protocol from asciilifeform's pov was ddos resistance, concretely: maximizing rate at which martians, dupes, & stales can be thrown out.
15:53 asciilifeform i.e. minimizing # of cpu cycles, within reason, req'd by this procedure
15:53 signpost yes, and even with large payloads it is better to be able to move them at all under attack than move at maximum throughput.
15:53 asciilifeform aha
15:55 asciilifeform ( throwing out a martian, in this case a packet where seal != H(Ks) for any Ks in yer wot , only requires hashing; however, tossing out a replay dupe requires decrypting
15:55 asciilifeform )
15:55 asciilifeform ( either that, or storing'em, as somebody suggested, which is a nonstarter, attacker can collect arbitrary many dupes from arbitrary interval for replay )
15:56 asciilifeform per doctrine of 'Nothing to the Stranger', absolutely nuffin is carried in plaintext. so you gotta verify seal & decrypt, potentially for each incoming packet, and this operation oughta be as fast as physically possible.
15:57 signpost cool, 100% agree that keeping this efficient trumps increasing packet size.
15:57 asciilifeform ( moar pedantically, per 'Nothing to the Snoop' rly )
15:58 asciilifeform a++
16:00 asciilifeform signpost: 1 way to think about it, is that when you process a packet, yer 'extending credit' to the sender, giving him a certain # of cpu cycles (and occupying a certain amt of storage while it happens)
16:02 asciilifeform ( per asciilifeform's napkin calcs, already current protocol, even in a hypothetical 'tight' compiled impl., will need 32 'opteron' cores to actually eat at 1g/s )
16:05 asciilifeform ... ergo if processing an incoming packet takes longer than the interval req'd to receive it, yer 'running a fractional reserve'.
16:06 asciilifeform asciilifeform would be that nobody's aboutta receive legit pest traffic at g/s, so this observation chiefly concerns liquishit from ddosers.
16:06 asciilifeform *would bet
16:08 asciilifeform see also this thrd.
16:08 dulapbot Logged on 2021-09-23 15:14:30 asciilifeform: PeterL: let's assume ethernet. so, in bytes, let's calculate 1 packet's mass: 5 (gap) + 4 (preamble) + 14 (eth header) + 20 (ipv4 hdr) + 8 (udp header) + 496 (pest) + 4 (ethernet crc) == 551
16:10 asciilifeform ( see also this piece, re actual maximum packet rate on ethernet, is considerably lower than above expectation )
~ 31 minutes ~
16:42 signpost extending credit is a great way to think about it.
16:42 * signpost will read linked poast upon return from errands, bbl!
~ 5 hours 31 minutes ~
22:14 asciilifeform signpost: thinking about it, one only needs to decrypt 8 bytes (well, 16, being the serpent block size) past the nonce, to see timestamp. but still.
← 2022-07-25 | 2022-07-27 →