#asciilifeform | 2022-07-26

peers that care to do so reply with an offer to supply a stream of encoded "check blocks" in OC terms (encoded message blocks).

02:03

requester then confirms that he wants such a stream, and the encoded chunks are supplied as direct messages to him.

02:04

there are details that would be included in each step which I skimmed over, but this is a sketch.

02:04

a nice thing about using OC is you don't have to have any of the peers coordinate, i.e. "A pls send M1-M3, B pls M4-6" etc

02:05

they generate their own streams and they're useful together.

02:05

efficiently, even.

02:06

my pyencoder is up on blog

02:06

I stopped working on this because I don't want to dance around the GIL to make it faster.

↖

02:07

there are tests included there which explain a lot, and which all pass.

02:07

I've got a WIP lisp version on my desk that doesn't work yet, but if there's interest I'll try to get it grunted out soon.

~ 6 hours 13 minutes ~

08:21

signpost: makes sense. if you make it a broadcast tho, will be riotously expensive, as l2+ would have to somehow pass the replies to the requester, and the only way to do so is to broadcast right back

08:21

dulapbot

(pest) 2022-07-26 asciilifeform: http://logs.nosuchlabs.com/log/pest/2022-07-25#1010638 << imho could make sense to have a lubytron where can ask >1 peer for $frag; but not l2+ (cuz ruinously, 'geometrically' expensive, and the traffic will be pure noise to all but the 1 intended recipient)

08:26

... could in principle have it so lubyism request aint rebroadcast (i.e. goes strictly to l1)

~ 5 hours 39 minutes ~

14:06

http://logs.nosuchlabs.com/log/asciilifeform/2022-07-26#1112252 << if this is the case does it make sense to continue work on blatta? I suspect most (including myself) will prefer to run a lisp station when one is eventually published. At the moment I can't think of a situation where one could run a python implementation that one couldn't also run a lisp implementation.

14:06

dulapbot

Logged on 2022-07-26 02:06:02 signpost: I stopped working on this because I don't want to dance around the GIL to make it faster.

14:06

I understand that it's good for the health of the network to have many implementaitons, but python ultimately seems like a dead end

14:09

*implementations

~ 15 minutes ~

14:24

it's not impossible to work around, I just didn't want to. multiprocessing with queues would probably work fine for both OC and blatta.

14:24

I don't think it's time to throw anything away yet.

14:25

*

signpost made some progress towards sharing a numpy array backed by shared memory between multiprocessing instances for ocpy

14:25

aight. will keep hacking away on PROD then.

14:32

thimbronion: atm blatta is the flagship, fwiw

14:32

afaik almost all folx on pestnet, currently using it.

14:33

'bird^H^H^Hblatta in the hand is worth 2 in cl' or how did it go.

14:33

asciilifeform: hey not like I can delete it from the internet lol

14:34

sure but imho makes sense to carry on with it, if sumbody posts a clean cl impl can always switch then

14:34

supposing thimbronion has the cycles

14:37

*

thimbronion currently has nothing but cycles.

14:37

*

asciilifeform envious, lol

14:39

also, respectfully, fuck this purism. we should be so lucky as to have so many implementations of a necessary item that we can dismiss some.

14:39

:)

14:40

^

14:46

also my py decoder can do 10MB in 1.3sec; it's not like it's terrible on single core, just not gonna saturate a 1gbps line.

14:46

*on this machine

14:46

3ghz i7 lappy

~ 35 minutes ~

15:22

notbad

15:23

signpost: is that 10MB of input chunks, or of payload (if the latter, what mass of chunks appeared in the test?)

~ 19 minutes ~

15:42

10MB of payload in 1300 byte chunks. that'd require discovery of MTU, which we can leave aside (though I'd benefit from understanding the rationale against doing MTU discovery). with 10MB payload cut into 324 byte chunks, 5.9sec to decode.

15:46

signpost: good q :

15:46

signpost: main arg against mtu discovery is the obv one

15:46

dulapbot

Logged on 2022-04-01 14:28:09 asciilifeform: verisimilitude: per pest conception, you wanna be able to ensure unfraggedness even if pesting from moving truck and wardriving, which precludes 'clever' path mtu discovery etc

15:47

signpost: the other is, of course, that it adds moving parts / complexity, for veehhery questionable gain

15:49

yeah, not the kind of thing to put on the table at this stage.

15:49

*

signpost hacking on the lisp version atm, will be interesting to compare performance.

15:49

as it is, one can pest immed. from any working ipv4 pipe

15:50

*

signpost doubtful he's anywhere near wrung all performance possible out.

15:51

signpost: other thought is, if you have heavier packets, verifying seal & decrypting costs you moar.

15:52

primary focus of protocol from asciilifeform's pov was ddos resistance, concretely: maximizing rate at which martians, dupes, & stales can be thrown out.

15:53

i.e. minimizing # of cpu cycles, within reason, req'd by this procedure

15:53

yes, and even with large payloads it is better to be able to move them at all under attack than move at maximum throughput.

15:53

aha

15:55

( throwing out a martian, in this case a packet where seal != H(Ks) for any Ks in yer wot , only requires hashing; however, tossing out a replay dupe requires decrypting

15:55

)

15:55

( either that, or storing'em, as somebody suggested, which is a nonstarter, attacker can collect arbitrary many dupes from arbitrary interval for replay )

15:56

per doctrine of 'Nothing to the Stranger', absolutely nuffin is carried in plaintext. so you gotta verify seal & decrypt, potentially for each incoming packet, and this operation oughta be as fast as physically possible.

15:57

cool, 100% agree that keeping this efficient trumps increasing packet size.

15:57

( moar pedantically, per 'Nothing to the Snoop' rly )

15:58

a++

16:00

signpost: 1 way to think about it, is that when you process a packet, yer 'extending credit' to the sender, giving him a certain # of cpu cycles (and occupying a certain amt of storage while it happens)

↖

16:02

( per asciilifeform's napkin calcs, already current protocol, even in a hypothetical 'tight' compiled impl., will need 32 'opteron' cores to actually eat at 1g/s )

16:05

... ergo if processing an incoming packet takes longer than the interval req'd to receive it, yer 'running a fractional reserve'.

16:06

asciilifeform would be that nobody's aboutta receive legit pest traffic at g/s, so this observation chiefly concerns liquishit from ddosers.

16:06

*would bet

16:08