#pest | 2023-02-09

pest - 9h 7m

Show Idle (>14 d.) Chans

← 2023-02-08 | 2023-02-10 → ↓

09:49	phf	it's still code some corner issues and the code needs to be cleaned up, but pest.lisp now incrementally places messages based on their netchain/selfchain and timestamps
09:50	phf	*still got some
		~ 30 minutes ~
10:21	asciilifeform	phf: neato
10:27	phf	test
10:34	phf	i'm by the way getting a constant stream of addresscasts http://paste.deedbot.org/?id=fEJ2
10:47	asciilifeform	phf: per spec, these only get sent to 'cold' peers; likely bug sumwhere
10:47	asciilifeform	( unless these are actually 'cold' vs. phf, impossible to tell from here )
10:48	asciilifeform	phf: does your prototype eat these (and unable to 'warm' $peer on acct of need for nat hammer..?) or notyet ?
10:49	phf	well, my counterparties have a bunch of cold peers, so they are just spamming everyone with addresscast requests
10:49	asciilifeform	a! that'd make sense
10:50	phf	i'm sure if you were to turn on or hack in packet logging in blatta you're going to get the same dilluge of addresscast requests
10:50	asciilifeform	probably
10:51	phf	i suspect pest traffic is addresscasts by weight at present
10:51	phf	*bulk of
10:52	*	asciilifeform not devised any way to avoid this, given a seekritkey-based pest, that'd be compatible with 'only peers have any biz knowing yer addr' dictum
10:52	asciilifeform	phf: nearly certainly tru
10:54	phf	well, i've proposed that "who you're looking for" should be part of the message, so at the very least peers can coordinate broadcast
10:55	*	asciilifeform recalls thrd. key problem there is that it leaks info publicly that perhaps the sender would not want (identities of 'cold' peers) leaked
10:56	*	asciilifeform also not convinced that it'd cut down the bw guzzle substantially -- you still gotta fwd the broadcast even if marked peer aint in your l1, he could be elsewhere in l3+
10:57	asciilifeform	notion behind addrcast was that the peer oughta receive it if he's anywhere (within max bounce) on the net.
10:59	phf	sure, but without very explicit algorithm, that's not at present in the spec, you're going to get blatta implementations, where peers spam network with constant addresscast requests
10:59	phf	well my experiment showed 2/3 reduction
11:00	phf	i mean "did peer X appear on the network within the last few seconds?" asked every few seconds is definitely not the way to go about it
11:02	phf	and the "cold" peers leak can be delt with in other ways, for example you have to scrub the name if you're rebroadcasting. this at least allows storngly connected peers to "coordinate" addresscasts
11:03	phf	at present you have, and i'm directly peered to dulapbot, deedbot and awt, a handful of stations yelling at each other about a small number of peers that have probably been down since forever, and will probably not come back anytime soon
11:05	phf	of course the main problem though is that blatta is a on backburner, so an iteration cycle on issues like that, that need to be explored and tuned, counts in weeks if not months.
		↖ ↖
		~ 37 minutes ~
11:43	asciilifeform	phf: i recall sumbody (possibly phf) suggested 'only send addrcast if saw the peer alive via hearsay'. may be worth considering
11:44	asciilifeform	(the tradeoffs, imho, obv.)
11:44	asciilifeform	would cut down on the 'shouting to people who aint coming any time soon' thing.
11:45	asciilifeform	possibly would not work well for bots, however, which rarely speak
11:58	*	asciilifeform still not satisfied that anyffin other than the current bw-guzzling algo would work reliably in a setting where folx connect through short-lived/unreliable pipes ('wardriving' etc)
11:58	asciilifeform	then again, it remains to be seen what, if anyffin, worx there.
11:59	asciilifeform	ideal is that links to peers oughta heal, after ip change, asap.
12:00	asciilifeform	( and whether or not you / the peer are particularly 'talkative' )
		~ 19 minutes ~
12:19	phf	i don't think current algorithm is related to wardriving, though
12:20	phf	you have to types of connection nated and direct, when a wardriver goes online he attempts to establish direct connections, and he always have to have at least one
12:20	phf	*two types
12:20	phf	having established that direct connection, he can send addresscast to all peers, ~and at that point he's done~
12:21	phf	because whoever's online and nated, will respond to addresscast and go into nat handshake
		↖
12:22	phf	but anybody who's offline, will do similar algorithm once he's online
12:22	phf	so purely to reestablish connectivity to all available peers, there's no need to spam network
12:24	phf	current algorithm prods for peers that are explicitly offline, with the goal of catching them as soon as they appear online
12:27	phf	but if you're peered with someone, they have as much an incentive in finding you, as you're in finding them, so more likely than not, when they are ready to talk, they'll either connect to you directly, or send an addresscast
12:28	phf	so if the question is ~just~ network self-healing, then you can addresscast at specific events, and then very rarely, and achieve desired result
12:29	phf	e.g. addresscast when you hear from peer on hearsay, when peer's "last packet" timeout goes beyond some threshold, when you come online yourself and can't direct connect to peer. there might be one or two more events like that, but that'll give you effective coverage
12:33	phf	􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and then not par
		↖ ↖
12:33	phf	􏿽ticularly care that some of my l1 are getting hearsay
12:40	phf	(to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)
		↖
12:47	phf	http://logs.bitdash.io/pest/2023-02-09#1022809 << *whoever's online, nated, or had his ip change since wardrive went offline
12:47	bitbot	Logged on 2023-02-09 12:21:36 phf[awt\|deedbot]: because whoever's online and nated, will respond to addresscast and go into nat handshake
		~ 30 minutes ~
13:18	asciilifeform	phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed
		↖
13:19	asciilifeform	http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
		↖ ↖
13:19	bitbot	Logged on 2023-02-09 12:33:08 phf[awt\|deedbot]: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and then not par
13:23	asciilifeform	current algo is indeed braindamaged in that it makes no attempt to determine whether 'i can't see $peer' is because ~own~ ip changed, or the latter wandered away.
13:23	asciilifeform	indeed in the former case oughta refrain from spamming addrcast.
13:24	*	asciilifeform will amend draft spec re subj
13:25	asciilifeform	( concretely: if you know that $peer was able to reach you at $addr, and prods indicate that $addr not changed since -- no reason to spam )
13:25	asciilifeform	kudos to phf for pointing this out .
13:26	asciilifeform	the 1 possible case not handled is if $peer somehow lost his AT cache. but not sure why he would w/out also losing keys
		~ 27 minutes ~
13:53	asciilifeform	http://logs.bitdash.io/pest/2023-02-09#1022818 << this also imho good idea (tho would need to think about when precisely oughta do it)
13:53	bitbot	Logged on 2023-02-09 12:40:22 phf[awt\|deedbot]: (to that last point, there's a fairly obvious algorithm for message "validation". whenever you establish a direct connection with l1, you can also walk over your log and for any packet that's hearsay from that peer, do a getdata directly to peer)
13:55	asciilifeform	( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )
		↖
13:57	asciilifeform	... and does blatta even currently store 'yes, i sent this' mark ? ( cuz if not, buncha stuff will get marked bogus. maybe this behaviour will need to be version-gated, then )
14:00	asciilifeform	http://logs.bitdash.io/pest/2023-02-09#1022816 << hammer prolly oughta default to off, aha
14:00	bitbot	Logged on 2023-02-09 12:33:08 phf[awt\|deedbot]: 􏿽also when wardriving, i'd probably have some kind of "hush" mode enabled. i wouldn't want entire pest network bombarding some potentially sensitive router with port discovery packets, that'll light up like christmas tree on any ids. i'll just connect to direct ips, and then not par
		~ 5 hours 30 minutes ~
19:30	phf	http://logs.bitdash.io/pest/2023-02-09#1022821 << yeah my pest instance is "mobile", so there's a clear point when need to do bookkeeping after suspend or network rotation, since there's a brief period when socket starts throwing system level exceptions
19:30	bitbot	Logged on 2023-02-09 13:18:30 asciilifeform[4]: phf: worth moar thought. esp. the very good point that station itself knows (for so long as can reach net at all, and receives prods) that its external ip changed
		↖ ↖
19:32	phf	but you're right can have similarlogic in prod receipt
19:35	phf	􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to green, whi
		↖
19:35	bitbot	Logged on 2023-02-09 13:19:54 asciilifeform[4]: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
19:35	phf	􏿽ch will initiate some elaborate discovery mechanism
19:37	phf	http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"
		↖
19:37	bitbot	Logged on 2023-02-09 13:19:54 asciilifeform[4]: http://logs.bitdash.io/pest/2023-02-09#1022816 << important to recall, tho, that you can't directmsg (or, when we've the luby thing -- warez) to anyone with whom direct peering is broken
19:37	phf	because if you don't see them, they most likely don't see you either, which will initiate discovery on their end also
19:38	phf	so if a peer "went silent" is aggressive discovery ever necessary?
19:42	phf	that was re http://logs.bitdash.io/pest/2023-02-09#1022832
19:42	bitbot	Logged on 2023-02-09 13:55:19 asciilifeform[awt\|jonsykkel\|deedbot]: ( and, in particular, wat-do if $peer does ~not~ return a 0bounce copy. mark it in red in log ? )
19:48	phf	asciilifeform, signpost, crtdaydreams, in related is any of you running a station that's doing addresscasts? because all the addresscast packets that i'm getting have ("awt" "unpx" "jonsykkel") as speakers
19:49	phf	theoretically i should be peered with all three of you, but i don't have dc established
19:55	phf	actually running a historgram for a few minutes, gives me (("jonsykkel" 56) ("awt" 179) ("unpx" 24))
19:57	phf	(("jonsykkel" 98) ("awt" 265) ("unpx" 42))
20:00	phf	(("jonsykkel" 126) ("awt" 326) ("unpx" 54))
20:03	phf	(("jonsykkel" 168) ("awt" 446) ("unpx" 72))
20:05	phf	(("jonsykkel" 210) ("awt" 584) ("unpx" 90))
		~ 21 minutes ~
20:27	phf	(("signpost" 79) ("jonsykkel" 504) ("awt" 1066) ("unpx" 215))
		~ 1 hours 1 minutes ~
21:28	asciilifeform	phf: can't speak for the rest, but currently still on blatta 9973 (where, iirc, not yet addrcast)
21:34	asciilifeform	http://logs.bitdash.io/pest/2023-02-09#1022842 << ( excluding the obv 'switched off box' ) afaik would just about always be one of a) at least one of you got new dynamic ip, or b) at least one of you is under nat and lost ephemeral port state
21:34	bitbot	Logged on 2023-02-09 19:37:11 phf[awt\|deedbot]: http://logs.bitdash.io/pest/2023-02-09#1022822 << well, more general question is under what circumstances and why would your peer "go silent"
21:35	asciilifeform	( if anyone can think of an unrelated 'c', plox to write in )
21:36	asciilifeform	( i suppose 'one or both of you moved box to new static ip' could count as 'c', but let's count it as 'a', lol )
21:37	asciilifeform	and imho it would make sense not to spam addrcast if 'you know it wasn't you' and $peer ought still have valid addr for you
21:39	asciilifeform	the remaining open q afaik is wat-do if you know it was you, but sending 1 addrcast not results in peer returning ( because, say, his station down just then ). how often to spam.
21:43	phf	in that last case why not wait till peer comes online and starts sending his own addresscast to find you?
21:44	asciilifeform	http://logs.bitdash.io/pest/2023-02-09#1022839 << imho requiring manual intervention for addrcast is an ugh -- some stations are automated/headless, e.g. dulapbot & similar
21:44	bitbot	Logged on 2023-02-09 19:35:34 phf[awt\|deedbot]: 􏿽http://logs.bitdash.io/pest/2023-02-09#1022822 << that might be a reasonable trade off, when you have some kind of interface feedback. peers are green or red, if you pop up on the network to say "hello" OR if you need to dc someone specific, click to take him from red to green, whi
21:44	asciilifeform	phf: maybe asciilifeform misunderstands, but sounds uncomfortably like 'if two trains meet on the tracks, neither shall move until the other has started'(tm)
21:45	phf	yah i can see that
21:47	phf	in the ideal scenario, you have two stations A and B, A changed ip address and B went offline. A sends out addresscast on account of changing ip, but B is offline. eventually B comes online, sends addresscast on account of coming, and not being able to connect to A.
21:47	phf	but
21:48	phf	if B is "offline" due to network issues in A->B direction, then B would never know anything was amis
21:48	phf	actually no, B will know that A's offline, because A eventually will stop sending garbage, etc.
21:48	asciilifeform	aha! B may be 'offline' only with respect to A.
21:49	asciilifeform	correct, but neither will be in 'i just booted, so let's send addrcast' state
21:49	phf	right
21:50	phf	this needs more thought, but at least "my ip is still the same, i'm not going to spam" is a solid change
		↖
21:50	asciilifeform	( 1 common, asciilifeform suspects, cause -- ephemeral nat state lost for 1 particular peer )
21:50	*	asciilifeform nods
21:50	asciilifeform	defo belongs in spec.
21:51	phf	(("signpost" 161) ("jonsykkel" 1401) ("awt" 2659) ("unpx" 599))
21:52	asciilifeform	( for thread-completeness -- the ~other~ purpose of addrcast is to make it so that a new peering where both sides are present on same pestnet not requires exchanging AT entries at all. so same q, how much to spam )
21:53	phf	the idea is that if i connected through signpost only, but i have peering with awt and unpx, i should be able to get later's ATs without explicit negotiation?
21:54	asciilifeform	supposing 1 and/or the other peers with signpost -- correct
21:54	phf	by explicit negotiaton i mean operator asking on channel, etc.
21:54	phf	right
21:54	asciilifeform	aha
21:56	asciilifeform	is mechanical (and confidential) substitute for 'ask him on pestnet what is his ip'
22:00	phf	just use fibonacci :D
22:02	*	asciilifeform in similar vein once suggested 'just send an 'ignore' to whole ipv4 space' lel
22:02	asciilifeform	on decent pipe, not even takes so long
22:02	asciilifeform	won't cure nat tho (quite the opposite..)
22:03	asciilifeform	iirc 'spam all of ipv4' takes ~40min on a gb/s pipe (if you have a decent map of the lacunae in the space, that is)
22:05	phf	also calling the person over phone and asking him for his new ip
22:06	asciilifeform	fwiw initial pest bringup already requires an out-of-band talk, so yes
22:07	asciilifeform	getting an addr, tho, unlike key exchange, oughn't require talking to $peer directly, is the point tho.
22:07	asciilifeform	merely talking to someone he's already able to talk to.
22:12	asciilifeform	( apropos, in the 'spam ipv4 thread', iirc noted that if it suffices to find ~one~ of your peers, then if you have n of'em, bringup of AT-less but properly keyed station is suddenly practical. only need to hit 1. )
22:15	asciilifeform	may seem idiotic, but can think of at least 1 scenario where it'd win -- e.g. sumbody who switches on station very infrequently
22:16	asciilifeform	'i was at sea for a year and where did erryone go'
22:17	asciilifeform	in principle, pest oughta take ~erry~ possible advantage of the fact that you only need to get 1 packet through to establish a link w/ peer.
22:18	phf	i like the idea of leaving pest bots in various corners of the internet
22:19	asciilifeform	phf: a la deedbot on dulapnet ? wainot
22:20	*	asciilifeform recalls thrd re www doors etc
22:20	asciilifeform	intended use there, tho, is drawing in n00bs
22:21	phf	right, you still need an addresscast. you can e.g. get a temp peer with deedbot, and then broadcast through it
22:22	asciilifeform	correct. simply pointing out that in principle not necessary for existing (i.e. keyed) peer to find such a corner, even in worst case where both sides ip-drifted in such a way that can reach no one
22:23	asciilifeform	~40min of spam cannon and yer back in biz
22:23	asciilifeform	(on acct of 'bday paradox', likely much less )
22:25	asciilifeform	i.e. hypothetically, pest letsya treat the net as a (sloow) broadcast medium, if necessary.
		↖
22:26	asciilifeform	possibly could even have client where omit manual AT entry entirely...
22:28	asciilifeform	sounds like a lul, but think, yer on 'mobile' station, ip changes maybe multiple times each day, and you meet somebody and want to exchange keys. not even makes sense to give him an at entry.
22:29	asciilifeform	a fully grown pest client oughta include the cannon, and then can peer even if yer both 'vagrants' with 'no fixed place of residence'(tm)
22:30	asciilifeform	( tho at least 1 side oughta be un-nat'd, obv )
22:33	phf	yeah, that nat problem makes it not as elegant as it could be
22:34	asciilifeform	for so long as pestnet not consists entirely of hobos , still worx
22:36	phf	in reality will probably really on a handful of heavily connected nodes, like deedbot and dulapbot
22:37	phf	like once i work out the kinks(tm), will stand up a logging station like btcbase, which will probably end up being one of just a handful of peers for the roaming station
22:39	*	asciilifeform considered a 'slave mode' which merely fwds packets, but this is tricky for the obv. reasons
22:40	asciilifeform	in 0xfb, instead proposed 'slave mode' where msgs coming through $peer (who you know to be one of your personal bots) are treated as 0bounce. but this afaik not implemented anywhere atm.
22:40	asciilifeform	(and not handles direct msgs in any way)
22:41	phf	it's fairly trivial to cook up a relay bot out of the bits in pest.lisp
22:42	asciilifeform	( from e.g. asciilifeform's pov, a bounce-1 via dulapbot is precisely as authentic as a bounce-0 )
22:42	phf	right
22:43	asciilifeform	the elegant pill is to make it simply a one turn of knob setting, with stock clients, rather than requiring a dedicated bot proggy
22:43	asciilifeform	then -- set up bot in proper dc, and is as good as sitting there yerself
22:44	asciilifeform	( the obv. downside is that if said box is stolen, you gotta issue new keys out of band. )
		~ 23 minutes ~
23:07	phf	i remember mp at some point objected to everyone idling on bouncers, after having voiced themselves, on account of opsec. he wasn't too insistent on it, so it didn't go anywhere, but the problem remains.

← 2023-02-08 | 2023-02-10 → ↑