Show Idle (>14 d.) Chans
← 2023-02-12 | 2023-02-14 → ↓
01:43 jonsykkel unpx: crtdaydreams: AAAAAAAAAAAAACHHHHHHHHHHTUNG!!!!!! CRITICAL update, fixing unset netchain: http://zzz.st/progz/
↖
01:47 jonsykkel i will now, as the only reasonable course of action, commit sudoku in town square as atonement for this mistake
~ 1 hours 3 minutes ~
02:50 cgra jonsykkel, will you provide outdoor town square photos of 1) clean sudoku page, 2) same sudoku completed, as proof?
02:53 jonsykkel cgra: ofc, with timestamps and ID visible
03:05 cgra responsibility appreciated
03:07 unpx hm, my station missed a message after upgrade. Will try to debug this.
03:09 jonsykkel unpx: which message in particular?
03:11 jonsykkel pls to paste tail of smalpest log
~ 31 minutes ~
03:43 unpx cgra's before yours, but looks oke
~ 1 hours 25 minutes ~
05:08 cgra awt, peering info for you if want. i might have some blatta details to ask about, but perhaps not generally interesting points
05:18 cgra http://logs.bitdash.io/pest/2023-02-12#1023114 << address cast somewhat useful here, because my included AT entry above is already outdated
05:18 bitbot Logged on 2023-02-12 14:41:07 awt: I spent a lot of time getting it to a functional state and I would really like to see it get used.
~ 1 hours 18 minutes ~
06:36 cgra if, as specced in current 0xfa draft (4.2.1.3, 4.2.1.2), timestamps in a chain were never decreasing, would it guarantee a reasonable order of messages if ordered primarily by timestamp and secodarily by chain dependency? or were there known issues with this appro
↖
06:36 cgra ach, too?
06:40 cgra there's that waiting of peer whose clock is running fast, which i'm aware of, but asking specifically about ordering problems
~ 23 minutes ~
07:04 cgra re peer with fast-running clock: footnote 21 states that you have to wait the time difference before sending your next-in-chain message. any obv downside to just add the difference to your new message's timestamp and send immediately instead?
~ 2 hours 49 minutes ~
09:53 asciilifeform cgra: that part of the spec defo needs over9000 moar thought. (adding to your timestamp, for instance -- then it's same as if yer own clock runs fast, with the obv consequences!)
10:00 asciilifeform is also likely to result in msgs being mysteriously discarded , when sumbody else's reply 'outruns' yours (client would need to notice this and reissue ?) -- gnarly
10:00 asciilifeform ( worse yet, discarded but ~not by all stations~ , depending on who outruns whom )
10:01 awt cgra
10:01 asciilifeform the 'monotonic timestamp' thing in 0xfa will prolly have to go.
10:01 awt cgra: will peer
10:01 cgra asciilifeform, what do you mean by 'outrun'?
10:02 cgra awt, ty!
10:14 asciilifeform cgra: 'outrun' i.e. 2 stations issue replies with same netchain (to same prev msg) and some net participants receive 1 prior to the other
10:16 asciilifeform suppose both have same timestamp (or the 'winner', at particular receiver, has greater one than the 'loser'). nao the 'loser', from that station's pov, is malformed
10:16 asciilifeform this is a catastrophic lul because potentially splits the net, in the sense where there now exists a msg that is valid on some stations but not others
10:18 asciilifeform (and yes, current thought is that malformed msgs get stored anyway, and displayed with special markings, as they may otherwise end up getdata'd infinitely, if they were discarded. but still ugh)
10:19 asciilifeform validity of a msg should not depend on the moar or less random order in which packets may be received.
10:21 asciilifeform 1 possible pill would be to keep the monotonicity requirement, but 'loser' notices that he 'lost' and auto-reissues $msg
10:21 asciilifeform gnarly tho
10:21 cgra asciilifeform, you mean no two netchains may refer to same message? aren't they racing like that independent of fast clocks?
10:23 asciilifeform cgra: per current 0xfa predraft, 'NetChain of a Message may not refer to one with a Timestamp greater than its own'
10:24 * asciilifeform may be mistaken re the implication, will have to think
10:24 asciilifeform come to think of it, pretty much errything above is fulla shit, so nm
10:25 * asciilifeform still waking up, lol
10:25 cgra hehe, right
10:25 cgra asciilifeform best caught off guard -- right off bed
10:37 cgra asciilifeform, was thinking that the main consequence of your clock running fast is your messages propagate as getdata only, once too much time difference. if otoh, you wait to catch up the difference, might end up with a slower propagation. and faster propagation --> faster 'whose clock's off here?'
10:45 cgra http://logs.bitdash.io/pest/2023-02-07#1022462 << btw, one consideration: you'd lose message reception times and break the ordering of old messages. old messages have two problems: occasional non-monotonic timestamp progression, and unreliable netchain path (contains zero netchain and netchain=selfchain messages). these to
10:45 bitbot Logged on 2023-02-07 13:56:31 asciilifeform[5]: cgra: fwiw asciilifeform has no plans to migrate anyffin other than keys from blatta db ( wainot let station sync from net the proper way ? )
10:45 cgra gether leaves the reception time the best sorting criteria, especially for high-uptime stations
~ 20 minutes ~
11:05 phf 􏿽http://logs.bitdash.io/pest/2023-02-13#1023172 << netchain/selfchain is an explicit claim by the sender about what he's implicitly replying to. the problem with `who cares about small drift` argument, is that it is primarily semantically an issue in a near-realtime communication, w
11:05 bitbot Logged on 2023-02-13 06:36:22 cgra[jonsykkel]: if, as specced in current 0xfa draft (4.2.1.3, 4.2.1.2), timestamps in a chain were never decreasing, would it guarantee a reasonable order of messages if ordered primarily by timestamp and secodarily by chain dependency? or were there known issues with this appro
11:05 phf 􏿽here small drift will be counter acted by packet delay, but the semantics rely heavily on correct ordering. in other words in cases of `and turns out she was a fish!` `lol!`, that lol really should go after the `joke`
11:12 cgra phf, if 'monotonic timestamp', 'lol' sender must've seen the joke already, so will have a >= timestamp. if equal timestamps, netchain decides order
11:15 phf ah, i got the idea, but i don't think it's in spec. the way i read it monotonic timestamp is per-machine, but there's no requirement that timestamp must be >= lastmessage, or am i missing it?
11:18 cgra phf, in 0xfa predraft, i'm looking at sections 4.2.1.2 and 4.2.1.3
11:20 phf ah, interesting, that seems like a new requirement, or maybe i just punted on it
↖
11:20 cgra fun fact, looking at this today, tried to search "Timestamp greater than" from every spec version and didn't find. turned out firefox pdf search tripped on link + italic, or some detail in it, and didn't match
↖
11:20 bitbot Logged on 2022-11-15 10:54:31 jonsykkel: pg.22 "Note that the NetChain of a Message may not refer to one with a Timestamp greater than its own" << this efectively requires cloks to be synced within seconds, doesnt it. how to know if someone else is fast or you is slow
11:21 cgra phf, first time introduced in that doc afaik
11:22 phf well, considering that i've re-read entire knuth to implement the graph solution, and this requirement basically eleminates sort order as a problem...
11:28 cgra right :P
11:36 phf but on other hand i think ascii is smoking crack :> per jonsykkel a packet from a station with (possibly temporarily) broken clock results in entire network doing ???
~ 38 minutes ~
12:14 cgra phf, was this a pest comment, got log link handy?
↖
12:18 asciilifeform phf: confirmed re crack, it doesn't of course permanently push ~erryone's~ time fwd
12:18 asciilifeform will only 'push' until there's a natural pause
12:20 asciilifeform still potentially headache, if e.g. asciilifeform's clock is 14min 59s 'fast' vs erryone else's, nao any immed. replies have a strong chance of expiring before being received
12:21 asciilifeform http://logs.bitdash.io/pest/2023-02-13#1023208 << not new, iirc that was in the 1st copy of 0xfa 'pre-draft' asciilifeform posted
12:21 bitbot Logged on 2023-02-13 11:20:13 phf[awt|deedbot]: ah, interesting, that seems like a new requirement, or maybe i just punted on it
12:22 asciilifeform (which was why asciilifeform was puzzled re phf's dig into graph walkers)
↖
12:22 asciilifeform may yet turn out to need the graph walker tho, not clear atm to asciilifeform that we don;t in fact
12:32 phf http://logs.bitdash.io/pest/2023-02-13#1023215 << your own link, http://logs.bitdash.io/pest/2023-02-13#1023209
12:32 bitbot Logged on 2023-02-13 12:14:46 cgra[jonsykkel|signpost]: phf, was this a pest comment, got log link handy?
12:32 bitbot Logged on 2023-02-13 11:20:33 cgra[jonsykkel|signpost]: fun fact, looking at this today, tried to search "Timestamp greater than" from every spec version and didn't find. turned out firefox pdf search tripped on link + italic, or some detail in it, and didn't match
12:34 phf http://logs.bitdash.io/pest/2023-02-13#1023221 << graph walker idea predates the current spec, and i just thought it was "cool" :) which is also how e.g. btcbase has a very very fast grep over logs
12:34 bitbot Logged on 2023-02-13 12:22:08 asciilifeform[4]: (which was why asciilifeform was puzzled re phf's dig into graph walkers)
12:37 phf also imho a graph walker solution is appropriate for a lisp implementation, on account of a kind of retro-grandeur that's inherent to all the approaches that those guys took. bit twiddling fighting for each byte c virgin vs my chat program requires a 3 hour GC every night genera giga-chad
~ 21 minutes ~
12:58 phf i already learned something as a result, in khan if you replace the set of all nodes S with a priority queue, you get the secondary constraint for when graph doesn't enforce one "for free"
12:58 phf which in our case is netchain/selfchain for graph, and timestamp for priority queue
13:02 phf i can theoretically ignore the timestamp entirely, and use arrival order for secondary constraint
13:05 awt cgra: address cast seems to have worked using the key you gave me
13:08 cgra apparently!
13:13 asciilifeform graph walker will surely come in handy sumwhere.
13:13 asciilifeform (maybe replace the rubbish one in vtron... )
13:16 asciilifeform re timestamps -- asciilifeform still thinking about this possible algo
13:16 bitbot Logged on 2022-12-25 15:59:26 asciilifeform[jonsykkel|awt|deedbot]: for thread-completeness, anuther solution could be to actually track clock diff. b/w you and $peer, and treat binary msgs as expiring in e.g. 1min from timestamp (adjusted for the diff) rather than 15.
13:17 asciilifeform ( i.e. actually track the clock drifts, rather than always sitting on ~erry~ msg for up to 30min-epsilon )
13:19 asciilifeform ... if a peer is 'warm', yer clock can be thought of as effectively synced
13:25 phf some kind of running average on clock_time vs packet_time_stamp across all peers
13:25 asciilifeform imho averaging wouldn't getcha anyffin but a slow collective drift
13:27 asciilifeform instead track delta per peer. if $peer is 'warm', you now have effectively synced clocks, within bounds of oscillator noise
13:27 phf well, we're already agreeing that packets reporting 15 minute difference from their actual time ain't a big deal. what's a little drift among friends
13:28 asciilifeform lol 14min 59s drift 'no big deal', 15 -- goes to /dev/null and ends up having to be getdata'd (which in turn requires at least 1 to get through, at least a prod, to trigger)
13:29 asciilifeform but what you ~actually~ want is simply to be immune to replays. which can get (for 'warm' peer) by tracking the actual delta, and nao your expir time with respect to that peer can be e.g. 10sec, or whatever multiple of the ping time with that peer yer comfortable with, rather than +/-15min
13:29 asciilifeform ( for msg from previously 'cold' peer, still need sumthing like the old logic , tho, afaik )
13:31 asciilifeform the point of this, if not obv, is simply to unclutter the antireplay buffer.
↖
13:31 asciilifeform ( 'filter', in 0xfa predraft, sec. 5.3.1. )
13:34 asciilifeform ( orig thrd re subj )
13:34 bitbot Logged on 2022-12-25 12:39:07 asciilifeform[5]: jonsykkel et al : runnable example of asciilifeform's filter . and somewhat braindead demo coad for same.
13:34 phf ah yeah i just ignore this kind of implementation policy details :}
13:35 * asciilifeform also 'i'ma ignore the boring parts', is how we ended up with the lulnurseries etc. headaches
13:35 asciilifeform at some pt gotta unignore'em, to get usable proggy
13:36 phf well, no lulnurseries are a a result of somebody explicitly not ignoring your prescriptions!
13:36 asciilifeform other side of same medal, neh
13:37 asciilifeform i.e. asciilifeform not properly worked out implications of proposed algo, result is a barfalicious rube goldbergism
13:42 phf well, it's more like i have a live system that's being constructed as it's being used, so i don't have to do it athena style. i can e.g. observe real effects and adjust things as needed
13:42 phf right now the entire history of pest, pushed into a hash-table, returns a result in under ms
13:42 asciilifeform in general imho this is sound, but in actual practice potentially painful, so makes sense to at least a little think-before-doing
13:44 phf in fact doing a vector store, so one array for hash, one array for message, where indexes are the same, will give a naive hash query of walking the hash array until you find the right one in under ms even for the whole history of republican logs
13:46 phf and if you run out of performance space there you do what them industrial routers do and just bloom filter the hell out of it
13:46 asciilifeform no doubt thing small enuff that almost fits in l1 cache, for nao
13:46 asciilifeform orig. thrd was re wat-do when gb's of warez moving b/w peers, tho
13:46 asciilifeform then -- no longer '1ms'
13:47 asciilifeform tracking clock deltas would letcha keep the antireplay guarantee while escaping sudden need for buying shoebox fulla ram stix
13:47 phf but we don't have implementation of gb's of warez
13:48 asciilifeform (recall that filter , per current predraft spec, stores hashes of ~all~, not only chained, msgs )
13:48 phf right now the most we know about it, is that it's probably going to be luby and there's going to be a special inline message type to announce it
13:48 asciilifeform phf: indeed we haven't yet. but will, and the moving parts gotta work correctly under warez conditions
13:49 asciilifeform key there is that these msgs will still end up hashed and hashes kept in filter (otherwise yer open to replay)
13:49 asciilifeform unless (as someone -- phf? -- proposed?) they can be discarded quickly via a different heuristic in the case of replay
13:52 asciilifeform fwiw asciilifeform expects that once warez starts moving regularly, most replay attemps featuring randomly stolen traffic will logically involve bin msgs
13:53 asciilifeform (given as they'll constitute the bulk of pest traffic)
13:54 asciilifeform atm, noshit, 'uncatchable joe', and we can't reply on 'naturally occurring' jokers to test the antireplay logic, will need to do it deliberately
13:54 asciilifeform *rely on
13:55 asciilifeform ( and for that matter, most of the possible 'misbehaviour space' of the protocol remains unexplored )
13:56 phf but wait if you want timestamp to be anti-replay, than it has at be > rather than >=
13:56 asciilifeform ( and see also. this is 1 place where arguably gotta get it right ab initio, rather than iteratively )
13:56 bitbot Logged on 2023-01-29 12:41:45 asciilifeform[4]: whereas if, hypothetically, next yr there is a pestnet with 400 people, and someone discovers 'magick packet' that floods it to the point of unusability, even after 'fix', 395 of'em will go back to 'telegram' or whatever and that'll be it.
13:57 asciilifeform phf: atm we have in 4.2.1.1 'Any Message bearing a Timestamp more than 15 minutes away, in either direction, from the Time at a receiver’s station at the time of its receipt, is deemed stale...'
13:58 asciilifeform so already > . asciilifeform's examples upstack were malformed.
~ 18 minutes ~
14:16 phf http://logs.bitdash.io/pest/2023-02-13#1023247 << i guess the thinking is that further restricting time buffer will let you store less data in filter. so instead of it being a +-15m it's essentially +15m
14:16 bitbot Logged on 2023-02-13 13:31:20 asciilifeform[jonsykkel|awt|deedbot]: the point of this, if not obv, is simply to unclutter the antireplay buffer.
14:17 asciilifeform +/- 1min, potentially, for most msgs
14:17 asciilifeform or even tighter
14:17 phf but that's at the expensive of either broken timestamp, or strong awareness of the counterparty's clock
14:18 asciilifeform would only need to track the delta vs. each warm peer
14:18 phf well, no you have to track delta which is a probabilistic value, that's split between time drift and packet delay
14:19 asciilifeform phf: idea is to reduce the effective window to span any plausible packet delay
14:19 asciilifeform (factoring out the long-term drift when possible)
14:20 phf i got the idea, i'm just clarifying that it's not "just timedelta" it's a time delta + packet delay
14:20 asciilifeform for a msg coming from 'cold' peer, naturally will need the full window
14:20 asciilifeform right
14:23 phf how does hearsay factor into this?
14:24 phf l1 station will have to update hearsay timestamp, or else you're also tracking hypothetical drifts with hypothetical l>1
14:24 asciilifeform in asciilifeform's notion, would retain the full window for hearsays. which aint much of a problem , as bin msgs cannot be hearsayed
14:24 asciilifeform the reduced window contemplated for bin msgs strictly
14:24 asciilifeform (they're what threatens to bloat filter to multi-GB)
14:26 phf i think i'm going back to considering luby packets as a totally special thing, that will be explored once it's out
14:27 phf the crack fumes from pg county might be drifting too close to your house!
14:27 asciilifeform ~all we know about'em so far is that they'll be in the bin msg class, which suffices to discuss q of timestamps
14:27 asciilifeform but no particular reason to hurry
14:28 phf there are other potentially unpleasant replay packets, like addresscast
14:28 phf or prod
14:28 asciilifeform fwiw asciilifeform not convinced that this, rather than phf's earlier proposed algo ('find fast way to detect replayed luby that not relies on timestamp') is the correct pill
14:28 asciilifeform remains to be seen.
14:29 asciilifeform phf: they'd get the full window (and not eat over9000 ram)
14:29 asciilifeform but in general asciilifeform would prefer the algo with fewest moving parts, ideally would treat all msg timestamps rather than exempting some types from staleness entirely
14:30 phf see i want a mechanism that doesn't involved message timestamps at all :)
14:30 * asciilifeform currently thinking 'narrow window where obviously possible'
14:30 asciilifeform phf: if you devised one, plox to post
14:31 phf in fact it's unlikely that i'll ever implement the 15 minute window, i'll just make my hash lookup really really really fast
14:32 asciilifeform if you've fast enuff ssd, could even work at line rate, conceivably
14:33 asciilifeform (until it burns a hole, lol)
14:34 asciilifeform thing is, only chained msgs, per spec, actually get to live on disk
14:34 asciilifeform errything else only lives in filter (until expires)
14:35 asciilifeform what asciilifeform thought , is that phf devised a way to enforce 'no message is valid if seen a second time' dictum ~without timestamps~ entirely
14:35 asciilifeform that'd be over9000 nifty.
14:36 asciilifeform (timestamp is in the validity criteria strictly to limit the mass of hashes that have to be stuffed into filter, if not obv)
14:37 phf google tryna stay relevant http://glyf.org/screenshots/goog.png http://glyf.org/screenshots/goog2.png
14:37 asciilifeform if erryone had 'infinite' ram (and somehow with o(1) -- parallel? -- lookup) -- would not need any such thing..
14:38 asciilifeform lol wtf is happening to the window in that screenshit ??
14:38 phf special effects!
14:39 * asciilifeform not seen the pictured spam , just yet
14:40 phf http://glyf.org/screenshots/goog3.png
14:40 asciilifeform phf: bloom is potentially exactly what's needed. ( asciilifeform not up to date on subj, will refresh )
14:42 phf i think bloom is inside out, can say "possibly not in set" and "definitely not in set"
14:42 * asciilifeform associates bloom filter with a braindamaged implementation in ye olde prb, but this aint bloom's fault, seems to be precisely what oughta have in a fast pestron
14:42 phf but yeah, bloom is what they put in routers and such for when need to guard bajilion something with "should i even check"
14:42 signpost god, this twee-ass shit. bunny planet.
14:43 phf signpost, it's really really bad
14:43 asciilifeform watsa 'twee' ?
14:43 * asciilifeform can guess
14:43 signpost "I'm just a widdle harmless code monkey; isn't Engineering Fun?"
14:43 phf asciilifeform, i recon originally reference to a stereotypical disney bird, mocked in "who framed roger rabbit"
14:43 asciilifeform a hm
14:44 phf it's a tiny adorable over the top cartoon bird that goes "twee twee twee". and it has all the most adorable manerisms, and it's so sacharine that you want to shoot the damn creature
14:45 asciilifeform a, yes, that one
14:45 * signpost has a brief flashback to hatespeech training in same tone
14:45 * asciilifeform recalls 'terrorism training'. 'a grenade falls through yer office window. what position should you take' etc
14:46 asciilifeform errybody in salt mine luvved these, they were a break from the monotony of the actual salt mining
14:47 asciilifeform prolly is half of what drives the various idjit 'training quizes' etc
14:47 signpost sounds way more fun. anyway I wont derail you gents further.
14:49 asciilifeform ( answer was 'lie down when machine gun firing through window, but crouch if grenade', theoretically frags form a 'hourglass' pattern. but lulzy if q is posed re 4 m^2 'office', 'fish in barrel' lol )
14:49 * asciilifeform prefers the ww1 answer : 'what do you do if a shell lands in yer trench?' 'jump up 20m and scatter yourself around'
14:51 phf signpost, yeah i but asciilifeform's mines are a lot more grownup than ours, he can't fully appreciate just how disgusting this bullshit is
14:51 asciilifeform this was possibly the most abjectly 'dilbertian' mine asciilifeform served in
14:51 asciilifeform after that, escaped, to 'house arrest'
14:52 phf well, i finished one task and now it wants me to log in into my official google account
14:53 asciilifeform since then, the only lultraining encountered was, one time, 'prevention of money launderers'. was even moar hilarious than the 'terrorism'
14:54 asciilifeform phf: you may already be a winner!111(tm)(r)(c)
14:54 asciilifeform googlewagen dispatched.
15:09 phf apparently it's a wellknown thing, i left feedback along the lines of "i ain't loging in and you suck". now i can tell people that i almost was a google developer.
~ 7 hours 30 minutes ~
22:39 asciilifeform phf: asciilifeform recalls similarly lulzy 'employment olympiads' in 2010s. iirc there was even 1 conducted by nsa.
22:40 asciilifeform some of'em are somewhere in the old #t log.
22:42 * asciilifeform at one pt found himself participating in one, 'reverse this 13337 fresh chinese trojan', while doing it realized that the perp co was likely farming out its workload to applicants, phree labour, lol
22:43 asciilifeform in theory is possible to run entire biz that way, and someone, somewhere is doing it
22:44 asciilifeform 'they insist on working for phree, wainot let'em'
22:48 asciilifeform ( the 'prize', ftr, is invariably an abject 'entry' jerb, the kind suitable for 19yo )
← 2023-02-12 | 2023-02-14 → ↑

Random(pest) | Download hourly DB snapshot | Get Source Code

Theme: Light | Dark | System Default