Show Idle (>14 d.) Chans
← 2023-01-15 | 2023-01-17 → ↓
07:54 cgra asciilifeform, why isn't version and command included in the pest message hash? why would: less room for funnybiz
07:58 cgra there's at least one instance: somebody could send a direct "hello" first to victim, then broadcast same text with same timestamp. now victim's broadcast reception is stuck at uniqueness constraint fault. i suppose you, alternatively, could handle the case by separating direct and broadcast logs
08:00 cgra "hello" as in first ever message, and could even in theory happen accidentally if done rapidly on same second
08:05 asciilifeform cgra: the random padding in message text makes this scenario ~impossible
08:06 asciilifeform (in message text, in unused bytes of speaker field, and, obviously, the chain hashes)
08:07 asciilifeform ... a valid direct msg can't have same chain hashes as any broadcast msg, thinkabout it
08:08 asciilifeform hash covers from timestamp to the end of the packet.
08:10 asciilifeform (fwiw current pestrons will accept a msg with arbitrary chain hashes, however)
08:12 cgra the first direct has zero chain hashes, like the first broadcast
08:12 cgra and i suppose a malefactor can pad with zeroes
08:13 cgra ahh, broadcast doesn't have zero net chain, if not fiddled. only first msg of the net...
08:14 asciilifeform what would that do, if victim doesnt pad with zeroes
08:16 cgra asciilifeform is it possible that broadcast zero net-chain will be(come) spec-acceptable in the future? besides for a couple of first-of-net messages
08:18 asciilifeform current spec is silent re subj. but imho oughta reject a broadcast msg with zero netchain unless yer db is empty. a new speaker on net oughta receive at least 1 msg before speaking. (and prod makes it simple to getdata for 1 even if no one has spoken in a while)
08:24 cgra right, so broadcast and direct cannot collide then
08:29 cgra another case: not sure how much funny to be had, but a peer could respond to a getdata with false commands. extra code checks if nothing else
↖
08:32 cgra re version, maybe some trick between incompatible message versions becomes possible, if version fiddled. cannot picture a concrete example though.
↖
08:35 cgra while at it (compatiblity-breaking (borderline-?)nitpicks): why specify 'speaker' field in messages, if not going to use it: ignore, getdata, prod, keyoffer, keyslice?
↖
08:36 cgra and another: why follow text messages' format in some other message types even if not going to use chain hash and speaker fields?
↖
~ 1 hours 4 minutes ~
09:41 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020487 << imho obv. but perhaps oughta be included in spec -- a getdata reply oughta go 'where was expected' and never elsewhere
09:41 bitbot Logged on 2023-01-16 08:29:50 cgra[deedbot]: another case: not sure how much funny to be had, but a peer could respond to a getdata with false commands. extra code checks if nothing else
09:41 asciilifeform (regardless of cmd code)
09:44 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020488 << in principle, an old pestron (e.g. 1 that not knows about multipart) can getdata for an incompat. msg and choke on it. there's little to be done about this imho
↖
09:44 bitbot Logged on 2023-01-16 08:32:37 cgra[jonsykkel|signpost]: re version, maybe some trick between incompatible message versions becomes possible, if version fiddled. cannot picture a concrete example though.
09:44 asciilifeform (does say 'prototype' on the box, neh)
09:45 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020489 + http://logs.bitdash.io/pest/2023-01-16#1020490 << compatibility artifact. (originally was gonna have'em in all msgs, to simplify parsing. in new spec, are absent only in bin msgs)
09:45 bitbot Logged on 2023-01-16 08:35:41 cgra[jonsykkel]: while at it (compatiblity-breaking (borderline-?)nitpicks): why specify 'speaker' field in messages, if not going to use it: ignore, getdata, prod, keyoffer, keyslice?
09:45 bitbot Logged on 2023-01-16 08:36:57 cgra[jonsykkel|signpost]: and another: why follow text messages' format in some other message types even if not going to use chain hash and speaker fields?
09:46 * asciilifeform in general proponent of mandatorily-present, fixed-length fields -- eliminates broad classes of exploitable bugs
↖
09:49 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020494 << apropos, anuther item that oughta be included in spec -- 'broken heart' to represent a getdata'd msg that for 1 reason or anuther invalid and not displayed. (iirc discussed previously)
09:49 bitbot Logged on 2023-01-16 09:44:07 asciilifeform[5]: http://logs.bitdash.io/pest/2023-01-16#1020488 << in principle, an old pestron (e.g. 1 that not knows about multipart) can getdata for an incompat. msg and choke on it. there's little to be done about this imho
09:50 asciilifeform (i.e. avoid infinite getdata attempt)
09:51 phf i think your thinking about getdata is linear. there's technically no guarantee that anybody is going to respond to your getdata request, at any guaranteed time
09:51 asciilifeform ^ entirely correct
09:52 phf so the process should be upfront assumed to be fully desynchronized: you're going to have "holes" in your log, potentially indefinitely, and you're going to make getdata requests, until you personaly decide to give up, and they might come back at any random time in the future
09:52 asciilifeform ( among other reasons, a msg could even contain garbage where the chain hashes oughta be, and so no one will ever supply a msg with that hash , no matter what )
09:53 asciilifeform ^ for this reason, getdata attempts gotta be finite (or will clutter the list of hashes exempt from timestamp check 4evah )
09:54 asciilifeform (so no, not 'any' time in future)
09:54 asciilifeform otherwise yer open to memory exhaustion attack
09:55 phf well, any time in the future, even after you already gave up, or before you did
09:56 asciilifeform if a getdata response shows up after you've already removed the hash from getdata list (i.e. exempt from timestamp) -- tough cookies
09:56 asciilifeform it gets thrown
09:56 phf well, it's tough cookies ~for you~ so obv gotta weigh your options
09:56 asciilifeform ( how long to keep'em around -- up to operator )
09:56 asciilifeform aaha
09:57 asciilifeform atm spec not mentions broken hearts, or the need to represent holes in log (permanent or otherwise) -- but oughta
10:00 * asciilifeform recalls early pest thrd with a briefly returned oldtimer (ben_vulpes?) where the latter argued that spec is a bloat warcrime because mentions not only protocol, as in classical rfcs, but tries to be 'complete bakeable recipe'
10:01 asciilifeform ^ imho not avoidable -- a p2p protocol includes good measure of 'social expectation' of station behaviour, on top of packet format per se
10:01 phf i am frankly of similar opinion, ought to separate "recommendations for author" from "spec". e.g. details of various nursery buffers, or even things like "mandatory" timeouts
10:03 phf because the irc nursery behavior turned out to be entirely broken as soon as we started test running it, even though the base protocol can be made to work
10:16 phf you like to defend even your bad decisions like they are strokes of genuis, but if original pest wasn't written perfect like athena from the brow of her father, then awt wouldn't be saddled with a lot of "legacy" code, that clearly didn't pan out
10:18 asciilifeform phf: there's no doubt bad decisions, and would rather find out what they are nao, than in hypothetical 'later' where >dozen folx tuned in
10:18 asciilifeform fwiw in current draft spec no 'mandatory' constants, but only 'recommended'.
10:19 phf like e.g. i'd still like to see a poc packet management code, that can put or reject any arbitrary packet into the backlog
10:20 phf which to me appears like in the worst case it's actually a topological sort on a graph, that can arbitrarily reorder log order
10:20 asciilifeform phf: plox to expand re above
10:23 * asciilifeform entirely on board re 'nursery buffers were idjit idea', threw'em entirely out in 0xfa, were forced mistake on acct of ircism
10:24 phf well, the order of packets in selfchain are fixed, we know as much. but the order of packets in netchain are subjective to each speaker
10:25 asciilifeform entirely correct (and, on top of this, notionally in gui client can set arbitrary netchain, to make 'reply')
10:27 * asciilifeform in his attempt at pestron, aims to display msgs in timestamp order, but with clickable arrows to self/net chain predecessors
10:27 phf and the construction of netchain is the artifact of "what's the last packet i got"
10:27 asciilifeform by default -- exactly that
10:29 asciilifeform ( and on a net even a little larger than the current, there's not even any notion that erryone will see the same broadcasts; so inescapable )
10:29 phf so the points at which multiple parallel subjective netchains link up are fully driven by random packet loss behavior
10:29 phf loss/delay
10:29 asciilifeform that, and bounce decay (on larger net)
10:30 phf now under normal circumstances, this will be near realtime
10:33 phf but lets say one station is on a unreliable connection. it's lost subchain (by either fully arriving, or having to be reconstructed with getdata) will have to be inserted into past log, by essentially random injection. either as a large, expanding chunk "there's 10 messages missing five screen pages ago"
10:34 phf or smaller subchunks "there was 2 messages missing here" "there was 5 messages missing here", depending on how many messages got through to the unreliable station in the process
10:34 asciilifeform right; the station, having plugged itself back into the wall, emits a msg with selfchain pointing to the latest msg sent 'into unplugged cable'; peers getdata for it; and so on
10:35 asciilifeform aha, is possible that some got through, rather than 1 continuous interval of 'unplugged'
10:35 phf because the unreliable station might be getting some packets, but not having any packets come out (was happening to me a couple of times, because internet)
10:37 asciilifeform right
10:38 phf but then if it happens to multiple stations, you can have all kinds of chaotic behaviors
10:39 phf 􏿽for example you can have two parallel chains A and B (two stations "talking to themselves"), but the chains actually got observed by a third station which was also talking to itself C. because there's nothing connecting A and B you will place them arbitrarily, depending on who you
10:39 phf 􏿽heard from first, but when C arrives it might say that your order of placement was actually incorrect
10:42 asciilifeform fwiw asciilifeform suspects that 'erryone agrees on canonical order' is impossible.
10:43 phf that is true, but the point is more that any moment anyone can chime in with "this there arbitrary order is actually fixed, because i saw it happen", at which point you have to rearrange your own log
10:43 asciilifeform why wouldja 'have to' rearrange own log ?
10:44 phf well, if you're trying to retain the guarantee that packets are "sorted" in the dependency of netchain
10:47 asciilifeform likely the best that can be done is to stow 'obvious' (e.g. fills in existing brokenheart, with 1 packet or linear chain thereof) getdata replies in the obvious place, whereas all else is displayed in chrono order (with clickable arrows to nominal predecessors)
10:49 asciilifeform but yes, stations with poor connectivity will often end up emitting chains of msgs which appear as clusters of 'replies' where was not intended
10:49 asciilifeform nfi what a hypothetical pill against this would look like tho.
↖
10:58 phf see, i thought that some kind of heuristic insert is basically a requirement. probably you're correct and in a strongly connected pest network these issues won't manifest
10:59 phf a public log on account of being batch can always just reorder the whole graph arbitrarily no matter how much it costs
11:02 phf but in a realtime setting with a homegenous network, i think some amount of cleverness as to where you insert the packet on arrival will make the conversation smoother. but maybe that's not even a reasonable approach, because we want time "new" messages to always be at the bottom, unless they are getdata
11:04 asciilifeform phf: ^ this was asciilifeform's notion -- 'always at bottom if new', and ui lets you walk the netchain; rather than 'threaded display' a la heathen chats (e.g 'slack') where indeed can sit & scratch yer head re 'in what order to display'
11:05 phf http://logs.bitdash.io/pest/2023-01-16#1020554 << well, at the baseline, if you make a graph of netchain/selfchain dependencies and then flatten it with topological sort, whatever subset of packets you have it will give you a correct global order
11:05 bitbot Logged on 2023-01-16 10:49:42 asciilifeform[jonsykkel|awt|deedbot]: nfi what a hypothetical pill against this would look like tho.
11:06 asciilifeform the q of 'how to display recently filled holes' will need to be answered ('unread' marks on msgs?) but needs empirical massage to see what actually is most edible for people
11:07 phf so in the most degenerate casey you could maintain a graph of your packets, new packets do a graph insert, and your render does a topological sort with a full rerender on each new packet :>
11:08 phf that'll give you an absolutely correct "answer follows question" ordering
11:08 * asciilifeform considered this, tho with a 'window' of e.g. 'last month' to keep cost in reasonable bound
11:09 * asciilifeform not has any notion that he currently knows 'final solution to how to best display pest log', expects to try a # of approaches
11:10 phf and on the other end of this design space you have "insert all packets to end as they arrive, getdata or not"
11:10 asciilifeform for that matter, atm still fighting the gui lib rather than even touched subj
11:10 bitbot Logged on 2023-01-09 23:04:17 asciilifeform[jonsykkel|deedbot|awt]: meanwhile, lulzily, found that the wx widget most suited to displaying log lines is atrociously slow on gtk/linux (blazing fast on crapple, for all the good that does asciilifeform) so will have to 'draw' that one (and handle text sel., etc somehow) after
11:11 asciilifeform phf: on disk, they go precisely 'as they arrive'
11:11 asciilifeform anyffin else promises to be nightmare
11:12 * asciilifeform threw in the towel for nao re custom db, and baking in, for the time being, sqlite, as awt did
11:13 asciilifeform so long as you can pull out arbitrary msg by hash in o(~log n), can live
11:18 phf right you're piggybacking on index b-tree
~ 59 minutes ~
12:17 asciilifeform at some pt, will likely end up with sumthing like this for msg store
12:19 asciilifeform but is even possible that storing parsed (i.e. with indices of antecedents, rather than hashes) representation a la nqb is over9000 faster (and can still pull out original msg by index)
12:20 asciilifeform ( i.e. if you know that given hash H points to msg # N on disk, store N instead of H, and store H 'next to' msg (or in brokenheart, if not have the msg) )
12:22 asciilifeform ( to rephrase -- no doubt that faster, but may be fast enuff that no particular reason to use hand-sewn db rather than sqlite )
~ 2 hours 39 minutes ~
15:01 * cgra wouldn't at this point consider other than easy-to-change db (ie. sqlite), given lack of his vision of how the db is really going to end up when 'done'
~ 59 minutes ~
16:00 cgra can asciilifeform give any other reason to not hash version and command with msg, other than cuz already ~set in stone?
~ 18 minutes ~
16:19 cgra i suppose, if non-zero net-chains were enforced (unlike now), you could distinguish direct and broadcast from each other by the net-chain value and wouldn't have to 100% trust the command value.
16:22 asciilifeform cgra: can't think of any convincing reason , other than where it'd be a 1st class bitch to retrofit
16:28 cgra does asciilifeform see possible somekinda 'reboot' where accumulated and identified, saddening wrinkles in spec get straightened? i keep thinking of this being still way experimental, and just a dozen or so d00ds 'just chatting', while seeing this command kind of nuisances somewhat pile up
↖
16:39 cgra in my limited imagination, how'd such a reboot possibly look like: new log era declared, newpest net forms, oldpest net is abandoned over time... or both nets go on, and eventually names are being called.
16:39 phf but why?
16:41 cgra just to get a cleaner spec
16:41 phf why would it be cleaner?
16:42 cgra phf, well for instance, that command field. could actually use it in full
16:43 cgra and perhaps the field arrangement would be easier to remember in some cases. a group of binary messages now have a speaker field which isn't supposed to be used
16:43 cgra for example, handful of other similar things
16:45 phf yeah, but you have to make a case for why the change makes it are cleaner, and so far, you're arguing from the opposite
↖
16:45 * cgra could overestimate the eventual severity, though. not really saying 'need reboot now'
16:46 cgra phf, for example getdata message has a speaker field, which is documented as 'do not use'. why isn't it cleaner if this field didn't exist in getdata message?
↖
16:50 cgra i'm looking at the pre-draft spec, above example in section 5.4.4.2
16:59 asciilifeform cgra: thinking about it, really oughta revise that (and similar) subsects -- old pestrons expect the obv value for speaker (given as they dunhave the 'filter' and ~will~ think that a msg sent to $peer and somehow mirrored back is 'own' )
17:00 asciilifeform ( peer->self and self->peer aint mechanically distinguishable w/out either speaker field or filter (where a mirrored msg will be zapped immed.) )
17:02 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020582 << can picture changes that could justify the headache of 'reboot' but cosmetic fiddles imho aint in that category
17:02 bitbot Logged on 2023-01-16 16:28:35 cgra[jonsykkel|signpost]: does asciilifeform see possible somekinda 'reboot' where accumulated and identified, saddening wrinkles in spec get straightened? i keep thinking of this being still way experimental, and just a dozen or so d00ds 'just chatting', while seeing this command kind of nuisances somewhat pile up
17:02 bitbot (asciilifeform) 2022-01-26 asciilifeform: thimbronion: current notion, summarized: we bite off 64byte from payload, and get 2 new fields, call'em 'unlock' and 'lock'. lock == h256(errything else in msg, incl. 'unlock', and unrevealed 32byte turd 'S'.); unlock = 'S' from yer previous msg.
17:03 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020592 << from asciilifeform's pov, protocol is in fact cleaner with fixed-length, fixed-position errything (and in fact was reluctant to propose 'binary' msg types where the usual fields entirely missing)
17:03 bitbot Logged on 2023-01-16 16:46:54 cgra[jonsykkel|signpost]: phf, for example getdata message has a speaker field, which is documented as 'do not use'. why isn't it cleaner if this field didn't exist in getdata message?
17:03 bitbot Logged on 2023-01-16 09:46:06 asciilifeform[jonsykkel|deedbot|awt]: in general proponent of mandatorily-present, fixed-length fields -- eliminates broad classes of exploitable bugs
17:04 asciilifeform the protocol is in fact ~less~ complex with fixed fields (that may be meaningful or not in given case, but their presence prohibits 'frame shifting')
17:04 cgra asciilifeform, right. and if binary messages didn't exist, no such dummy fields didn't exist either.
17:04 asciilifeform ^ could be debated, but imho not justifies 'reboot'
17:05 asciilifeform bad enuff that the multipart msg format breaks back-compat.
17:06 cgra asciilifeform, frame shifting atm hebrew to me. how to grasp?
17:06 asciilifeform term from geneticists
17:06 asciilifeform i suppose the nearest commonly-used electronic equiv. is the 'nop sled'
17:07 asciilifeform i.e. eater eats from unexpected place, with exploitable effect
17:08 * asciilifeform must brb
17:16 cgra does 'frame shifting' explain why getdata format "timestamp:8,noise:64,speaker:32,wanthash:32:noise:292" is better than "timestamp:8,wanthash:32,noise:388" would be?
17:18 * cgra to be clear, not saying either that this alone would warrant a reboot. now just trying to grasp apparent fundamentals he wasn't aware of
~ 30 minutes ~
17:49 cgra http://logs.bitdash.io/pest/2023-01-16#1020590 << phf, in arguing that hashing here (pieces of preamble) and there (all message fields) was cleaner than just hashing there, i conflated "cleaner spec" and "cleaner impl". cleaner impl in the sense that can trust command of a msg being of same origin the msg contents are.
↖ ↖
17:49 bitbot Logged on 2023-01-16 16:45:09 phf[deedbot|awt]: yeah, but you have to make a case for why the change makes it are cleaner, and so far, you're arguing from the opposite
~ 1 hours 31 minutes ~
19:20 phf http://logs.bitdash.io/pest/2023-01-16#1020613 << something came up, so i had to run away, apologies; but i'd like to address this thread extensively, so i'll have to come back to it tomorrow
↖
19:20 bitbot Logged on 2023-01-16 17:49:05 cgra[jonsykkel]: http://logs.bitdash.io/pest/2023-01-16#1020590 << phf, in arguing that hashing here (pieces of preamble) and there (all message fields) was cleaner than just hashing there, i conflated "cleaner spec" and "cleaner impl". cleaner impl in the sense that can trust command of a msg being of same origin the msg contents are.
19:24 phf (as in irl intervened so fast i didn't even have time to "afk", just as i was in the middle of writing things)
~ 31 minutes ~
19:56 asciilifeform cgra: there's anuther (entirely aside from above) 'seekrit' explanation -- asciilifeform was attempting to design pest proto for, among other things, clean fpgaization
19:56 asciilifeform ... where a field that is or isn't used is simply set of 'wires', whereas if you allow a 'thing to be in different places' you gotta mux
↖
19:57 asciilifeform throwing speaker/nethash/chainhash outta binary packets was a concession to bw conservation (these, presumably, nao would need separate logic path on fpga...)
20:01 asciilifeform call it megalomania if want, but asciilifeform would like to have at some pt nics, routers, radios, etc. that natively speak pest 'at line rate' with 0 cpu utilization.
20:06 asciilifeform http://logs.bitdash.io/pest/2023-01-16#1020613 << recall that cmd and version are covered by the seal and thereby just as authenticable to $peer as the rest of packet
↖
20:06 bitbot Logged on 2023-01-16 17:49:05 cgra[jonsykkel]: http://logs.bitdash.io/pest/2023-01-16#1020590 << phf, in arguing that hashing here (pieces of preamble) and there (all message fields) was cleaner than just hashing there, i conflated "cleaner spec" and "cleaner impl". cleaner impl in the sense that can trust command of a msg being of same origin the msg contents are.
20:06 asciilifeform the only thing they aint covered by is msg hash (and still not obvious to asciilifeform what'd be gained by doing so)
20:08 asciilifeform ... is why left'em out of msg hash to begin with (... not because drunk)
20:09 asciilifeform version in particular not belongs in msg hash any more than bounce does. ( is intented to be version of the pestron sending the msg in all cases. rather than descriptive of the particular msg per se )
↖
20:10 asciilifeform i.e. it will be of current pestron, even when relaying an old msg originally received and logged to disk by an older one
20:10 asciilifeform and this is proper, because it is there to tell counterparty what kind of msgs the sender knows how to eat.
20:11 asciilifeform ( and ditto when relaying a broadcast received from peer running older version of protocol )
20:11 asciilifeform hence, gotta be mutateable.
← 2023-01-15 | 2023-01-17 → ↑

Random(pest) | Download hourly DB snapshot | Get Source Code

Theme: Light | Dark | System Default