Show Idle (>14 d.) Chans
← 2022-12-28 | 2022-12-30 → ↓
08:15 jonsykkel correct solution is obvious
08:15 jonsykkel zero lolcats in chain, only hashes
08:15 jonsykkel if none of ur peers wants to store lolcat on drive, u are fucked
08:15 jonsykkel nice and simple
08:15 jonsykkel then clients would have setting like "download all <3mb cats auto", and "store all <3mb cats forever" if can spare the disk
08:15 jonsykkel puting them inline in chain seems like annoying way to force evryone to store 5000 lolcats or nothing. then when disk full u gota start deleteing evrything from old end rather than just the cats
↖
08:15 jonsykkel which seems sily given avg cat is 3000x the mass of avg text message
08:15 jonsykkel if do the operator decides if inline or not, inevitably sombody will "i think this here 2000x2000 jpg reencoded as png is integral to disscusion" then u are stuck forever with ball of shit weighing same as 3years of text log
~ 2 hours 38 minutes ~
10:53 asciilifeform jonsykkel: this was moar or less asciilifeform's argument. atm it may seem like 'wainot jpg in chain' because net is small, errybody has luxurious net pipes, disk/cpu to spare, etc.
10:55 asciilifeform but all-to-all flooding of multi-MB pile (and demand to walk through it when standing up new station or resyncing a downed one) is a solid ugh imho.
10:56 jonsykkel massive ugh indeed
10:56 asciilifeform ideally, we'll have a reasonable direct-share mechanism for non-humanreadable payloads, and get ~same effect as 'in chain' for things people actually want to keep around.
10:56 bitbot Logged on 2022-12-28 19:09:50 asciilifeform[jonsykkel|awt|deedbot]: per asciilifeform's lights, the way to get the tar itself oughta be to ask one's l1 for the hash, as broadcast; someone will have it, or at least have permitted his station to receive it from a peer who ... recurses
10:58 * asciilifeform will introduce a direct packet type, 'getbinwithhash'
10:58 jonsykkel puts responsibility of deciding which blobs to store on the net rather than the guy pushing potentially large # of cats
10:58 asciilifeform once we have with-what.
11:00 jonsykkel guess one could make arg that someone can push megabytes of text in same way, but will not happen by accident in that case
11:01 * asciilifeform could even see having coupla-kB bins in chain, for thumbnail pics; these wouldn't bloat much moar than fat multipart texts. but not even convinced atm that this is needed
11:01 phf that solution has already been proposed, at the beginning of original discussion. the exploration was whether or not the other ~additional~ behavior can work
11:01 asciilifeform phf: aha
11:02 jonsykkel sure, just giving 2c on problems of additional behavior
11:04 asciilifeform 1 possible pill that hasn't yet been mentioned is a separate chain for broadcast bins (with separate db etc)
11:05 asciilifeform ( still suffers from the 'if not errybody propagates'em, availability will suck' problem tho )
11:05 jonsykkel wat would be advantage of that in comparsion to hash links in main chain?
11:06 asciilifeform jonsykkel: notmuch afaik. mentioned for thread-completeness.
11:06 jonsykkel aight
11:07 signpost morning all.
11:07 signpost jonsykkel: at least worth considering why pest protocol can't communicate the whole pest spec (which includes illustrations) as something represented in the chain.
11:08 * signpost finds this bandying of "obvious" lazy on the part of all.
11:08 signpost "I am a narcissist and wish you to submit to my bare opinion" is the entire content.
11:09 signpost good, we can all do that one again when we conquer earth
11:09 jonsykkel morning
11:10 signpost at any rate, seems like most (all?) agree that binwads of some size need to be skippable, but there's disagreement on whether that line is at size >0 or somewhere higher
11:10 jonsykkel only cuz of practical considerations, cant have infinite sized chain, so line must be drawn in sand somewhere and seems to me simplest possible place to put it at "no blob in chain"
11:10 signpost can't have infinite sized chain, so need to kill everyone in 2100
11:10 signpost not being a dick here, just saying this is more an engineering tradeoff subj than cut and dry.
11:11 jonsykkel better to kill them in 2100 than in 2025
11:11 * signpost cannot disagree, lol
11:12 jonsykkel to some extent exagerating cut and dryness of conclusion for comedic efect
11:16 jonsykkel also, final spec already poised to be reasonably complex, makes sense imo to value simplicity where posible if nothing else
11:18 signpost yep. fwiw I don't see that image inlining actually forces anyone to retain images. I can hack my client to silently discard all but a before-after marker around the images.
11:21 signpost this is perhaps made more cumbersome but nothing prevents it any more than "headers-only" mode in prb
11:21 jonsykkel true, would still have to walk them during backwards sync though
11:22 jonsykkel making full sync take watever number of times longer than otherwise would
11:23 signpost not a proposal. using the example to call the representation of the image arbitrary when talking about whether to retain.
11:23 phf that's a technical detail though, we're having an ontological conversation about what the log is, whether or not some kind of binary blobs are its essential properties. is it a pdf or txt
11:23 signpost yup, agreed. was about to switch to the social question.
11:24 signpost what kind of artifact a pest station accumulates (and leaves for others to find, perhaps) is the design question.
11:24 * signpost would go full zealot and call a pest station's job to record History.
11:24 signpost or at least be the primary index into same.
11:25 phf from technical perspective you can devise a fairly simple schemes for storing binary blobs separately and just enough meta to recover their packets. so you have A -> B -> <meta for a pdf> -> G on drive, which becomes A->B->C->D->E->F->G in flight
11:25 signpost mhm
11:27 phf and if you deleted a pdf, and somebody asks for getpacket C,D,E,F you fail to respond
11:28 phf but that of course means that /somebody/ has to keep the entire log at all times, which is when you run into the whole 10GB of log with all kinds of lolcats
↖
11:29 jonsykkel if expect that some will store inline blobs and some wont, doesnt this defeat purpose of puting blobs in the chain in first place
11:30 jonsykkel u end up with same availiability problem if enough ppls decide to not store cats
11:34 signpost would it be useful to refocus on the question of push vs pull, rather than storage? that seems more important to me.
11:34 phf i was going to get to that :)
11:34 signpost let's say phf gets the pic of bill gates getting pissed on by epstein's girls
11:35 signpost but he's about to be captured by the enemy
11:35 signpost etc
11:36 phf deleting blobs becomes an equivalent of tearing pages out of a book, because it doesn't fit on your shelf, but what that signals is `your book is missing parts`.
11:41 phf 􏿽i guess overall my argument hinges on authorial intent, and for their to be an intent. that is concsious construction of a shared document. the case of 2000x2000 jpg reencoded as png or even the general case of not understanding this complicated point that i'm trying to make. you
11:41 phf 􏿽can't prevent somebody from attaching linux.iso into their word document, in other words, but i guess i don't really care about those people? what i'm trying to accomplish is if somebody is making some illustration heavy argument, i don't want to discover that all the pics got lost
↖
11:41 phf 􏿽 because of technicality, because first-flight-1.png got equated to gzdoom-patched-nudes.tgz
11:41 phf and to some extent the counterargument is `we can't have a rich document, because some idiot is going to attach linux.iso inline`
11:46 jonsykkel big difference is that if numskull attaches fedora27.iso to important word document it can be removed at any point in time without riping pages from book
11:47 signpost to note where there isn't disagreement, sounds like nobody objects to an author being able to specify the intent that an image is inline in the log.
11:48 signpost two questions remain. one's not super interesting to me, which is whether the byte array rides along with the inline item's hash, or lives elsewhere.
11:48 signpost the other is whether this binwad's hash, byte-array, or both are pushed rather than my station having to ask to retrieve them.
11:49 signpost the bill-gates-piss-pic example is the best argument I can give for why have push.
↖
11:51 signpost there's preserving for posterity sure, but also the question of whether a station will ever be able to get important contextualizing bits out to others before perishing.
11:51 phf signpost, the two are tied together intimately, because the question becomes `we have a mechanism for delivering log that has properties X, do we want binary blobs to fully share those properties`
11:52 signpost seems very useful for some binary blobs to have.
11:55 phf right, even in trivial cases of a non-stationary station. post a lolcat, close the lid
12:00 phf this end of the conversation is abstract though, because if binary blob messages don't use the same mechanism as broadcast messages we no longer have anything concrete to go by.
12:00 phf the only mechanism discussed so far is having some kind of announcement message that is equivalent to broadcast, but as a payload it has id of file that you can then use to query some binary delivery mechanism. but we don't know anything about this binary delivery mechanism yet
↖
~ 17 minutes ~
12:17 asciilifeform interesting thread
12:18 asciilifeform http://logs.bitdash.io/pest/2022-12-29#1019867 << will observe, not errythng gets lost at same rate. e.g mp's s&m links mostly 404, but, otoh, 100% of fg schematics still up
12:18 bitbot Logged on 2022-12-29 11:41:06 phf[jonsykkel|awt]: 􏿽can't prevent somebody from attaching linux.iso into their word document, in other words, but i guess i don't really care about those people? what i'm trying to accomplish is if somebody is making some illustration heavy argument, i don't want to discover that all the pics got lost
12:19 asciilifeform if sumthing is genuinely interesting, even nao folx will keep copies around even when takes some elbow grease. not to mention when takes ~0 in a hypothetical sane p2p apparatus
12:20 asciilifeform so imho there's a natural filter, which worx rather well when material is very easy to copy (not speaking here of '80s with their bolix tapes 'sank in atlantis' etc)
12:22 asciilifeform http://logs.bitdash.io/pest/2022-12-29#1019874 << no reason why not have pushed-jpg in ~direct~ msgs, to l1. and a 'push this photo of gestapo behind my chair to erry peer!' cmd, if one likes
12:22 bitbot Logged on 2022-12-29 11:49:02 signpost: the bill-gates-piss-pic example is the best argument I can give for why have push.
12:24 asciilifeform ( from philosophical pov, pest supports both push ('here's a packet') and pull ('hey, i heard there was a $hash, nao plox getdata $hash') fwiw )
12:25 asciilifeform http://logs.bitdash.io/pest/2022-12-29#1019880 << there was a napkin-level thread which cannot atm find, where 'luby has overhead, so only makes sense for $size, so what response to getbinwithhash oughta be is a tree of hashes until leaves represent frag with $size'
12:25 bitbot Logged on 2022-12-29 12:00:05 phf[awt]: the only mechanism discussed so far is having some kind of announcement message that is equivalent to broadcast, but as a payload it has id of file that you can then use to query some binary delivery mechanism. but we don't know anything about this binary delivery mechanism yet
12:26 asciilifeform no concretes tho, afaik these will require empirical #s
12:28 phf is getbinwithhash going to follow the same 1-for-1 policy?
12:29 asciilifeform phf: the notion iirc was that it won't, when comes down to level of 'leaf' (you get n luby packets, of which you need m, or until say 'enuff'. again need concretes to say what n/m oughta be)
12:30 asciilifeform 1 of the reasons why the luby transfer absolutely gotta be direct-only (tho can be from >1 peer in parallel, sending diff frags)
↖
12:30 asciilifeform if you flood-route'em , no amt of net bw will ever suffice
12:31 asciilifeform ( why luby to begin with? precisely so can be asynchronous and order-not-matters and occasional-loss-not-matters )
12:33 asciilifeform the reason why 'need empiricals' is that there are unknowns ( what, for instance, happens if the sender and receiver have pipes of severely mismatched bw ? )
12:34 phf here, catch!
↖
12:34 asciilifeform aaha
12:36 phf there's other unknowns, like for example i'm sending only one broadcast out(but i send ignore to three other stations, so they send me stuff back), and there are whole periods of dark, why? who knows. when my packets are silently lostt.
12:36 asciilifeform ideally oughta have some way to tell sender roughly what speed of ball yer able to catch; and conversely, for sender to be able to 'slow the balls'
12:37 asciilifeform phf: fwiw asciilifeform's original spec was braindamaged, in that it asked for getdata to be issued to the particular peer who had sent the msg that triggered it
12:38 asciilifeform this will give perma-holes if specifically a<->b link is lossy (which is how a missed msg from b to start with)
12:39 asciilifeform theoretically, tho, if erry station correctly handles getdata and prod -- you can 'talk into unplugged cable' for an hour and eventually folx will see what you had said, tho
12:40 asciilifeform but afaik we aint there yet
12:44 phf that's vaguely how it works at the moment
12:44 phf which is the purpose of http://logs.bitdash.io/log-search?q=from%3Aphf+bump&chan=pest :D
12:45 asciilifeform rright, but evidently not 100% (tho asciilifeform not knows whether phf's station currently implements 100% getdata & prod)
12:46 phf i send out getdata for whatever packet to all comers
↖
12:47 phf and i've never gotten prod request from anyone, but i handle prod responses
12:48 asciilifeform hmm not even jonsykkel ? iirc his sends prods
12:48 phf i'm not peered with him yet, because i still can't remember my gpg password :D
12:49 asciilifeform aa
12:49 phf i'm kind of getting concerned, but i was also sick for a week, so hopefull once my head clears out a bit…
12:49 asciilifeform could try setting up a smalpest locally to test prods, if you've the cycles
12:49 phf y tho
12:50 asciilifeform phf: worst case, 1 of us takes a drive to where the other is, and 'key party', asciilifeform remembers what phf looks like
12:50 asciilifeform (and hopefully vice versa)
12:51 shinohai phf: "You need a man that knows how to handle your prod responses ...."
12:51 asciilifeform lol
12:51 phf if i permanently forgotten the password that would be Very Bad™. i have backups keys encrypted with it, etc.
12:52 asciilifeform hopefulyl it's still there sumwhere in phf's head, and will float
12:56 phf http://logs.bitdash.io/pest/2022-12-29#1019909 << http://glyf.org/screenshots/pest-getdata-prod.png
12:56 bitbot Logged on 2022-12-29 12:46:27 phf[awt]: i send out getdata for whatever packet to all comers
12:57 phf i mean, it's not a very complicated logic!
13:06 asciilifeform phf: does your pestron have provision to retry a getdata if not answered in $interval ?
13:07 * asciilifeform trying to implement his earlier 'broken heart' idea, where when one finds a hash pointing 'nowhere', a db entry is made to represent the fact, and then asynchronously getdata's are issued to try & replace the placeholder
13:08 asciilifeform ( rather than synchronously shooting one out immediately when see the hash, and then 'fughet about it' )
13:08 asciilifeform phf's logit ftr is correct per current spec tho
13:08 asciilifeform *logic
13:09 asciilifeform also phf is that a bolix screenshit ?
↖
13:15 phf http://logs.nosuchlabs.com/log/pest/2022-12-29#1019993 << that part is totally not to spec. i don't ever send getdata immediately, just plop the packet into the backlog. and in (periodic) handler there's logic to spool getdata for everything that's missing.
13:15 dulapbot Logged on 2022-12-29 13:05:22 asciilifeform: phf: does your pestron have provision to retry a getdata if not answered in $interval ?
13:15 asciilifeform a hm
13:15 asciilifeform wasn't obv from the photo
↖
13:23 phf 􏿽http://logs.bitdash.io/pest/2022-12-29#1019935 << i have two parallel tracks in code, very simple operator driven conditional stuff. so e.g. i call (prod) -> there's prod response -> it triggers getdata -> resulting packet goes into packet log. if that loop fails i don't care much
13:23 bitbot Logged on 2022-12-29 13:15:26 asciilifeform[6]: wasn't obv from the photo
13:23 phf 􏿽because i'll just rerun it manually. the second track is automatic stuff. the packet log maintains holes, there's a code in periodic handler to try and fill those holes with getdata requests. it's basically all-in on the broken heart idea.
13:24 phf http://logs.bitdash.io/pest/2022-12-29#1019931 << yes
13:24 bitbot Logged on 2022-12-29 13:09:10 asciilifeform[jonsykkel]: also phf is that a bolix screenshit ?
13:25 asciilifeform neato
← 2022-12-28 | 2022-12-30 → ↑

Random(pest) | Download hourly DB snapshot | Get Source Code

Theme: Light | Dark | System Default