| 14:13 |
PeterL |
hello from blatta-9988 ! |
| 14:17 |
PeterL |
http://paste.deedbot.org/?id=_Lid << had it crash, not sure why? |
| 14:19 |
PeterL |
this was just as I tried to connect with my IRC client |
| 14:25 |
jonsykkel |
hello |
| 14:30 |
PeterL |
shinohai: looks like it works |
| 14:35 |
jonsykkel |
9988 test |
| 14:48 |
PeterL |
shinohai, billymg: it looks like messages from shinohai are not ending up in the logs.bitdash.io log |
| 14:52 |
awt |
PeterL: the crash you saw is likely due to "gamma ray" connection to your irc port. Blatta will try to use some info from the client that get's created and then crash because the nick, for example, not there. |
| 14:54 |
awt |
I made a small effort to only allow one client connection but ran into some issues. |
| 14:54 |
awt |
shinohai: yes this is possible |
| 14:56 |
awt |
I make sure to do a test press before releasing |
| 14:58 |
awt |
So once billymg updates the logger station, we shouldn't be missing any logs due to any known issues. |
| 14:59 |
awt |
Hopefully my logs being public will help with debugging various NAT'd stations not receiving packets from each other. |
| 15:02 |
PeterL |
I'm getting a 404 when I tried to load your logs? |
| 15:02 |
awt |
PeterL: http://share.alethepedia.com/blatta/logs/current |
| 15:02 |
PeterL |
ok, that one works |
| 15:03 |
awt |
PeterL: updated the link in the 9988 release article. |
| |
~ 25 minutes ~ |
| 15:28 |
* |
asciilifeform aboutta try 9988, will prolly end up with new ephemeral port tho, grr |
| 15:30 |
asciilifeform |
helloworld |
| 15:30 |
awt |
I see you |
| 15:31 |
* |
asciilifeform sees timestamps from shinohai and awt in AT; but notyet billymg or PeterL |
| 15:32 |
asciilifeform |
PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725 |
| |
↖ ↖ |
| 15:32 |
asciilifeform |
hmm lemme try the bot nao... |
| 15:32 |
asciilifeform |
http://logs.bitdash.io/pest/2021-11-17#1000745 |
| 15:32 |
bitbot |
Logged on 2021-11-17 15:32:01 asciilifeform: PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725 |
| 15:32 |
awt |
asciilifeform: that's the port I have for you now in my AT |
| 15:32 |
asciilifeform |
soo awt the bot silence wasn't, turns out, on account of broken deduper |
| 15:33 |
asciilifeform |
(given as bot gets asciilifeform's hearsays, but asciilifeform does not get bot's) |
| 15:35 |
awt |
asciilifeform: I received the bot echo |
| 15:35 |
awt |
possibly there's a record in my log of my station forwarding that on |
| 15:36 |
asciilifeform |
i'd assume it's a bounce limit thing, but that would not explain why it indeed does work -- in asciilifeform->bitbot direction strictly |
| 15:41 |
awt |
asciilifeform: in my log I can see where I received the echo from billymg, but I don't see it being repacked and forwarded as I do with other surrounding messages: http://share.alethepedia.com/blatta/logs/@40000000619521a2069255bc.s |
| 15:44 |
awt |
lemme restart with a ridiculously high max bounce limit |
| |
↖ |
| 15:47 |
awt |
http://logs.bitdash.io/pest/2021-11-17#1000756 |
| 15:47 |
bitbot |
Logged on 2021-11-17 15:44:21 awt: lemme restart with a ridiculously high max bounce limit |
| 15:49 |
awt |
asciilifeform: you should have received it this time: 2021-11-17 10:50:13.108495 [71.191.220.241:50725] <- 8aff65d3cb7554f8 |
| 16:02 |
awt |
also note that afaik billymg has not updated to 9988, so quite possibly still the timestamp/dedupe issue. |
| 16:15 |
billymg |
running 9988 |
| 16:16 |
PeterL |
asciilifeform: that is the port that I am sending to, but still appears nothing is going through |
| 16:16 |
awt |
nice! |
| 16:16 |
billymg |
the bot is also on 9988 |
| 16:16 |
billymg |
as of a minute ago |
| 16:17 |
billymg |
shinohai: weird, looks like your messages still aren't getting logged |
| 16:17 |
awt |
Ok no dropped logs so far. |
| 16:18 |
awt |
billymg: perhaps try setting MAX_BOUNCES to something way higher like 10 |
| 16:18 |
billymg |
btw, if i look at my /at now, jonsykkel, bitbot, and awt all have timestamps. no one else does though |
| 16:20 |
PeterL |
so shinohai-peterl-jonskkel-billymg-bot is four bounces? |
| 16:20 |
billymg |
awt: those MAX_BOUNCES would be on my billymg station, correct? |
| 16:20 |
awt |
My current at: http://paste.deedbot.org/?id=TUAS |
| 16:20 |
awt |
billymg: yes |
| 16:22 |
billymg |
my current at: http://paste.deedbot.org/?id=lXRD |
| 16:23 |
awt |
My understanding of the code is that the original message goes out with bounces=0. The count is then incremented by 1 and rebroadcast by each recipient. So by the time it reaches the bot it should be=3. |
| 16:23 |
asciilifeform |
btw asciilifeform's AT still as it was ~hr ago |
| 16:23 |
asciilifeform |
(well, with fresh timestamps for awt and shinohai , but otherwise) |
| 16:24 |
* |
asciilifeform testing bot : http://logs.bitdash.io/pest/2021-11-17#1000745 |
| 16:24 |
bitbot |
Logged on 2021-11-17 15:32:01 asciilifeform: PeterL: asciilifeform's new ephemeral port appears to be (via awt's very handy www debuglog) 50725 |
| 16:25 |
asciilifeform |
*bounces |
| 16:28 |
PeterL |
and I just received my own message back as an echo... |
| 16:28 |
billymg |
ok restarted my station with MAX_BOUNCES=9 |
| 16:29 |
asciilifeform |
PeterL: per spec, hearsays have an embargo interval, so that a station has a chance to receive immediate msg (if one is on its way) and failing that, to bounce the copy with the fewest bounces (did i put this in spec? possibly not?) but iirc we don't have the embargo logic yet |
| |
↖ |
| 16:29 |
billymg |
shinohai: wanna try the logging? |
| 16:29 |
* |
asciilifeform testing echo: http://logs.bitdash.io/pest/2021-11-17#1000783 |
| 16:29 |
bitbot |
Logged on 2021-11-17 16:29:04 asciilifeform: PeterL: per spec, hearsays have an embargo interval, so that a station has a chance to receive immediate msg (if one is on its way) and failing that, to bounce the copy with the fewest bounces (did i put this in spec? possibly not?) but iirc we don't have the embargo logic yet |
| 16:33 |
awt |
PeterL, then I sent the message back to you after I got it from shinohai: 2021-11-17 11:30:18.055166 [162.247.151.243:55565] <- b3e3bf6cba250cd8 |
| 16:34 |
awt |
WTF |
| 16:34 |
billymg |
!. uptime |
| 16:34 |
bitbot |
billymg: time since my last reconnect : 0d 0h 1m |
| 16:35 |
billymg |
^ that command crashes the blatta instance the bot is connected to |
| 16:35 |
awt |
My station should have discarded the one from shinohai as a dupe. |
| 16:35 |
awt |
billymg: stack trace? |
| 16:37 |
billymg |
one sec, gonna try again |
| 16:37 |
billymg |
test |
| 16:37 |
billymg |
ok, logging |
| 16:37 |
billymg |
!. uptime |
| 16:37 |
bitbot |
billymg: time since my last reconnect : 0d 0h 0m |
| 16:41 |
billymg |
ok, restarted the logging instance, nobody !. the bot! |
| |
↖ ↖ |
| 16:43 |
shinohai |
>.> |
| 16:43 |
billymg |
^ hey, the increased MAX_BOUNCES seem to have fixed shinohai's logging |
| 16:44 |
shinohai |
sweet |
| 16:44 |
asciilifeform |
billymg: this aint actually a fix tho. (see http://logs.nosuchlabs.com/log/asciilifeform/2021-11-17#1066384 ) |
| 16:44 |
awt |
Other outstanding issue is why some peer's can't directly communicate, while others can. |
| 16:44 |
asciilifeform |
aha |
| 16:45 |
billymg |
awt: would it make sense to have 'max-bounces' as a startup flag? |
| 16:46 |
billymg |
since spec calls it out as being user configurable |
| 16:46 |
asciilifeform |
billymg: per spec, 'CUT' cmd sets bounces. but i dun recall if implemented yet in awt's proggy |
| 16:46 |
billymg |
asciilifeform: ah, gotcha |
| 16:46 |
asciilifeform |
billymg: thing is, 3 bounces oughta work, if the proper embargo logic were there |
| 16:47 |
asciilifeform |
(while 10 or even 9000 will actually fail some % of the time given the current situation) |
| 16:47 |
asciilifeform |
... and produce massive bw guzzle |
| 16:47 |
asciilifeform |
remember that bounce cut only affects ~your~ station's acceptance of incomings |
| 16:49 |
PeterL |
does it accept acceptance of incoming or decision to pass along farther? |
| 16:49 |
PeterL |
s/accept/affect |
| 16:49 |
awt |
Also, might be helpful to log messages dropped due to bounce cut off. Currently blatta just silently doesn't rebroadcast. |
| 16:49 |
asciilifeform |
PeterL: acceptance of incoming (with the latter gated by the former naturally) |
| 16:49 |
shinohai |
awt: mirror of blatta vpatches now live: http://btc.info.gf/devel/irc/pest/blatta/ |
| 16:50 |
awt |
ty shinohai |
| 16:50 |
asciilifeform |
awt: i suspect that if you log bounce counts (of both rejected and accepted msgs) will find the effect discussed earlier |
| 16:50 |
awt |
awt: yes highly likely |
| 16:50 |
asciilifeform |
(i.e. msgs accumulating bounce much faster than they oughta per the net's topology, on account of deduper zapping the low-bounce and even immediate copies in favour of high-bounce ones) |
| 16:51 |
asciilifeform |
it's why asciilifeform specified the embargo thing |
| 16:51 |
awt |
asciilifeform: I'm getting there lol |
| 16:51 |
asciilifeform |
aite |
| 16:51 |
* |
asciilifeform genuinely must bbl |
| 16:52 |
PeterL |
so, regardin gtheephemeral ports thing, what happens if you point something at the port I specified to blatta instead of the ephemeral port it is sending from? |
| 16:54 |
shinohai |
now that i have own mirror the beauty of kelvin versioning is apparent. |
| 16:55 |
awt |
PeterL: the packet will never reach that port because it's behind a NAT. |
| 16:56 |
PeterL |
shinohai: but doesn't that just list everything in reverse order? Wouldn't it make more sense to list it the other way around? |
| 16:58 |
shinohai |
just as easy to map $LATEST to mean lowest integer found in `patches/` neh ? |
| 16:58 |
shinohai |
dupes! |
| 17:01 |
awt |
saw it too |
| 17:02 |
shinohai |
`test` |
| 17:04 |
awt |
Weird. Completely swapped out dupe detection -- behaves almost exactly the same. |
| 17:05 |
awt |
Some how some messages are being packed in such a way that their hashes differ, while the message body is the same. |
| 17:05 |
awt |
possibly |
| 17:13 |
asciilifeform |
awt: do you have proper zero padding for unfilled portions of fields? |
| 17:21 |
awt |
asciilifeform: messages and handles are padded with the " " character using .ljust |
| 17:21 |
asciilifeform |
mnope, per spec gotta be 0 |
| 17:22 |
awt |
unused fields are padded with the null character |
| 17:22 |
awt |
er, 0 |
| 17:22 |
asciilifeform |
messages and handles too, per spec, gotta be |
| 17:22 |
awt |
asciilifeform: noted |
| 17:22 |
asciilifeform |
tho the lack of this wouldn't explain if indeed you got multiple hashes for same msg |
| 17:23 |
awt |
also why intermittent? |
| 17:23 |
asciilifeform |
awt: is it possible you included a field in the hash which oughtn't to be included ? |
| 17:24 |
asciilifeform |
( see http://logs.nosuchlabs.com/log/asciilifeform/2021-11-15#1066154 ) |
| 17:25 |
asciilifeform |
the other possible explanation is that the proggy modifies fields prior to rebroadcast somewhere (per spec, nuffin in above list ought to ever be modified) |
| 17:25 |
awt |
asciilifeform: this is what is hashed: struct.pack(MESSAGE_PACKET_FORMAT, int_ts, self_chain, net_chain, speaker, message.body) |
| 17:25 |
awt |
net_chain currently always 000 |
| 17:25 |
asciilifeform |
hm why is it message.body ? |
| 17:25 |
asciilifeform |
what else is in 'message' ? |
| 17:26 |
awt |
timestamp, speaker, etc.. |
| 17:27 |
asciilifeform |
so why not message.self_chain, message_net.chain, ... etc ? |
| 17:27 |
awt |
command |
| 17:27 |
asciilifeform |
hm? |
| 17:28 |
asciilifeform |
awt: i rec to put an assert (or equiv.) that outgoing rebroadcast has same hash it came in with, or similar. then will likely find where it gets diddled. |
| 17:28 |
awt |
asciilifeform: check out get_message_bytes() in infosec.py -- will possibly make more sense. |
| 17:28 |
* |
asciilifeform prolly oughta reread whole thing before commenting further |
| 17:29 |
awt |
asciilifeform: there is also the possiblity that the messages are being originated with different contents somehow. |
| 17:30 |
asciilifeform |
awt: how do you mean? origination is a 1-time thing |
| 17:30 |
asciilifeform |
(per spec, it's what happens when you enter a line in console) |
| 17:31 |
awt |
asciilifeform: as implemented the message is packed for each peer. |
| 17:31 |
awt |
results should be identical but possibly not |
| 17:31 |
awt |
certainly were not before 9988 |
| 17:31 |
asciilifeform |
if this is the case, i suspect you end up with different timestamps periodically |
| 17:32 |
asciilifeform |
rly oughta pack 1ce |
| 17:32 |
asciilifeform |
pack once, then cipher/sign to ea. peer |
| 17:32 |
awt |
attempted to address the multiple timestamp issue in latest patch by using the same timestamp for each pack() call. |
| 17:33 |
awt |
But yes the message should just be packed one time |
| 17:33 |
awt |
for the sake of sanity |
| 17:33 |
* |
asciilifeform expects this bug will go away when cleaned up as above |
| 17:34 |
asciilifeform |
and likewise you oughtnta be repacking when rebroadcasting (can't recall if doing this tho) |
| 17:34 |
awt |
asciilifeform: am, because currently message packing is in the same func as encryption |
| 17:35 |
asciilifeform |
heh then prolly also mutilates when rebroadcasting, lol |
| 17:35 |
awt |
quite possibly |
| |
~ 4 hours 15 minutes ~ |
| 21:50 |
PeterL |
http://logs.bitdash.io/pest/2021-11-17#1000799 << *Magic packet* ! |
| 21:50 |
bitbot |
Logged on 2021-11-17 16:41:39 billymg: ok, restarted the logging instance, nobody !. the bot! |