| 15:36 |
billymg |
http://logs.bitdash.io/pest/2021-12-28#1001402 << yeah, the rk, while great for a blog and even the crawler, doesn't have enough ram to get the full (all chans) log history into memory. so if there's a long query going on while the bot tries to write a new line the connection sometimes times out |
| 15:36 |
bitbot |
Logged on 2021-12-28 20:36:13 signpost[billymg]: db connection issue? |
| 15:38 |
billymg |
i was going to attempt a fix but put it on the back burner thinking i'd have a faster box soon anyway, not anticipating the global collapse issues would affect the wait time |
| 15:38 |
bitbot |
(asciilifeform) 2021-12-02 asciilifeform: billymg: waiting on crate of ram, sadly (atrocious delays in shipments) |
| |
~ 2 hours 51 minutes ~ |
| 18:30 |
asciilifeform |
billymg: oddly enuff, asciilifeform ran his logger on identical rk for 8+mo, and db never fell |
| |
~ 41 minutes ~ |
| 19:12 |
signpost |
billymg: needing all log lines in memory for a search seems weird |
| |
~ 1 hours 47 minutes ~ |
| 20:59 |
billymg |
signpost: i was just speculating on a potential solution, since the disk on the rk is slow and could be causing some of the long queries that would cause another connection to timeout. see here when i first brought up the issue |
| 20:59 |
bitbot |
(asciilifeform) 2021-09-17 asciilifeform: billymg: i strongly suspect your proggy is closing it (on, e.g. timeouts or whatever externally-induced eggog) and not reopening |
| 20:59 |
billymg |
and yeah, as asciilifeform points out, might not be the logotron but actually the crawler that is generating some lengthy queries |
| 21:01 |
billymg |
but can try for yourself on my logger's www, the "search all chans" can take a while |
| 21:03 |
signpost |
what db does it sit atop? |
| 21:05 |
billymg |
postgres 10.3 |
| 21:06 |
signpost |
hmm. happy to help eyeball queries if you want. |
| 21:06 |
signpost |
(no rush, just offering) |
| 21:06 |
signpost |
iirc in postgresql writes should only stack behind writes; reads should never block unless locks are being used. |
| 21:07 |
* |
signpost has had plenty of cases where an `explain analyze` on a query yielded a multi order of magnitude speedup |
| 21:08 |
billymg |
signpost: sure, all the code for the crawler is here: http://billymg.com/bitdash-crawler-vtree/ |
| 21:08 |
billymg |
and the logotron is linked there as well, basically asciilifeform's |
| 21:10 |
billymg |
the crawler will, like the bot, sometimes also need to be restarted when its postgres connection times out and closes |
| 21:10 |
billymg |
not 100% consistent, sometimes bot loses, but crawler keeps going, sometimes the opposite |
| 21:13 |
billymg |
i didn't spend more time debugging because i figured i could just throw more hardware at it. i've been working on some other things, including a new blog theme that's almost ready to publish, and a "v2" of the crawler www, which is probably a bit further out |
| |
↖ |
| 21:13 |
signpost |
curious that postgresql is timing out. |
| 21:14 |
signpost |
billymg: btw got that gpggram from me? |
| 21:14 |
billymg |
signpost: with the peering info? |
| 21:14 |
signpost |
yar |
| 21:14 |
billymg |
yup, i tried peering yesterday but my blatta version is out of date (8885) and i'm not at a box where i can press 8883 |
| 21:15 |
signpost |
ah right, sorry, saw that in logs. |
| 21:15 |
billymg |
s/888/998 |
| 21:15 |
* |
signpost had a gas leak in house last night, groggy |
| 21:16 |
signpost |
will be putting a proper detector in the attic now. |
| 21:16 |
signpost |
and ground floor |
| 21:16 |
billymg |
signpost: in the new NH fort? |
| 21:17 |
signpost |
haven't bought up there yet, real estate market sucks in winter. this is the tx house. |
| 21:17 |
billymg |
ah, gotcha |
| |
~ 38 minutes ~ |
| 21:56 |
deedbot |
msg from deedbot. |
| 21:57 |
signpost |
!!help |
| 21:57 |
deedbot |
http://deedbot.org/help.html |
| 21:57 |
signpost |
anyone else able to see that? |
| 21:57 |
signpost |
neato, it's in the logs. |
| 21:58 |
* |
signpost agrees with asciilifeform that the wot ought to be reimplemented as part of pest-tronics, but til then, deedbot will be accessible here too. |
| 22:04 |
asciilifeform |
hm not seeing it in weechat list |
| 22:04 |
asciilifeform |
( or hm, awt , does that currently show only peers ? ) |
| 22:05 |
asciilifeform |
!!help |
| 22:05 |
asciilifeform |
noworky |
| 22:05 |
signpost |
hm, weird. |
| 22:05 |
asciilifeform |
defo saw broadcasts from it earlier |
| 22:06 |
asciilifeform |
e.g. deedbot[signpost|awt] | msg from deedbot. |
| 22:06 |
signpost |
!!help |
| 22:07 |
signpost |
interesting, my first call to help had output at least for me. looks like I'll have to massage it moar. |
| 22:07 |
signpost |
!!help |
| 22:07 |
deedbot |
http://deedbot.org/help.html |
| 22:07 |
deedbot |
http://deedbot.org/help.html |
| 22:07 |
signpost |
seems like content of the NAMES list matters to it, and yeah, only direct peers appear in NAMES |
| 22:08 |
signpost |
(in blatta) |
| 22:08 |
signpost |
this seems correct, since hearsay oughtn't fill up a table of nicks. |
| |
~ 28 minutes ~ |
| 22:37 |
asciilifeform |
it's annoying to not be able to tabcomplete a reply to a hearsay tho |
| 22:37 |
asciilifeform |
imho oughta be at least a configable knob |
| |
~ 17 minutes ~ |
| 22:54 |
asciilifeform |
(where, if enabled, erryone who spoke within N hours ago shows up in nicklist) |
| |
~ 45 minutes ~ |
| 23:40 |
signpost |
yeah, in reconsidering it, peers either spam you or don't, no need to avoid another line or so to populate nicks. expiration definitely needed. |