Show Idle (>14 d.) Chans
← 2020-01-23 | 2020-01-25 → ↓
00:19 whaack argh it looks like asdf is the memory hogging culprit, not quicklisp itself
~ 1 hours 58 minutes ~
02:18 auctionbot S#1077 O=17mn LB=None E=2020-01-29 05:43:03.567830 (117h28) >>> Dell R610 PE Server ships from U.S. (Server-A) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
02:18 auctionbot S#1078 O=17mn LB=18mn E=2020-01-29 05:43:42.679910 (117h28) >>> Dell R610 PE Server ships from Uruguay (Server-B) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
02:18 auctionbot --- end of auction list, 18mn total bids ---
~ 2 hours 28 minutes ~
04:47 whaack diana_coman: EOD Report: Stuck to schedule, except I didn't have time for my Spanish study. For TheFleet I ran into a dead end trying to find a way to reduce the memory my program uses. asdf at a minimum consumes ~25MB. I tried to load asdf -> load my program -> unload asdf, but afaik there's no way to "unload" in CL. On another front, I investigated and discovered the reason for the join/disconnect dance. I was disconnecting becau
04:47 whaack se the servers were sending malformatted irc replies. My bots logic is to reconnect when it hits an error, so it would get stuck in a cycle of connect->disconnect from bad reply->reconnect. Going forward I plan to not reconnect when I receive a malformatted reply. Also, I will put a limit on num reconnects per time window.
04:50 whaack And I read through a good chunk of cl-irc. I was going through some of the utility functions in the code that handle formatting, string parsing, encoding/decodings, etc. That was mind numbingly boring. There were certain large functions/macros that I did not grok and will have to tackle again. Tomorrow I will read more of the high level code.
~ 38 minutes ~
05:28 * jfw facing the puzzle of reporting two fails while not being too hard on self nor being too slow about the reporting.
05:30 jfw first: I'd intended to start before dinner or at least directly thereafter, which didn't happen
05:35 jfw *start writing. I ended up doing first all manner of other things that had been put off - laundry, dishwashing, a phone call home, a walk. (perhaps that displeasure being relative point is proven already! - not that those things are especially displeasurable, in fact some the opposite so idk)
05:43 jfw I did start though and not so late as to be tired, so, second: the log summarizing went poorly. I found myself going back through the text, trying to note each point of conversation briefly yet faithfully. I did not get far at all, by input line count or output word count
05:48 jfw I thought maybe I should instead try to work from memory, but... there's so many unconnected / loosely connected points, I think I'd just end up with some kind of random sample of what happened to stick
05:51 jfw another thought - ignored butterfly perhaps - was to (re)read one of the ossasepia log summary articles to get a better sense of how it could be done
~ 1 hours 27 minutes ~
07:18 auctionbot S#1077 O=17mn LB=None E=2020-01-29 05:43:03.567830 (112h28) >>> Dell R610 PE Server ships from U.S. (Server-A) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
07:18 auctionbot S#1078 O=17mn LB=18mn E=2020-01-29 05:43:42.679910 (112h28) >>> Dell R610 PE Server ships from Uruguay (Server-B) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
07:18 auctionbot --- end of auction list, 18mn total bids ---
07:26 diana_coman http://logs.ossasepia.com/log/ossasepia/2020-01-24#1015980 - this type of situation (too many/too quick reconnects) sounds like something you'd want reported to you/flagged too.
07:26 ossabot Logged on 2020-01-24 00:58:41 whaack: se the servers were sending malformatted irc replies. My bots logic is to reconnect when it hits an error, so it would get stuck in a cycle of connect->disconnect from bad reply->reconnect. Going forward I plan to not reconnect when I receive a malformatted reply. Also, I will put a limit on num reconnects per time window.
07:28 diana_coman http://logs.ossasepia.com/log/ossasepia/2020-01-24#1015984 - ahaha; well, if you get to cleaning ovens, the whole house and start looking for maybe digging as well the garden you don't have, *then* you'll know for sure you're truly avoiding there something!
07:28 ossabot Logged on 2020-01-24 01:46:20 jfw: *start writing. I ended up doing first all manner of other things that had been put off - laundry, dishwashing, a phone call home, a walk. (perhaps that displeasure being relative point is proven already! - not that those things are especially displeasurable, in fact some the opposite so idk)
07:30 diana_coman jfw: summarising is not an easy task at best of times (because it means basically that you fully digested the whole thing and are then able to re-tell any/all of it at whatever level of detail you choose; there's a LOT going on to end up with a good summary, let alone an excellent one)
07:31 diana_coman add to that the fact that #t logs are not an easy read either and so take the time to appreciate and enjoy the steps you took on that long road really
07:33 diana_coman yes, it takes time; no, there's no hurry nor pressure on *this* front so don't add it yourself where it doesn't belong
07:35 diana_coman jfw: so listen, do your reading of #t to catch-up and then for writing go ahead with the practice of summarising what you read of #t but *without* pressure on the result being "a faithful summary of all points" or whatever else; let it be exactly a summary of your recollection (if you need to, brand it as such, it's your own blog and you can write there "what happened to stick" too, what!)
07:38 diana_coman jfw: even better, take the writing on the #t logs as the fun part of the catchup and so write it whichever way you'd like to, be it picking and pointing what you enjoyed reading or poking fun at parts or whatever else!
07:39 diana_coman the log-summary can turn in log-satire even; I'll probably enjoy reading it all the more for that!
07:44 diana_coman whaack: how long do you think it would take you to write down a schema of what your bots/full setup do exactly? e.g. the pseudocode of it all.
07:48 diana_coman jfw: btw, from the sounds of it, what makes you avoid/postpone/drag your feet there is not perceived difficulty but outright perfectionism getting in the way.
~ 4 hours 30 minutes ~
12:18 auctionbot S#1077 O=17mn LB=None E=2020-01-29 05:43:03.567830 (107h28) >>> Dell R610 PE Server ships from U.S. (Server-A) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
12:18 auctionbot S#1078 O=17mn LB=18mn E=2020-01-29 05:43:42.679910 (107h28) >>> Dell R610 PE Server ships from Uruguay (Server-B) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
12:18 auctionbot --- end of auction list, 18mn total bids ---
~ 27 minutes ~
12:46 diana_coman now why does my blog spam keep asking me -and in Romanian too! - to order birthday cake in Kazan of all places and things to order.
~ 1 hours 44 minutes ~
14:30 whaack diana_coman: I'm not sure, I would say 4-6 hours to be safe. It'd be a good exercise. Though I first want to apply some fixes for the problems discovered during the test run.
~ 24 minutes ~
14:54 diana_coman whaack: all right, apply the fixes first and then schedule somwhere the write-up of what's in there too.
~ 27 minutes ~
15:22 whaack diana_coman: okay i scheduled the writeup for tomorrow. i'll try to begin later today though if time permits
15:23 diana_coman whaack: wait, so are you done with the fixes and otherwise figuring out the cl part and everything else?
15:24 whaack diana_coman: I have not applied the fixes, but I have a good idea of what needs to be done and I don't think it will take that long.
15:24 diana_coman ok.
15:25 whaack The difficult tasks left are (1) fixing the memory issue, or figuring out a way to sidestep the problem (2) creating the function (get-all-channels-i-am-connnected-to)
15:29 whaack For (2) my plan is to parse the db and determine that we are connected to a channel if a JOINED-CHANNEL row for that channel appeared more recently than the latest DISCONNECTED-FROM-CHANNEL row. The problem with this is that if there is an unclean disconnect, then the function will return false positives.
15:35 whaack jfw: Can you explain, "You'll need to decide whether to apply initial or future rounds of yum updates. " ?
15:36 whaack diana_coman: Would it be an appropriate time to write my own V, and then use it to install items on my new comp? Or should I use an existing tested V and leave the exercise of writing my own for a later time?
15:39 diana_coman whaack: uhm, re that plan on get-all-channels etc - it seems to me you should first set out to write properly your understanding of the whole thing and only then look at that and identify the best way to do what you want done
15:39 whaack diana_coman: ok
15:40 diana_coman solving a problem is always a matter of a. properly identifying what the problem IS in the first place b. scoping it out c. figuring out the options to address the problem d. choosing one option (with clear reasons too!)
15:40 diana_coman it should never be "oh, this looks like it might solve it, let's go"
15:42 diana_coman whaack: how about manually v-ing for now, it's not like you need a ton of presses of very complicated trees, is it?
15:42 diana_coman and damn it, now I really need to recall the name of that rrmartin story.
15:43 diana_coman whaack: if you are done with reading&understanding that cl part you are using, then write up what you have and then add to it what problems you are trying to solve + what options you see
15:44 whaack it should never be "oh, this looks like it might solve it, let's go" << this gave me a real chuckle
15:46 whaack ^ do you m ean the cl-irc part?
15:46 whaack mean*
15:46 jfw diana_coman: will give it a try with the low-pressure log summary, thanks
15:47 diana_coman whaack: yes, cl-irc part
15:48 diana_coman jfw: cool.
15:48 whaack alright, no i am not done reading & understanding it, i will continue today
15:48 diana_coman whaack: good; that comes first; then the write-up; then decision/discussion; only after all that any implementation.
15:49 whaack diana_coman: ack
~ 22 minutes ~
16:12 jfw whaack: re yum updates - Red Hat publishes errata in the form of advisories, updated source RPMs, and binaries for paid customers. CentOS grabs the updated sources, applies the occasional rebranding type patch, and builds their free binary RPMs. These live in a separate "updates" repository while the base repository in theory doesn't change from release time. Sometimes the problems fixed are
16:12 jfw minor; sometimes they're security related; sometimes those might even be exploitable in your usage. Obviously this is not a V-like process, so the decision is necessarily kinda blind: do you trust the old packages with more-known holes or new ones with unknown holes?
16:14 jfw Standard "best practices" in "the industry" based on vendor's advice is to always install updates timely - if you're compromised by known bug it's seen as lazy system administration, whereas if compromised by the inherent bug of 'always take updates' policy then who coulda possibly predicted.
16:15 jfw That clarify the mess at least?
16:19 whaack jfw: yes it does, thank you.
16:20 jfw and would you have anything to add/correct there diana_coman? I recall you distinguish 'updates' from 'patches' but not sure how that works out in CentOS case
16:23 diana_coman jfw: re CentOS tbh I do not upgrade and that's that; the whole old-holes vs new-holes can easily be stated in quite a few interesting ways (perhaps the most appropriate being the good ol' are you with the party -and so we'll commiserate at your loss though ofc you'll lose- or are you not with the party and so you are to blame for (all) loss(es).
16:24 diana_coman re updates vs patches what/where do you mean?
16:26 jfw ah, vpatch vs upgrade was the distinction rather; http://logs.ossasepia.com/log/ossasepia/2019-11-05#1008861
16:26 ossabot Logged on 2019-11-05 16:02:55 diana_coman: jfw: out of pure curiosity now: do you find yourself upgrading software much those days?
16:28 diana_coman at any rate and to state it clearly: it's not that I consider CentOS 6 to be some great OS or anything of the sort. It's just that I don't currently have anything better to put in its place and it is at least not changing aka not creating more work for no reason.
16:34 jfw right. I like the 'party' comparison
16:36 jfw whaack, would you like any more input from me on the CL memory thing or are you set for now with tracking down what the code does and what might be the problem?
16:37 whaack jfw: I would love input on the CL memory problem
16:38 jfw ok, well 25MB supposedly due to asdf is an improvement on 70MB noted previously, do I understand right?
16:38 diana_coman jfw: btw, your description above was fun to read anyway; my "write it as satire and have fun" earlier was not random either.
16:38 jfw ty diana_coman
16:39 whaack jfw: yes, and that was from running my script with 'sbcl --script my_progam.lisp' (with (require "asdf") as the first line instead of running my script with 'sbcl --load my_program.lisp' (--load automatically loads an init file )
16:40 jfw whaack: a tricky thing with that 25MB, depending how you're measuring - might not be asdf itself but parts of sbcl that got swapped in to load it (just guessing, I'm not up on sbcl internals), or garbage collection overhead
16:40 jfw one way to look at it though, is that a CL environment is like an operating system in its own right, with memory-resident compiler even
16:42 jfw so the way it 'wants to be used' is with internal scheduling - internal processes/threads or some async thing
16:42 jfw so that overhead is paid once and shared by all processes. Running multiple UNIX processes, as IIRC you're doing, works but is kinda like loading a separate Linux kernel for each shell
16:43 jfw Doesn't this bot support threads?
16:45 whaack jfw: yes. And if I can't reduce the memory overhead I am planning on redesigning the system to run all in one process
16:46 jfw why would that be a 'redesign'? too much state in global variables or something?
16:46 whaack (i meant redesign the system to run in one unix process, still using multiple threads)
16:47 jfw well I guess the way it spawns up bots for the many networks would have to change at least.
16:47 jfw (understood)
16:48 diana_coman whaack: why did you choose separate unix processes vs single process?
16:49 jfw If it really is ASDF eating all that RAM though, the next step on that route could be removing that and explicitly specifying the whole buncha .lisp files to load and in what order, as I think you were planning anyway.
16:49 whaack diana_coman: Two major reasons. One was for fault tolerance, since my program is writing to log files, parsing messages from random networks, etc. I figured there was a lot of opportunity for something to go wrong, and I didn't want a problem with one network crashing all the other networks.
16:51 whaack diana_coman: The second reason is that there is a cap on threads-per-unix-process and web-sockets-per-unix-process. This problem is likely fixable - I believe there are commands / settings I can change to increase these caps.
16:52 jfw 'web-sockets'? O.o
16:53 diana_coman uhm, for the first reason, if it crashes, it's the code that crashes so the fault is in there and as a result, not much gain from separating.
16:53 diana_coman it's not the network that crashes after all, but your code.
16:55 whaack diana_coman: right. it's not a great reason. it's similar to the ~ "i'm going to throw this all in a try/catch cuz i can't conceptualize all the things that go wrong"
16:55 diana_coman anyways, for now do the figuring out and the write up and the problem(s) description + options you see and we'll work out a proper process from there.
16:56 whaack (which I do anyways, surrounding the code that reads messages from the network)
16:57 diana_coman whaack: thing is, they will go wrong whether you conceptualize them or not; and the apparent-defense is worse than no defense in that it masks the problem until it becomes a beast.
16:59 whaack jfw: yup i could figure out the order of the .lisp files, but I think I'm going to pass on that task. and web-sockets i guess is the wrong terminology lol. there's a max number of sockets-per-process
17:00 whaack diana_coman: yup. the reconnect-disconnect dance was an example of a masked problem that bit me
17:01 diana_coman myeah.
17:02 jfw whaack: ah ok, saw the dashes and figured those were the literal symbols. There's also a Unix-level limit on file descriptors per process (which category includes sockets): ulimit -n
17:10 whaack jfw: I have to review what a file descriptor is / how "files" work under the hood. My understanding is that there is a table mapping integers to files. The integers in this table are the file descriptors. Files are anything that can be written to or read from. So they can be a block of address space in storage, or a buffer from a connection over a network.
17:13 jfw not a bad approximation; one missing level is the 'open file object'; these can be shared between processes. The file descriptor table is per-process
17:14 * whaack did not know the file descriptor table was per process
17:17 jfw 'block of address space in storage' - not if you're thinking of that as a contiguous region on disk, it's more indirect than that, but the kernel presents it as contiguous address space, yes
17:18 auctionbot S#1077 O=17mn LB=None E=2020-01-29 05:43:03.567830 (102h28) >>> Dell R610 PE Server ships from U.S. (Server-A) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
17:18 auctionbot S#1078 O=17mn LB=18mn E=2020-01-29 05:43:42.679910 (102h28) >>> Dell R610 PE Server ships from Uruguay (Server-B) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
17:18 auctionbot --- end of auction list, 18mn total bids ---
17:19 whaack jfw: ok, i wasn't sure either way
17:19 jfw 'buffer from a connection' - the 'buffer' doesn't quite belong, it's more of an implementation detail. Could be as small as one byte (perhaps on a serial port)
17:23 jfw example of per-process FDs: FD 1 is considered 'standard output'; one process might have it connected to an xterm (via pseudo-tty driver), another to a regular file (as in a shell redirection), another to /dev/tty1, another to /dev/null etc.
17:25 * jfw bbl.
17:26 whaack ok ty
~ 2 hours 8 minutes ~
19:35 * jfw is back btw.
~ 44 minutes ~
20:19 whaack I am getting an error "Determing IP Information for eth0..." failed. [Failed] after running "service network start" I guess I am missing a driver. My /etc/sysconfig/network-scripts/ifcfg-eth0 file has the [following contents] [http://paste.deedbot.org/?id=aX_B]
20:23 whaack I also get spammed RTNETLINK answers: File exists
20:25 jfw whaack: I suspect you've got some basics to learn here too. ifconfig and route commands, DHCP
20:25 whaack yes i do
20:26 jfw the sysconfig scripts are fine but merely automate lower level tools that you will still need for figuring out what's going on.
20:31 jfw add 'ping' for sure too, then in the more advanced level of the toolkit there's arp, netstat, tcpdump. Then there's name resolution which is controlled principally by /etc/hosts and /etc/resolv.conf and there's various tools for DNS testing.
20:35 jfw iptables will be a thing to learn about down the line I expect.
20:41 whaack jfw: alright, i will make a note to look into all those tools. for now am I correct in thinking that the error I'm receiving is because I am missing a driver?
20:42 jfw I think not.
20:42 jfw ifconfig will have the answer to that though.
20:42 * whaack wishes he had man pages
20:43 jfw lol. well there's man pages online. Or on your mac, though there are slight differences.
20:44 whaack lol the man pages on the mac are dangerous! i got tripped up by a slight difference for some command a while back
20:46 whaack (specifically: the mac man page for crontab doesn't mention that crontab will fail if the crontab file doesn't have a newline at the end, nor does the tool give a warning.)
20:47 whaack the result from ifconfig brings up eth0 with RX: packets:3259, so it looks like there is some life there, i will invsestigate
20:50 jfw possibly very simple problem: did you try 'service network restart' ? perhaps it was already working but the scripts don't detect the double-start?
20:51 whaack yeah i did try that
~ 32 minutes ~
21:23 whaack looks like the problem is with dhclient not being able to get an ip address. the output from "ip link" and "mii-tool" makes me believe i am connected to my router
21:24 jfw First base to cover is whether it's even expected to get one, i.e. that the router is serving DHCP. I would imagine so if it's a typical home setup.
21:25 jfw Next, you could run dhclient in the foreground to see its output (would also go to syslog normally but uncertain as yet that you have that captured)
21:27 jfw re 'makes me believe', I'm less familiar with the 'ip' suite, it's a newer linuxism; the expected ifconfig output would include UP and RUNNING. Also worth checking for blinky lights on the network port.
21:34 whaack jfw: lights are blinky. dhclient eth0 -d gives me a few "DHCPDISOCVER on eth0 to 255.255.255.255 port 67 interval 7 (xid=0x1745f5ef) lines, and then No DHCPOFFERS received.
21:34 whaack I went to my router's page and I believe it is servering dhcp
21:38 jfw well, we could rule out DHCP problems for now by manually setting an otherwise unused IP/mask in the correct subnet and see if router pings
21:41 jfw hm, I'm also recalling that centos has some firewall rules enabled by default, though not especially strict. Possibly SELinux too
21:44 jfw on the former, 'iptables -F' should flush all the rules until next boot; on the former, check 'getenforce', either Disabled or Permissive is ok
21:48 whaack jfw: I set a static ip address and ran "ping 192.168.1.1" then I got "from 192.168.1.9 icmp_seq=2 Destination Host Unreachable" for four lines and then after ping: sendmsg: No buffer space avaialable
21:52 jfw the no buffer space thing is odd, does make me wonder about the driver or hardware. Maybe do some looking through dmesg for clues related to eth0
21:53 jfw anyone else watching have ideas at this point? I'm running low
21:59 whaack I see "eth0: no IPv6 routers present" in dmesg
22:02 jfw that's normal (for an ipv6-loaded kernel)
22:04 jfw something in there should name the chipset/driver which could be an avenue of research.
22:06 jfw I'm going afk; might be one of those times to break on this, do more reading, come back later with fresh mind
22:06 whaack getenforce was set to Enforcing, I Switched it to Permissive and restarted my connection, but that did not help
22:06 whaack yup i'm going to put this down until tomorrow
22:06 whaack thanks for the suggestions
22:06 jfw ahh ok. Well that'll avoid other troubles at any rate.
22:07 jfw iirc 'setenforce' writes to filesystem and so is persistent across boots
22:07 jfw and you're welcome.
22:18 auctionbot S#1077 O=17mn LB=None E=2020-01-29 05:43:03.567830 (97h28) >>> Dell R610 PE Server ships from U.S. (Server-A) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
22:18 auctionbot S#1078 O=17mn LB=18mn E=2020-01-29 05:43:42.679910 (97h28) >>> Dell R610 PE Server ships from Uruguay (Server-B) http://blog.mod6.net/2020/01/physical-specifications-for-the-bitcoin-foundations-servers/
22:18 auctionbot --- end of auction list, 18mn total bids ---
← 2020-01-23 | 2020-01-25 → ↑

Random(ossasepia) | Download hourly DB snapshot | Get Source Code

Theme: Light | Dark | System Default