Show Idle (> d.) Chans

Results 1 ... 15 found in asciilifeform for 'from:billymg crawler'

billymg: trinque: fwiw my working on the btc network crawler and new www to display obvious centralization of network is to attract more hands
billymg: asciilifeform: from there if you added some watchglass methods for getting blocks i could then incorporate those into the crawler (if that is what you meant by trying to analyze block propagation)
billymg: as soon as that's up will publish genesis for the crawler portion
billymg: my goal in making this crawler is to get more "bitcoiners" running trb nodes, and i suspect some al gore / nate silver stats and infograffix will widen the pool of those who see there is a problem
billymg: the ~8500 "actual" nodes number seems to be inline with what heathen trackers report as the total number of nodes on the network (~9k), so i suspect the crawler is nearly complete in mapping out reachable nodes
billymg: good morning, asciilifeform. my crawler seems to have hit a spam vein on the network, total unique IPs in the db exploded over the last two days to over 100k (these all come only from what a node returns in a 'getaddr' request). of those, when subsequently interrogated, only 8580 respond with a valid version message
billymg: i initially looked at it with the idea of repurposing for my crawler, barfed at 1001 dependencies pulled in, then remembered, "hey, watchglass does this"
billymg: asciilifeform: lotta hoops to jump through in setting it up:
billymg: the updated version of the crawler has been humming along nicely since last night, it's now up to ~4900 nodes discovered (heathen sites report over 9000)
billymg: << this wasn't even on my mp-wp todo list but since using postgres for the crawler it's now jumped near the top
billymg: from here i'm just going to proceed with making the crawler send the correct version message depending on whether it's trying to reach a trb or prb node. but yes, perhaps could write it so it tries once with '99999' then tries with '70001' and records result
billymg: asciilifeform: ah, interesting. now i'm wondering what the crawler could do to coax a node into sending a 'heathen command' in a reasonable amount of time
billymg: asciilifeform: i think whaack was working on a replacement block explorer. this thing i'm building is much simpler, just a network crawler
billymg: my reason for doing so was because both bitnodes and stopped tracking trb with their crawlers. i used to be able to check from time to time to see how many trb nodes are out there
billymg: asciilifeform: i wrote a simple btc network crawler that uses watchglass for node probing and dumps the results into a postgres db. it's been running since yesterday afternoon, here are some stats so far: