#trilema | 2017-07-02

Unless you go full synthetic on the fuel and they why the fuck not drill for natural gas then

00:18

<asciilifeform> mircea_popescu: afaik there is no konsoomer nife. there are only industrial. << In UK "consumer" sets are being marketed for solar crowd

00:20

hola

00:20

sina: got any better ideas for comparing program runtimes than perf?

00:21

ben_vulpes: dtrace?

00:22

gprof?

00:22

I heard good things about systemtap but never used it myself

00:22

ben_vulpes: what is wrong with perf for your usecase?

00:23

sina: i wouldn't know, is why i'm asking

00:23

Isn't dtrace that Sun Microsystems thing that came with Solaris 10?

00:23

BingoBoingo: yah

00:23

it's pretty awesome

00:24

you can run it on fbsd

00:24

systemtap is basically "dtrace for linux" I guess

00:24

but again, I haven't used that one

00:25

mircea_popescu: around? any time to play with gossipthing?

00:26

ben_vulpes: can we roll back and start at the usecase?

00:26

why to compare mpfhfhfhfhfhfs!

00:30

ben_vulpes: oic. and why not just using black box testing?

00:30

e.g. "how long does it take to hash document of N bytes size in M bits hash" with varying N and M?

00:30

...

00:30

yes

00:30

eg runtime?

00:31

is that all you want to measure? why not just use `time` then?

00:31

something something not great subsecond resolution or so the various reddits say?

00:32

obvious counterargument is that "don't bother with subsecond executions, dork"

00:32

well

00:32

here is the thing, it depends on how anally you want to measure

00:32

because, for example, python and lisp, probably most of the time will be spent in starting the runtime/interpreter than actual computation, unless you're doing larger sized stuff

00:33

so if you really want super precise, apples <=> apples comparison, you would need to instrument performance on a per lang basis, no?

00:33

of the main loop iterating through M

00:33

no i do intend to black box it

00:34

runtime startup is a cost of the program, innit?

00:34

depends!

00:34

mk go on, on what does it depend

00:34

if I make an mpfhf daemon, then no

00:34

(for example)

00:34

aye

00:35

that might be a fairer blackbox test?

00:35

anything else?

00:35

lol, well, depends!

00:35

well, I dunno too much about lisp, does it "JIT" for long running programs?

00:35

i wouldn't keep a lisp runtime hanging around just on the offchance i want to hash things

00:35

i've no idea what you mean by that in this context

00:36

but extending from what i know of java's jit (not much), no. the whole file (at least in the tests i'm running) is compiled.

00:36

for example, pypy is much faster than cpython for long running programs, because it Just In Time compiles

00:36

same as LuaJIT vs Lua

00:36

right

00:37

faster in steady state runs or faster to compile?

00:37

faster in the run because its no longer "interpreting"

00:37

its reading something closer to native compiled code

00:39

dun think that's really in play here

00:39

ben_vulpes: http://wotpaste.cascadianhacker.com/pastes/wBiP0/?raw=true

00:39

as an example

00:40

buddy how do you think i've been comparing things?

00:40

a stopwatch?

00:40

ben_vulpes: sorry. I am referring to pypy JIT vs python

00:41

just as an example of "anything else?" re "depends!"

00:41

<+ben_vulpes> runtime startup is a cost of the program, innit? <<

00:41

sina: how does the paste relate to the JIT thread?

00:42

"cython wouldn't know the input type without chasing pointers all over the place"?

00:43

ben_vulpes: you asked what else "it" can depend on, where "it" == whether or not runtime startup is a cost of the program or not

00:44

my point was, either you *really* care about evaluating the actual main loop, which is a fair apples/apples

00:44

or you are OK with a blackbox, in which case can just use time and avoid calling it with small loops where the runtime startup cost dominantes `time`

00:45

y/n?

00:45

ah yeah i thought we'd put that thread to bed with http://btcbase.org/log/2017-07-02#1678389

00:45

Logged on 2017-07-02 04:33 ben_vulpes: no i do intend to black box it

00:46

black box doesn't have to be invoking from CLI each time tho, you could write a daemon around each impl and measure how long it takes to return a value

00:46

could, yes

00:47

anyway, it does sound like `time` should be fine

00:49

adequately 'apples' from my pov; not particularly interested in imaginary performance comparisons of subsets of compiled programs

00:54

from reading https://stackoverflow.com/questions/12593768/how-is-lisp-dynamic-and-compiled it looks like GNU CLISP compiles down to bytecode, which I guess will be probably performance equivalent to pypy

01:02

it does sound like lisp is doing something JIT-like, except you can "re-JIT" at any time during execution?

01:05

sbcl lays down asm?

01:05

ah ok

01:06

i think

01:06

ben_vulpes: did you use sbcl the other day when you mentioned golang impl was faster than lisp impl?

01:06

cos that'd be pretty interesting

01:06

definitely has hooks for disassembling a given function, so it would be nice if it also compiled *to* asm

01:06

sina: aye, i did

01:07

hmm

01:07

go's some 3-4x faster incl. runtime

01:08

well, I guess lets see how the benchmarks play out over a larger dataset, maybe it evens out over a certain bitlength or bytesizer

01:10

myeah 'tis what i'm thinking

01:10

well who knows, who'm i to make guesses like that

01:10

but i am definitely interested to see how performance plays out on large sets

01:12

ben_vulpes: will you include pypy?

01:13

could yeah

01:14

ben_vulpes: I am about to head out the door so currently not the best time, but if it would help in anyway I can donate some compute to the effort

01:14

can leave me a note with anything you require

01:14

mk neato

01:14

no rush

01:14

i'm well supplied with compute tho, thx

01:15

cool

01:15

well

01:15

I am out. hope all have a wonderful day

01:18

in other optimizations: https://i.imgur.com/h7WcmU5.gifv

~ 16 minutes ~

01:35

http://www.contravex.com/2017/07/01/le-150ieme-partie-i/ << » Contravex: A blog by Pete Dushenski - Le 150ième – Partie I

~ 21 minutes ~

01:57

ben_vulpes or you could just instrument your impl to read time at start and end ?

02:00

sure

02:02

i'd call it good

02:06

you mean start and end of actual hashing routine?

02:06

why disregard runtime startup time?

02:12

because you're timing the actual impl.

02:20

hm

02:22

not "unix tool as it may or may not be used in the future"

02:24

nobodyu cares about shitix.

02:24

rightright

02:28

had like five followups, all of which are probably answered by "this is probably one of those things worth doing rigorously"

02:42

heh

02:43

well if i time just the hash impl it does squeak in under the go implementation

~ 16 minutes ~

02:59

http://qntra.net/2017/07/bitcoin-network-mining-difficulty-drops-0-43-percent/ << Qntra - Bitcoin Network Mining Difficulty Drops ~0.43 Percent

03:10

BingoBoingo: gotta a grass q for ya

03:10

mulch or compost?

03:11

Your lawn? Compost. Don't mulch your lawn, it needs to breath.

03:11

!~ticker --market all

03:11

jhvh1

BingoBoingo: Bitstamp BTCUSD last: 2416.74, vol: 10266.02584088 | BTC-E BTCUSD last: 2358.591, vol: 2944.95989 | Bitfinex BTCUSD last: 2333.4, vol: 14881.3570013 | BTCChina BTCUSD last: 2543.63455, vol: 7386.44530000 | Kraken BTCUSD last: 2397.731, vol: 4991.08122316 | Volume-weighted last average: 2402.67930284

03:20

BingoBoingo: ty

~ 23 minutes ~

03:44

your grass is welcome

03:55

a not-great thing about this chart is that hash length is in bits but message length is in bytes: http://cascadianhacker.com/wp-content/uploads/2017/07/100.png

03:56

sina if you can get your implementations to print <execution_ms>\n<hash> that'll save me a bit of fiddling

03:58

https://github.com/sinner-/mpfhf-python/blob/master/LICENSE << hey sina ever read http://trilema.com/2015/a-new-software-licensing-paradigm/#selection-75.0-83.128 ?

~ 4 hours 24 minutes ~

08:22

ben_vulpes: now enabled "python mpfhf.py <message> <bits> <time|notime>" (https://github.com/sinner-/mpfhf-python/commit/a8c37e51e3a893785519052f939c222c2178168a) and "mpfhf-golang -message <message> -bits <bits> -time <y|n>" (https://github.com/sinner-/mpfhf-golang/commit/674a502a93c63de8dacbb07e5eb9a62eb5dbee58)

08:23

http://btcbase.org/log/2017-07-02#1678478 << re this request

08:23

Logged on 2017-07-02 07:56 ben_vulpes: sina if you can get your implementations to print <execution_ms>\n<hash> that'll save me a bit of fiddling

08:27

example: http://wotpaste.cascadianhacker.com/pastes/84afn/?raw=true

08:31

asciilifeform: thoughts https://micropython.org/

08:44

sina: garbage

08:44

even let's suppose it weren't python3 ( which it is )

08:46

python2 was also garbage, infix pseudolisp with globalinterpreterlock, broken lambda, 1,001 eager idiocies. apparently i write a handful of throwaway rubbish proggies in it ( was less atrocious than perl, so i threw out perl) but now i'm condemned for life to hear about it

08:46

and to have folx suppose that i somehow want to perpetuate it.

08:47

and, worse, to introduce it to places that happily worked without idiotic interpreter ( e.g. microcontroller ) at all

08:50

http://btcbase.org/log/2017-07-02#1678460 << how about we roll the boot time ( to shell!! ) of your cmachinekernel, how about?

↖ ↖ ↖ ↖

08:50

Logged on 2017-07-02 06:06 ben_vulpes: why disregard runtime startup time?

08:50

if counting lisp runtime load, why not the c runtime.

08:52

http://btcbase.org/log/2017-07-02#1678431 << not only does it, but there is not even an interpreter in there as fallback ( see the old sbcl vs cmucl threads )

08:52

Logged on 2017-07-02 05:05 ben_vulpes: sbcl lays down asm?

08:56

http://btcbase.org/log/2017-07-02#1678356 << there are afaik no acceptable autoprofilers in existence at all. they all do this idiotic thing with statistical sampling rather than actual per-line timer ( because apparently the year is eternally 1980 and there is no highres timer, or wat.)

08:56

Logged on 2017-07-02 03:58 ben_vulpes: in re benchmarking, is 'perf' a reasonable thing to use?

08:57

( see 'blackhole revealer' discussion from coupla mo. ago )

09:00

http://btcbase.org/log/2017-07-02#1678477 << i have nfi why mircea_popescu went with ascii-010010010111.. for the output format

09:00

Logged on 2017-07-02 07:55 ben_vulpes: a not-great thing about this chart is that hash length is in bits but message length is in bytes: http://cascadianhacker.com/wp-content/uploads/2017/07/100.png

09:01

imho oughta be hexascii like earthlings use

~ 44 minutes ~

09:45

http://phuctor.nosuchlabs.com/gpgkey/479C78D67322671E964668B28E0CC778B7E5CBB10EB315D3E438860DA9014D18 << Recent Phuctorings. - Phuctored: 1612...5087 divides RSA Moduli belonging to '160.39.90.80 (ssh-rsa key from 160.39.90.80 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US NY)

09:45

http://phuctor.nosuchlabs.com/gpgkey/479C78D67322671E964668B28E0CC778B7E5CBB10EB315D3E438860DA9014D18 << Recent Phuctorings. - Phuctored: 1497...2787 divides RSA Moduli belonging to '160.39.90.80 (ssh-rsa key from 160.39.90.80 (13-14 June 2016 extraction) for Phuctor import. Ask asciilifeform or framedragger on Freenode, or email fd at mkj dot lt) <ssh...lt>; ' (Unknown US NY)

~ 1 hours 25 minutes ~

11:11

http://btcbase.org/log/2017-07-02#1678500 << possibly artifact of how my first prototype did output; easily changed

11:11

Logged on 2017-07-02 13:01 asciilifeform: imho oughta be hexascii like earthlings use

11:13

or no, it dates to his prototype

11:15

but if you'll excuse me, i'm going to go make a hash of breakfast

~ 20 minutes ~

11:36

btw ben_vulpes your mphash seems to use some shitlibrary that 1) i dun have 2) won't install via quicklisp

~ 55 minutes ~

12:31

http://btcbase.org/log/2017-07-01#1678237 << and you're running it

12:31

Logged on 2017-07-01 23:36 sina: if you write a systemd unit file with "User=0day", it launches the process as root. Pottering sez: "not a bug"

12:32

http://btcbase.org/log/2017-07-01#1678246 << total fucking insult to the heavens.

12:32

Logged on 2017-07-01 23:52 asciilifeform: and nao bernstein, henninger ( this is what, 3rd paper since she was attached to him ) 'unhappened and rehappened' it

12:34

sina "The Bitcoin network has more than 6,000 nodes," << lost interest at that point.

12:36

and this isn't just mp being hoity toity. the point here is that the sort of superficial schmuck who imagines bitcoin has 6k nodes, is also the superficial schmuck who imagines if bitcoin is framed through usg owned internet, that'll "just oiccur". it won't just occur, the same day there's a nuclear blast on capitol hill, no questions asked.

12:38

historically, the best way to "inexplicably" die suddenly was to attempt to attack groups of strictly selected, very determined, technologically superior people. bitcoin is no exception, whatever the hallucinations of the "we are talking about it therefore involved in it" crowd may show.

12:39

(you are aware, yes, usg vulnerable to nuclear blasts, latest studies show ?)

12:40

http://btcbase.org/log/2017-07-02#1678472 << word.

12:40

Logged on 2017-07-02 07:11 BingoBoingo: Your lawn? Compost. Don't mulch your lawn, it needs to breath.

12:41

http://btcbase.org/log/2017-07-02#1678479 << to my eye the worst part of it is that it's very badly drawn. a) about half of the Y space is actually used, which is terrible. b) all the same color, they melt together, can't tell apart. can use color gradient ? (yes, on blue, not on red, can't see red).

12:41

Logged on 2017-07-02 07:58 ben_vulpes: https://github.com/sinner-/mpfhf-python/blob/master/LICENSE << hey sina ever read http://trilema.com/2015/a-new-software-licensing-paradigm/#selection-75.0-83.128 ?

12:42

but other than that, looks like exponential on mlen and perhaps linear on hlen ?

12:43

exponential on mlen << that dun look good...

12:44

kinda badly chosen cutoffs too, i don't specifically care re diff between 40 byte and 70 byte message. make it log on that side and do 16, 128, 1024, 8192 and 65536 byte messages, for 32, 256, 2048 bit hash lengths as a standard of testing.

12:44

if can't hash a 1GB message , or even 1MB, in less than geological time -- not very useful, sadly, algo

12:44

asciilifeform i can't really visually saw that appart, but looks like it's a hlen ** b mlen or such.

12:44

( a 512byte msg doesn't need to be hashed... why would you )

12:45

asciilifeform useful for different things. apparently all of modern computing comes to "adjust your expectations". what do you need 1gb codebases for ?

12:45

a right, nothing.

12:45

mircea_popescu: think back to what is the point of a hash to begin with

12:45

vpatches seem historically to go about 512 - 65535 bytes or so

12:45

if that'd covered, i'm happy.

12:47

originally algo ( in the 1st , ancient version, suggested by asciilifeform in http://btcbase.org/log/2016-12-24#1589879 thread ) was to destructure inputs for rsa signing

12:47

Logged on 2016-12-24 01:02 asciilifeform: incidentally, since (to borrow the lament of turing's school headmaster) 'the room already stinks of mathematics', i'll share a tidbit that i promised folx some half year ago and promptly forgot:

12:48

( what the derps call, maliciously misnamedly, 'padding' )

12:48

that op doesn't even need much more than a kb or so

12:49

1 of the things i've been curious re mphash is bits-of-message-touched vs message-length

12:49

and bits-of-output-flipped-per-bit-of-message-flipped vs message-length.

12:49

should be measured yes.

12:50

also ( and should be theoretically possible to calculate ) what is the worst-case cycle count.

12:50

and in other lulz, #trilema OODA loop : "hey, make an X" "what kind an X ?" "JUST MAKE AN X!" *picks arbitrary params a through w, makes such an X. "here you go" "o cool, now set b to this and q to that, link l to k and make c double d" "here." "holy shit the submarine now flies! and lays eggs! good job man!"

12:51

asciilifeform what's the 1kb figure ?

12:53

really a few kb

12:53

depending on rsa modulus length

12:53

right.

12:54

( N bit rsa modulus carries 'naked' payload of N bits )

12:54

yes.

12:54

basically, there's a large number of perfectly useful applications for 64kb capable hash. and upon examination it may be discovered that larger sizes not supported is not a bug

12:54

but a feature.

12:54

can -- if you must -- merkle tree

12:55

or other things. and if you don't must, you FUCKING DIDN'T NEED THE GB.

12:55

large part of usgization of normal processes. "oh, does your msg app support VIDEO ?" bitch, i don't want to watch overweight dudes with their cats braying into a mic.

12:56

"but this would make it take mbps to use!" "right."

12:56

"does it at least support unicode ?" "no. you may not teach reading to arabs, it's a sin."

13:00

btw here's another ! , potentially similarly bowel-loosening ! ( to the usual suspects ) hash algo on entirely different principles :

13:01

1) generate a single-use rsa public modulus, M

13:02

2) take the payload bitstring P, and calculate E = nextprime(P)

13:02

3) E is now your public exponent !!

13:03

this is computationally worse than mpfhf though. imagine, nextprime(todays log).

13:03

actually not so bad.

13:03

if you have a hardware gmp or something

13:03

no, even on pc.

13:03

try it sometime.

13:03

anyway i'll omit steps 4-6 and leave as exercise.

13:04

but theoretically this scheme -- while algebraic -- is as strong as the rsa used in the wrapping layer.

13:04

this aside ; i'm kinda loath to mix rsa in both padding and encryption. for all anyone knows they resonate

13:04

(above is pure magical thinking)

13:05

if you were to use == keys -- they actually would

13:05

but you wouldn't.

13:05

trivial cases aside. may be hidden class there

13:05

there's a (sadly nameless) principle, where a thing that ~someone else~ could easily do to your ciphertext, is not any added problem if you do it yourself

13:06

( 'someone else' being someone with 0 knowledge of the plaintext, naturally )

13:06

certainly.

13:06

funnily enough, last time we had a related thread, i looked for it in schneier 1995 where i THOUGHT i learned it

13:06

and -- mega-surprise -- it ain't there...

13:07

original printed material ? or did it get expunged ?

13:07

original

13:07

weird.

13:07

it was never there

13:07

i have nfi where, if anywhere, it lives on paper.

13:07

possibly i dreamed it.

13:07

anyway -- if you're making both the payload and the hash on the same machine...

13:07

asciilifeform it's well known, i had conversations re this in 2012. prolly can fish out quote.

13:08

lol why not 'on the same street'

13:08

asciilifeform have you seen the rngs ?

13:08

mircea_popescu: i can't imagine it wouldn't be well known. but i was digging for a 'schoolbook' description of it, which i thought existed.

13:08

this incidentally is a very undiscussed topic, suspiciously. what is the effect of generating BOTH the padding for a message and the key that's used on the same... historical debian / kochatron /etcv

13:09

mircea_popescu: one nuance is -- it is folly to plan FOR a broken rng. 'what to do if a shell lands in my trench? jump twenty metres and scatter yerself around'

13:09

the plan 'for broken rng' is to have 7 running from 7 batteries.

13:10

this is true.

13:10

'jesus bolt'

13:10

was jesus nut neh ? heli thing ?

13:10

aha yes

13:10

the item that holds the airscrew on.

13:10

ya. nut.

13:20

incidentally theoretically you don't need the nextprime(), all you need is to compute that gcd(e, p-1) == gcd(e, q-1) == 1.

13:21

( if it isn't -- make a new e, or a new m=p*q, whichever you prefer... )

13:30

http://trilema.com/2015/sooo-fetlife-is-butthurt/#comment-122323 in other lulz

13:34

*

mircea_popescu imagines alf's 7-prop helicopter...

13:37

http://trilema.com/2017/qntra-sqntr-june-2017-statement/ << Trilema - Qntra (S.QNTR) June 2017 Statement

13:44

mircea_popescu: http://btcbase.org/log/2015-03-31#1080340 << we had thread !

13:44

Logged on 2015-03-31 02:52 asciilifeform: i will now take the liberty of calling the hypothetical machine 'freyacopter'

13:45

flies equally well through molasses!

~ 1 hours 13 minutes ~

14:58

http://www.contravex.com/2017/07/02/le-150ieme-partie-ii/ << » Contravex: A blog by Pete Dushenski - Le 150ieme – Partie II

~ 2 hours 51 minutes ~

17:50

jurov

http://www.explo.yt/inspirobot.jpg << tmsr got its motto?

~ 51 minutes ~

18:41

http://btcbase.org/log/2017-07-02#1678507 << "bit-smasher"? came in from ql just fine for me. i only use it as a crutch to beat strings into bit-vectors.

18:41

Logged on 2017-07-02 15:36 asciilifeform: btw ben_vulpes your mphash seems to use some shitlibrary that 1) i dun have 2) won't install via quicklisp

18:55

consider importing just the function / class you use

19:01

ben_vulpes: i haven't 'updated' ql since 2011 and never again intend to !

19:01

hence no shitbitter or wateveritwas

19:02

updates. never again.

19:02

no genderolade kthx.

~ 50 minutes ~

19:52

yes this refrain does ring a few bells

19:55

i will probably just implement the slice that i actually need

19:57

in other news, 65536 byte message, 256 bit hash took some 2.47 hrs

20:01

and heningerisms: "In this paper we demonstrate a complete break of RSA-1024 as implemented in Libgcrypt." https://eprint.iacr.org/2017/627.pdf

~ 15 minutes ~

20:16

ben_vulpes that's pretty substantial. s size ?

~ 53 minutes ~

21:10

that'd be the other point of interest, "how big does S get"

~ 35 minutes ~

21:46

!!deed http://lobbesblog.com/billing/2017/q2/Minigame_payment_receipt.txt

21:46

accepted: 1

21:46

!!deed http://lobbesblog.com/billing/2017/q2/diana_coman_payment_receipt.txt

21:46

accepted: 1

21:46

!!deed http://lobbesblog.com/billing/2017/q2/mircea_popescu_payment_receipt.txt

21:47

accepted: 1

~ 20 minutes ~

22:07

lobbes are you going to not let delinquents auction till they pay ? :D

~ 18 minutes ~

22:25

mircea_popescu, hehe. I was thinking of implementing that once n00bs start using the thing. Make it Wot-based at some level; disabling auction service for people lower down in WoT once delinquent x amount of time

22:25

I gotta say, I'm experiencing the whole "WoT make decisions easier" thing in practice now. I.e. I know hanbot is good for it, and since I trust danielpbarron's rating of Birdman, I also trust that he, too, will be good for it

22:40