#pest | 2024-12-27

awt: what's a galley fulla ukrs cost these days, anyway?

14:49

(awtho) No idea? $40/hr? Honestly no clue

14:50

gpu farm also costs sumthing neh

14:53

*

asciilifeform experimented, like errybody else, with gptism at the tail end of his commercial work days, but it was uniquely terrible fit for the line of work he was in

14:54

i.e. moar or less the exact opposite of 'generate 100kB of boilerplate for www app'

14:54

(awtho) I'm using Cline in Visual Studio. First plugin I've found that will work at the project level rather than just the file level.

↖

14:55

(awtho) Makes sense. I'm delving deep into platform-based liquishit here.

14:56

the days of 'get $$ for www boilerplate' are prolly numbered. sorta like how starving immigrants could get paid to wash dishes in restaurant, but prolly nowhere nao, they use machines

14:57

(while the days of 'get $$ to fix what gpt users subtly fucked' not yet arrived, nor obvious that they will, likely things will simply stay subtly fucked indefinitely)

14:59

(awtho) Scratching my own itch here. I frequently get emails from customers with the same issues. Trying to put something in front that does some triage before I have to deal with it.

15:01

a, the 'q&a chatbot' a la lulazon et al support ?

15:01

(awtho) Over email, but yes.

15:03

*

asciilifeform would expect it'd be infuriating to folx w/ 'over roomtemp iq', like lulazon's, but not knows who is the customer base, so can't say for sure

15:03

(awtho) Not over roomtemp iq

15:03

*

asciilifeform would rather read an ordinary text faq than fight with a bot, errytime

15:03

a then wainot.

15:04

(awtho) Yes specifically happens with the customers that are at roomtemp IQ.

15:05

there's a species of humanoid that want to 'feel served' 1st, while wants actually to resolve a question a distant 2nd

15:06

for these, 'eliza'-level 'ai' would have prolly sufficed, not sure wai the bots had to wait for gptism etc

15:06

(awtho) It's just regular people who don't understand image resolution and transparency and how that translates to the quality of a finished product.

~ 2 hours 24 minutes ~

17:31

http://logs.bitdash.io/pest/2024-12-27#1034622 << neat, have you also tried Continue? if so, how do they compare?

17:31

bitbot

Logged on 2024-12-27 14:54:51 discord_bridge[billymg]: (awtho) I'm using Cline in Visual Studio. First plugin I've found that will work at the project level rather than just the file level.

17:33

i've been using Continue in VS Code backed by a locally running model, "DeepSeek-Coder V2 Lite". it's been great for certain things, also handles spanish localization pretty well

17:34

i've not tried having it write a full app though, mostly just use it to generate boilerplate snippets

17:38

i think what i like most about it is that it makes writing webshit a lot less tedious. it acts as a sort of motivator, by removing the "ugh" hurdle to sitting down and making progress

17:38

but it's possible that wears off over time

17:39

(awtho) I haven’t tried Continue. I’ll check it out.

17:40

(awtho) Yeah Cline made a bunch of boring work disappear: oauth, email threads, writing tests…

17:40

are you using an API for the GPTism or running something locally?

17:42

(awtho) I’m using Claude via api.

17:43

(awtho) Running into the token limits is slowing me down on tasks with a lot of context.

17:48

how much vram you got?

17:49

(awtho) On my MacBook pro? Lol

17:50

(awtho) I have a very heavy duty box at home but don’t know the stats.

17:51

https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF << probably won't match the Claude API but the whole thing fits on a 24gb consumer card

17:52

(awtho) I’m gonna try it.

17:54

i'm using "Q8_0" quant type. since i've not tried any of the API models i'd be curious to hear how you think it compares in your use case

18:01

i serve the model locally with: https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#llama-server, which the Continue plugin talks to

~ 2 hours 9 minutes ~

20:11

awt, billymg : any speshul sauce req'd to try these on crapple? ( asciilifeform atm has an 'm2 ultra' w/ 192GB ram that's mostly idle, lol )

20:22

asciilifeform: can build the llama.cpp thing for metal https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#metal-build

20:22

billymg: ty!

20:23

*

asciilifeform wonders how performance would compare to the moar typical pcie gpu

20:23

from there can play around with any .gguf format model you find on huggingface

20:24

*

asciilifeform has the box simply on acct of it having been, at the time bought, 'fastest gcc box $ could buy', but not yet even tried gpuism of any kind on subj

20:26

asciilifeform: not sure, i think if you compare to top of the line nvidia cards it's a lot slower, but that's assuming the person with the nvidia card has enough vram to fit the whole model. usually they don't, whereas on your mac you've got the 192gb of 'unified' ram that the gpu cores can use

20:26

sadly haven't a recent nvidia card to compare to, but oughta suffice for simple experiments

20:27

nvidia intentionally gimps the vram on consumer cards for this reason, so if you want more than 24gb at the moment you gotta pay ~4x what a 4090 costs for the A6000 (ada model)

20:31

if the gpttrons could do, for the sorta things asciilifeform does, what they do for 'make me http server boilerplate' folx, the cost wouldn't seem excessive to asciilifeform

20:31

but afaik atm lolno

20:37

afaict it really is just an advanced autocomplete, so only works for things that have been written before

20:38

not necessarily useless on complex problems simply from this fact. theoretically, could be the proverbial 'sufficiently smart compiler'(tm)

20:39

*

asciilifeform suspects that there are plenty of 'low hanging fruit' simple patterns in all kindsa domains that haven't been usefully, systematically mechanically detected yet.

20:43

afaik the irons req'd to actually train gptistic models de novo, tho, costs $maxint

↖

20:44

*

asciilifeform if not for this, would've tried already the mega-obvious application: to sniff for interestingly weak rngs...

20:45

all the coinz sitting in addrs genned on winxp etc., potentially, waiting to be excavated...

20:52

'hypertrophied autocomplete' in principle could make short work of keys that came outta gimped rng. (considering that it aint the least bit difficult to bring up a 2009-style ms victim's box and get as many samples as could ever want)

20:57