This has been the biggest boost for me. The number of choices available when facing a blank page is staggering. Even a bad/wrong implementation helps collapse those possibilities into a countable few that take far less time to think about.

Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers.

> The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.

For me, my skepticism of using a statistical text generation algorithm as if it were a search engine is because a statistical text generation algorithm is not a search engine.

Recently I did some tests with coding agents, and being able to translate a full application from AT&T Assembly into Intel Assembly compatible with NASM, in about half an hour of talking with agent, and having the end result actually working with minor tweeks isn't something a "decent search engine a la Google circa 2005." would ever been able to achieve.

In the past I would have given such a task to a junior dev or intern, to keep them busy somehow, with a bit more tool maturity I have no reason to do it in the future.

And this is the point many developers haven't yet grasped about their future in the job market.

  > the conversational interface, for some reason, seems to turn off the natural skepticism that people have

People are expecting perfection from bad spec

Isn’t that what engineers are (rightfully) always complaining about to BD?

I've definitely also found that the poor code can sometimes be a nice starting place. One thing I think it does for me is make me fix it up until it's actually good, instead of write the first thing that comes to mind and declare it good enough (after all my poorly written first draft is of course perfect). In contrast to the usual view of AI assisted coding, I think this style of programming for tedious tasks makes me "less productive" (I take longer) but produces better code.

Not really, not always. To anyone who’s used the latest LLMs extensively, it’s clear that this is not something you can reliably assume even with the constraints you mentioned.

They don't

> Garbage in = garbage out generally.

Generally, this statement is false

> When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well.

Keyword: can.

They can also not perform really well despite all the management and materials.

They can also work really well with loosey-goosey approach.

The reason is that they are non-deterministic systems whose performance is affected more by compute availability than by your unscientific random attempts at reverse engineering their behavior https://dmitriid.com/prompting-llms-is-not-engineering

No they don't, they generate a statistically plausible text response given a sequence of tokens.

"UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges" Also "The use of faulty AI is not new for the health care industry."

It would actually have been more pernicious that way, since it would lull people into a false sense of security.

But I actually can’t imagine how you can teach someone to code if they have access to an LLM from day one. It’s too easy to take the easy route and you lose the critical thinking and problem solving skills required to code in the first place and to actually make an LLM useful in the second. Best of luck to you… it’s a weird time for a lot of things.

*edit them/they

TBF, trial and error has usually been my path as well, it's just that I was generating the errors so I would know where to find them.

It's hard to remember what it was like to be in that phase. Once simple things like using variables are second nature, it's difficult to put yourself back into the shoes of someone who doesn't understand the use of a variable yet.

People with a minimum amount of expertise stop asking for advice for average circumstances very quickly.

When talking with reasonable people, they will tell you if they don't understand what you're saying.

When talking with reasonable people, they will tell you if they don't know the answer or if they are unsure about their answer.

LLMs do none of that.

They will very happily, and very confidently, spout complete bullshit at you.

It is essentially a lotto draw as to whether the answer is hallucinated, completely wrong, subtly wrong, not ideal, sort of right or correct.

An LLM is a bit like those spin the wheel game shows on TV really.

> At least with humans I have some basic level of trust, so that even if I don't understand the code at that level, I can scan it and see that it's reasonable.

If you can't scan the code and see that it's reasonable, that's a smell. The task was too big or its implemented the wrong way. You'd feel bad telling a real person to go back and rewrite it a different way but the LLM has no ego to bruise.

I may have a different perspective because I already do a lot of review, but I think using LLMs means you have to do more of it. What's the excuse for merging code that is "off" in any way? The LLM did it? It takes a short time to review your code, give your feedback to the LLM and put up something actually production ready.

> But every piece of LLM generated code I've seen to date hasn't been trustworthy once I put in the effort to really understand it.

That's why your code needs tests. More tests. If you can't test it, it's wrong and needs to be rewritten.

My approach is to describe the task in great detail, which also helps me completing my own understanding of the problem, in case I hadn't considered an edge case or how to handle something specific. The more you do that the closer the result you get is to your own personal taste, experience and design.

Of course you're trading writing code vs writing a prompt but it's common to make architectural docs before making a sizeable feature, now you can feed that to the LLM instead of just having it be there.

Yep - my number 1 use case for LLMs is as a template and example generator. It actually seems like a fairly reasonable use for probabilistic text generation!

IMO, they're still useless today, with the only progress being that they can produce a more convincing facade of usefulness. I wouldn't call that very meaningful progress.

LLMs can only give you code that somebody has wrote before. This is inherent. This is useful for a bunch of stuff, but that bunch won't change if OpenAI decides to spend the GDP of Germany training one instead of Costa Rica.

Programming has become vastly more efficient in terms of programmer effort over decades, but making some aspects of the job more efficient just means all your effort it spent on what didn’t improve.

no i don't remember that. They are doing similar things now that they did 3 yrs ago. They were still a decent rubber duck 3 yrs ago.

I think we are, especially with executives mandating the use LLMs use and expecting it to massively reduce costs and increase output.

For the most part they don't actually seem to care that much about software quality, and tend to push to decrease quality at every opportunity.

Yeah we shouldn’t and I limit my usage to stuff that is easily verifiable.

But there’s no guardrails on this stuff, and one thing that’s not well considered is how these things which make us more powerful and productive can be destructive in the hands of well intentioned people.

I'm betting they're the most senior people on the team.

But he should've know that people would jump at the opportunity to contradict him and should've written his comment so as not to admit such an easily-contradictable interpretation.

I genuinely think this could be great for toys that kids grow up with i.e. the toy could adjust the way it talks depending on the kids age and remember key moments in their life - could be pretty magical for a kid

I have been drawing all my life and studied traditional animation though, so it’s probably a little bit of nature and nurture.

But these endless claims that the fact they're "just" predicting tokens means something about their computational power are based on flawed assumptions.

So the argument goes: LLMs were trained to predict the next token, and the most general solution to do this successfully is by encoding real understanding of the semantics.

It's even crazier that some people believe that humans "evolved" intelligence just by nature selecting the genes which were best at propagating.

Clearly, human intelligence is the product of a higher being designing it.

/s

> it was a decent, if probabilistic, rubber duck

How is it a rubber duck if it suggested where the problem was?

Isn't a rubber duck a mute object which you explain things to, and in the process you yourself figure out what the solution is?

Look, we see the forest. We are just not impressed by it.

Having unlimited chaos monkeys at will is not revolutionizing anything.

There's no guarantee a technology will take off, even if it's really, really good. Because we don't decide if that tech takes off - the lawyers do. And they might not care, or they might decide billing more hours is better, actually.

When I graduated high school, I had never been or knew anyone who had ever been on the internet at all. The internet was this vague "information superhighway" that I didn't know really what to make of.

If you are of a certain age though you would think a pointless update to react was all the change ever coming.

That time is over and we are back to reality.

Or maybe they just know the nitty-gritty inherent limitations of technology better than you.

(inb4: "LLMs can't have limitations! Wait a few years and they will solve literally every possible problem!")

Which means it's back to being a very useful tool, but not the earth-shattering disruptor we hoped (or worried) it would be.

Recently I converted all the (Google Docs) documentation of a project to markdown files and added those to the workspace. It now indexes it with RAG and can easily find relevant bits of documentation, especially in agent mode.

It really stresses the importance of getting your documentation and processes in order as well as making sure the tasks at hand are well-specified. It soon might be the main thing that requires human input or action.

Wrote me:

- a SPI deserializer that sets a bit after 12 bits read in, to trigger a prefetch

- an SDC constraints file for the deserializer that correctly identified the SPI clock and bus clock as separate domains requiring their own statement

- a test bench that validated both that the prefetch bit was being set, and that it was being set at the proper time relative to the SPI clock

- a makefile with commands for build, headless test, and debug by loading the VCD into a waveform viewer

It always feels like LLMs can do lots of the easy stuff, but if they can't do everything you need the skilled engineer anyway, who'd knock the easy things out in a week anyway.

Nearly every part of the tool flow I just described, I would consider “tricky to get right”. Been doing this for ~15 years and it’s still tough to bootstrap something like this from scratch. ChatGPT-4o did this for me from zero in about 15 minutes.

I won’t lie: I love it. I can focus on the actual, bigger problems at hand, and not the tricky little details of HDLs.

People are either deluding themselves or ignorant of the capabilities of frontier models if they don’t believe LLMs offer a speedup in workflow.

I personally believe that most of the doubt and cynicism is due to:

1) a pretty big collective identity crisis among software professionals, and

2) a suspicion that LLMs make it so that anyone who is good at articulating the problem precisely no longer needs a software engineer as a translation specialist from specs to code.

I say this as an EE of ~15 years who’s always been able to articulate what I want, specifically, to a firmware counterpart, who then writes the code I need. I can turn years of practice in this skill into great prompts for an LLM, which effectively cuts out the middleman.

I really like it. It’s helped me take on a lot of projects that are just outside of my innate level of capability. It’s also helped me learn a lot of new things about these new software adjacent areas. ChatGPT is a great tutor!

Any skilled developer with a decade of experience can write prompts that return back precisely what we wanted almost every single time. I do it all day long. "Claude 4" rarely messes up.

Personally, I'm more interested in the political angle. I can see that AI will be disruptive because there's a ton of money and possibly other political outcomes depending on it doing exactly that.

An LLM is more like outsourcing to a consultancy. Results may vary.

Tools and systems which increase productivity famously always put everyone out of a job, which is why after a couple centuries of industrial revolution we're all unemployed.

"AI" is the latest iteration of snake oil that is foisted upon us by management. The problem is not "AI" per se, but the amount of of friction and productivity loss that comes with it.

Most of the productivity loss comes from being forced to engage with it and push back against that nonsense. One has to learn the hype language, debunk it, etc.

Why do you think IT has gotten better? Amazon had a better and faster website with far better search and products 20 years ago. No amount of "AI" will fix that.

Depends on the context. You have to keep in mind: it is not a goal of our society or economic system to provide you with a stable, rewarding job. In fact, the incentives are to take that away from you ASAP.

Before software engineers go celebrate this tech, they need realize they're going to end up like rust-belt factory workers the day after the plant closed. They're not special, and society won't be any kinder to them.

> ...and even wiser to be in charge of and operate the replacement.

You'll likely only get to do that if your boss doesn't know about it.

This "vibe coding" seems just another way to say that people spend more time refining the output of these tools over and over again that what they would normally code.

That was the way I saw it for a while. In recent months I've begun to wonder if I need to reevaluate that, because it's become clear to me that scaling doesn't actually start from zero. By zero I mean that I was naive enough to think that all programs, even the most googled programmed one by a completely new junior would at least have, some, efficiency... but some of these LLM services I get to work on today are so bad they didn't start at zero but at some negative number. It would have been less of an issue if our non-developer-developers didn't use Python (or at least used Python with ruff/pyrefly/whateveryoulike, but some of the things they write can't even scale to do minimal BI reporting.

That's because other people are making those working well. It's like how you don't care about how the bread is being made because you trust your baker (or the regulations). It's a chain of trust that is easily broken when LLMs are brought in.

In this example the idea that losing the muscles that make you great at hoeing" seems kind of like a silly thing to worry about

But I think there's a second order effect here. The kid gets a job driving the tractor instead. He spends his days seated instead of working. His lifestyle is more sedentary. He works just as many hours as before, and he makes about the same as he did before, so he doesn't really see much benefit from the increased productivity of the tractor.

However now he's gaining weight from being more sedentary, losing muscle from not moving his body, developing lower back problems from being seated all day, developing hearing loss from the noisy machinery. His quality of life is now lower, right?

Edit: Yes, there are also health problems from working hard moving dirt all day. You can overwork yourself, no question. It's hard on your body, being in the sun all day is bad for you.

I would argue it's still objectively a physically healthier lifestyle than driving a tractor for hours though.

Edit 2: my point is that I think after driving a tractor for a while, the kid would really struggle to go hoe by hand like he used to, if he ever needed to

You can let it self-drive, but you'd probably learn nothing, and it will actually take you longer. Put an expert driver behind the wheel, and they'll drive faster and only use automation features for the boring parts.

I think it is about how utilitarian the output is. For food no one cares how the sausage is made. For a painting the story behind it is more important than the picture itself. All of Picasso's paintings are famous because they were painted by Picasso. Picasso style painting by Bill? Suddenly it isn't museum worthy anymore.

No one cares about the story or people behind Word, they just want to edit documents. The Demo scene probably has a good shot at being on the side of art.

It eludes all of those who died in the process : those still alives are here despite bad IT, not due to a bad IT

Poker tests intelligence. So what gives? One interesting thing is that for whatever reason, poker performance isn't used a benchmark in the LLM showdown between big tech companies.

The models have definitely improved in the past few years. I'm skeptical that there's been a "break-through", and I'm growing more skeptical of the exponential growth theory. It looks to me like the big tech companies are just throwing huge compute and engineering budgets at the existing transformer tech, to improve benchmarks one by one.

I'm sure if Google allocated 10 engineers a dozen million dollars to improve Gemini's poker performance, it would increase. The idea before AGI and the exponential growth hypothesis is that you don't have to do that because the AI gets smarter in a general sense all on it's own.

We did. Why do you think labor laws, unions, etc. exist? Why do you think communism was appealing as an idea in the beginning to many? Whether the effects were good or bad or enough or not, that’s a different question. But these changes have demonstrably, grave consequences.

At least if looking at this specific portion: You can have a 20-30 file code base, with 5-10 classes in the context, in full, and the rest of them filtered in a sensible way. Even a model with a 200k context window can handle this.

The output definitely can stray, but it's not the norm in my experience. Of course, if the output does start to stray, it needs to be snipped in the bud. And the fixes can range anywhere from working but bad code, to very close how you'd written it yourself, if you've clearly described how you want the code to be written.

If you're trying to fix a specific bug, for example, but don't provide thorough logs on what is happening in the code, it's much more likely the output will stray towards some average of what the problem could be, rather than what it actually is in the current code.

And this is absolutely not to say that LLMs could do what Antirez is doing. There is a massive amount of variation in how deeply people think about the code they're reading or writing.

> you will almost certainly be replaced by an llm in the next few years

So... Maybe not. I agree that Hacker News does have a generally higher quality of contributors than many places on the internet, but it absolutely is not a universal for HNers. There are still quite a few posters here that have really bought into the hype for whatever reason

In fact, maybe most of has have been replaced by LLMs already :-)

Also, so many people said the same thing about chess when the first chess programs came out. "It will never beat an international master." Then, "it will never beat a grandmaster." And Kasparov said, "it would never beat me or Karpov."

Look where we are today. Can humanity adapt? Yes, probably. But that new world IMO is worse than it is today, rather lacking in dignity I'd say.

Stockfish NNUE was announced to be 80 ELO higher than the default. I don't find it frustrating. NNs excel at detecting patterns in a well defined search space.

Writing evaluation functions is tedious. It isn't a sign of NN intelligence.