My approach to building large technical projects
233 points by akalin 2 years ago | 27 comments- ZephyrBlu 2 years agoBeing able to prioritize building projects like this is a skill in itself, and I think it's a really important one.
People seem to chronically undervalue optimizing for shipping something. Mitchell does say "I don't talk about shipping", but shipping a demo for yourself is still shipping!
It doesn't matter how good you are if you can't stick at something for a long period of time. You're probably not going to be able to do the thing for long enough to generate big ROI.
Optimize for something usable. Go one step at a time. If you get discouraged or lose motivation it's game over, you gotta prevent that at all costs.
- lucasfcosta 2 years agoDefinitely true. That's something my co-founder talks a lot about.
If you just stick at it for a few months, projects get huge. It's incredible how much code gets written when you look 6-months back. If you had to look forward and try to guess, you'd never imagine this much would have happened.
- kqr 2 years agoWhat do you mean by not being able to guess, specifically? I seem to start growing code at 1000 lines per week, decreasing to 200 after a small number of months, yielding around 15000 lines after six months. For reference, this is about half the size of Wolfenstein 3-D. Is that not a fair guess?
- kqr 2 years ago
- lucasfcosta 2 years ago
- norir 2 years ago> My goal with the early sub-projects isn't to build a finished sub-component, it is to build a good enough sub-component so I can move on to the next thing on the path to a demo.
The one caveat I'd add to this is to make sure to panic (throw an exception or assert false) at all unhandled edge cases in the good enough subcomponent. It's often easy to write conditionals for the happy path without a corresponding else for the unhappy path. You don't need to implement all branches initially, but you will likely be glad you added those panics, which have the bonus side effect of showing you exactly where the unhandled edge cases are, as you expand the sub-component to handle those edge cases.
- bob1029 2 years agoNotImplementedException("reason") is my greatest ally when adventuring into the dark forest. I will sometimes build the entire prototype as a big NIE experience and then decide on which path(s) to implement.
Getting deep into details/code when you aren't quite sure of your surroundings can be extremely dangerous for the health of a fledgling project.
- bob1029 2 years ago
- thijser 2 years agoNice approach. It feels very similar to "tracer bullet" development (I think coined by the pragmatic programmer book), where you get something end-to-end as quickly as possible and then start iterating on the parts. (https://www.swaroopch.com/tracer-bullet-development/ explains it too)
- sharemywin 2 years agoThis video on youtube is pretty cool on the subject of big projects:
How to Achieve Your Most Ambitious Goals | Stephen Duneier | TEDxTucson https://www.youtube.com/watch?v=TQMbvJNRpLE
- 2 years ago
- yakshaving_jgt 2 years agoCool that he's still writing code. Not sure I would if I became a billionaire.
- blowski 2 years agoI would probably write more code as I wouldn’t need to sit in meetings! That said, I’d be pickier about who I wrote code for.
- Firmwarrior 2 years agoYeah dude, I would spend all day writing scripts to optimize my World of Warcraft character instead of fixing obscure OS bugs
I'd probably do it on a megayacht while cruising around aimlessly too..
- AnimalMuppet 2 years agoWho you wrote code for? If I were a billionaire, I'd be done writing code for other people. (I mean, users, sure. Not employers, though...)
- ZephyrBlu 2 years agoWorking in a company with other people can be way more fun than coding solo.
- ZephyrBlu 2 years ago
- Firmwarrior 2 years ago
- shard 2 years agoIn a previous job, the owner of the company would do all the fun parts of the engineering -- the brainstorming, the rough specs, the prototypes -- and leave the details and grunt work to everyone else. He was having a ball.
- sanderjd 2 years agoI think I'd write a lot more code, but it would be very different than the code I write now.
To me, this is a much more relatable reaction to being a billionaire than all the ones who become obsessed with politics and posting and grievances.
- blowski 2 years ago
- samsquire 2 years agoMy work is nowhere near as impressive as Terraform, Consul and Packer. Hashicorp products show insight and vision. They get things done. So take my thoughts as I haven't done something as impressive as that.
I generally work on small technical side projects that are achievable and do something interesting or useful. Then I start another one in a different subject or problem but also useful by itself with the desire of plugging/joining them together. Because they're all interrelated. They're technical demonstrations, not products.
I play in IPython Notebook or replit with no goal of shipping something then I when I have something satisfying I move it to its own repository on GitHub, then I add features increasingly until I am satisfied I understand what I intended. This is not product thinking, it's building to understand something that you think is interesting or useful.
A few months ago on HN I wrote a comment talking about my dream to create a JIT Compiler - similar but much less serious than V8, JVM or the CLR. I also talked about a multithreaded runtime that can schedule lightweight threads similar to Golang or Erlang. I want to understand asynchrony, parallelism, coroutines, multithreading and locks enough to abstract them away from developers. My JIT compiler is incomplete: it can JIT compile MOV and ADD instructions and printf without parameter passing being done but not much else. When I feel inspired to work on the next piece of work on it, I'll keep working on it. At the moment I'm more interested in parallelism and asynchrony runtimes so my attention is there.
I've noticed that there is a task that we put off with the desire to come up with an elegant solution so we don't have to do it. Such as a refactoring or yak shaving to support something we need to support to do what we want to do.
I've been learning about delimited continuation passing and algebraic affects.
With some of the core counts on AWS you could process many requests in parallel and yield on IO so you could get some extremely impressive requests rates were the software well designed. I think 600,000 requests per second is top of the line for a single c++ server on the fortune TechEmpower benchmarks.
I want to make my own things and understand them well. I also want to learn other project's codebases and understand them well enough to work and think in them.
If you have a side project, I encourage you to work on it. I am always interested in people's writeups of their side projects.
There is so much work to do, it's not just 1 thing you need to learn and understand and think in (the programming language) there's a 1000 other tasks that need to be done to get anything shipped or useful for other people to use generally. The "getting past the recognising you have 1000 tasks to do for the first time" is really difficult in just working and forging ahead and getting stuff done. If only I could tackle all those 1000 tasks the same way I do the things I am good at, rather than not experienced or effective at.
- throwawaaarrgh 2 years agoNote he doesn't mention a user story. Large technical projects apparently let you ignore what the user wants, use cases, dependencies, stakeholders, design reviews, feature changes. Instead you just decide what you want and wait to see if the user wants it too.
- jt2190 2 years agoWhat users want and User stories (in the agile iterative software sense) are different things. At their best user stories represent problems to solve for the end user, but rarely have I been able trace back a user story to something that an end user actually wanted because nobody bothered to ask them.
Not talking to customers is extremely common. Everyone’s just too busy, including the customer. It’s kind of the “eat healthy and exercise” of business: Everyone knows they should do it, yet few actually do.
- arcbyte 2 years ago> At their best user stories represent problems to solve for the end user, but rarely have I been able trace back a user story to something that an end user actually wanted because nobody bothered to ask them.
This is such a universal problem in the industry. It's also the source of the endless reinvention cycles we go through.
Everyone hates user stories now because no one actually does the other work involved - talking to the users to get the stories in the first place. So it becomes an unproductive chore and people grow to hate it. Someday, someone will discover talking to the user and they'll have their own favorite little way of writing down the conversation and they'll come up with their own title for it. It will start getting hyped and eventually we'll all be doing it - except we still won't be talking to the users and the next generation will hate it. And it will repeat.
It's the same with User Stories, OOP, and every other concept in computer science since the 50s.
- kubanczyk 2 years ago> It will start getting hyped and eventually we'll all be doing it - except we still won't be talking to the users and the next generation will hate it.
By the way, user stories are an iteration of that. I remember:
- when user stories were a novelty, everyone hated actors and use cases,
- when actors and use cases were a novelty, everyone hated functional requirements.
- kubanczyk 2 years ago
- xoac 2 years agoIt's not like the users know what they really want though. You talk to a bunch of Salesforce users and they'll tell you that what they want is Salesforce+. This may not be what you want to be building as your product.
- fishnchips 2 years agoBase on my experience (product leadership role) when talking to customers it's most important to focus on their problems and challenges. They will obviously suggest solutions but you will want to be super careful with following these - think your Salesforce+ or the famous "faster horse".
- fishnchips 2 years ago
- arcbyte 2 years ago
- n4r9 2 years agoMy impression is that the article's advice is more fundamental than all of that (e.g. it could just as easily apply to personal passion projects), yet can easily be integrated with it (e.g. iterative work->demo->feedback phases).
- ZephyrBlu 2 years agoHe mentions something far better: getting the product into the hands of users ASAP so you can iterate on it.
- yardshop 2 years agoTwo thoughts, one, his approach is orthogonal to that. This is a way to move yourself forward on any project, no matter where the design comes from.
Two, he IS the user! His example is the terminal emulator he is writing for himself. He says to write something that you will use yourself, as a good way to generate your own interest and involvement in it. Also, it seems like good practice to write something for the user you probably have the best communication with!
I used some of these ideas to get over the hump on a project today. Just get the simplest thing going to give a useful result, and that helps break the log jam!
- bsnnkv 2 years agoDescribes basically all of my large technical personal projects. Definitely the happiest I have ever been writing code and building things.
- readthenotes1 2 years agoWell, he write stuff for developers so he might know the space well enough to start his work,
unlike the recent story of a developer trying to disrupt coffee delivery
- sanderjd 2 years agoSounds nice!
- jt2190 2 years ago