A software engineering manager guide to measuring an engineer’s performance

114 points by jonaldomo 5 years ago | 108 comments
  • weliketocode 5 years ago
    > It is not realistic to give an early software engineer defects within the first few months of a project assignment. So this skill area might not apply until a little later in their career.

    what?

    Please don't listen to this. Give junior engineers bugs right away. It forces them to set up their environment for debugging and begin to understand the flow of the project.

    Even if he/she needs to be guided to the solution, this is a GREAT litmus test for the standards of your documentation, and ease of environment setup.

    • lnsru 5 years ago
      This!!! Nobody gained any skill from writing vanilla code or “hello world” type programs. Experience comes from spending days and sometimes weeks on ugly bugs. Bugs are perfect to level-up juniors quickly and find ones with weak motivation. Because having no progress for hours is really really frustrating. Went recently through this cycle teaching colleague. Found out, there is no motivation at all.
      • bitL 5 years ago
        Some people thrive when bugs are assigned to them early, some people thrive when whole new functionalities are assigned to them. Usually intersection of sets of these two kinds of people is close to empty and the worst mistake is to assume bugs first are good or new stuff first is good automatically.
        • 0xEFF 5 years ago
          When you give someone new functionality to implement and they don’t have experience fixing defects, they’ll likely add more defects along with the new functionality.
          • TimTheTinker 5 years ago
            Our team always assigns a few cosmetic bugs to new developers during their first week - only because it helps them learn their way around our codebase.

            Where they proceed from there depends on their strengths and interests.

            • 5 years ago
              • wolco 5 years ago
                I like both as long as they as smaller items. Bugs will help you to understand what is there and new functionality can be a good bridge where a new person can leverage existing experience.
              • z3t4 5 years ago
                I was very fortunately that my skill level followed the complexity of problems, only several years into my career I had to deal with systems level complexity and scaling issues, working as a high level web developer. Some things are just magic for a beginner. As a manager you should make people feel confortable, and some people do not have the mindset for constant self improvement. And some people you need to protect or they would over-work themselves. The trick is to have a load/capacity balance. Take games for example you usually progress the difficulty, you do not start with the end boss, or most players would quit, and those that pass would find the rest of the game boring.
              • gregmac 5 years ago
                Exactly. Fixing bugs is one of the best ways to learn a new codebase. Assuming you have a process for code reviews and some form of testing in place, it's not like there's much risk of a bad change getting out.

                I'd actually lean the other way: No one should be adding new features to an existing codebase without having successfully done some fixes first.

                • fatnoah 5 years ago
                  >Please don't listen to this. Give junior engineers bugs right away.

                  Right?!? That's how I get every junior developer into the code. Hell, that's how I get myself into the code at a new job.

                  • lunias 5 years ago
                    Agreed, this makes no sense to me (sounds like something an "engineering manager" might recommend). If not defects, then what? Surely it's not the recommendation that someone who struggles to fix bugs should be given greenfield work...
                    • JamesBarney 5 years ago
                      Definitely this. Bugs are great for new devs. Nothing teaches why code quality matters and what types of code quality matter like investigating bugs and why they happen.

                      Also jr. devs require lots of sr. dev time to verify their work. But bug fixes are usually easier to verify then features because the ratio of code written to investigation is much smaller.

                      • yalogin 5 years ago
                        Yes. Agree completely. Giving new hires bugs is the best way to bring them up to speed. They get to delve into the whole software stack, interact with people, understand data flows, familiarize themselves with code names if any, and generally get an appreciation and understanding of the product. This is my go to move for new hires.
                        • dfeojm-zlib 5 years ago
                          Heck, you should try to give interviewees real bugs so they're solving real problems (not to exploit them but to see how they think and work) instead of theoretical chalkboard problems or standard brainteasters.

                          Certainly, guide them towards easy-ish bugs rather than complicated ones right away, because there is something to risk management of new people attacking gradually-increasingly critical code / difficult bugs. Of course you probably wouldn't give a random interviewee root access to a credit card processing system, but you should ratchet-up the level of trust incrementally as fast and as frequent as a staff member has proven themselves (treat people as adults, not kids).

                          • lmilcin 5 years ago
                            Debugging stuff is still my favorite task when I join a new team. This is absolutely the best way to quickly learn about the project.

                            It is also important to select right kinds of bugs for newcomers.

                            • nilkn 5 years ago
                              I couldn't imagine having a new developer go several months without looking at a single bug. Sure, I'm not going to give them something that I know is going to be really difficult and frustrating, but I generally try to get people in the swing of development within the first week.
                              • NicoJuicy 5 years ago
                                Yeah, our software manager gives everyone new new projects, that of course are never completed.

                                Lol, some "managers"

                                • jammygit 5 years ago
                                  When I was an intern I spent most of the year fixing bugs, between other assignments. It was the most beneficial educational experience I could have had. I got really good at it by the end too.

                                  There isn’t a better way to learn a codebase and new tech than one issue chosen by you at a time

                                  • techslave 5 years ago
                                    I think this comment misses the point by being myopic.

                                    The great majority of devs are not very good. (Same for managers.) Here at HN you are only seeing the pretty good to great devs. Not boot camp grads and unranked school grads.

                                    The other thing is that this comes from the management school of management. One where managers mentor ICs and managers drive skill growth. As opposed to the SV ideal (rarely met) where mgmt gets out of the way. They remove blockers, not give opportunities. They communicate information, not hoard it.

                                    So, from the management school of management style, managing lackluster devs (the 90% that you reject at phone screen), it’s true. These devs get in over their head when the first thing they have to do is actually understand the spaghetti codebase. This is not a road to success for these devs.

                                    With that POV and that reality in mind, the advice is sound.

                                    • closeparen 5 years ago
                                      Debugging requires an extremely thorough understanding of the project, so much that you can not only find each responsibility in the code, but anticipate their emergent properties, what mistakes might have been made, and how their interactions might go wrong. Debugging a system successfully is the culmination of successful on-boarding, not a starting point.

                                      Doing the actual code changes to fix a bug which has already been analyzed and localized to one or a few sites in the codebase is a bit more reasonable.

                                      • jolmg 5 years ago
                                        > Debugging requires an extremely thorough understanding of the project

                                        That's precisely why it's great for a starting point. It's work that mostly consists of research of the current code, to look for inconsistencies that cause a well-defined misbehavior. It involves writing little code and therefore involves little decision-making if any, so you avoid the newcomer making changes or adding stuff that, in ways, may be inconsistent with how things are done in the project.

                                        • commandlinefan 5 years ago
                                          > an extremely thorough understanding of the project

                                          Right - so no time like the present to get started understanding!

                                          • biehl 5 years ago
                                            Right. But sometimes this clashes with new developers expectation of actually being able to produce an acceptable solution and getting it merged within a reasonable amount of time.

                                            Have those expectations never clashed for you?

                                            • closeparen 5 years ago
                                              You can get started by doing smaller and better-defined projects first, rather than going for ~weeks before knowing whether you have any traction.
                                          • 5 years ago
                                            • e40 5 years ago
                                              Agreed. We do this and it has worked out amazingly well.
                                            • guitarbill 5 years ago
                                              > The Lake Wobegon Strategy famously coined by Google and Peter Norvig claims that you should always hire above your team average. Doing so increases the quality of your team.

                                              Ugh, not this again. Obviously don't hire people who aren't good at their job. But most improvements come from investing in your team.

                                              > are they putting their code up for review [...]

                                              Missed one: Are their code reviews/pull requests high quality? I.e. do they go out of their way to document how they tested it? Reproduction steps? Do they invest time in making code reviews as easy to review for other people as possible? Or does their code reviews always take multiple rounds of review due to sloppiness?

                                              • Consultant32452 5 years ago
                                                Investing in workers is hard. I'm in a mentoring/leadership position. I approach this in two ways. One is the general mentoring everyone gets during regular stuff like code reviews. The other way is sometimes I see someone who I think has real promise and kind of take them under my wing, give them lots of 1on1 attention, etc. They improve exponentially and then leave. One guy I mentored had been with the company for over 10 years. No one had ever "invested" in him. He had a great attitude and work ethic, but terrible skills. I taught him how to program. He left the company and doubled his salary. That was a year and a half ago. This week he called me to thank me because he was changing jobs again and doubling his salary again.

                                                So my current employer went from a low skill high dedication worker to no worker because I invested in him. I'm going to continue help/mentoring people because I find it fulfilling, but if I'm honest it's bad for my employer. They currently pay good salaries in line with the market. There's no way they can justify giving a person who's been with the company for 10 years 4x salary growth in 2 years. How could they even reasonably measure the market value of his skills changing so much so quickly?

                                                • jimbokun 5 years ago
                                                  > They currently pay good salaries in line with the market.

                                                  > There's no way they can justify giving a person who's been with the company for 10 years 4x salary growth in 2 years.

                                                  One of these must be false. According to "the market", this employee is now worth 4x previous salary, so then how can your company be paying "in line with the market?"

                                                  • Consultant32452 5 years ago
                                                    Ok, so it's more like the business is in the market for a $80k/yr developer, and not a $300k+ developer. The words changed but the result is the same. The work they're doing, it honestly doesn't make sense to pay anyone that much.
                                                  • indigochill 5 years ago
                                                    Maybe it's not so bad for the employer in the long term if it leads them to realize "Hey, we need to do better about assessing the value of our employees."

                                                    I'm curious what the data point is that leads you to think your company's paying a good salary if the hard-working, motivated, skilled people can easily find work that pays quadruple, because that sounds like the market is saying otherwise (unless the missing variable is location, like moving from Nebraska to Silicon Valley or something).

                                                    • yowlingcat 5 years ago
                                                      It's not bad for your employer. It's the truth. Your employer is bad for your employer if they let someone remain at the company for 10 years, paying salary without investing in said employee nor treating them like an investment.
                                                      • stcredzero 5 years ago
                                                        How could they even reasonably measure the market value of his skills changing so much so quickly?

                                                        Let the market do it. When someone comes with an offer, give a counteroffer and convince them to stay.

                                                        There's no way they can justify giving a person who's been with the company for 10 years 4x salary growth in 2 years.

                                                        Why not? Markets are better at pricing things than bureaucracies.

                                                        • jvagner 5 years ago
                                                          But this isn't usually the norm.

                                                          If this dev's trajectory is true, it's also something worth discussing with your management. There might be candidates for this kind of trajectory in your current dev pool, and any feelings contrariwise are to the detriment of your company.

                                                          Too often there's an emotional component to how employees are viewed. The employee today will be the same employee forever, if seen through the lens of a limited manager.

                                                          I'll say this: if you can do this 2-3 times, you should find your way to an org that can support and appreciate the lift you're able to provide. You shouldn't be holding yourself down to the poor standard of your current employer.

                                                          • Consultant32452 5 years ago
                                                            Honest question: are there places where you can be a mid-level developer making $80k/yr and over the course of two years get raises and promotions lifting you up to $300k+? I've never heard of such a thing happening. In my experience, industry-wide, the only way to make that kind of raise is to leave.
                                                          • Paul-ish 5 years ago
                                                            > There's no way they can justify giving a person who's been with the company for 10 years 4x salary growth in 2 years. How could they even reasonably measure the market value of his skills changing so much so quickly?

                                                            If he was doing the same job I could see why he would get the same pay. If he had taken on greater responsibilities or become more productive though, why not?

                                                          • srikbs8095 5 years ago
                                                            "Most improvements come from investing in your team"

                                                            Very well said. Completely agree. Which is why I think the ability to "learn" as you go is so valuable. You don't want to hire a Know-it-all. You want to hire a Can-learn-it-all.

                                                            • tracer4201 5 years ago
                                                              > “But most improvements come from investing in your team.”

                                                              I agree on this but you have to cut your losses at some point. Some people just aren’t motivated. A surprise pay raise, catering lunches, taking them to an offsite, etc whatever motivational strategies you have may provide some short term result, and that could be sufficient if your goal is just X and it’s in the line of sight, but sometimes raising the bar on your team means finding a different spot for the unmotivated folks, and if they’re not investing even in that, then cut your losses and let them go.

                                                              • jimbokun 5 years ago
                                                                > A surprise pay raise, catering lunches, taking them to an offsite

                                                                I don't think any of those things will increase team productivity at all.

                                                                Mentoring and training and encouraging learning new things and constantly giving new challenges and expanding responsibility and tying company financial success directly to employee paychecks, are the kind of investments that can improve a team's performance.

                                                                • rifung 5 years ago
                                                                  > Some people just aren’t motivated.

                                                                  Is it really that people aren't motivated or that they aren't motivated by the incentives you're offering?

                                                                  > A surprise pay raise, catering lunches, taking them to an offsite, etc whatever motivational strategies you have may provide some short term result

                                                                  I imagine what might be more effective is actually talking to them and figuring out what they actually care about.

                                                                  • mieseratte 5 years ago
                                                                    Absolutely agree. Some people are just beyond saving. Sometimes this a matter of skill, and sometimes a matter of will but the result is the same. If you're hiring pipe-hitters and grafting them onto a JV squad the results will often not be what you'd hoped for, unless you have a particularly high-tolerance for projects being over (time-)budget.

                                                                    Having personally burned myself down in such an environment, a good interview question to ask is if everyone has gone through the same hiring process and if not ask what the previous process was and why things have changed.

                                                                  • kevan 5 years ago
                                                                    You could reasonably argue that the highest impact investments a manager can make in a team are hiring better people and firing worse ones.
                                                                    • jimbokun 5 years ago
                                                                      Why do you say that?

                                                                      Hiring people is very expensive. Finding and qualifying leads, creating job description, interviewing, on boarding, orientation, training, introduction to and getting familiar with the existing code base. Firing requires distributing that team member's work to other team members.

                                                                      This can take months and a big investment from existing team members, to go from wanting to hire someone to a fully productive team member. And that's assuming your hiring process never accidentally hires someone even more incompetent than your "poor" team members.

                                                                      Compare that to investing in improving existing team members to help them improve. Is that really more expensive than hiring and firing?

                                                                      • ses1984 5 years ago
                                                                        Attrition is a thing. You have to always be hiring, or very ready to hire, anyway.

                                                                        Also accidentally hiring the wrong people can be devastating. It's better to identify and get rid of bad hires right away.

                                                                      • commandlinefan 5 years ago
                                                                        > hiring better people

                                                                        You say that like it's some sort of epiphany, but do you honestly believe that there's anybody who wasn't trying to hire better people in the first place? If the problem is the quality of the people you've been hiring, maybe your boss should consider replacing you with somebody who's better at finding better people to hire, eh?

                                                                        • brianpgordon 5 years ago
                                                                          I've been on many interview debriefs where the discussion came down to a question of our team's appetite for taking on a more relatively junior hire vs holding out for a more solid candidate. Some "weaker" hires can be OK, but in order to be fair to them (and the team) you have to make sure you have the resources to develop them by giving them the opportunity to work with more experienced people and get one-on-one mentoring. So yes, the extent that you want to "hire better people" is very much an intentional judgment, and you can calibrate your hiring process accordingly based on your current team composition and the work on your plate.
                                                                          • wolco 5 years ago
                                                                            I wonder if that happens. Some hiring processes are flawed so they filter out the better candidates at the HR levels and the engineers filter out the candidates who are left over. This can go on for a long time until the company is so understaffed they hire anyone and it works out..
                                                                          • yowlingcat 5 years ago
                                                                            Careful. How did the team get there in the first place? Why would better people want to join if the manager can't coach an existing team? You can argue that it doesn't matter and you just need to clean house. But that runs the risk of cover up the root cause and not addressing it.
                                                                            • JamesBarney 5 years ago
                                                                              Not having a team because you can't staff it, and having terrible moral because of frequent firings also impact team performance.
                                                                              • uwuhn 5 years ago
                                                                                Hiring better people doesn't mean much if you can't get them to stick around for more than a year.
                                                                            • perlgeek 5 years ago
                                                                              > Code reviews are probably the first thing a new engineer can start doing in a new role.

                                                                              That really depends on what you want the code review to achieve.

                                                                              Catch typos? Yes, a new engineer can do that.

                                                                              Check if code fits into the existing architecture, adheres to the invariants of the code base, uses base libraries idiomatically etc? I don't think a new engineer can contribute that from the start.

                                                                              > Creating metrics through issue trackers and time sheets

                                                                              Which metrics? It's far too easy to create metrics that are easy to measure, rather than metrics which actually increase the business when optimized for (and developers will optimize for / game a metric when it's used to assess their performance).

                                                                              • shados 5 years ago
                                                                                Code reviews have 2 purpose. Help the person whom's code is being reviewed improve quality/catch issues, BUT ALSO the reviewer gets familiar with the code being pushed and learn stuff. It's super important for new engineers to review code as soon as possible for the later.
                                                                                • jolmg 5 years ago
                                                                                  > BUT ALSO the reviewer gets familiar with the code being pushed and learn stuff.

                                                                                  The reviewer is supposed to determine what goes in or not, so how can this be an opportunity to learn from what goes in? They're the one who's supposed to determine that! Is your idea of a reviewer someone who just spectates all code going through? It's like saying people who don't know a subject should grade work from students taking a class in that subject, as it's a great opportunity to learn from what gets turned in.

                                                                                  • BeetleB 5 years ago
                                                                                    >The reviewer is supposed to determine what goes in or not

                                                                                    This is not a given, and in fact, I would not recommend it.

                                                                                    For small code reviews with only one reviewer, the reviewer gives feedback and they have a conversation about it, but with only one reviewer the developer should have the final say. The problem with giving the final say to a single reviewer is you'll get plenty of pointless code changes to suit the reviewer's individual preferences. Don't waste time on what is a perfectly reasonable difference in opinion.

                                                                                    If you have more than one reviewer, but still a small code review, the final say should be with the reviewer(s) - if their is consensus (you could also go with voting, with its pros and cons). Having multiple reviewers prevents the likelihood of individual preferences dominating.

                                                                                    If it's a significant code review, the final say should be with an experienced moderator.

                                                                                    • tkxxx7 5 years ago
                                                                                      > The reviewer is supposed to determine what goes in or not, so how can this be an opportunity to learn from what goes in?

                                                                                      You specify multiple reviewers. That's it.

                                                                                  • vinceguidry 5 years ago
                                                                                    Maybe have a senior review the code afterward so the new guy can see what's being expected?
                                                                                    • jolmg 5 years ago
                                                                                      So the new guy is just watching the reviewer? That sounds neither productive, nor an efficient way to get familiar with the project.
                                                                                      • rleigh 5 years ago
                                                                                        That's not what they said.

                                                                                        You learn by doing, and also by example. So they can do the review themselves, but also see what other people pick up on. It sets expectations, and also shows how other people think.

                                                                                        I always find it interesting when I do the same review as one of my co-workers. We might pick up on the same things, but often they are completely different. It shows the value of having multiple perspectives on a team. Not just for review, but for design, implementation, validation, debugging, and the rest. It widens people's perspectives, serves as a means for ongoing improvement and learning, and makes the team better as a whole. We all have different backgrounds and expertise, and having people share and pass on some of their knowledge and skills is highly valuable.

                                                                                    • ashishuthama 5 years ago
                                                                                      A new engineer can learn a lot from being on code reviews earlyon. Anything from the team coding style to the application architecture.
                                                                                      • jolmg 5 years ago
                                                                                        A reviewer is not there to learn. If anything, they're there to teach. What good is a reviewer that can't grade others on their adherence to the coding style and application architecture of the project? They're supposed to determine what goes in or not.
                                                                                        • wolco 5 years ago
                                                                                          They wouldn't be the only one reviewing or reviewing at all. It would help them understand the process if they see it in action.
                                                                                          • stevens32 5 years ago
                                                                                            Reviewers not willing to learn anything during a code review is probably an easy tell for a crappy code review culture
                                                                                      • Dayshine 5 years ago
                                                                                        "Process improvements" is a bit weird. You measure software engineering performance by their dev ops skills?

                                                                                        It's a completely different skillset, and you're probably only measuring how eager people are to have breadth of knowledge or how familiar they are with your specific system.

                                                                                        Similarly, the explanation for "Debugging and troubleshooting complicated issues" is a bit odd. Why is knowing where the log files are, and knowing how to do complicated test setups for your specific environment a performance measure. Again, that's not really measuring skill but familiarity.

                                                                                        The other two measures are just "Do they complete tasks" and "Do they follow code review guidelines", neither of which are very good measures beyond pass/fail.

                                                                                        The conclusion is: > Once a set of skill areas for a role is landed and agreed upon you will want to make sure your team knows in advance what they are being measured on

                                                                                        So, to do well in your business, I need to pick easy tasks, snipe code reviews, make pointless CI tasks and spend all my time learning the build/test processes not actually developing the product? :)

                                                                                        • sanderjd 5 years ago
                                                                                          Your comment seems to fall into a common fallacy, that "developing the product" is entirely done by writing the code. That isn't true, there is no product if it is not built, tested, deployed, and debugged. Lots of programmers consider this "pointless" grunge work, but there is a reason it tends to be picked up by the more senior engineers on the team. This sort of work has more foundational impact than just writing feature code; it benefits all features written in the future.
                                                                                          • Dayshine 5 years ago
                                                                                            No, of course other things are important.

                                                                                            But if you measure only ancillary things, that's a pretty bad measure.

                                                                                            I don't expect every one of my team members to understand the entire build system, testing setup, logging system. That's a waste of their time. They should know some, and perhaps one of the areas in depth.

                                                                                            • sanderjd 5 years ago
                                                                                              My point is: Those things aren't ancillary. They aren't a waste of time. Good developers can figure out how to use logs to debug issues. They can figure out how to fix the build and deployment system when it's broken. They can figure out how to set up testing environments. If not them, then who? Managers look for people who are self sufficient. All of this stuff is a necessary and important part of the job.
                                                                                          • eropple 5 years ago
                                                                                            > It's a completely different skillset, and you're probably only measuring how eager people are to have breadth of knowledge

                                                                                            In my experience, this skillet being measured is the critical indicator for high-quality development talent at a healthy shop. You might need one deep-magic wizard for a particular area. You need breadth everywhere. You need to kill "that's not my job" stone dead when you see it.

                                                                                            Writing code is easy. And it's usually an additive process, even when the code is subtractive. All the other stuff, the being-a-person-in-a-community stuff (because that's really what it is), is multiplicative. So I pretty unashamedly hire for curiosity and for breadth. When it's necessary, I contract out hard expertise until curious people can develop it internally.

                                                                                            • jimbokun 5 years ago
                                                                                              > You measure software engineering performance by their dev ops skills?

                                                                                              Absolutely! That is the "dev" in dev ops.

                                                                                              I think there are few things that increases the productivity of a dev team more than improved processes.

                                                                                              > Why is knowing where the log files are, and knowing how to do complicated test setups for your specific environment a performance measure.

                                                                                              Because if you don't know those things you are much less helpful to your product team.

                                                                                              • AstralStorm 5 years ago
                                                                                                You'd find a cushy job in a few corporates by doing just that.

                                                                                                Someone else then gets to do hard tasks and gets bad reviews because they're slower at this and the numbers do not tick up.

                                                                                              • januzis 5 years ago
                                                                                                It seems to me there are two sides to engineer's performance: the ability and the productivity. Ability measures how complex tasks can an engineer solve and how well can he/she execute, and the productivity measures the actual amount of work done. Able programmer is not necessarily productive, and productive programmer might not be able to do tasks of high complexity.

                                                                                                As a technical lead, I feel that I'm able to judge the ability of individual team members, but I'm having a hard time objectively judging productivity. Simple count of PRs doesn't really tell the whole story, and some tasks look simple in hindsight, when in reality it took a lot of effort to find a good solution. There are also a lot of other complications I'm not going to dive into, but the end result is that it's hard to have an objective productivity evaluation based on the engineer's output only.

                                                                                                I'd be interested to know how other people evaluate individual productivity?

                                                                                                • jrumbut 5 years ago
                                                                                                  People say that no metric works but I think almost any metric works, for example PR count.

                                                                                                  You, a human being, would never actually confuse the engineers with 0 PRs because they spent the last month playing online poker with the engineers with 0 PRs because you entrusted them with developing an automated deployment system for a legacy application and starting a mentorship program.

                                                                                                  Many metrics will spot the outliers. But what is the business value of knowing when an engineer rated 82.13 vs 84.51?

                                                                                                  • sam0x17 5 years ago
                                                                                                    Yeah but what about the highly skilled employee who spends 90% of their time playing online poker and spends 10% of their time producing output on-par with other people in the org that work 100% of the time. There is almost always one of these in any given org.
                                                                                                    • januzis 5 years ago
                                                                                                      I would actually be OK with it, as long as I can objectively say that even though someone is slacking most of the time, he/she is very productive in short bursts, so the overall productivity is on par with the team average.

                                                                                                      The problem is that if you don't have an objective approach to evaluating productivity, all sorts of biases come in, e.g. if I see someone coming to the office at 11am and leaving at 4pm, I would subjectively rate their productivity lower, even though objectively the results might be the same as for someone who comes in early, and leaves late.

                                                                                                      • Smithalicious 5 years ago
                                                                                                        Well are they getting paid 10 times as much as the others?
                                                                                                      • januzis 5 years ago
                                                                                                        Yes, the extreme cases are easy to spot, but the nominal cases are not that minuscule, more like one developer being twice more productive than the other, which, I feel, should be recognized and rewarded accordingly.
                                                                                                      • JamesBarney 5 years ago
                                                                                                        This is especially tough because one important development skill is the ability to solve complex problems with simple solutions. This can sometimes make them look less productive because they've made a hard problem look easy.

                                                                                                        One technique I've found that helps a little is having everyone individually estimate tasks before hand. Over time you can notice who is completing tasks faster or slower than the average. Of course this is just one data point that needs to be weighed against a variety of others.

                                                                                                        • januzis 5 years ago
                                                                                                          Measuring against estimates incentivizes over-estimating, especially if the team is small, and team members know roughly who will do what during the estimation process. Big complicated tasks also tend to be under-estimated, so they would be avoided, because they would usually influence the perceived productivity negatively.

                                                                                                          I would like to find a feasible approach to productivity evaluation using the output only (PRs, reviews, basically all the data points the article mentions), because I feel that’s the only way of creating an environment, where team members can proactively take a bit more time when it benefits the end result, without fearing any negative repercussions.

                                                                                                          • JamesBarney 5 years ago
                                                                                                            I don't think developers spend that much time gaming metrics unless they are objectively measured against them.

                                                                                                            And you would notice if someone's estimates are always higher for the work they are assigned compared to others.

                                                                                                        • yowlingcat 5 years ago
                                                                                                          You talk to them and the people they work with or whose work depends on them. How quickly, autonomously and completely do problems get solved?

                                                                                                          Metrics are a distraction unless in the service of answering that question.

                                                                                                          • januzis 5 years ago
                                                                                                            That’s the default, I guess, my main concern is that it’s very subjective.
                                                                                                            • yowlingcat 5 years ago
                                                                                                              Sure -- some parts are, but some parts aren't. It's intrinsic to the nature of working in a team or coaching a team. If you notice weak spots consistently, is it a fluke? Probably not. If you notice complaints about the ability of the team, division and org to collaborately smoothly, is that a fluke? Again, probably not. By the time you fine-tune all the weak spots properly, you have a very high functioning team. But it takes a lot of iteration.
                                                                                                        • caymanjim 5 years ago
                                                                                                          > Be a better management with transparent performance reviews and quick feedback with on focus areas.

                                                                                                          This article contains dozens of grammatical errors (starting right off with the title). Whenever I read something like this, I'm so distracted by the errors that I can't even focus on what the author is trying to say. The English language is dying.

                                                                                                          • jolmg 5 years ago
                                                                                                            It's not just English. :(
                                                                                                            • 5 years ago
                                                                                                            • techslave 5 years ago
                                                                                                              i took it to be ESL author and gave some slack in that regard.
                                                                                                            • sytelus 5 years ago
                                                                                                              > Ability to write source code that adheres to specifications

                                                                                                              This article reads like a series of bad advice from 1990s.

                                                                                                              - No, engineers shouldn't be writing code that "adheres" to specifications. They should study, understand, question and contribute to problem statement and approach at all times (also called "requirements" in pre-2000s).

                                                                                                              - Manager shouldn't be at center of evalauting performance but rather establishing process, standards and collecting feedback and metrics.

                                                                                                              - Bonuses are inherently evil and would always motivate individuals to exploit short term gains at the expense of long term sustainibility. Any performance evaluation strategy must keep this issue front and center at all times.

                                                                                                              - Large part of performance feedback shouldn't come from managers but peers

                                                                                                              - Performance reviews should never be entirely metrics-driven. No finite set of metrics tell the full story and all metrics are susceptible at gaming.

                                                                                                              - Don't treat new comers as incapable of fixing bugs or do X but not Y. Don't create class system of seniors vs juniors. Titles cause more troubles then they are worth.

                                                                                                              • jimbokun 5 years ago
                                                                                                                I think there's only one metric that is really effective for measuring and evaluating software teams.

                                                                                                                Tie their compensation to the product's financial success.

                                                                                                                This means, to the greatest extent possible, everything relevant to the product's success must be owned by the team and become their responsibility. Maybe there are some cross cutting concerns that should be the responsibility of a group separate from any product team, but those should be rare and require a strong justification.

                                                                                                                This will have an amazing effect of clarifying prioritization of what to work on and figuring out how to deliver it as quickly and reliably as possible. Suddenly the whole team will be in the loop about what features are most important to the customers. Suddenly the things blocking new features demanded by the customers from shipping will be cleared away.

                                                                                                                I think for any other metric you can devise, either intentionally or unintentionally, employee behavior will be optimized for satisfying the metric, and not customer satisfaction with the product.

                                                                                                                • Rooster61 5 years ago
                                                                                                                  > Tie their compensation to the product's financial success.

                                                                                                                  This is an atrocious idea in anything much larger than a start-up. It leaves the engineering team beholden to bad decisions made in other facets of the company (sales, marketing, upper management). It's exceedingly frustrating to write a solid, stable, performant program only to have marketing or sales push it as something that it is not and nosedive the company due to customers calling bullshit.

                                                                                                                  Start-ups are somewhat immune to this because of the lower barrier of communication between departments, as many people will be wearing multiple hats due to there being more things to do than people to do them.

                                                                                                                  This is very visible when executives begin choking off benefits across the board. Nothing kills morale quite like losing your bonus because another department isn't doing its job properly. I've seen more than one mass exodus from a company as a result of this.

                                                                                                                  • jimbokun 5 years ago
                                                                                                                    > It leaves the engineering team beholden to bad decisions made in other facets of the company (sales, marketing, upper management).

                                                                                                                    I mean, you're screwed anyways if those people can't do their jobs. No sales means no revenue means engineers taking a pay cut or getting laid off.

                                                                                                                    So the sales and marketing and product management responsible for your project, need to be on the same product team as the engineers, and have their compensation tied to the product's success.

                                                                                                                    If the engineers see themselves as the natural enemies of sales and marketing, the product is probably already doomed.

                                                                                                                  • sytelus 5 years ago
                                                                                                                    Very bad idea. This almost always incentivizes to exploit for short term financial results at the expense of long term sustainability. For example, you can cut down customer support department to save money which would increase your profit (and therefore bonus for an employee who made that decision). In longterm you probably won't have customers. Executive pays often follow this rule and you can find vast number of stories how they ruined companies for short term gain and bonus cashouts.
                                                                                                                    • jimbokun 5 years ago
                                                                                                                      OK, but if you substitute "product's long term financial success" for "product's financial success", I think the point still stands.

                                                                                                                      Could add in factors for customer retention and customer satisfaction, for example.

                                                                                                                      Everyone on the team should share responsibility for figuring out what will make the customer's happy with the product and how to deliver on those things.

                                                                                                                      • sytelus 5 years ago
                                                                                                                        The criteria "product's long term financial success" doesn't have a lot of meaning. Does it mean my bonus is withheld until 5 years? What if something else goes wrong in such long time frame (like economy tanks) which wasn't my fault? What if someone unrelated contributes to 5-year success and I get to share the glory?

                                                                                                                        The point is that measuring performance off of single metric is incredibly tricky. In the field of reinforcement learning this is known as reward function engineering and its one of the hardest thing to get right for even well specified problems. Employee actions would need to have short term as well as long term focus (exploitation vs exploration). Most actions have consequences that won't reveal itself completely until sometime in future. The current "state of the art" technique is to use stock vest schedule with hope that employee would care to grow his/her future stock vest.

                                                                                                                    • januzis 5 years ago
                                                                                                                      This doesn’t solve the issue of individual team member performance evaluation, even if it improved the overall performance of the team, you still wouldn’t know who are under and over-performers.
                                                                                                                      • jimbokun 5 years ago
                                                                                                                        Because that is counter productive to increasing productivity.

                                                                                                                        Having team members focused on getting themselves ranked higher than their teammates, means incentivizing them to not help each other or do something they might not get credit for, even if it's in the best interest of the product or the team.

                                                                                                                        • januzis 5 years ago
                                                                                                                          Even if you've tied team compensation to the results of the company, you still have to decide how it will be shared among the team members. So how do you go about compensation, bonuses, etc.? The same for everybody?
                                                                                                                    • dfeojm-zlib 5 years ago
                                                                                                                      I think of software developers having multiple qualities:

                                                                                                                      - coolness (cooperation, proactiveness, professionalism)

                                                                                                                      - carefulness

                                                                                                                      - integrity (ethics)

                                                                                                                      - morale

                                                                                                                      - cadence (speed) of work

                                                                                                                      - skills competencies (matrix)

                                                                                                                      - grit (badassery)

                                                                                                                      - estimated time to completion multiplier

                                                                                                                      • pmiller2 5 years ago
                                                                                                                        All I see here is a bit of advice on what to measure, and nothing on how to measure it. The problem with measuring software engineer performance has always been in the how, not the what, so this article is just noise, IMO.
                                                                                                                        • slaymaker1907 5 years ago
                                                                                                                          Lake Woebegon is a horrifically bad strategy under even the most basic of assumptions. There are not enough engineers in the world available for hiring to create a single large tech company under such a strategy even with zero turnaround. A fixed level of achievement can give you similar quality (assuming you do have some level of attrition) but at a much faster rate.

                                                                                                                          The best strategy after looking at various strategies under simulation is to focus on developing the people you already have since fixing your quality through hiring is very difficult and expensive.

                                                                                                                          • yowlingcat 5 years ago
                                                                                                                            It's a substitute for a strategy: in reality, it is a wishful thought masquerading as a strategy. It doesn't give you any strategic framework for achieving that by hiring, mentoring and retaining in a competitive landscape. I find it of limited use and anachronistic.
                                                                                                                          • 5 years ago
                                                                                                                            • gorzynsk 5 years ago
                                                                                                                              I know how to fix most problems with measuring an engineer's performance. The best solution is to remove "manager" from measurement. Lead developer would speak with all team members to assess performance of colleagues. Those are those who know exactly who makes their work harder and who helps them everyday whether by wise advice or by leaving clear code and thought through architecture.

                                                                                                                              Managers will argue that they have so much responsibilities that they cannot code together with teh team, but I again would say that the problem may be reduced with reduction or higher management. Bussiness part shall talk with engineers on feasibility of their vision and ideas without proxies who are so in the middle that they neighter understand bussiness nor technology.

                                                                                                                              • alexbanks 5 years ago
                                                                                                                                Yikes.
                                                                                                                                • backtobecks 5 years ago
                                                                                                                                  Are we really going to pretend that managers in sillycoin valley even bother to be objective like this?
                                                                                                                                  • dang 5 years ago
                                                                                                                                    I'm sure not, but could you please stop posting unsubstantive comments here?