Estimating Software Projects - Jacob Kaplan-Moss
Series: Estimating Software Projects| jacobian.org
Writing| jacobian.org
Ways I’m available to help| jacobian.org
Applications are open for the 2025 Django Software Foundation Board of Directors – you can apply until October 25th. So, in this post I’ll do two things: try to convince you to run for the board, and document my goals and priorities for 2025.| jacobian.org
There are three types of interview questions: behavioral, hypothetical, and trivia. Behavioral questions are the gold standard; they’re the most effective at predicting job performance. Hypothetical questions can be useful in certain circumstances, if used correctly. Avoid trivia.| jacobian.org
The fifth and final part of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today’s question is the most difficult-to-ask of the series, but also one of the most valuable: asking a candidate to discuss one of their weaknesses.| jacobian.org
Part 3 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: making sure candidates align with organizational values of diversity, equity, and inclusion.| jacobian.org
So you’ve found a security issue in an open source project – or maybe just a weird problem that you think might be a security problem. What should you do next?| jacobian.org
If you’re new to tech – say, less than 5 years in the field – you should take career advice from people who’ve been in the industry more than 10-15 years with enormous skepticism.| jacobian.org
A series sharing some of the questions I use when I interview for technical roles. I’ll unpack the question, when to ask it, and how to evaluate answers.| jacobian.org
In part 1 of this series, I briefly covered quantitative risk measuring – assigning a numeric value to risk, like “$3,500”, rather than a qualitative label like “medium” – only to quickly recommend against trying it. In this final sidebar, I want to come back to this topic. I’ll spend a bit more time explaining what I see as the pros and cons of quantitative risk measurement – why you might or might not want to use numeric values over more simple risk matrixes.| jacobian.org
When you look at a likelihood/impact risk matrix, you might notice that “medium” appears twice – once as high-likelihood/low-impact, and once as low-likelihood/high-impact. These two “mediums” aren’t at all the same!| jacobian.org
In the real world, accidents happen when a series of small missteps align to create severe consequences. This is something we call the “Swiss Cheese Model”: imagining a systems failure as a set of “holes” in our layers of defense that all line up to create a series accident.| jacobian.org
Subscribe for updates| jacobian.org
Risk is usually defined as the product of two factors: Likelihood and Impact. However, some disciplines include a third factor: Exposure. What’s that about, and when is it useful?| jacobian.org
I spend what feels like a majority of my waking hours thinking about risk. Professionally, I’ve worked in computer security for fifteen years (and software engineering more generally for longer).So my work days are filled with questions about risk: how risky is this vulnerability? How dangerous is it to launch this new feature if it hasn’t gotten a proper security review yet? How much risk is left after we do that review? And then after work, all my hobbies take place outdoors in the wild...| jacobian.org
So you’ve identified a risk — now what do you do about it? Here’s a simple framework to help frame discussions about risk mitigation. It’s intentionally very simple, a basic starting point. I’ll present a more complex framework later in this series, but I want to lay more of a foundation before I get there, so we’ll start here.| jacobian.org
Contact me| jacobian.org
If you — as an individual or a group — are re-assessing your digital security posture in light of the US election results, I’m available to help. I’m offering free digital security check-ups to anyone who feels like they need it now.| jacobian.org
What would the Django Software Foundation look like if we had 4x our current budget?| jacobian.org
There have been massive developments in AI in the last decade, and they’re changing what’s possible with software. There’s also been a huge amount of misunderstanding, hype, and outright bullshit. I believe that the advances in AI are real, will continue, and have promising applications in the public sector. But I also believe that there are clear “right” and “wrong” ways to apply AI to public sector problems.| jacobian.org
Sumana challenged me to apply the principles of Leave No Trace to engineering leadership, so here we go.| jacobian.org
A new principle I’m trying to follow: we should be paying more for independent media. How I got there, and a list of the media I’m paying for.| jacobian.org
Welcome to FictionalSoft! I hope your first week is going well? Great. As you start to find your feet, I want to make sure we have a shared understanding of what success looks like here. Apologies in advance if I’m telling you something you already know, but it’s important to be explicit about this early. You were hired to write code. Many developers make the mistake and think that their job stops there. That’s not true. In fact, you have two jobs: …| jacobian.org
I planned and started to write a series about personal productivity systems, but I’m abandoning the series. Here’s why.| jacobian.org
One of the main responsibilities of a leader/manager is helping their staff develop. Mentorship, coaching, and sponsorship are import tools in the staff development toolbox. Good leaders should be adept in all three, and know when (and when not) to use each. In my work with new managers, I sometimes see confusion about these three different tools, and I see people using them in the wrong circumstances. So here’s a glossary, a high-level explanation of what these three things are, how they d...| jacobian.org
I was invited on the Sustain podcast to discuss my recent rant about open source sustainability. I talked about my reaction to the criticism that open source maintainers receive when they take funding, and how this is a personal issue for me – maintainers aren’t abstract ideas to me, they’re my friends. We discussed my call for a more expansive definition of open source, and got into some of the nuance about some of the problems this can cause.| jacobian.org
I was on the Django Chat podcast to about Django’s history, the creation of the DSF, my recent return to the DSF board and my goals there, and the things I’m excited about for Django going forward. Here are some highlights from the interview.| jacobian.org
I’ve been through close to a dozen reorgs. This article contains the advice I wish I’d been given earlier in my career when I didn’t yet have that experience. Reorgs are disruptive, and nobody really tells you what to do in the wake of one. It’s easy to feel adrift, scared for your future, and uncertain about how to behave. Some of that fear is warranted: your job security probably goes down in the months following a reorg. But confusion and chaos aren’t necessarily signs that the r...| jacobian.org
Something missing from this series on estimation, until now, has been a discussion of how to “break down” a project into a well-defined task list. I’d not previously written about this because, to me, it’s largely intuitive. But it isn’t for everyone, so this post fills the gap, and explains in detail how I break down projects into a task list.| jacobian.org
If you have a problem with maintainers getting paid then you have a problem with me and I suggest you let that one marinate.| jacobian.org
How do you understand what engineers are doing with that time? How do you know if they’re working on the “right” things? Here’s how I suggest answering those questions.| jacobian.org
Today I Learned…| jacobian.org
The RFC process has been a huge success in defining the standards that run the Internet, but naively adopting this process for your company is a mistake. RFC processes tend to fail at most organizations because they lack a clear decision-making step.| jacobian.org
Managers often talk about performance as a static thing. We say that someone is a “high performer” or “low performer”, as if performance is a fixed attribute of their personality. This fixed mindset is a mistake. Performance is contextual: how well you perform your job is deeply dependent on the conditions around you.| jacobian.org
When is it the right choice to hire full-time staff, and when should you use consultants or contractors instead? The short answer: hire consultants for guidance, contractors for execution, and employees for stability and flexibility. For the long answer, read this article.| jacobian.org
A pattern of short tenure – multiple short jobs over a small period of time – can be a red flag, but necessarily. I’ll dig into short tenure in this article. What’s “short”? What’s a “pattern”? When you’re thinking of switching jobs, when and how should your tenure factor into the decision? For hiring managers, when is it reasonable to be concerned about job tenure, and what should we do when we become concerned?| jacobian.org
Work sample tests are an exercise, a simulation, a small slice of real day-to-day work that we ask candidates to perform. They’re practical, hands-on, and very close or even identical to actual tasks the person would perform if hired. They’re also small, constrained, and simplified enough to be fair to include in a job selection process. Work sample tests are a critical factor in effective hiring. Interviews aren’t enough; hiring without work sample tests risks selection people who exce...| jacobian.org
There’s a Right Way to conduct job interviews: one-on-one, with a single interviewer per interview session. If you need multiple interviewers (you probably do), schedule multiple sessions, each one-on-one. The alternate approach, panel interviews – having multiple interviewers in a session at once – is almost always a bad practice. It increases stress on the candidate, risks measuring the wrong things, and doesn’t lead to better results. Avoid panel interviews: they don’t work.| jacobian.org
Most of the time, reference checks go very well: you only turn up information that confirms your decision to hire this candidate. But sometimes — maybe about 10% of the time — the reference will tell you something concerning. Here’s what to do if that happens.| jacobian.org
Part two of my reference check series, covering the nuts and bolts of conducting a reference check. When should you check references? How many? How should you contact references? What questions should you ask?| jacobian.org
Reference checking isn’t optional: it can save you from making a big mistake. Reference checks are your last line of defense against hiring a jerk.| jacobian.org
“What metrics should I use to measure my engineering team’s performance?” Believe it not, there is a Right Answer: the so-called DORA metrics.| jacobian.org
I’m starting a new series on professionalism: the set of workplace behaviors that are generally expected at work. These behaviors are largely unspoken, but they do exist: there are consequences for violating them. In this series, I aim to write down some of these rules and explore their implications. Eventually, I hope to have a solid list of what “professional behavior” really means.| jacobian.org
Ask A Manager had a wild story a week ago. A company interviewed someone, hired him, but when he showed up for work … it was a totally different person. A friend asked, “if this was your hire, and you manager asked you to change your hiring practices to prevent this, what would you do?” Nothing. I would do nothing. Here’s why.| jacobian.org
It’s tempting to air your grievances at your exit interview. Don’t. There’s almost no upside to speaking up, and tremendous potential downside. Avoid exit interviews if you can. If you must go, be totally bland; say nothing negative.| jacobian.org
This is the final post in my series on work sample tests. It’s a wrap-up post: I’ll address a few random points I couldn’t quite fit in elsewhere, and answer some questions from readers.| jacobian.org
If you want to be a good manager, you need to accept that your behavior is under a microscope. You need to watch your behavior carefully and pay attention to what that behavior communicates.| jacobian.org
What makes a work sample test “good” – fair, inclusive, and with high predictive value? Here’s my framework: eight principles that, if followed, give you a great shot at constructing a good work sample test.| jacobian.org
The work sample tests I’ve covered in this series so far all involve software development. But what about roles that don’t involve day-to-day coding: roles like security analysis, penetration testing, technical support, bug bounty triage, project or program management, systems administration, technical operations, and so on? For those roles, I turn to simulated, “lab”-style environments. Here are some examples of that kind of test.| jacobian.org
For most software engineering roles, the best work sample test will be some combination of the exercises I covered earlier in this series. But not every role; there are some circumstances where other types of tests fit better or are better at revealing some critical piece of information relevant to hiring. This post covers one of them: a “reverse” code review, where instead of you reviewing the candidate’s code, you have them review yours.| jacobian.org
I tend to prefer asynchronous work sample tests. The flexible scheduling of asynchronous exercises (i.e. “work on this whenever you like”) works better for the majority of candidates. But for some candidates, and some roles, synchronous exercises work better. By “synchronous” I mean: work sample tests that are explicitly scheduled, and that has both the interviewer and the candidate working directly together at the same time. In these cases, I often turn to pair programming.| jacobian.org
Coding homework is my default work sample test: I use it for all engineering roles unless it’s obvious that another kind of exercise is better. There are good reasons to make homework-style work sample tests the default: they’re relatively easy to construct, they scale reasonably well to large hiring rounds, they’re accurate simulations of real work, and easier than most other kinds of tests to construct in a way that maximizes inclusivity. Here’s how to conduct a coding homework work sample test.| jacobian.org
Good hiring processes try to maximize inclusivity and predictive value, but unfortunately, work sample tests bring these goals into conflict. There’s always a tradeoff between predictive value and inclusivity. The guiding principle of work sample tests is: construct a test that balances predictive value and inclusivity. Fair work sample tests will be predictive enough to give you a high degree of confidence that you’re making a good hire, while also being designed to be as accessible to a...| jacobian.org
Earlier this year, I wrote a series on interview questions. Good interview questions are one key to hiring well, but they’re not the only key. Today, I’m starting a new series on another critical factor in effective hiring: using work sample tests, aka practical exercises. This is part 1: what are work sample tests, and why do we need them?| jacobian.org
Three simple tricks product I’ve picked up that help me be more than completely useless when I need to wear a Product hat.| jacobian.org
Wrapping up my series on delegation with an example: how to delegate meeting attendance.| jacobian.org
Sometimes, programming feels like magic: you chant some arcane incantation and a fleet of robots do your bidding. But sometimes, magic is mundane. If you’re willing to embrace the grind, you can pull off the impossible.| jacobian.org
For years I’ve noodled around with various setups for a Python development environment. A couple of years ago I wrote about a setup I finally liked; this is an update to that post. Bad news: this stuff still isn’t stable, and I’ve had to make some changes. Good news: the general concepts still hold, and the new tools a generally a bit better. If you’re curious about the changes and why I made them, there’s a section at the very end about that.…| jacobian.org
It’s looking increasingly likely that the COVID-19 pandemic will cause a recession. It’s easy to think we might be immune from the effects of a global recession, but my experience is that tech companies are quick to cut staff, especially engineers, in the face of declining markets. I hope I’m wrong, but I don’t think I am. Either way, it’s not going to hurt to prepare.| jacobian.org
What’s the best way to train folks to conduct job interviews? I have a process I’ve used for about five years that seems to work well. It’s loosely based on the “see one, do one, teach one” methodology used by many medical schools.| jacobian.org
How should you structure a larger engineering organization, one with dozens (or hundreds) of engineers? There are many tradeoffs to consider, and no single right answer. But, there are some structures that work better than others.| jacobian.org
The next time you have an important proposal to make, don’t wait until the big meeting to ask for support. Here’s how to gather feedback and build consensus beforehand, so you can make that big meeting into a non-event.| jacobian.org
Part 1 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: asking candidates to explain a topic at multiple levels. This is one of my favorite questions to ask for engineering roles; strong performance on this question correlates very highly with high job performance on my teams.| jacobian.org
Part 2 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today: measuring a manager’s ability to lead projects and manage them effectively.| jacobian.org
Part 4 of my Unpacking Interview Questions series, where I share one of the questions I use when I interview for technical roles. Today, an oldie-but-goodie: looking into a candidate’s ability to disagree and resolve conflict professionally.| jacobian.org
I believe that autonomy is one of the most important values of effective organizations. But I also think it’s a value that’s misunderstood and misapplied. In this post, I’ll (1) define what I mean by “autonomy”, (2) explain what autonomy isn’t, and (3) try to articulate why autonomy, as an organizational value, leads to higher effectiveness.| jacobian.org
In early January, Sumana Harihareswara and I started getting together a few times a week to help each other write more. This model of working together on separate-but-related projects has worked really well for us, and we wanted to share our model and our details to encourage other folks to give it a shot.| jacobian.org
Within an organization, there are three kinds of power structures: role power, relationships, and expertise. Understanding these kinds of power — how they’re built; how they’re wielded; ethically and otherwise; what they can and can’t accomplish — is key to understanding organizations at a systemic level and maximizing your effectiveness at work.| jacobian.org
I had a call a few weeks ago with a friend and fellow engineering manager, and we spent most of it talking about someone on her team who wasn’t responding well to feedback. He was performing several parts of his job pretty poorly, but when each time she told him that his work wasn’t acceptable, he pushed back. He argued, sometimes loudly, and refused to make the changes that she was asking for. My friend came to me pretty frustrated, not entirely sure how to respond to this guy. Most mana...| jacobian.org
Some conversations about my previous piece brought me back to one of the earliest lessons I learned in my management career. It’s a realization that’s embarrassingly obvious in hindsight: treating people fairly doesn’t mean treating everyone the same.| jacobian.org
When organizations are performing well, the whole is more than the sum of its parts. That’s the whole point of building teams: together we can accomplish more than if we work solo. But many organizations get stuck: suddenly, the whole becomes less than the sum of its parts! When this happens, it can be difficult to understand what’s going on: everyone can be working hard, and yet as a whole, the team just seems to be treading water. When this happens, I’ve found a useful model for under...| jacobian.org
On the fundamental purpose of middle management: context down, information up.| jacobian.org
Here are three models that I like for delivering feedback. Each is valuable on its own and would make a great starting point for anyone who wants to build their feedback muscle. Together, they highlight some common factors in effective feedback models and show off a couple of “special features” that can help your feedback be particularly effective.| jacobian.org
Managers need to understand the difference between praise and positive feedback. Feedback is one of the most important tools in your management toolbox, and an absolute must for any manager who wants to be effective. Praise is a useful tool, but it doesn’t directly drive performance improvement the way feedback does. If you’re accidentally giving praise when you think you’re giving positive feedback, you won’t see the results you expect.| jacobian.org
Last time, I explained that, although estimating software project timelines is hard, you should do it anyway. With that background, I want to go into some detail and share the technique I use when I need to develop a project timeline. I don’t believe there’s a single “correct” technique; this is one system that works well for me. However, my system does have one critical characteristic that I believe any effective estimation technique should have: it captures both time and uncertainty.| jacobian.org
For an organization to succeed – to reliably and consistently deliver great products that customers want – Product and Engineering need to work well individually, but more importantly, they need to work well together. The working relationship between these two organizations starts with the relationship between their two leaders. If these two individuals don’t have a strong individual working relationship, the team relationship is doomed. This article covers what a strong relationship be...| jacobian.org
My foundational principle of delegation: “give away your toys”. Look to delegate the work you love, not the stuff you dislike or dread.| jacobian.org
Your gut instinct is probably to wait to delegate some work until you’re fully confident that the person can handle it. This is often a mistake. Instead of withholding a delegation opportunity from someone because they might fail, you should instead create a situation where failure will be safe.| jacobian.org
