Timely Reviews
Better Feedback More Quickly
Good pull request reviews are relevant, timely, and actionable. When reviewers are intentional, understanding the goals of the review, and ensuring that reviews have context, reviews are more likely to be relevant and actionable.
Reviews should also be timely. Actionable and relevant feedback is good, but your goal is to deliver software, and delays caused by slow feedback can hurt your team.
Being timely requires:
an agreement on what timely means, and
an environment that enables the team to meet the expected latency.
A shared definition of timely can be trickier than you expect, and team structure and dynamics can have a significant role in enabling or discouraging timely feedback.
Timeliness
While quicker is usually better, people disagree on what an ideal latency for a review is. Some delay is inevitable, but past experience can lead people to be a bit more forgiving of delay than they need to be.
The speed at which a review can be completed is constrained (as shown in the figure below) by the time it takes for:
Automated checks to run
A reviewer to read through the code and consider feedback. Even if your reviewer has context for the project, it will take some time to become familiar
A reviewer to context switch to the review.
Unless you are pair programming — which implies continuous review — that time won’t be zero, though it should be small.
While “immediately” might not be reasonable, you have control over a few steps, in particular, how long it takes for a review to start and how long a review takes.
Regardless of the constraints on how quickly a review can happen, people and teams have different opinions about what’s ideal. I asked a small sample of people on LinkedIn what their ideal code review cycle time would be, and the answers I got were split between “less than 30 minutes” and “1 to 4 hours,” with some people saying even 4 hours to 1 day would be best.
I didn’t ask follow-up questions about why people chose the longer values, but based on other conversations, I suspect past experience influences people’s expectations; when people are used to cycles taking days, “4 hours to 1 day” might seem great. But we don’t have to settle for that.
People have adapted to long review cycles by trying to make the time they wait more productive by working on other projects. But even if the time spent waiting for feedback is “productive,” there are still costs:
the cost to context switch between the new task and the code you submitted for feedback
the cost of the delay in integrating your changes to other developers who need to work with them
the cost of the feature being delayed.
Research on flow and context switching points to a feedback latency of less than a day as an upper bound for integrating code after a review, and teams can aim for better, ideally, less than an hour. Achieving this is in part a matter of will, but there are some challenges.
Obstacles to Timeliness
There are many possible reasons for slow feedback cycles, but they all come down to ownership and priority.
There are three structural challenges to getting initial feedback promptly:
Needing a review from someone on another team.
Needing a review from someone on your team who has other priorities.
Defaulting to asynchronous reviews.
When I wrote about contextual pull requests,I described the value of ensuring there is a Feature Partnerwho has the context to give prompt feedback and whose work includes delivering the code you are working on. This can also be a forcing function to start reviews quickly and to have a tight feedback loop.
The format of the feedback matters as well. Pull Requests are often asynchronous: you share a review, someone comments, you respond, and so on. While asynchronous feedback has a place when teams are distributed or if the feedback is straightforward, it also has costs, especially when the feedback cycle is slow to get started. A conversation via Pull Request comments can lead to delays between each comment/reply cycle, stretching out the review process unreasonably.
Enabling Timeliness
There are some structures that you can put in place to address some of the obstacles to quick feedback:
Establish working agreements about how quickly people should address reviews. The time can vary by team, but consider starting with “1 hour”. The key thing is meeting your agreement.
When you do asynchronous reviews, but resist doing things that block progress: rather than “requesting changes,” approve with comments, trusting your colleague to merge only after your comments are addressed.
When there is an issue that needs discussion, plan to meet (virtually or in person) to review the comments in real time
For each of these, the logistics are straightforward, but you may encounter organizational or team cultural resistance; however, the added value can make it worthwhile. In some cases, such as when a team is distributed across time zones, some delays may be inevitable, but you can organize work so that, in most cases, reviews happen in a timely manner.
There are other things you can try, such as committing to defaulting to synchronous reviews or having periodic scheduled review blocks where the team focuses on giving and receiving feedback. Both of these can work, but they carry their own challenges to flow. Regardless of what you try, be open to new things and evaluate your process periodically.
What’s Next?
Once you agree on the value of a more rapid feedback process and begin making changes, gather data to determine whether you are still on track. Anecdotal data, such as gathering impressions about feedback, is valuable, but as we noted, people often adjust expectations, so some quantitative metrics are more valuable, including:
Time to first comment: If the wait for a review frequently exceeds the team’s working agreement, work to identify the issue. It could be over-commitment, or perhaps the expectation is unreasonable
Time between interactions (if you do asynchronous reviews): Asynchronous reviews can be useful for simple comments, but don’t work well for conversations. If the team feels that it takes too long to have a feedback conversation, consider defaulting to synchronous reviews.
Time from Code Completion to Merge: This is the key metric for an agile code line. If Code Review becomes the bottleneck in integration, understand why.
You can probably identify other metrics that give you insight into the process.
As with all metrics, when you diverge from expectations, be curious about why rather than making assumptions. If you see a review cycle diverging rapidly from your goals, encourage the teams to understand why and address any roadblocks.
Feedback from colleagues is valuable. Give the work of giving feedback the appropriate priority. You may be surprised by how much more engaged and productive a team feels when you improve the review cycle.