Everything I Have Learned about Making Engineering Performance Reviews Efficient, Effective, and Fun

Kortina, 23 Oct 2017


When I ran our first performance review cycle for about a dozen engineers a little over a year ago, I had never participated in an official performance review myself, either as a manager or a report.

I was a bit skeptical that the whole thing might be a bunch of corporate bureaucracy, only necessary at large companies for incentivizing armies of employees with invented ladders of meaning.

I wondered why all feedback could not just be in realtime, ad hoc, based on performance wins and successes as they happen.

I was also a bit daunted by having to write reviews for over a dozen people.

Having run several of these now, however, I have come to really value and enjoy our review cycles, and I wanted to write up everything I’ve learned about the process along the way.

TOC (since this is a very long post):

  • Getting Started
  • The Basic Components
  • Upward Review
  • Self Review
  • Peer Reviews
  • Manager Review / Compilation
  • Grouping and Summarizing Direct Quotes from Peers
  • Commentary on Key Themes and Strategies for Future Growth
  • Review Summary and Compensation
  • Delivering Reviews
  • Final Thoughts

Getting Started

When we spun up our first review cycle, having never done one before, I did what I usually do when I’m in a role doing something new: I interviewed some people who had done this before.

The two people I interviewed were Raylene Yung, an engineering lead @ Stripe (formerly @ Facebook, and also an advisor to Fin), and my co-founder, Sam, who had lots of experience with performance review cycles when he led a large product org at Facebook.

Btw, if you’re an engineering manager, I highly recommend checking out Step 4: Edit and iterate on your outline of Raylene’s post on how she runs performance review cycles. There are some great tips in there for making the review more focused.

Input from Raylene and Sam led to most of the shape of our current engineering review process at Fin.

The Basic Components

Our review cycle for each employee consists of the following:

  • a review of your manager
  • a self review
  • 3 peer reviews
  • a review from your manager

We use Google Forms to collect all of the reviews.

#protip Google Forms doesn’t reliably save your progress, so better to write your reviews in a doc and paste the answers into a form, rather than write a long review in Google Forms directly and risk losing your work.

Upward Review

The manager review consists of the following questions:

What did this person do well?

What could this person be doing better?

These are the most open ended of all our review prompts, leaving room for people to give feedback of any kind.

Here’s an example of some good upwards feedback:

Sometimes I wish I understood the process by which certain decisions were reached. In all-hands meetings, I’ve always walked away impressed by how clear and thoroughly reasoned objectives and goals are (i.e., I know why we have that objective and how we chose it vs. others). I don’t always feel this way in more micro-objectives or sometimes I feel that we choose to just “do it all” instead of picking specific things to focus on.

Self Review

The self review consists of the following questions:

Looking back over the last few months, what are you most proud of?

What do you think are your biggest areas of opportunity to grow? What are you hoping to achieve in the next quarter both in your role and as a team member?

Examples of some good answers to the first question:

ML Tagging & the docker pattern for this service: It forced me to learn the infrastructure more and productionalized a tool on a new stack (python), with tests, with a new AWS setup, and more cross-pollination with analytics. Buttoning it up to a point where analytics can build off it, beefing it up with tests, and making it “learn” from new data over time was very very fun.

Or:

We created an environment that was supportive, challenging, and fun, and carved out projects that interns were genuinely interested & excited in — and we will have a very high offer:intern ratio! I applied our learnings from last summer’s intern program and took on a diffuse set of responsibilities that included the coordination of onboarding efforts, guiding interns on project selection & scoping, reviewing PRs, organizing performance reviews, organizing meetings / office hours, and scheduling events (with lots of help from Laura and awesome intern mentors).

Examples of some good answers to the second question:

I’d like to improve code velocity on larger projects. I haven’t taken enough time to write out a plan and get feedback so that I can build as quickly and confidently on larger projects compared to smaller ones.

Or:

I have a hard time knowing when to rewrite vs refactor vs add to the mess. Sometimes the right thing to do is patch the old code, sometimes the big rewrite is worth it. I find it harder to justify the time for the larger refactors, so end up somewhat frustratedly working with the existing architecture.

Or:

Outside of planning and communicating — another big opportunity for me to grow here is to invest more time in doing external research on how companies manage projects or deploy infrastructure — I think that there’s a lot to learn from others on these fronts.

Peer Reviews

I use the term ‘peer’ loosely here. We basically use one form for all feedback from any other team member, so we may collect reviews not only from other engineers, but also from people you are mentoring or people in other functions outside of engineering that you collaborate with.

Some key questions when considering how you want to use peer feedback are whether or not you want to show it directly to people and whether or not you want to show attribution of the feedback.

We choose not to show attribution of specific feedback to reviewees, because we think some feedback people only feel comfortable sharing when there is no attribution, and often, this is some of the most important feedback for reviewees to see.

Although we do not show attribution of feedback to reviewees, we do show attribution of feedback to the managers compiling reviews. This seems like a middle ground that leads to the most feedback being shared and provides the most information to managers who might want to dig deeper on specific issues with authors of some feedback.

Peer reviews consist of the following questions:

When was this person at their best this quarter? What are some shining moments as a product/software engineer? As a teammate? Give concrete examples.
What are some ways this person could improve as a product/software engineer? As a teammate? Again, give concrete examples.

NB: software engineers are not only responsible for shipping code, but also for helping build a great engineering organization, and we explicitly ask everyone to consider this when writing reviews.

Before we send out the peer review forms, we also ask each person this question:

Are there any areas of responsibility outside the scope of day-to-day engineering (eg, on-call, code review, CI, site reliability, security, recruiting, etc) that you are shepherding and want feedback on?

Then, before you write a peer review for a teammate, you lookup in a directory whether there’s anything else they are responsible for that you should give feedback on.

Examples of good peer feedback look like:

I appreciated Rob’s commitment to refactoring the phone call code. It has been a notoriously thorny and difficult feature with a lot of agent pain attached, and he pushed for setting aside time to clean it up when nobody else was pushing for this as a priority.

Or:

As the primary iOS developer, it might be good to think about roping some more people into iOS work. Rob is still a knowledge silo for big parts of the iOS app, which means that more of the feature and bug work on iOS falls on him.

#protip when sourcing specific examples for peer reviews you are writing, it can be helpful to mine data exhaust from collaboration tools to jog your memory. Places to consider include:

  • Github Pull Requests
  • Code Reviews
  • RFCs
  • Engineering Blog Posts
  • Feature Release Emails
  • Post Mortems
  • Eng Retro Notes
  • KanBan Boards (we use Airtable)
  • Slack History

Manager Review / Compilation

Here is the part where I ultimately spend most of my time, and where I think our specific process for reviews adds a lot of leverage.

When compiling formal reviews for each report, I use the following template:

2017-H2 Review

NAME,

I’ll start with some of the feedback from the team.

Praise

GROUPED_QUOTES

Ideas for Improvement

GROUPED_QUOTES

Looking through the feedback from the team and your self review, I’ll offer some commentary.

COMMENTARY_ON_KEY_THEMES_AND_STRATEGIES_FOR_FUTURE_GROWTH

- Kortina

Review Summary

Meets Some | Meets Most | Meets | Exceeds | Greatly Exceeds | Redefines

Grant: ______ Shares (standard vesting schedule)

Comp Increase: $_____ / yr_

The goal of this template is to focus my time on where I can specifically be the most valuable to the person being reviewed, which is to help them synthesize the feedback, understand from the team and company perspective how they have been performing, and prioritize the most important opportunities for personal growth.

When writing someone’s review, I first paste this template into a new Google Doc, and then at the very top, I paste in all of the praise and critical feedback from peers, as well as all of the self review, from their respective Google Forms summary spreadsheets.

Grouping and Summarizing Direct Quotes from Peers

The first key leverage point is to directly pull quotes from peers and summarize them thematically. This looks like the following:

You’re willing to take on important projects that need love / have historically been neglected:

I appreciated Rob’s commitment to refactoring the phone call code. It has been a notoriously thorny and difficult feature with a lot of agent pain attached, and he pushed for setting aside time to clean it up when nobody else was pushing for this as a priority.

Rob is willing to take on and execute on impactful projects like phone calls, request entries, checklists

#protip formatting is important here, to distinguish quotes from peers from your commentary as a manager.

I try to scrub / cleanup some of the stylistic peculiarities that might unnecessarily reveal the author, but also will include specific details that may in some cases reveal some information about the author, eg, “When Rob and I worked on project X, ….” narrows the scope of potential authors. But, given that people choose their reviewers, there is some amount of pseudo anonymity anyway, and I strive to be judicious about which quotes of this kind I include. Often, there are concrete examples important for the reviewee to hear, where any attempt at transcribing / anonymizing would not dramatically change the amount of information revealed, and I typically find including the details worth the trade.

Commentary on Key Themes and Strategies for Future Growth

After compiling all of the Praise and Ideas for Improvement from peers, I have a thorough understanding of what the team thinks. I then walk through these along with the self review, and address key points, explaining whether or not my opinion matches the team’s or the reviewee’s on a specific issue, and then offer some suggestions for improving on that point in the future.

This might look something like:

I don’t think you need to be responsible for fixing every iOS bug, but given that historically you have been the point person for iOS, I don’t think anyone else feels empowered to update the process here. At the very least, I’d setup a better / more transparent system for tracking iOS bugs and time to resolution, but even better, float the idea by a few others on the team of having them help out with fixing the iOS bugs and doing some of the cards.

After offering this detailed play by play of the peer and self reviews, I add some higher level thoughts of my own about how each person can grow or take on more responsibility.

For example:

Second, a place you could start taking on more responsibility / ownership is getting more involved in and vocal about staffing infrastructure projects. There has been some concern from the team that we are spending too much time on dev tooling projects (and some concern from others that we are not investing enough in dev tooling or fixes to key systems that would enable everyone to move faster). From my perspective, it feels like we have had a nice mix of work in flight, but in many ways I don’t feel as close to the technical needs as you are, so it would be great to have another judicious opinion from you about when we are spending too much or too little time on infra work.

Review Summary and Compensation

Finally, I’ll fill in the review summary.

We use the following rating scale:

  • Meets Some: Serious gap between performance and expectations. Almost certainly entails a performance improvement plan.
  • Meets Most Not performing to level of expectations. Problematic and requires concrete improvement.
  • Meets: Great work. We have high expectations so this is a solid rating.
  • Exceeds: Great work that exceeds all expectations, with respect to things like depth or breadth or sheer volume of contribution.
  • Greatly Exceeds: Exceeds all expectations, to a surprising extent, eg, has accomplished a shocking amount of work relative to the rest of the team given their level.
  • Redefines: Performed industry changing work: for a senior engineer, this might be pushing an open source project that Facebook ditches React for.

It’s important to note that the rating is based on expectations for the individual, so it’s a moving target, harder to get higher scores the better you are.

In addition to the rating, once per year we do comp adjustments. (Sidebar: it’s really liberating for both reports and managers to do comp adjustments only once per year, rather than constantly diving into comp negotiations ad hoc. It also results, I believe, in more fair treatment across employees, rather than simply benefiting the more vocal, as you can imagine might happen otherwise).

There are 2 components to comp adjustments, equity and base cash compensation increases.

We do equity ‘refreshers’ each cycle, granting more equity to employees each year they work at the company. This equity is subject to our standard vesting schedule, and the way we roughly think about it is “were we to hire this person today, what would their initial offer package include as an equity grant?” The other thing we consider when sizing equity grants is the amount of progress we have made as a company, the goals we have hit, and the questions we have answered about operations or product market fit. Many other startups might only do equity grants as part of offer packages, and then offer more equity only once the vesting for the initial grant completes (if at all). We think, however, that committing to regular annual equity refreshers results in more fair compensation for all employees in the long run.

While the ratings are based on expectations for the individual given past performance and their individual level of expectations, compensation is more grounded in how an individual contributes value to the team and company goals relative to the rest of the team.

There are two key principles governing our basic philosophy around compensation. (1) Since the team comes to work everyday ultimately determines the success of the company, they should be well compensated and dealt in to the financial upside of the success of the company. (2) If everyone were to see everyone else on the team’s comp, everyone should feel good about the fairness of it.

Delivering Reviews

It usually takes me about a day to compile all of the reviews (I can do one in 45 minutes or so), then I meet with everyone in person to discuss them.

I use Fin to schedule 45 minute meetings with each report (spread over 1–2 days).

#protip It’s important to sit side by side with someone in 1:1 meetings between reports and managers. Sitting across a large table can create an adversarial atmosphere.

I ask each person to bring their laptop with them to the review meeting, and I don’t send them the review until they walk in. I imagine myself reading a formal review, seeing things I don’t understand or maybe don’t agree with, and then losing sleep over it until the opportunity to discuss it, so I try to just save everyone some of this stress.

Once someone is in the meeting, I then share the Google Doc with them and tell them, “Take your time reading through. I’ll reread again while you read it, and then we can discuss everything once you are through.”

Then, once they finish reading, I ask, “Is there anything you have questions about? Things that are surprising? Things that resonate?” They talk through any of these questions or concerns, and I offer more context about the feedback and try to help them understand and prioritize what I think is the most important set of takeaways for them to grow and succeed.

Sometimes these meetings can take twenty minutes, and sometimes they can take up to an hour. I find forty-five minutes is usually more than enough time to budget, however.

Final Thoughts

I’m generally very wary of any form of process, as it can easily just add overhead or over optimize focus on metrics to the point of forgetting top level goals.

But process done right can also be a tool that automates repetitive details and liberates you to focus mental energy on things which most require your attention.

Given our current team size and stage of company (and I underscore this premise, because process should always be evaluated and reevaluated given its context), I find our review process consistently helps me learn about the needs of individual team members, extract broader themes and team needs from patterns across reviews, and leaves me really excited about the quality of our team and our potential to do great things going forward.


If you run your review process differently or have any other thoughts or reactions based on this document, I’d love to hear from you. I generally learn a lot from talking shop about eng org stuff like this.

Also, email me if our engineering team sounds like one you’d be interested in joining. We’re hiring.

If you’re interested in working with us, check out Jobs at Fin.