Git Analytics for Engineering Leads

This document provides information on how Engineering Leads can best leverage LinearB to support efficiently their activities overseeing their assigned squads, their reports, and the Engineering team as a whole.

The document is organized around their responsibilities to make it a practical guide, i.e., each section is one of the areas for which they are responsible.

Software Delivery Accountability

Using Git analytics can play a significant role in efficiently delivering software, ensuring a high bar for code quality, and detecting risks and bottlenecks. Using a tool such as LinearB centralizes data that usually live in different tools (VCS, project management, etc.), thus making it more efficient for Engineering Leads to oversee several projects simultaneously.

Sprint Efficiency

Since squads work in one-week or two-week sprints, efficiency must be regularly assessed and monitored, i.e., monitoring Cycle Time should not be done only once per sprint or per month. Any performance issue in one sprint can impact consecutive sprints, thus delaying the overall delivery of the products.

Cycle Time is the gold standard metric to assess sprint efficiency. As part of monitoring Cycle Time, Engineering Leads must also monitor its subset of metrics:

• Coding Time
• Pickup Time
• Review Time

An efficient Cycle Time has the following characteristics:

• Consistently between 2-3 days on average for each sprint.
• Each metric composing Cycle Time is in the efficient range, thus colored in light or dark green.

When a metric composing Cycle Time is not within the optimal range, the corresponding part in the Cycle Time is colored in either orange or red.

The following are usual causes for inefficiencies in the metric composing Cycle Time:

• Coding Time

  Inefficiencies can originate in development and product management. For the former, developers might be slower due to personal technical and collaboration aptitude or skills. As for product-management-related issues, there might be deficiencies in the backlog items for the sprint, e.g., lack of documentation on the product features, unclear acceptance criteria, etc.

• Pickup Time

  Inefficiencies can only originate in development, and it signifies that code review is not prioritized or performed on time. The Team Lead and developers do not dedicate enough time to review all open pull requests during the sprint.

• Review Time

  Inefficiencies can originate in development and product management. For the former, developers might have opened pull requests promptly, but they might need to rework the submitted code significantly due to deficient technical or collaboration skills. Similar to Pickup Time, long review time can also signify that code review is not prioritized enough. Issues in Pull Request Size can also negatively impact the code review efforts. As for product-management-related issues, similar to Coding Time, there might be deficiencies in the backlog items uncovered after developers’ completion of the code, thus requiring rework.

Each project and squad have specificities that might differ from the list of causes listed above. However, the fundamental principle is that Engineering Leads must investigate any deviation from the optimal range defined at the team level and collaborate with all team members of the squads — the Product Manager, the Team Lead, and the developers — in resolving the issues at the earliest.

Backlog Efficiency

Backlog efficiency can be broken down into assignment of the backlog items and composition of the backlog in terms of task types (features, chores, and bugs).

Backlog Assignment

There is no strict enforcement in the way backlog items are assigned to developers. A Team Lead can either assign all backlog items when the sprint starts, progressively as the sprint progresses or let the developers pick up the tasks by themselves. The lack of convention in this regard is intentional to leave room for the various leadership styles of Team Leads and accommodate the diversity of environment the squads have to interact in with clients.

However, to deliver sprints efficiently and reliably, Engineering Leads must ensure that the load of the backlog is spread equally among all developers. In practice, every developer should have more or less the same amount of backlog items and points to deliver and work on various types of tasks (features, chores, and bugs). Having only one or a few specific developers doing most of the work or working on only one type of task is causing heavy dependencies, creating unfairness, and preventing group ownership of the code. Indeed, if the assigned developer is on leave, the sprint is at risk if no one else can take over. Unfairness can be perceived by both the assigned developers as they do more work than others but also by the other developers in the squad as they can feel the negative impacts of being benched or relegated to a proverbial second tier. Eventually, if developers are not working on all areas of the application, it takes significantly more time for them to know every part of the codebase.

To deliver backlog items, developers have to open pull requests to merge their changes. Therefore, Engineering Leads can monitor the efficiency of the backlog assignment in the table on the People page.

The People page provides a summary of several metrics for all individual contributors in a team.

An efficient Backlog Assignment has the following characteristics:

• All individuals opened a similar amount of pull requests.
• All individuals committed a similar volume of code changes (expressed in lines of code) within an appropriate delta.

Backlog Composition

While Product Managers own the backlog, Engineering Leads collaborate with them closely to support the efficient development and delivery of products. As part of their collaboration efforts, Engineering Leads can monitor the type of tasks delivered on the Investment Profile page.

The above diagram displays the distribution of tasks by their type by sprint.

An efficient Backlog Composition has the following characteristics:

• Features represent the majority of the type of work delivered.

  By widely accepted standards, only features have Agile points attached to them, thus representing the squad’s velocity. To continuously deliver business value to end-users, delivering features must remain the focus at all times, even in the initial sprints of a greenfield project. If a sprint contained only chores and bugs, the velocity of the squad would effectively be zero, thus posing an issue in the services the company offers to clients. Engineering Leads must ensure the squads are always focused on delivering value at all times.

• Chores and bugs are part of every sprint.

  Software application constantly needs to evolve, change and be maintained, thus require work outside of delivering features. Scheduling chores and bug fixes in every sprint is preferred to avoid having sprints focused on performing fixes and maintenance, thus stopping iterating on the features that bring value to end-users. Engineering Leads can play a significant role in helping the squad find the complex balance between increasing functionality and improving the codebase.

Code Review Efficiency

In addition to monitoring the time taken by the squad in code reviews via Review Time - a quantitative measure at the squad level - Engineering Leads must also monitor the metrics gathered at an individual level in the table on the People page:

• Review Depth

  Efficient code review depth is crucial to maintain a high bar for code quality. Indeed, squads could have an efficient review time while having a too low code review depth, thus endangering code quality.

• Number of completed reviews

  All members of the development team in the squad must perform both development and code review. As a result, all developers must participate in the code review process, regardless of their role in the squad or seniority level.

  It is inefficient and unfair if only one or two developers — for instance, only the Team Lead or the most senior developer — perform all code reviews. In the long term, it negatively impacts one of the core purposes of code review beyond code quality: group ownership of the codebase. Indeed, by reviewing pull requests, developers increase their overall knowledge of the codebase. By not reviewing the work of others, their knowledge of the application is limited to what they worked on.

An efficient Code Review has the following characteristics:

• All individuals have a review depth of 4 or above on average.

• All individuals performed a similar volume of code reviews within an appropriate delta.

  The Team Leads usually have a higher volume than the developers as they must review all pull requests since they are in charge of code merges. In contrast, developers are encouraged to review as many pull requests as they can. Once pull requests are approved by the required number of reviewers, a pull request is merged. As a consequence, it can happen that developers did not review some of the merged pull requests.

Risks and Bottlenecks Detection

As an observability platform, LinearB automatically tracks specific risks and bottlenecks. When defined thresholds are crossed, LinearB triggers notifications that can be viewed on:

• LinearB’s Dashboard for each project.

  

  The list of notifications contains the pending risks and bottlenecks that Engineering Leads must investigate.

  The Engineering Lead for the project is responsible for acknowledging the notification on the Dashboard by checking the checkbox for the specific alert upon completing their investigation.

  

  Once an issue has been investigated, it can be closed.

• Slack in the activity channel for the project.

  

Refactor/Rework Risk

The “Work at risk” notification tracks branches with more than 100 code changes and a rework or refactor above 50% by default.

Such branches can have negative impacts on the Cycle Time since both Coding Time and Review Time will increase as a result. Indeed, extensive refactoring/rework is usually harder to complete and review.

Engineering Leads must ensure that the work is:

• Not unplanned.

  Refactoring work is standard and, in fact, healthy in software development as long as it is planned as part of the sprint. Such planning can take the shape of a chore to tackle technical debt or a feature for which it is known that refactoring is necessary.

  If the work is unplanned, there can only be two options. In the first case, if the work is unplanned and required — the need might have appeared when development started, thus providing a better understanding of the feature —, the squad needs to assess the impact it will have on the overall sprint. For instance, some stories might need to be removed or altered. In the second case, if the work is unplanned and not required, the Engineering Lead and Team Lead must collaborate to ensure that the refactoring/rework stops as it endangers the delivery of the sprint.

• Not caused by a development workflow issue.

  Issues in understanding or strictly following the acceptance criteria defined in a story can result in developers working outside the intended scope. In such a case, the Engineering Lead can improve the situation by clarifying the scope of work and recommending a different path of implementation. The sooner a work-in-progress pull request is opened, the more convenient it is to address such issues on the version control system application (e.g., GitHub). If no pull request is opened, the Engineering Lead must not wait and review the branch in their preferred development environment.

Whether the work is unplanned or caused by a development workflow issue, the faster the handling of the situation, the better the odds are that the problem will not negatively impact the sprint.

Deficient Code Review Risk

The “Merged without review” and “Merged with basic review” notifications track pull requests that receive an insufficient number of reviews and amount of review time, respectively.

Such branches signify that the squad has an inefficient code review, thus shares all the observations details in Code Review Efficiency as it negatively impacts the code quality.

Engineering Leads must ensure that the merged branches do not pose serious code quality risks. There can be cases when minor or expedited reviews are required, such as release pull requests or urgent hotfixes. Outside of these specific cases, Engineering Leads must ensure that the development team in the squad improves their code review process to avoid such issues from happening in the future.

Code Review Bottleneck

The “Review request hanging” and “Long review” notifications track pull requests that take more than three days to get either picked up for code review or merged following the code review.

Code review is often the culprit of longer Cycle Time and therefore must be monitored closely. Engineering Leads must ensure that:

• The squad prioritizes code review.

  An opened pull request represents work that is marked as completed by a developer. Therefore, it has more value than un-started and even work-in-progress work. Each merged pull request gets the squad closer to delivering the sprint.

• Lengthy discussions in pull requests find a quick resolution.

  The merging of pull requests can be delayed because developers have repeated and prolonged exchanges of review comments. While a healthy amount of review comments is beneficial to code quality, lengthy and too frequent discussions at the pull request stage can also signify issues to address. Engineering Leads must investigate and assess if there is an opportunity to unblock a stalemate or refocus the squad on priorities.

Engineering Process Ambassador

Similar to a SCRUM master, Engineering Leads ensure and enforce that all squads at the individual level follow development processes and conventions.

Version Control

Regularity of Git Commits

The average of daily commits for each individual is part of the Throughput Metrics dashboard.

Engineering Leads must ensure that developers in a squad commit their work daily and throughout the day to ensure:

• Small and frequent commits are made instead of a single commit for an entire day of work. The latter practice is a violation of the team conventions for using Git.
• All work-in-progress work is saved to the version control system outside the local environment, thus cannot be lost. One of the core benefits of using a service like GitHub is to ensure decentralized development work can happen without any risks of losing completed work. Another advantage to regularly push work-in-progress code is that it can be reviewed early by the other developers, thus detecting implementation issues early.

Size of Pull Requests

The average pull request size for each individual is part of the Quality Metrics dashboard.

Release Management

On the LinearB Dashboard, the Cycle Time shows if a release was made in the sprint for each project.

A release was made in the above diagram for Cycle Time since Deploy is in dark green.

A release was NOT made in the above diagram for Cycle Time since Deploy has the text “N/A”.

While Deploy Time is not actively monitored, Engineering Leads must ensure that every squad makes a release in every sprint, i.e., merge all code to the main branch, tag the main branch using semantic versioning and create a release version on the version control system platform.

True to Agile standard methodologies practices, each sprint is a product increment. As a convention, each increment is versioned and released. Releasing the product increment is essential to receive feedback and continue iterating on a product efficiently. Failure in such practice is an engineering process deviation as it results in adverse outcomes and increases risks similar to the Waterfall development model and Big Bang releases.

Nurture and Assess Developers

Engineering Leads can have detailed insights into the activity of the developers they manage (i.e., their reports) and how best they can support them for their individual growth. Using Git analytics brings not only efficiency to Engineering Leads but also to their reports. Since LinearB centralizes data that usually live in different tools (VCS, project management, etc.), Engineering Leads can more easily oversee all their reports working on different projects. Using a tool also enables monitoring to be asynchronous and continuous. Instead of disruptive check-ins in person or via chat (e.g., “How are you doing on the project today?”), Engineering Leads can observe, cross-reference, and record discussion points for punctual check-ins, One-on-Ones, Peers Feedback, and Skill Appraisals.

Activity Insights

Engineering Leads do not necessarily oversee the squads to which their reports are assigned. In practice, this means that they do not always take part in the day-to-day development activities of all their reports. Using LinearB, Engineering Leads can have detailed technical insights on the development activities of their reports without being “physically” part of the squad. Such data is crucial in providing context and support information to best coach and mentor their reports.

LinearB provides several graphs providing a breakdown of a developer’s activity on each sprint. Having such a level of analytical data allows understanding better on which activity a developer is facing no issue, spends more time, or, worse, is stuck.

Individualized Cycle Time

Like for squads, Cycle Time is the gold standard metric providing tangible data for all stages of development through its subset of metrics, namely Coding Time, Pickup Time, and Review Time.

LinearB provides an individualized Cycle Time on the People page.

Since the individualized Cycle Time metric is computed in “real-time” for each sprint, Engineering Leads can identify the stages in which their reports are performing well and, inversely, in which they need to improve.

Individualized Work Breakdown

Similar to the balance of a backlog for a sprint, developers need to have a balance in the type of work they do in each sprint. The types of work completed by developers fall into the following buckets:

• New work: brand-new lines of code added to the codebase.
• Refactor: changes to any code older than a couple of weeks (by default, three weeks).
• Rework: changes to any code that was recently added to the codebase

LinearB provides an individualized Work Breakdown on the People page.

An efficient Work Breakdown has the following characteristics:

• New work represents the most prominent type of work as it usually provides additional value.
• Rework represents the most minor type of work since it usually results in delays. Changing recently added code can signify development issues in the implementation stage caused by the developer’s deficiencies in technical skills, planning skills, or a combination of both.

Individualized Workload

At the organizational level, developers are dedicated to a single client project to deliver quality work efficiently. At the individual level, it is also crucial that each developer is focused on a limited set of tasks at any given time. Such a practice benefits the individuals and the squad. Indeed, Developers have a balanced and sane workload, thus reduced stress and increased productivity. As for squads, focusing on completing work-in-progress first before starting new work enables them to deliver working code continuously.

Engineering Leads can monitor the Individualized Workload from the LinearB Dashboard for each project.

The above table lists the information for the current sprint.

The above table lists the information for past sprints.

During the current sprint, Engineering Leads must pay attention that their reports have:

• Few active branches at the same time.

  Focus on a single task; thus a single branch is usually preferred. A high number of active branches can signify that developers are multitasking, thus juggling too much work-in-progress.

• An efficient number of completed branches (i.e., merged branches).

  Within an appropriate delta, each developer must deliver a similar amount of work at any given time. If developers have a much lower number of completed branches, it might signify they have issues in having their work approved by the team, thus cannot start working on new tasks.

• A low number of carried-over branches.

  Not completing the work-in-progress within the sprint can signify individual technical deficiencies that must be addressed and, in some cases, over-tasking. Regardless of the causes, a high number of carried-over work impact developers and the squads negatively since it delays the start of new work in the current sprint.

Data Source for Individual Growth

Regular feedback and assessments are crucial elements for individual growth. Git analytics can support Engineering Leads to provide both unbiased and actionable information.

During One-on-Ones and Peers Feedback, Engineering Leads can use the individualized information to provide feedback on two key responsibilities of developers: coding and code reviews. The numerous diagrams and data points listed in this document can be reviewed with the developers to give them insights into how they perform. Instead of feeling about output and impact, individuals can have actual data that can be compared across time. For instance, code review is an oft-hard-to-master skill as it involves technical skills and an efficient personal workflow, so it is common for many developers to focus on improving in that area. Engineering Leads can thus use Git analytics to illustrate the deficiencies and review them regularly with the concerned developers.

As for assessments, Git analytics provide another layer of information that can be used in validating objectives and skills on the Developer Track. Continuing on the above example about code reviews, Engineering Leads can create objectives for developers to grow their skill in that specific area. As part of validating the objectives, improvements on Review Depth and Number of completed reviews, as detailed above in Code Review Efficiency, can support the Engineering Leads to be confident in their assessments.

High-Focus Situations

While Git analytics provide insightful information at any given time, there are critical moments when Engineering Leads must pay closer attention in the oversight of the reports:

• When a new team member joins their first project.
• When a developer starts a new project.

Both situations are sources to several challenges for the concerned developers, thus requiring higher-than-usual oversight. For new team members, it is crucial to best support their onboarding as they need to learn and adapt to numerous new processes. The first assignment to a client project might even be part of the probation plan, so prompt actions are usually required to ensure the developers perform as expected. As for seasoned team members, the start of a new project also means an onboarding period of one to two sprints, during which they must learn the codebase and rapidly ramp up their velocity. Therefore, Engineering Leads must pay close attention to how their reports are handling the change of domain and, sometimes even, the change of stacks (e.g., starting a new project on a new stack for the first time).

Long Term Analysis

Acting upon individual analytical information, such as reaching out to the developer, providing technical help, or mentoring support, is at the discretion of Engineering Leads.

The diversity of situations, projects, and individuals make it practical to have a single escalation process. However, as a general rule, deficiencies in specific areas that occur punctually and solve themselves do not require the same attention as recurring issues in several sprints. For instance, a developer might work on more challenging tasks in a sprint, causing Coding Time to increase in that specific sprint. Such a case differs significantly from a developer consistently over the efficient range of 3 days on Coding Time in several sprints. The observation of repeated deficiencies is usually evidence of a deeper issue to address.

Unlike fulfilling their sprint delivery responsibility, for which prompt actions are required, as detailed above in the first section of this document, the nurturing of individuals must be viewed in the long term. By continuously monitoring their reports over months and years, Engineering Leads can identify improvements as they happen and other areas to strengthen as they appear.

Insights and Reports to the CTO

Just like Engineering Leads, the CTO regularly monitors the Git analytics metrics for all projects and developers. However, since Engineering Leads are in charge of the day-to-day oversight of the client projects, the CTO relies on them to get detailed information on:

• Deviations in the delivery, quality, and throughput metrics for each squad.

  Engineering Leads are accountable to directly know the issues causing the squads not to meet the expected efficiency ranges. As explained above, Engineering Leads must investigate promptly and support the squads to solve the issues by monitoring the metrics closely.

• Performance issues for their reports.

  As their direct managers, Engineering Leads are best placed to provide detailed information on why a specific individual has issues delivering work as expected. Through one-on-ones, Engineering Leads can also uncover issues and work with their reports to address particular shortcomings.

In addition to providing information on issues, Engineering Leads can also use LinearB to demonstrate how well the squads perform. Analytics are reliable data that can be used as evidence, and celebrating victories is as essential as addressing issues.

The CTO and Engineering Leads meet every two weeks for a retrospective of their respective projects and reports. This session is the critical moment for the CTO to ask questions and collect feedback from the Engineering Leads.

Conclusion

Git analytics play a significant role in enabling Engineering Leads to oversee their assigned squads and developers efficiently. It is a crucial tool to:

• Scale the Engineering team to a larger group of individuals while not sacrificing efficiency and quality.
• Enable Engineering Leads to do proactive work instead of reactive work. No one likes to put out fires, especially when it could have been avoided if one had spotted the smoke early on.
• Provide reliable and actionable data for Engineering Leads to support the squads and developers.

Like any tool, Git analytics is not a silver bullet on its own, but it can surface actionable trends and critical data points for further investigation. Engineering Leads must still collaborate with Product Managers and Team Leads and review the backlog details on the project platform to get additional data points and build a complete picture of the situation. The significant difference is that this process can happen earlier and with fewer frictions than without using Git analytics.