Managing New Standards Adoption

Can you measure the adoption and impact of new best practices?

Software organizations constantly evolve their development technology and best practices. In order to manage change, it is critically important to measure the adoption and impact of these advancements. This is difficult to do in large, distributed R&D environments. Often, the only option open to teams is to report on this manually, which is time consuming, subjective, and inaccurate.

Due to its patented flexibility and accuracy, Semmle engineering analytics is often used to analyze best practices compliance at any level of the organization—portfolio, location, team, project, or individual. It works by analyzing the engineering data and artifacts that teams generate every day. Semmle is supremely flexible and able to measure customers’ unique definitions of “best practices.”

Ad hoc question from the VP of R&D

For example, a global head of software development asked whether Semmle could show if test-driven development (TDD) was being practiced by all projects in their portfolio. Studies have shown TDD can reduce software errors by 50%, so the customer had made it standard practice and invested in training and enablement across the software organization.

With TDD, you would expect the number of test methods to rise with the amount of new code created. The Semmle visualization below shows that all but two of the projects Semmle evaluated appeared to be adopting TDD.

Tests-per-project

Semmle analytics can provide insights with an even finer granularity. The Semmle dashboard below shows the number of tests introduced per individual developer. There is one developer on the team (ID #1040) who is adding lots of new code, but few new tests. The insight Semmle provided to the vice president is very actionable. Employee 1040 can be managed to get on board with TDD for greater compliance and project quality.

Tests-per-developer

Tailoring Semmle to analyze the way your team works

One of the great advantages of Semmle analytics is the inherent flexibility. It can be customized to analyze your unique way of operating and evolving. For example, Semmle did not include an out-of-the-box analysis for test-driven development compliance. But within a few minutes of the customer executive raising the question, team members used Semmle’s powerful query language to answer the question. The query below finds test methods in Java source files; the customer ran it across snapshots of their source code to show how the number of test methods in the code changed over time.

from CompilationUnit f, int n
where n = strictcount(TestMethod test | test.fromSource() and test.getCompilationUnit() = f)
select f, n
order by n desc

 

Download PDF