Customer asks “Do we have the right mix of contract developers?”
In order to meet staffing resource demands for new applications or to lower costs, many software organizations supplement their full-time employees with on-shore or off-shore contract developers.
In an effort to optimize contractor costs and productivity, a customer used Semmle engineering analytics to analyze contractor performance along several dimensions. The company employs over 5,000 contract developers, based in many different countries, employed through a number of software consulting vendors. Given the large scale of the customer’s development organization, contractor staffing optimizations can have a tremendous impact on payback.
Finding 1: Teams from the same vendor do not perform uniformly
Firstly, the customer used Semmle to evaluate the code changes (quantity, type, quality, and impact) for each consulting vendor by project. The visualization below shows that different contractor teams from the same vendor (E) do not perform uniformly. When code (or another project resource) breaks a rule of coding best practices (as defined by each customer), Semmle analysis triggers an alert. The Vendor E team working on project 2 made many changes to the code, but also introduced many alerts as defined by their measures of good practices. The Vendor E team working on project 1 made many fewer code changes, but introduced very few issues.
One could assume that more code = more quality issues, which could explain why the project 2 team introduced more quality problems than the project 1 team.
With Semmle, you don’t have to assume
The customer explored that assumption by looking more closely at the data for Vendor E to examine individual contractors and the ratio of their good coding practice problems to lines of code changed (added, modified, deleted).
Finding 2: Location matters
The Semmle visualization above shows the results of the customer’s drill down view of the developers from Vendor E. It showed that:
- Project 2 was staffed by developers from Japan, Spain, New Zealand, and Austria.
- The developers in Japan, produced higher quality code than their co-workers in Spain, albeit, at a slightly higher daily consulting rate.
Finding 3: Higher price doesn’t mean higher quality
The contractors based in Austria produced code that was similar in quality to the contractors in Japan, but at almost twice the daily consulting rate.
Semmle leads to better quality and lower contractor costs
Based on the insights provided by Semmle, the customer optimized contractor cost and productivity by replacing contractors from Spain and Austria with the more moderately priced and quality-minded contract developers from Japan.