Benchmarking in High-Security Environments

Benchmarking is certainly important (Goetsch & Davis, 2021), but making it happen in a sector where secrets must be kept involves “creative” solutions, or only making extremely broad comparisons that do not have value to opponents. For example, Gebicke & Magid’s (2010) global study doesn’t compare specific defense systems, but they do compare force size, tooth-to-tail ratio, and so on.

Another type of comparison that doesn’t involve sharing secretive information is between military education institutions. For example, V. Kravets (2024) compares Ukraine’s higher military educational instructions, but instead of comparing technical proficiencies in military science, her goal is to determine the feasibility of including management activities into those institutions. Aren't things going bad enough for Ukraine?

Because of the need to avoid classified information becoming public, the benchmarking that could be used by companies like Rheinmetall AG against BAE Systems or General Dynamics is fraught with difficulties that aren’t encountered in civilian industries. For example, in any company that has IT infrastructure (which means all companies), benchmarking various IT components (like databases or servers) is possible because different industries use the same IT components, and so the benchmarking partners need not be competitors. For example, Google’s Gmail and the fictitious Gaggle dot Com’s GaggleMail are competitors, so no mission-critical information should pass between them. Gaggle dot Com is not a competitor of X (formerly Twitter), so it is OK to benchmark their databases, for example. And this database benchmarking can involve direct comparison of databases made by the same company (like Microsoft) or comparisons of databases made by different companies (Microsoft vs Oracle).

It's not clear whether the same benchmarking would happen in the IT departments of artillery manufacturers since there are all sorts of proprietary IT components. But one can benchmark various systemic quality measures like lean or six sigma standards against other companies, without giving away classified information.

In a 1999 paper by Yarrow & Prabhu, three different modes of benchmarking are presented: metric benchmarking, diagnostic benchmarking, and process benchmarking. Metric benchmarking is the comparison of “apples with apples” performance data. Process benchmarking “involves two or more organizations comparing their practices in a specific area of activity, in depth, to learn how better results can be achieved.” And diagnostic benchmarking “seeks to explore both practices and performance, establishing not only which of the company’s results areas are relatively weak, but also which practices exhibit room for improvement.”

I would like to make guesses of the types of benchmarking done at companies like Rheinmetall AG, without knowing anything about artillery manufacturing! Metric benchmarking could be done on an IT component level (everybody uses databases), but exact benchmarking may be precluded because proprietary software is used. In that case, process benchmarking would still be possible. Diagnostic benchmarking seems to best describe comparisons of six sigma measurements. But I don’t know anything about six sigma, either!

For metric comparisons, then, companies like Rheinmetall AG must enter into consortium agreements with other defense manufacturers as you stated. It isn’t clear how security would be maintained, even if the data is anonymized. In the absence of consortium agreements, one would have to look to different industries. For example, for information about specific tolerances, one would have to compare data from, say, civilian pipe manufacturers. This is probably a one-way transfer of information.

References

Gebicke, S. & Magid, S. (2010). Lessons from around the world: Benchmarking performance in defense. McKinsey & Company. https://www.mckinsey.com/~/media/mckinsey/dotcom/client_service/public%20sector/pdfs/mck%20on%20govt/defense/mog_benchmarking_v9.pdf

Goetsch, D. L. & Davis, S. B. (2021). Quality management for organizational excellence: Introduction to total quality (9^th ed.). Pearson.

Kravets, V. (2024). Development strategies for higher military educational institutions of Ukraine: analysis based on benchmarking. Честь і закон, 2(89), 74-82. https://chiz.nangu.edu.ua/article/download/309198/300732/714412

Yarrow, D. & Prabhu, V. (1999). Collaborating to compete: Benchmarking through regional partnerships. Total Quality Management, 10(4-5), 793-802. https://doi.org/10.1080/0954412997820