Urgent change needed in Defense processes for major acquisitions
The failed $1 billion Super Seasprite helicopter project was a low point in Australian government markets. It seems incomprehensible that the Ministry of Defense could ever reproduce it.
Detailing a rapid deterioration in regional security, the 2020 Strategic Defense Update highlights that Australia cannot afford to spend time or money on projects that do not provide combat capability. effectively and efficiently required. The scope and complexity of capabilities required for Australia’s security are increasing, while lead time is decreasing. Supply chain issues are leading to new pressures to design and manufacture more in Australia. The defense must be an effective “savvy buyer”, as envisaged by the first principles reviewand it is essential that decision makers can assess “whether risks and interdependencies have been identified and managed”.
Test and evaluation (T&E) is a key systems engineering tool for identifying risks throughout the capability lifecycle, and Defense has long Strategies describing why it is important and detailing how it should be used in force acquisition, sustainment and generation.
Following several reviews, there have periodically been “new” avenues to establish (or recover) and maintain an effective T&E capability. As the various defense procurement and capability manuals have been updated, a consistent theme has been the vital role of T&E in informing risk-based decisions.
In recommending a smart buyer approach, the first review of principles assumes that Defense can use a T&E process to assess whether risks and interdependencies have been identified and managed. T&E was recently recognized as one of the top 10 Sovereign Defense Industry Capability Priorities.
Given this consistent focus, it should surprise taxpayers that nearly every defense procurement review offers a negative evaluation how Defense deals with T&E. Concerns include difficulties in defining, creating and maintaining an experienced workforce; the lag and build-up of project-by-project experience that makes it difficult to effectively apply T&E early in the capability lifecycle; a lack of coordination between the different entities that are stakeholders in defense T&E (including industry); a lack of coordinated investment in T&E infrastructure; and a lack of accountability to ensure that projects engage T&E and meaningfully consider all subsequent reporting.
The first review of the principles highlighted the need to reinforce and distance a continuous contestability function that works throughout the capability development life cycle, from concept to disposal.
It transferred responsibility for establishing and managing requirements to the Vice Chief of Defense and service chiefs under a strong and independent contestability regime.
For contestability to be effective, the risk identification function must be independent so that the assessment is carried out without bias or influence, intentional or not. Independence also ensures that the risk assessor has a voice—not a veto—heard at every decision-making level in the capability lifecycle. Defense and ultimately the Cabinet Committee on National Security should always make the final risk-based decisions, as they are responsible for providing military response options to the government.
The Force Posture Review recognized that Defense needs to ensure that committed people with the right skills are in the right positions. Competency is a matrix of qualifications and experience relevant to the task at hand. Those who have worked competently as operational commanders or maintenance engineers may not be competent to assess technical performance, integration, and certification risk.
Risk assessors working within Defense face various barriers, individual or organizational, which determine whether their voice is truly heard. The risk must be assessed and the results taken into account by the decision-makers. Given the costs and national security implications, the taxpayer deserves to know this is happening, despite the commercial and security considerations of full transparency. There is already a good model for this. The Office of the Inspector General for Intelligence and Security conducts regular audits of the national intelligence community as well as specific investigations and reports to the relevant minister and the Joint Parliamentary Committee on Intelligence and Security.
Measures to identify and manage risks and interdependencies should be professional and responsive throughout the capability lifecycle. This is also true for ready-made products that can be used by an ally. In the Australian environment, a capability may be the best option to buy, but those who operate and manage it, the government and the taxpayer deserve to know that what is being bought may not be capable of all that one hope for it. This might require additional funding, or additional capacity might be required for certain tasks. This knowledge is important for operational planning, funding forecasting and even reputation management.
The US Congress has legislated for the independent supply of T&E mandatory annual report in Congress on all major defense acquisitions. A UK company, Qinetiq, provides technical support to the UK Ministry of Defence, including T&E. This model provides independence and emphasizes that industry can take the initiative to determine skill and training requirements, provide training, and manage testing ranges and infrastructure.
The strategic defense review should bring a different approach embracing these principles.
It is expected to recommend the establishment of a Defense Capability Assurance Agency (DCAA), an independent statutory body to assess the risks associated with the procurement and sustainment of materiel, which may include technical risk, systems integration risk, force integration risk, contract risk, process risk or even reputational risk. It would be headed by a director appointed by a board of directors and its staff would be made up of qualified and experienced T&E practitioners from defense and industry. The agency would report to the Minister of Defense and to parliament.
The DCAA would be required to assess risks at agreed points throughout the capability life cycle and its recommendations would be included in briefing notes to project managers, assurance agencies, the investment committee defense and the NSC. The agency would not have veto power, but would ensure that risk-based decisions are based on credible information.
The agency should have a long-term agreement with an Australian industry partner to provide deep domain expertise and a consistent global approach to T&E across the life of multiple platforms, environments and systems. The industry partner would not necessarily perform all of the T&E, but at a minimum would oversee the qualifications and professional standards of the workforce performing the T&E.
La Défense already has a good model for this approach: the technical airworthiness system. The Defense Aviation Safety Authority (DASA) provides assurance, through candidate suitability assessment and continuous auditing, that anyone working in this mission and safety critical area has the qualifications and appropriate experience. DASA also provides a range of specialist support services where in-depth domain expertise is required, such as aircraft structural integrity.
The industry partner would be expected to provide a regulatory function analogous to DASA, essentially acting as a DCAA regulator, and there would be a need to maintain an expert workforce with the ability to build, mentor, support and develop the practice T&E in support of defense capability. The industrial partner should also be responsible for the quality and operation of the T&E infrastructure and the effectiveness of training. Subject to an assessment of probity measures, the industry partner may be limited to selecting, auditing and managing contracted training providers or may also provide a range of in-house T&E training for industry stakeholders. defense and industry.
Scale and flexibility would be achieved if the DCAA relied on defense personnel or industry vendors with the appropriate qualifications and experience. This expertise in the T&E domain would be complemented by the recent operational experience of operators or engineers from the defense forces with relevant T&E training integrated into the DCAA. They could also verify compliance with qualifications and experience requirements for defense or civilian T&E personnel to comply with operational regulatory requirements.
Defense should be audited to ensure proper DCAA engagement and transparency in subsequent reporting of identified risks. This should be conducted by a small team with security clearances and subject to safeguards for confidential business information. He would work as part of an independent insurance office.
Individuals and units within Defense are professional and competent technical risk assessors, but the organizational inability of Defense to generate, maintain, perform tasks consistently and respond transparently to credible risk assessment in supply and sustainment is well known.
Defense Strategic Review should not seek to launch another review of T&E within Defence. The strategic imperative for Defense to be a smart buyer now should compel the government to initiate legislative reform to establish a capability assurance agency.