PDF Version

ADF Health April 2006 - Volume 7 Number 1

Viewpoint

Surgical audit on military operations: an evolving necessity

• Susan J Neuhaus and Jeffrey V Rosenfeld

Since the Vietnam War, Australian surgeons have been involved in an increasing number of deployments in support of war and peacekeeping/counterinsurgency activities. 1 They have made a valuable contribution to the health of Australian military personnel, and provided significant humanitarian assistance. However, surgical data capture on Australian military operations has been poor, with significant deficiencies in record-keeping. 1 This has created difficulty in developing a true picture of clinical surgical practice on operations. Frequently, surgical data come from widely disparate sources, all of which are retrospective and unique to the particular deployment. Many surgeons keep their own logbooks, but it is difficult to integrate these data in a meaningful or consistent way. Official surgical registers are limited in detail about the specifics of the operative procedure, sometimes inaccurate, and have no information on outcome.

Many of the surgical lessons of deployment are lost. Reporting of surgical workloads and analysis of lessons learned is often limited and rarely linked with wider health reporting processes. Australian Defence Force surgeons work in unique settings with implications for surgical training, administration and logistic support. It is important that ADF surgeons can demonstrate a commitment to ongoing improvement in surgical care and developments in "best clinical practice".

This article aims to overview the process of clinical surgical audit. We focus on the application of surgical audit to the operational health environment, and propose incorporation of surgical audit systems into all ADF deployments.

What is surgical audit?

Surgical audit is the systematic, critical analysis of the quality of surgical care provided. Audit involves:

• collection of clinical activity data and outcomes;
• analysis using performance indicators, accepted standards and outcome measures; and
• a confidential peer review process with feedback loops to redress any problems identified.

The key aims of audit are to:

• identify ways of improving and maintaining the quality of care for patients;
• assist in continuing education of surgeons;
• help guide appropriate use of health resources; and
• identify systems errors and thereby improve patient care.

Why consider surgical audit on operations?

Surgical audit is a normal part of surgical practice and a requirement of the Royal Australasian College of Surgeons (RACS). 2 Surgeons are expected to undertake or be involved in regular audit and peer review of their individual and department work. ADF health providers should not be exempt from this process.

Although the purpose of surgical audit is to improve standards in clinical surgical care, it has several other advantages.

Accuracy of data collection. Accurate data provide an important measure of workload and workload fluctuations. Information on patterns of injury encountered, proportion of Australian service personnel compared with local populations treated, and resource use (eg, bed-days) is important to health planners.

Opportunities for clinical learning and training needs analysis. Information about what surgery is performed can be used to guide surgical training, including specific training in subspecialty skills that may be required on operations.

Clinical outcome indicators. Clinical outcome measures should be comparable to accepted norms or benchmarks. These include measures of complications such as postoperative infection, unplanned return to theatre, and mortality. Historical benchmarks for most clinical indicators exist, but must be interpreted within the context of evolving conflict, developments in surgical care, and the austerity of the operating environment. 3 Outcomes should be assessed against standardised preoperative indicators, for example, injury severity scores or the American Society of Anesthesiologists’ assessment of preoperative physical status. The health of local populations may not reflect that normally seen in Australia, and this also needs to be considered.

Development of surgical protocols. Audit information can be used to develop new practice guidelines or define standards (eg, changes in antibiotic use, optimum time for "re-look" laparotomy).

Measures of operational effectiveness. These include not just numbers of patients treated, but length of stay and patient transfers to local hospitals. Increasingly, elective surgery is performed on longer overseas deployments. It is imperative that the ADF is proactive in demonstrating a willingness to undertake peer review and maintain the highest quality of surgical care in this setting.

Opportunities to identify and correct problems. Problems are inevitable in an operational setting. Audit allows these problems to be identified and measures taken to avoid them on future operations. Examples of correctable problems include equipment shortfalls and specialist skills deficits. However, any comparison with Australian civilian health system data needs to be tempered with an appreciation of the operational situation and the skills of the generalist performing the duties of a specialist surgeon in some circumstances. 4

Research. The purpose of audit is not research. However, the data collected can provide impetus for related research projects.

Audit is about making changes, not just collecting data. Operational surgical audit also provides an opportunity to disseminate the lessons learned to the wider military surgical community. It is imperative that audit information is subject to peer review. The appropriate peer review group is the Military Surgery Consultative Group. Peer review is a confidential, nonconfrontational, constructive process designed to "close the feedback loop". Feedback via the military module of the Definitive Surgical Trauma Care course or alternative educational sessions enables military surgeons to benefit from others’ experience. 4 It also encourages learning and may identify new patterns of injury or presentation, for example, with improvised explosive devices.

Specific considerations during "humanitarian assistance" operations

Increasingly, the ADF assists in stability and reconstruction operations. An important component of these operations is often provision of health support to local populations. International consensus standards for minimum performance in delivery of health care 5 have been designed to improve effectiveness of health care delivery in these situations and minimise inconsistencies between various providers (ie, military and humanitarian agencies). These standards, such as NAOMI indicators and SPHERE guidelines, 6 while still in their infancy, have helped clarify what constitutes "acceptable" practice. In future operations, the ADF will be expected to meet these guidelines and demonstrate transparency in its clinical practice.

Resource constraints are an important element of any military operation. The true cost of surgical services provided on operations is rarely calculable. The automated inclusion of, for example, Medicare item numbers in the audit program would enable indications of "cost value" of services to be estimated. These descriptors are well known to Australian surgeons and are incorporated into the current International Classification of Diseases (ICD) coding systems used by the World Health Organization. This may be particularly relevant on such stability and reconstruction operations, where cost recovery may be sought by the ADF from another governmental agency such as the Department of Foreign Affairs and Trade.

Methods of audit

The two major methods of audit information collection are traditional logbooks and computer-based audit programs.

Traditional operating logs have the advantage of simplicity, but their use is outweighed by their disadvantages, including inconsistent, non-standardised data entry, ability to be lost, reliance on allied health personnel to complete, and lack of operative data.

Computer-based audits are now widely used. 7 The most commonly encountered surgical audit systems are the Otago audit, Liverpool hospital audit and the audit program developed by Professor David Watters from the Department of Surgery, University of Melbourne at the Geelong Hospital, VIC. 8 Computer-based audit programs have a number of advantages:

• retrievable, storable data available for analysis;
• ability to interface with small personal digital assistant (PDA) units as part of a networked system 9 (Box 1);
• capability to transmit data back to Australia for data storage and collection;
• potential compatibility with current RACS audit programs;
• standardised data entry with use of preset "drop down" menus (Box 2);
• ease of de-identification (eg, as part of reports and returns); and
• ability to integrate with other aspects of care (eg, can be used to print hard copies of operative record or treatment summary to accompany a patient who is evacuated or transferred).

What are the requirements of a military audit program?

Any audit program used by military surgeons must meet the requirements of simplicity and operational robustness. It should also be compatible with RACS audit programs, enabling individual surgeons to integrate their own data into their personal audit. Data fields should be based on a modified minimum audit dataset (Box 3) and incorporate only data which are accurate and relevant. A good audit program can also form a case record, operation note and follow-up instructions. Simplicity and minimisation of data entry time can be enhanced by the use of drop down fields. This also assists in providing consistent entries.

Data entry should be the responsibility of the operating surgeon. Under times of intense workload, the audit can be completed later, provided the patient has been logged into the system. This is a useful strategy to ensure that patient information is accurate and not overlooked (a comparison exists with the tertiary trauma survey).

One major challenge with all audit systems is the reliability and completeness of the data entered. This requires the surgeons and staff to be committed to the process, which is not always easy to achieve. An audit officer should be appointed to assist with data entry and ensure compliance and completeness of the data set. A nurse clinician who has an interest in quality assurance and audit would be ideally suited to this role. The US Air Force has instituted this type of nursing officer appointment at the 332nd Expeditionary Medical Group Field Hospital in Iraq. Similarly, the British Army has embraced a clinical governance appointment for a senior nurse on operations. Having a weekly audit and clinical review/mortality and morbidity meeting in a military hospital setting should also be part of the review and oversight process. Data generated from this meeting feed directly into the computer-based audit.

Any audit system introduced on ADF operations should be simple to use and easy to learn for those with limited computer skills. "Off the shelf" computerised audit systems are available and could be modified to meet the needs of the ADF. However, there are potential problems with imported systems. Audit imperatives differ widely within and between disciplines and countries. These systems often contain mandatory fields, some of which may not be relevant in an Australian military setting. For this reason, it may be better for the ADF to develop its own database, specifically tailored to meet the needs of surgery in a military environment.

Conclusions

Surgical audit on military operations provides many potential benefits to the ADF. The most important of these is the opportunity for Australian military surgeons to promote a culture of clinical excellence and ongoing improvement.

We recommend that:

• Surgical audit should be considered a necessity on all ADF deployments.
• A suitable surgical audit program should be developed by the ADF in association with the RACS, the Military Surgery Consultative Group, and the Centre for Military and Veterans’ Health (University of Queensland).

Surgical audit and peer review are a part of routine surgical practice. The development of operational surgical audit would demonstrate the ADF’s commitment to ongoing improvement of standards of care provided to, and by, Australian servicemen and women.

Acknowlegements

We wish to acknowledge the assistance of Professor David Watters in the preparation of this paper and his permission to use images of the electronic audit system.

Competing interests

None identified.

References

1. Neuhaus SJ. Post Vietnam - three decades of Australian military surgery. ADF Health 2004; 5: 16-21.
2. Surgical audit and peer review. A guide by the Royal Australasian College of Surgeons. 2nd ed. Sydney: RACS, 2005.
3. Watters DA. Quality assurance in surgery: surgical audit in the developing world. P N G Med J 1993; 36: 120-125.
4. Rosenfeld J. Training the military surgeon: Definitive Surgical Trauma Course (DSTC) and the development of a military module. ADF Health 2002; 3: 68-70.
5. Driffmeyer J. Toward more effective humanitarian assistance. Mil Med 2004; 169: 161-168.
6. The SPHERE Project. Humanitarian charter and minimum standards in disaster response handbook. 4th ed. London: Oxfam Publishing, 2004. Available at: http://www.sphereproject.org (accessed Feb 2006).
7. Aitken RJ, Nixon SJ, Ruckley CV. Lothian surgical audit: a 15-year experience of improvement in surgical practice through regional computerised audit. Lancet 1997; 350: 800-804. Erratum in Lancet 1998; 351: 220.
8. Watters DA. Surgical audit tool. Available at: http://www.surgeons.org/ Content/NavigationMenu/FellowshipandStandards/ProfessionalStandards/ CPDresourcesandtools/David_Watters_Surgic.htm (accessed Feb 2006).
9. Watters DA, Morris O, Cotton J. Surgical audit on a PDA using eLogbook. Aust N Z J Surg 2005; 75(S1): A100.

(Received 6 Jul 2005, accepted 20 Jan 2006)

Lieutenant Colonel Susan Neuhaus enlisted in the Australian Regular Army as an undergraduate medical officer and has served in Cambodia as Regimental Medical Officer, Force Communications Unit, UNTAC, and in Bougainville as Officer Commanding, Combined Health Element, Peace Monitoring Group. She is a graduate of Australian Command and Staff College (Reserve) and is currently posted as a Reserve Officer to Defence Health Services Canberra.

Colonel Jeffrey Rosenfeld joined the Army Reserve in 1984 and is currently Chair of the General Surgery Consultative Group. He is the Professor of Surgery and Professor/Director of Neurosurgery at the Alfred Hospital and Monash University. He has served on seven operational deployments to Rwanda, East Timor, Bougainville, the Solomon Islands and Iraq. He has helped to develop and instructed on the Definitive Surgery Trauma Course and the Military Module. He was awarded the Geoffrey Harkness Medal of the Royal Australian Army Medical Corps in 2001 and has a strong research interest in traumatic brain injury.

Defence Health Services, Campbell Park, ACT.

Susan J Neuhaus, PhD, FRACS, Consultant General Surgeon.

Alfred Hospital, Prahran, VIC.

Jeffrey V Rosenfeld, MS, FRACS, Chairman, ADF Surgical Consultative Group.

Correspondence: Lieutenant Colonel S J Neuhaus, 15 Roberts Street, Unley, SA 5061.