Quality Accreditation and Data Validation
Following the introduction of the Quality Accreditation Programme, by Sam Nashef in 2000, there were 7 units which were accredited and certificated under this scheme. There was subsequently a lull in interest until discussions about public disclosure of unit and surgeon specific outcomes led to regular audit leads meetings and a further 7 units being visited. The purpose of these latter visits was to benchmark systems for data collection and validation, and to emphasise deficiencies in units to help obtaining resources for infrastructure - aspects of governance were left to individual units. Data Quality visit reports are available on the Society home page along with a model for best practice in data validation. It was originally anticipated by the Society that all units would be visited.
With time, however, it seems there is more understanding and familiarity with the process of cardiac surgical data management. Data submission to CCAD, and its subsequent presentation on a public portal, emphasise the importance of completeness, accuracy and validation of data. There’s no doubt that publication of outcomes on the Health Commission website is a potent driver to improving systems and thereby data quality. On the SCTS home page there is a model for best practice in data validation under quality accreditation.
A mechanism for undertaking Data Validation visits still exists and I would be pleased to hear from any units which feel they would like to pursue this.
Mark Jones
QUALITY ACCREDITATION PROGRAMME - BEST PRACTICE FOR CENTRAL CARDIAC AUDIT DATABASE (CCAD) DATA VALIDATION
The majority of cardiac surgical units are now routinely submitting data to CCAD. The publication of surgeon specific outcome data on the Health Commission Public Portal emphasises the importance of completeness, accuracy, and validation of this data.
The purpose of data validation visits is to review the local processes for data collection, collation, analysis and feedback including the quality assurance of all these systems.
Through the pilot visits (reports can be found on the Health Commission website) which have been undertaken by the Quality Accreditation Panel it has been possible to distil out the elements that appear to make the most robust systems.
The purpose of this document is to describe this “best practice” model in the hope that it may be of value to all cardiac surgical units.
Personnel
Minimal core personnel requirement for an effective audit structure consists of a Consultant Surgeon Clinical Audit Lead, supported by a Surgical Database manger, and audit clerk (s).
This grouping should interface closely with the Trust IT and Information department, and Clinical Governance structure.
Currently this will most likely sit in parallel with or be integral with the equivalent cardiology system.
It should be housed / located in a dedicated physical area geographically close to the main area of cardiac surgical activity. This promotes an environment conducive to communication, shared initiatives/ideas, common ownership of the data and audit process. The proximity of these areas of activity and relationships between key personnel should further promote data quality.
Software System and Network
The data collection system is likely to be supported by a commercially available cardiac surgery/cardiology software package (e.g. Dendrite, DataCamm, Tomcat) or a bespoke/ in house system.
There should be an easily managed interface with the Trust PAS system for import of patient demographics.
It is most likely to be PC based, possibly on the web, although some units still have paper based data entry systems. If paper based systems are used then consideration should be made to the potential for transcription errors
The essentials are widespread availability of PCs for data entry, at multiple stages of the patient journey.
Overview of Process
Underpinning any successful validation system is a statement of the strategy and principles for the management of a units data.
There should be a clearly understood and documented structure to the process of data entry, with clarity of the timing of different stages of data entry and transparency about individual responsibilities of members of the surgical and audit teams.
The system should include a clear description of the systems for X checks of data entry, data validation, reporting and feedback to consultant units, and the governance systems in place.
In general the SCTS minimum data set should be entered prior to patient admission, e.g. at the pre op clinic by a junior doctor or clinical nurse specialist. Incomplete data entry, especially of required data fields should be flagged up appropriately.
Inappropriate numerical entry should be prevented by some means of internal validation e.g. incorrect height / weight, sequential admission date and discharge date, elective surgery with IABP etc. A risk score should be automatically generated.
Entry of operative data should produce an operation note which can enable easy checking of the inputted variables and accompany the patient to the ITU.
Fully enabled systems with entry of discharge data and medication (with validation systems for drug doses/ interactions etc) allows a computer generated discharge summary, saving secretarial time and provides a contemporaneous discharge record.
A robust audit trail should exist for each stage of data entry including identity of entrant, nature of entry, time of entry and any change that has taken place. Hospital IT should ensure system security and back up.
Consideration should be given to a means of “locking” data after the completion (and validation) of the patient record.
Data Collection processes and Cross-Checks
In an ideal world all data collected on patients undergoing surgery would be subject to an independent correction and validation – few units have this capability.
The most important data relates to cardiac surgical activity, surgeon identifier, case mix, and mortality and there should be absolutely robust systems for checking these. Records with which to compare and check this data will exist in various forms - existing paper database systems, parallel local IT systems, and HES data
Possible sources to check against are for example
1. Operating Theatre lists, theatre logbooks, considering emergencies not on
elective theatre lists, and allowing for cancelled patients
2. CICU logbook for activity, deaths.
3. Cardiac Surgery ward records for discharges, transfers and deaths
4. Hospital mortality database
5. Contracted/ commissioned surgical activity
6. Perfusion records
7. Regular departmental audit/review meeting X checked for activity and deaths.
These checks may be manual, or facilitated through X check features of relational databases, or other software e.g. Access.
Regular feedback is key to improving the accuracy and validation of the cardiac surgical database record from these independent sources.
It is essential that there is a regular report, at least monthly, circulated to unit consultants for individual validation and feedback. This should form part of the system for data quality monitoring and also feed into unit and hospital systems of quality assurance and governance, according to preset levels of minimum performance. This can be basis of the regular unit review/ audit programme, and is an important way of accurately presenting unit activity and outcome data on the website.
There should be a regular in house notes validation exercise whereby a sample of notes is independently checked against the data base record.
CCAD
Data should be regularly downloaded to CCAD. The CCAD software runs a series of logic checks looking for flaws in dates of admission, discharge and operation or other issues. It will also highlight missing data.
For each unit, CCAD gives a table of discrepancies between ONS and tracked mortality data, the completeness of data for core variables, completeness of Euroscore fields, incidence of risk factors, and a data quality index. There is a comparison with equivalent “pooled” national data – this is a useful benchmark for individual units.
As important as submission of data to CCAD is the regular review of unit data on the CCAD database. This should be fed back to the contributing unit as a further check for accuracy and to promote confidence in the system.
CCAD for example gives the capacity to drill down to specific patient records to help validation processes, it has the ability to use ONS tracking to validate mortality, and illustrate long term survival. Audit leads and data managers and possibly all unit consultants should have access to CCAD software, this may further help units to improve the quality of their data.
Summary
The key to successful cardiac surgical data validation and management lies with the obsession, rigour and enthusiasm of the key members of the team. This leads to a culture of responsibility, and ownership of data and a corresponding pride in your unit’s outcomes.
This, of course, requires manpower and physical resource. Hopefully the data validation visits have been helpful in supporting the case for this investment.
Mark Jones
Lead SCTS Quality Accreditation Programme
25/2/2007