Neural Machine Translation Based Automatic CPT Classification System Using Surgical Procedure Text: Development and Validation Study (Preprint)

BACKGROUND Administrative costs for billing and insurance-related activities in the U.S. are substantial. One critical cause of the high overhead of administrative costs is medical billing errors. With advanced deep learning techniques, developing advanced models to predict hospital and professional billing codes becomes feasible. These models could be used for administrative cost reduction and billing process improvements. OBJECTIVE We have developed an automated anesthesiology Current Procedural Terminology (CPT) prediction system that translates manually entered surgical procedure text into standard forms using neural machine translation (NMT) techniques. The standard forms are calculated into similarity scores to predict the most appropriate CPT codes. While this system aims to enhance medical billing coding accuracy to reduce administrative costs, we compare its performance to that of machine learning algorithms previously developed. METHODS We collected and analyzed all operative procedures performed at Michigan Medicine between January 2017 and June 2019 (2.5 years). The first two years of data were used to train and validate existing models and compare the results from the NMT-based model. Data from the year 2019 (six-month follow-up period) was then used to measure the accuracy of CPT code prediction. Three experimental settings were designed with different data types to evaluate models. Experiment 1 uses the surgical procedure text entered manually in EHR. Experiment 2 uses preprocessing of the procedure text. Experiment 3 uses preprocessing of combined procedure text and preoperative diagnoses. The NMT-based model is compared with the SVM and LSTM models. RESULTS The NMT-based CPT prediction model compares favorably with SVM and LSTM models. The NMT model yielded the highest Top-1 accuracy on Experiment 1 and Experiment 2 at 81.64% and 81.71%, compared to the SVM model (81.19% and 81.27%) and the LSTM model (80.96% and 81.07%). The SVM model yielded the highest Top-1 accuracy at 84.30% on Experiment 3, followed by LSTM (83.70%) and NMT (82.80%). This Experiment 3 adding preoperative diagnoses shows 3.7%, 3.2%, and 1.3% increase of Top-1 accuracy over SVM, LSTM, and NMT in Experiment 2. For Top-3 accuracy, the SVM, LSTM, and NMT models achieved 95.64%, 95.72%, and 95.60% for Experiment 1, 95.75%, 95.67%, and 95.69% for Experiment 2, and 95.88%, 95.93%, and 95.06% for Experiment 3. CONCLUSIONS This study demonstrates the feasibility of creating an automated anesthesiology CPT classification system based on NMT techniques using surgical procedure text and preoperative diagnosis. Our results show that the performance of the NMT-based CPT prediction system is equivalent to SVM and LSTM prediction models. Importantly, we found that including preoperative diagnoses improved accuracy over using procedure text alone. CLINICALTRIAL Not applicable