We evaluated the accuracy of National Weather Service (NWS) Fire Weather mixing height estimates over the western United States corresponding to smoke plume events from 2006-2020. We developed a toolbox for measuring smoke-related aerosol mixing heights using Cloud-Aerosol LiDAR and Infrared Pathfinder Satellite Observations (CALIPSO) Vertical Feature Masks and Moderate Resolution Imaging Spectrometer (MODIS) atmospheric profiles. Similar to previous research, we found mixing height difficult to reliably estimate and verify (Fearon et al. 2015; Nauslar et al. 2016). While we did not find strong evidence supporting systematic biases in NWS estimates, we found dramatic differences between NWS forecasts and the mixing heights determined from CALIPSO Vertical Feature Mask (VFM) data. The disagreement between these two methods was especially evident at low mixing heights, which may be when accurate forecasts are most important given that these prognostications constrain authorization of prescribed burns. We found that, compared with CALIPSO VFM data, NWS tended to overestimate at lower mixing heights (<2.5 km) and underestimate at higher mixing heights (>3.0 km). Overall, the forecasts tended to underestimate mixing height, with a median relative error of -13.0% and a mean relative error of -3.34%. This underestimation is consistent with the results of Fearon et al 2015, which found that the Stull method underlying the forecasts also tended to underestimate mixing height. The mixing heights from MODIS atmospheric profiles generally cohered with NWS forecasts below 3 km, though no MODIS estimates were generated above 3 km for the smoke plume events in this study. Furthermore, the MODIS-derived mixing heights are limited to the discrete pressure levels in the atmospheric profiles, which have a vertical resolution of ~0.5 km.