Background: Fire simulators are increasingly used to predict fire spread. Australian fire agencies have been concerned at not having an objective basis to choose simulators for this purpose.

Aims: We evaluated wildland fire simulators currently used in Australia: Australis, Phoenix, Prometheus and Spark. The evaluation results are outlined here, together with the evaluation framework.

Methods: Spatial metrics and visual aids were designed in consultation with simulator end-users to assess simulator performance. Simulations were compared against observations of fire progression data from 10 Australian historical fire case studies. For each case, baseline simulations were produced using as inputs fire ignition and fuel data together with gridded weather forecasts available at the time of the fire. Perturbed simulations supplemented baseline simulations to explore simulator sensitivity to input uncertainty.

Key results: Each simulator showed strengths and weaknesses. Some simulators displayed greater sensitivity to different parameters under certain conditions.

Conclusions: No simulator was clearly superior to others. The evaluation framework developed can facilitate future assessment of Australian fire simulators.

Implications: Collection of fire behaviour observations for routine simulator evaluation using this framework would benefit future simulator development.