The 2016 QSM Challenge: Lessons learned and considerations for a future challenge design

Abstract

PurposeThe 4th International Workshop on MRI Phase Contrast and QSM (2016, Graz, Austria) hosted the first QSM Challenge. A single?orientation gradient recalled echo acquisition was provided, along with COSMOS and the ?33 STI component as ground truths. The submitted solutions differed more than expected depending on the error metric used for optimization and were generally over?regularized. This raised (unanswered) questions about the ground truths and the metrics utilized.MethodsWe investigated the influence of background field remnants by applying additional filters. We also estimated the anisotropic contributions from the STI tensor to the apparent susceptibility to amend the ?33 ground truth and to investigate the impact on the reconstructions. Lastly, we used forward simulations from the COSMOS reconstruction to investigate the impact noise had on the metric scores.ResultsReconstructions compared against the amended STI ground truth returned lower errors. We show that the background field remnants had a minor impact in the errors. In the absence of inconsistencies, all metrics converged to the same regularization weights, whereas structural similarity index metric was more insensitive to such inconsistencies.ConclusionThere was a mismatch between the provided data and the ground truths due to the presence of unaccounted anisotropic susceptibility contributions and noise. Given the lack of reliable ground truths when using in vivo acquisitions, simulations are suggested for future QSM Challenges.