Assessment of the 2021 WHO Mycobacterium tuberculosis drug resistance mutation catalogue on an independent dataset

We applaud the recent work from Walker and colleagues in constructing a knowledge base of mutations associated with Mycobacterium tuberculosis complex drug resistance.1Walker TM Miotto P Köser CU et al.The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: a genotypic analysis.Lancet Microbe. 2022; 3: e265-e273Summary Full Text Full Text PDF PubMed Scopus (7) Google Scholar The rigorous classification criteria make it an excellent foundation for other, more extensive catalogues and provide confidence in each mutation's grading stability. We anticipate that this catalogue will be rapidly and widely adopted for drug susceptibility testing. However, one of the limitations acknowledged by the authors is that "as no independent dataset was available to test the catalogue, sensitivity and specificity were assessed by predicting phenotypic resistance from the same data from which the catalogue was derived".1Walker TM Miotto P Köser CU et al.The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: a genotypic analysis.Lancet Microbe. 2022; 3: e265-e273Summary Full Text Full Text PDF PubMed Scopus (7) Google Scholar The main aim of this Correspondence is to assess the sensitivity and specificity of the 2021 WHO catalogue on an independent dataset. A secondary aim is to evaluate the benefits of combining this catalogue with an existing catalogue.We curated a dataset of 8321 M tuberculosis complex isolates with associated phenotypic profiles, which do not overlap with the isolates reported by Walker and colleagues (appendix p 2). We used Mykrobe (version 0·11·0)2Hunt M Bradley P Lapierre SG et al.Antibiotic resistance prediction for Mycobacterium tuberculosis from genome sequence data with Mykrobe.Wellcome Open Res. 2019; 4: 191Crossref PubMed Scopus (39) Google Scholar to obtain drug susceptibility profiles by genotype for these isolates, using all of the mutations in the WHO catalogue with a confidence grading of 1 or 2, plus their expert rules. Finally, we compared our results to those reported by Walker and colleagues on the training set (appendix p 6).We confirmed that the reported sensitivity and specificity results for amikacin, capreomycin, and moxifloxacin were equivalent to or better than those reported by Walker and colleagues (appendix pp 5–6). However, we found significantly lower sensitivities than reported for linezolid (38·2% lower), isoniazid (9·2%), streptomycin (8·7%), ethambutol (7·9%), pyrazinamide (5·3%), and rifampicin (2·5%; appendix pp 6, 8), indicating overfitting for these drugs. We also found significantly lower specificity than reported for levofloxacin (52·6% lower), ethionamide (8·8%), ethambutol (6·5%), streptomycin (4·8%), pyrazinamide (0·9%), isoniazid (0·8%), and rifampicin (0·6%). Lastly, we investigated the benefits of combining the WHO catalogue with an existing one, using the default catalogue provided by Mykrobe. For most drugs, combining the catalogues leads to improved sensitivity but decreased specificity. However, no drug had a statistically significant change when comparing the combined catalogue with both the WHO and Mykrobe catalogues alone (appendix pp 5-6, 8).In conclusion, we thank Walker and colleagues for their fantastic contribution to the progression of M tuberculosis complex diagnostics. We hope others take our independent assessment into account when deciding on the utility of the WHO catalogue.This online publication has been corrected. The corrected version first appeared at www.thelancet.com/microbe on July 11, 2022 We applaud the recent work from Walker and colleagues in constructing a knowledge base of mutations associated with Mycobacterium tuberculosis complex drug resistance.1Walker TM Miotto P Köser CU et al.The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: a genotypic analysis.Lancet Microbe. 2022; 3: e265-e273Summary Full Text Full Text PDF PubMed Scopus (7) Google Scholar The rigorous classification criteria make it an excellent foundation for other, more extensive catalogues and provide confidence in each mutation's grading stability. We anticipate that this catalogue will be rapidly and widely adopted for drug susceptibility testing. However, one of the limitations acknowledged by the authors is that "as no independent dataset was available to test the catalogue, sensitivity and specificity were assessed by predicting phenotypic resistance from the same data from which the catalogue was derived".1Walker TM Miotto P Köser CU et al.The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: a genotypic analysis.Lancet Microbe. 2022; 3: e265-e273Summary Full Text Full Text PDF PubMed Scopus (7) Google Scholar The main aim of this Correspondence is to assess the sensitivity and specificity of the 2021 WHO catalogue on an independent dataset. A secondary aim is to evaluate the benefits of combining this catalogue with an existing catalogue. We curated a dataset of 8321 M tuberculosis complex isolates with associated phenotypic profiles, which do not overlap with the isolates reported by Walker and colleagues (appendix p 2). We used Mykrobe (version 0·11·0)2Hunt M Bradley P Lapierre SG et al.Antibiotic resistance prediction for Mycobacterium tuberculosis from genome sequence data with Mykrobe.Wellcome Open Res. 2019; 4: 191Crossref PubMed Scopus (39) Google Scholar to obtain drug susceptibility profiles by genotype for these isolates, using all of the mutations in the WHO catalogue with a confidence grading of 1 or 2, plus their expert rules. Finally, we compared our results to those reported by Walker and colleagues on the training set (appendix p 6). We confirmed that the reported sensitivity and specificity results for amikacin, capreomycin, and moxifloxacin were equivalent to or better than those reported by Walker and colleagues (appendix pp 5–6). However, we found significantly lower sensitivities than reported for linezolid (38·2% lower), isoniazid (9·2%), streptomycin (8·7%), ethambutol (7·9%), pyrazinamide (5·3%), and rifampicin (2·5%; appendix pp 6, 8), indicating overfitting for these drugs. We also found significantly lower specificity than reported for levofloxacin (52·6% lower), ethionamide (8·8%), ethambutol (6·5%), streptomycin (4·8%), pyrazinamide (0·9%), isoniazid (0·8%), and rifampicin (0·6%). Lastly, we investigated the benefits of combining the WHO catalogue with an existing one, using the default catalogue provided by Mykrobe. For most drugs, combining the catalogues leads to improved sensitivity but decreased specificity. However, no drug had a statistically significant change when comparing the combined catalogue with both the WHO and Mykrobe catalogues alone (appendix pp 5-6, 8). In conclusion, we thank Walker and colleagues for their fantastic contribution to the progression of M tuberculosis complex diagnostics. We hope others take our independent assessment into account when deciding on the utility of the WHO catalogue. This online publication has been corrected. The corrected version first appeared at www.thelancet.com/microbe on July 11, 2022 This online publication has been corrected. The corrected version first appeared at www.thelancet.com/microbe on July 11, 2022 This online publication has been corrected. The corrected version first appeared at www.thelancet.com/microbe on July 11, 2022 All authors declare no competing interests. Supplementary Material Download .pdf (1.05 MB) Help with pdf files Supplementary appendix Download .pdf (1.05 MB) Help with pdf files Supplementary appendix The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: a genotypic analysisWe present the first WHO-endorsed catalogue of molecular targets for MTBC drug susceptibility testing, which is intended to provide a global standard for resistance interpretation. The existence of this catalogue should encourage the implementation of molecular diagnostics by national tuberculosis programmes. Full-Text PDF Open AccessCorrection to Lancet Microbe 2022; published online May 31. https://doi.org/10.1016/S2666-5247(22)00151-3Hall MB, Coin LJM. Assessment of the 2021 WHO Mycobacterium tuberculosis drug resistance mutation catalogue on an independent dataset. Lancet Microbe 2022; published online May 31. https://doi.org/10.1016/S2666-5247(22)00151-3—In this Correspondence, the data have been updated to reflect the expert rules applied in the linked Article. This correction has been made as of July 11, 2022. Full-Text PDF Open Access