We were able to make a retrospective comparison of the performance of the EuResist engine with 10 HIV drug resistance experts’ opinions on a set of 25 cases derived from patients harbouring drug-resistant virus. The Veliparib number of cases was deliberately limited so that it would take a reasonable amount of time for the participants to complete the study. As a cautionary note, it must be taken into account
that the cases were selected from the EIDB rather than from an external source, although these cases have never been used during the development of the EuResist model. Moreover, the EIDB, including data from more than 100 different clinics in four countries, is likely to represent great diversification in drug prescription attitudes and patient populations. Overall, the EuResist engine performed at least as well as the human experts. The lowest number of incorrect calls in the binary classification
of success and failure was in fact made by EuResist and by only one of the experts. To mimic clinical practice, the experts selleck products had access to the entire available patient history, including all CD4 cell counts and viral load measurements, past treatments and HIV-1 genotypes. It should be noted that the current version of EuResist does not include past viraemia levels and only simple surrogate markers of previous drug exposure, less detailed than those made available to the experts, are taken into account. Thus, the experts could consider some extra information over and above that considered by the expert system. However, it could be argued that the experts did not have any familiarity with the patients and the design thus failed to reproduce the real scenario where doctor–patient BCKDHA interaction plays a key role, particularly in assessing patient commitment to therapy. A prospective study comparing standard of care supplemented or not by the EuResist system is required to
evaluate appropriately the potential role of the engine in clinical practice. By design, this study did not allow assessment of whether (and by how much) taking into account the patient and virus data not included in the minimal TCE definition increased the accuracy of the prediction. However, such additional information has been consistently found to increase accuracy in several recent studies using rule-based or data-driven systems [13,18,19]. The correlation between the average quantitative prediction made by the experts and the quantitative prediction computed by EuResist was statistically significant. However, the agreement among the individual experts was rather low, both in the binary classification and in the quantitative score. This highlights the complexity of choosing an antiretroviral treatment in patients harbouring drug-resistant virus which results in frequent discordances in experts’ opinions.