An Evaluation of the Accuracy, Precision and Consistency of Expert HIV-1 Genotype Interpretation

• The GUESS Study: An International Evaluation of the Accuracy, Precision and Consistency of Expert HIV-1 Genotype Interpretation (ThOrB1385)
  Authored by A.R. Zolopa, L.C. Lazzeroni, D. Kuritzkes, A. Rinehart
  View the original abstract

This group of oral presentations dealt with topics around HIV drug resistance. HIV drug resistance has been reported to be present in upwards of 70 percent of patients who have been treated with antiretroviral medications. The number of acutely HIV infected people with evidence of drug resistant virus, who have not received HIV therapy, is around 10 percent.

Because of this problem, many countries have adopted guidelines for use of HIV drug resistance testing in clinical practice. HIV drug resistance testing comes in three varieties: genotypic testing, which looks at the genetic sequence of the virus for mutations that confer drug resistance; phenotypic testing, which looks at the ability of a patient's virus to grow in the presence of HIV drugs as a measure of resistance; and virtual phenotyping, which combines both by using the genetic sequence of a patient's virus and compares it to a database containing matching phenotypes and genotypes to generate a "virtual" phenotype for a given patient's strain of virus.

However, the interpretation of these tests is complicated and it is unclear as to what level of agreement there is among resistance experts in the interpretation of test results. It is also not clear what patient factors or variables associated with HIV care are important in deciding when to use resistance testing and in which patient groups.

Dr. Andrew Zolopa from Stanford University, CA, U.S. presented results from the GUESS study. This Web-based study asked the question: How well do resistance experts agree on the interpretation of resistance test results? Fifty resistance genotypes (which had evidence of different kinds of HIV drug resistance) for which phenotype data from Virco was available, were used in the study.

Five samples were also blinded duplicates. The genotypes were then given to 12 international HIV resistance experts who were blinded to the phenotype results. They were asked to interpret the genotype report and then estimate what the concomitant resistance phenotype would be, provide an activity score for each drug and state what an appropriate regimen might be for the given reported genotype. The level of phenotypic resistance was expressed as a fold-change (FC, or the amount of increased drug concentration required to inhibit virus) compared to a wild-type virus, which had no evidence of resistance.

The overall results indicate that each individual expert agreed with the true phenotype only 30-40 percent of time. The results were significantly better for 3TC (75 percent) and for non-nucleoside reverse transcriptase inhibitors (NNRTIs, 66 percent), perhaps because a single mutation confers drug resistance and thus makes it easier to interpret resistance. There was a systematic overestimation of nucleoside reverse transcriptase inhibitor (NRTI) phenotypic resistance. There was both over and underestimation of protease inhibitor (PI) phenotypic resistance.

Each expert's results were then combined to determine whether collectively the results would be improved by a consensus evaluation. The results were almost identical to the individual results reported above, meaning that consensus (or pooling the experts results together) opinion did not improve the results. The experts were then asked to rate the activity of each drug (on a score of 0-5, 5 being the highest activity) based on the genotype. The experts were correct an average 45 percent of the time, and were most accurate when scoring the NNRTI class. Again this probably reflects a single mutation causing class resistance, which makes it easier to interpret resistance in this class.

The experts demonstrated much greater agreement when it came to treatment recommendations. The experts agreed more than 80 percent of the time when it came to designing an appropriate treatment regimen based on the genotype and interpreted phenotype. There was greater disagreement when it came to using drugs like d4T, ddI, abacavir and amprenavir -- perhaps because of the greater confusion in the resistance testing community around interpretation of resistance to these drugs.

When the five replicate samples were analyzed, the experts were consistent about their treatment recommendations 90 percent of the time, but were less consistent (50-70 percent agreement between samples) about their activity score and the interpreted phenotype. This study is important because it demonstrates how complicated the interpretation of resistance results are and that experts do not generally agree on how to interpret the results of these tests. This has important implications for general HIV practitioners, who may have even greater difficulty in the interpretation of these tests. Clearly more data and more education are needed so that these expensive tests can be used effectively.