February 27, 2003
VaxGen may have overstated the effectiveness of its AIDS vaccine because it did not make the proper statistical adjustments to its data, an expert consulted by the company said yesterday.
Dr. Steven G. Self, a professor of biostatistics at the University of Washington, said that the company should have lowered the level of confidence with which it said that the vaccine appeared to protect blacks, Asians and other non-Hispanic minorities from infection by HIV.
VaxGen President Dr. Donald P. Francis said there was some debate among statisticians about the proper adjustments but called it a "a tangential issue." Even with the adjustments, he said, the results showing efficacy in minorities would still be statistically significant.
VaxGen reported that its finding for the minority subgroup had a confidence interval of 30.2 percent to 84.2 percent. That meant that there was a 95 percent probability that the actual efficacy of the vaccine in the subgroup was between those points. But Self said that by his calculation the low end of the confidence interval should be just above zero. That is because the company should have lowered its confidence to account for the fact that it analyzed multiple subgroups of patients. If the data are cut enough ways, some effectiveness can almost always be found, he said, so such "penalties" to the interval are taken to guard against false positive conclusions. "It's probably an honest error," Self said. But he said the fact that the lower bound was still above zero still provided "some marginal statistical evidence that there is some efficacy in that subgroup."
New York Times
02.27.03; Andrew Pollack