There is now a well-justified interest in rapid introduction of modern therapeutics for HIV-infected patients in resource-limited regions of the world. One of the major issues with this new effort concerns the adequacy of the medical infrastructure and the ability of the treatment force to supervise care. Based on documented benefit in the Western world -- measured in terms of reduced mortality, frequency of hospitalization, frequency of AIDS-defining diagnoses, cost of care, and adherence to guidelines -- we know that expertise is an important factor for successful HIV care. In fact, there is for HIV/AIDS a better-documented correlation between expertise and outcome than for any other disease. It is also clear that medical care historically has been "local" and must be done in a fashion in which local healthcare workers are in charge. Thus, it is imperative that local healthcare workers are empowered with HIV expertise.

Competencies in medical care include: patient care experience, medical knowledge, ability to transform scientific evidence to clinical decisions, communication skills, and professionalism. In general, these attributes include experience in clinical care, the ability to translate scientific observations into clinical decisions, and a record of high professional standards.

Equally important is performance on a written examination of medical knowledge and judgment. This is true for every specialty of medicine, and so it is with HIV care in resource-limited settings. The purpose of this "Perspectives" editorial is to define the guiding principles behind an International Association of Physicians in AIDS Care (IAPAC) proposal to deal with this issue: the certification component of the Global AIDS Learning & Evaluation Network (GALEN).

First, it is important to emphasize that examination performance is only one component of establishing clinical competency. Other criteria are training, experience, and professionalism. In terms of the examination process, there are some fundamental rules of medical competency testing which are summarized here.

Who Should Write the Examination?

Persons who have established expertise and a reputation in the field should write the examination -- which justifies this function. They should be experienced practitioners, able to critically review data relevant to the examination. There needs to be inclusion of experts from the regions where examinations will be given in order to reflect local practice standards. The GALEN Certification Committee, composed of clinicians and public health experts from many distinct geographical regions, is convened with these goals in mind. (A full list of GALEN Certification Committee members is available on IAPAC's Web site: www.iapac.org.)

What Should the Test Look Like?

There are several components as follows:

• Blueprint: The blueprint defines the content by category of information being tested. For example, there may be 10 percent of questions allocated to antiretroviral therapy, 10 percent for prophylaxis and management of opportunistic infections, etc. Physicians knowledgeable in clinical care of HIV, especially as it is practiced in resource-limited settings, defined the blueprint for the GALEN certification examination as articulated in Table 1.

• Question format: The time-honored examination format, based on psychometrics, is to use "type A questions," which have a stem and five options, only one of which is correct. The GALEN certification examination uses this format.

• Question characteristics: Most of the examination is composed of "questions of knowledge" and "questions of judgment." An example of a knowledge question is: "Which of the following is the most common side effect of nelfinavir?" A question of judgment is: "Which of the following is the most important test in a patient with fever, headache, ALC of 460 cells/mm³ and HIV infection?" There should also be some questions dealing with ethics, but it should be emphasized that this needs to be a small portion of the examination, not because ethics are unimportant, but because ethics is best documented by behavior rather than by performance on an examination.

• Question phrasing: This is not an examination of examination taking, but an examination of medical competence. All questions must be clinically relevant; all the necessary information that would be readily available at the bedside should be provided, and the examination should carefully avoid ambiguities. Double negatives should be avoided, for example: "Which of the following is not uncommon" would be a poor way to phrase an examination question. Terms such as "common," "rare," and "frequent," should be avoided because it is unclear, for example, if "uncommon" means less than 50 percent, less than 10 percent, or less than 1 percent.

• Consensus: The GALEN Certification Committee must agree on each answer, and each answer must be indisputably correct. The challenge here is great because there are so many clinically important questions whose answers are either arbitrary or subject to rapid change.

How Do We Know If It's a Good Test?

The following are generally accepted measures that will be employed by the GALEN Certification Committee:

• There should be a "practice run" of persons with established expertise who take the examination to determine if it is fair.

• There needs to be agreement that the questions are relevant to clinical practice. It is nice to know the replication rate of HIV, and how HIV enters the CD4 cell, but this information is not important for clinical care and it is, therefore, unfair to ask about it on the examination.

• There needs to be a determination of the "pass score." There are two methods to do this: First, there may be an arbitrary passing grade such as 75 percent of all who take the examination. An alternative method is the "Angoff" method in which a group of experts decide the number of people with reasonable medical competency that would get each question correct; this is then used to determine the pass score. There may be some questions that are weighted simply because of their relative importance. At its next meeting, the GALEN Certification Committee will determine a method for setting the GALEN certification examination's "pass score."

Question analysis: The perfect question will be answered correctly by about 75 percent of those writing the examination. Incorrect responses should be distributed relatively evenly among the four other choices. Having a question that 95 percent of participants answer correctly does not help the examination; likewise, a question that almost all participants miss should be avoided.

An additional method of analysis is to review the performance of the top 20 percent of examination takers for each question with the assumption that the "smart group" will do especially well on a "good" (read: well-constructed) question.

Post-hoc analysis: The GALEN certification examination will be evaluated using the methods above. Questions will be discarded if they are viewed, in retrospect, as too hard, or too easy. Questions would also be eliminated if the correct answer has changed, or new information has made that answer debatable. Sometimes a question simply does not perform well. In many such examinations, up to one half of the entire examination is discarded by this post-hoc analysis. Many questions are preserved because they do extremely well and can be used in subsequent years. One benefit of this tactic is to compare performance from year to year to know if the examination takers are getting better.

Re-using questions, of course, requires that the test is secure -- that the people sitting the examination cannot leave the examination area with a copy of the examination, for example.

Updating: The GALEN certification examination will be updated on a frequent basis; at least annually. This is to reflect new developments in the field, which are especially relevant to the topic of HIV clinical management, in which new developments occur often. Frequent updating will require the GALEN Certification Committee to communicate regularly and work hard.

Credentialing: It is emphasized that examination results are only one component of the credentialing process. Other facets of the credentialing process may include a training requirement, medical licensure, clinical experience, and documentation of professionalism. In the case of GALEN certification, we have determined that only physicians with a locally valid medical license and documented experience treating HIV-infected patients may sit the examination. The examination is just one component of the credentialing process, but a very important one. The hope and anticipation is that a good examination, created by appropriate experts in the field, will stand as testimony of medical competence and serve as the most important component of the GALEN certification process.

John G. Bartlett is Chief of Infectious Diseases at the Johns Hopkins University School of Medicine in Baltimore, and Co-Chair of the GALEN Certification Committee.