Nick Slack, who chaired a session on improving IRB functions and operations through the utilization of quality management programs, said of the conference that, “APREC was illuminating in that we were able to see first-hand how other human research protection programs around the globe are, in some ways, just catching up to US-based HRPPs, in others, outpacing the US, but overwhelmingly, struggling with the same issues that we are. The conference really left you with a sense of just how flat the world of subject protections really is.”

One clear indication of where Western HRPPs lag behind their Asian counterparts relates to the cultural importance placed on quality management programs. Whereas a typical research ethics conference in the US, for example, might have one or two talks dedicated to quality management, APREC offered approximately 10 such talks. If you consider the history of quality management, this observation is really not that surprising. Quality management programs emerged in the East as part of manufacturing and management philosophy decades before similar systems were deployed in the West. Thus, the idea that HRPPs in Asia are more focused on employing quality management systems in the practice of IRB review than their Western counterparts is not unexpected. An orientation to quality, which permeates much of Asian culture, is one place where Western HRPPs can take a lesson.

A unifying theme among conference attendees pertained to the difficult balance of performing IRB reviews of social and behavioral science research. IRBs and ethics committees across the world appear to be struggling to find the balance between conducting ethics reviews which are sufficiently protective of the rights and welfare of subjects, but not unreasonably burdensome to researchers. According to Cheryl Savini of the HRP Consulting Group, who gave a talk about The Importance of Having a Human Research Protection Program, “our global colleagues are facing the same challenges that we are in trying to apply regulations to social and behavioral science research that skew towards biomedical research. Unlike the Common Rule, which allows greater flexibility to review social and behavioral science research, much of the regulatory canon abroad is premised on Good Clinical Practice requirements, which were more clearly written for the review and conduct of biomedical research.”

AAHRPP accreditation is also emerging as an important topic in Asia. With organizations now accredited in China, India, Korea, and Singapore, APREC attendees were familiar with AAHRPP and appeared to be considering engaging the accreditation process for their respective organizations. APREC had two separate sessions on accreditation conducted by Dr. Peter Vasilenko, PhD, of AAHRPP and representatives from organizations in China, India, and Singapore (Pfizer Phase I Units), who had achieved the AAHRPP Seal. The growing importance of accreditation in Asia is yet another signal of the flattening world of human subject protections.

Jeffrey Cohen from the HRP Consulting Group will provide the keynote address and Cheryl Savini will make a presentation on applying the regulatory criteria for the approval of research. HRP representatives will be available to answer questions and answers from 4:30 pm – 8:00 pm at the “meet and greet” reception following the event.

An agenda for the University of Southern Maine Research Integrity Symposium is available here. We look forward to seeing you there!

Point One:
Researchers should talk to IRB administrators before submitting a study for review.

Take-Away Message for IRBs:
IRBs should be open to and encourage researchers to discuss their research with an IRB administrator prior to submission. This supports a positive, helpful perception of the IRB and it also improves the quality of the submissions.

Point Two:
Researchers should orient the IRB to the rationale and scientific/scholarly significance of the research being proposed. Researchers should provide examples to the IRB of similar studies and/or give evidence of research participants’ experiences in comparable studies.

Take-Away Message for IRBs:
IRBs should make determinations based on empirical evidence whenever possible. Asking researchers to cite relevant literature and to provide information about similar studies supplies the IRB with additional context to better inform their reviews.

Point Three:
Researchers should ensure that a comprehensive description of the consent process is provided in IRB application materials.

Take-Away Message for IRBs:
The consent process is more important than the consent document. A comprehensive informed consent process ensures that coercion and undue influence is minimized and that the choice to participate in research is well-informed and voluntary.

Point Four:
Researchers should attend IRB meetings, where possible.

Take-Away Message for IRBs:
Being able to communicate with researchers directly during the review process often leads to better and more informed IRB review. Questions can be answered through dialogue instead of through repetitious letters or email exchange. While it is not always feasible to have researchers attend IRB meetings, IRBs can usually arrange for researchers to be available by phone during the meeting.

Point Five:
Researchers should be familiar with the regulations pertaining to the protection of research subjects.

Take-Away Message for IRBs:
Training and education should be provided to researchers. When researchers understand the criteria that IRBs must consider in order to approve research, they submit better protocols and have a more positive perception of the review process. Another message the IRB can take away from this recommendation is that the IRB must always clearly communicate to researchers the basis, especially the regulatory basis, for all of its determinations.

Cheryl Savini, Chief Operating Officer of HRP Consulting Group, says, “IRBs and researchers really have a common goal: the efficient and ethical conduct of research. In as much as IRB review contributes to that goal, both groups must work together to ensure that the IRB review process is as comprehensive, informed, and collegial as possible. Keeping in mind how Dr. Stark’s principles translate to the IRB can promote a better review process for all parties involved.”

As consultants to a wide range of human research protection programs (HRPPs), biases are most clearly on view when we are engaged by an organization to conduct an external evaluation of their HRPP; we call these Program Evaluations. When the Program Evaluation begins, it is not unusual to hear statements like “we think we are doing a really good job, but we know that we have some minor things to fix.” Or, “our IRB is great, but our processes need some work.” Or, “we are having this evaluation to convince our investigators that the IRB is doing what it is supposed to do.” Often, these visits end in a much different tone: “It looks like we have more issues than we thought.” Or, “I guess we need to take a serious look at how we provide training and education to our IRB members.” And, “do we really need to report that to OHRP?”

One of the main goals of an external evaluation by a third party is to eliminate personal bias. We trust accredited universities more than those who are not accredited because we believe that a third party is in a better position to evaluate the university; we are confident that our dairy is safe when it passes USDA inspection; and we are more inclined to trust the safety and efficacy of drugs approved by the FDA. The same is true for HRPPs. Any organization hoping to gain an accurate picture of the strengths and weaknesses of its HRPP will more likely benefit from an objective, third party evaluation. In our experience conducting these evaluations, and, as much as we hate to be the bearer of bad news, we are seldom surprised when the reality of the HRPP’s performance does not match the collective opinions expressed by those individuals working in the program. Why? Because biases are unavoidable. To be exact, here are our most common findings:

1. IRB members are unfamiliar with the regulatory criteria for the approval of research found at 45 CFR 46.111 and 21 CFR 56.111: Arguably, if you are not considering the regulatory criteria for the approval of research, you might be an ethics committee, but you are not an IRB. Yet, it is striking to see how many IRBs fail to consider and are not familiar with these basic requirements for reviewing research. Not only does this raise a host of compliance issues, but it also means that the IRB is probably applying a lesser and inconsistent standard of protection than what is required by the regulations.

2. The IRB does not utilize the flexibilities in the regulations: One of the most cited complaints about IRBs is their lack of efficiency. It is hard to disagree with this sentiment when we observe IRBs sending research that is either exempt or eligible for the expedited procedure to the convened IRB for review. Not only does this create more work than what is needed, but we also see this turn into further compliance issues when the research comes up for continuing review.

3. IRB minutes do not record the required findings set forth at 45 CFR 46.115 and 21 CFR 56.115: There are two issues here. First, if the IRB is not making determinations required by the regulations, then the minutes are sure to be out of compliance. Secondly, from a regulatory standpoint, minutes are the only proof that you have that the IRB did or did not do its job. Poor minutes lead to increased risk of regulatory findings should OHRP or FDA come knocking at the IRB Office door.

4. IRB Files: An IRB file should allow for a complete reconstruction of the history of the IRB’s actions related to its review of a research project. Yet, there are many times where we find significant gaps in IRB files. Whether it is missing documentation related to continuing review or no correspondence between the IRB and the investigators, together with your minutes, an IRB file is your only evidence of the IRB’s action or inaction.

5. Investigators are unfamiliar with the types of projects that require IRB review: Poor training of investigators can be an HRPP’s biggest nightmare. Organizations that do not provide adequate initial and ongoing training to investigators are more likely to have research being conducted without IRB approval. At the very least, an adequate training program should prompt investigators to contact the IRB when they have questions rather than forgetting that there is an IRB at all. When you finally discover that your training program for investigators is inadequate, it might take a while to sort out the mess.

According to Cheryl Savini, COO of the HRP Consulting Group, “while it is often the case that a program is not performing as well as they thought, there are many times when we are pleasantly surprised to find that an HRPP’s performance is much better than what was described by the individuals working in the program.” But, whether you are over or underestimating how well your program is performing, biases can affect how we evaluate the functions and operations of our HRPPs. And, since no program is perfect and there is always room for improvement, the only way for deciding where you need to improve is by figuring out exactly where you stand. External evaluations, whether conducted by a consulting group, one of your regional partners, or an accrediting body like AAHRPP, can be of tremendous benefit for improving the functions and operations of your HRPP. The more difficult task is deciding whether you are ready to discover some things that might not match what you previously thought to be the case.

Tip #1: Get familiar with the AAHRPP Evaluation Instrument.

There is no document more central to the accreditation process than the AAHRPP Evaluation Instrument for Accreditation. According to Nick Slack, Director of Consulting Services at the HRP Consulting Group and former AAHRPP Accreditation Director, “the Evaluation Instrument is your roadmap for achieving accreditation. Each Standard/Element in the Evaluation Instrument includes a series of bullets under the headings ‘Required Written Materials’ and ‘Outcomes’ that directly reflect how AAHRPP will evaluate your policies and what your HRPP is doing in practice. If you ensure that your policies and processes are crafted around these bullets, your chances of earning the AAHRPP Seal and improving your program in the process will be high.”

To access AAHRPP’s recently updated Evaluation Instrument, please click here. According to AAHRPP, “the latest version includes some minor edits and the addition of changes in the Department of Defense requirements according to DOD Instruction 3216.02. Also, the requirements on financial conflict of interest in the new DHHS rule were added to Element 1.6.B.