Overview
The community reviewers project has proven to be one of the strongest and most significant
ways we have engaged our community in translational research. It is systematic, action-based, and incentivized. That is, we have a very specific ask of the community (to review grant applications) that occurs cyclically (at least annually for 2 different grant programs) and reviewers are compensated on a per-activity basis. It is a program that has remained strong, expanded to include other activities (facilitating community engagement studios, for example), and has grown with little administrative effort (current reviewers refer other community leaders to participate).
The community reviewers project started in 2016 when our leadership felt it might be a missed
opportunity to not include community voices as part of grant reviews for our basic science pilot applications. We have always had community representatives review our community-engaged partnered research (Campus Community Research Incubator, [CCRI]) grant applications, but this
was a new idea for our non-community-facing pilot grants. We decided to test this question about the impact of including community input on the grant review process by engaging 21 community reviewers alongside scientific reviewers in a 2-stage process of evaluating research
proposals (publication link). In Stage 1, reviewers scored proposals and, during Stage 2, two study sections (face-to-face discussion committees) were convened: one a mix of community reviewers and scientific reviewers, and one engaging only scientific reviewers.
Comparisons of reviews revealed little difference between scores of community reviewers and those of scientific reviewers, and that community reviewers largely refrained from critiquing scientific or technical aspects of proposals. Community reviewers overwhelmingly found the
experience to be a positive one that they would be willing to repeat. The findings suggested that involving community reviewers early in the grant cycle and exposing them to the entirety of the review process can bolster community engagement without compromising the rigor of
grant evaluations.
Following this initial test of the idea, we were invited to participate in a Community Reviewers Training Program that grew out of our CTSA External Reviewer Exchange Consortium (CEREC). Adopting a community reviewer training model developed by our partners at the University of Arkansas for Medical Sciences, where they have a long history of engaging the community in the review process, we recruited 19 community leaders to attend a 6-hour training on serving as a community reviewer for our Institute for Clinical and Translational Science (ICTS) pilot grant applications. The training and subsequent engagement with these community leaders has strengthened our working relationship into an action-based partnership. We compensate community reviewers for each application they review, and for each study section they attend. Over time, our community reviewers have gained confidence about sharing their perspectives with scientific reviewers, and the scientific reviewers find the input from the community members adds a valuable dimension to the review process. Community reviewers also have learned much about the research funding processes, and felt enriched for the ability to contribute so directly to the research agenda of the ICTS.
Additionally, the community reviewers project has strengthened our community engagement capacity in the following ways:
1. Led to a significant new partnership with our local Public Health Department during COVID to fund more than 2x as many CCRI projects as we can typically fund in a year;
2. Community reviewers have participated in facilitating Community Engagement Studios; one of the most important aspects of the Community Engagement Studio since skilled, neutral, third party facilitators can be hard to find;
3. Growth in the number of interested reviewers has spurred us to begin an annual training program for new community reviewers to join the team (and for existing reviewers to brush up on the grant review process if it has been a while or would just like to re-take the training).
2. Community reviewers have participated in facilitating Community Engagement Studios; one of the most important aspects of the Community Engagement Studio since skilled, neutral, third party facilitators can be hard to find;
3. Growth in the number of interested reviewers has spurred us to begin an annual training program for new community reviewers to join the team (and for existing reviewers to brush up on the grant review process if it has been a while or would just like to re-take the training).
If you are interested in learning more or would like to become a community reviewer, please
contact Robynn Zender at rzender@hs.uci.edu.