Introduction to MEC2IR: Guidance for Campbell Systematic Reviews


John Westbrook: Hello and welcome to today’s
webcast being brought to you by the Center on Knowledge Translation for Disability and
Rehabilitation Research at SEDL through the funding from the National Institute on Disability
and Rehabilitation Research in the U.S. Department of Education. This is John Westbrook, Director
of the Center and also Co-Chair of the Knowledge Translation and Implementation Coordinating
Group of The Campbell Collaboration. We are delighted to bring today’s webcast to you
featuring the work and resources of the Methods Coordinating Group of The Campbell Collaboration
or C2 as it’s often referred to. I know you will find it very useful, and I thank
this group very much for planning such an informative webcast for us. Okay, so without
further ado, I will turn it over to Dr. Ariel Aloe, Co-Chair of the C2 Methods Group. Ariel?
Ariel Aloe: Hello. It’s a pleasure to introduce our editorial board, which includes Terri
Pigott, Emily Tanner-Smith, and Josh Polanin. Terri and Emily are our Editors, and Josh
is our Managing Editor. Today they are going to speak about the methodological expectations
for Campbell Interventions Reviews. Let’s hear from them now. Terri Pigott: This is
Terri Pigott. What is the Campbell Collaboration Methods Coordinating Group? Who’s involved?
Well, here we have a bunch of pictures of people that are involved in our Campbell Methods
Group. We have two co-chairs. We have an editorial team consisting of a managing editor and two
co-editors. We have a number of chairs of our subgroups, which I will talk about in
a minute, and we also have an Advisory Board. So, what does the Methodological Group of
Campbell do? We support the production of Campbell systematic reviews. We provide methods,
peer reviewing and editing for all protocols and reviews that come through to Campbell
Collaboration. We also are all involved in research to improve the methodology of research
synthesis and meta-analysis. We are also involved in disseminating guidelines for state of the
art review methods. We have several subgroups, as I mentioned earlier, that provide advice
and guidance on specific methodological topics. These subgroups include one on economics,
one on equity, specifically equity methods to encourage review authors to discuss the
impact of interventions and reducing inequality of opportunity. We also have a group involved
in information retrieval; how to find studies for systematic review. We have a group entitled
Process Implementation looking at how to include methods about the implementation of an intervention
into a Campbell review. We have a Methods Group on statistical methods, and we also
have a subgroup that provides training in statistical methods for meta-analysis and
also for other aspects of a systematic review from information retrieval through data analysis.
What kinds of things can we provide you with? The Methods Group provides consultation on
these specific review methods. As I mentioned before: information retrieval, what kinds
of studies, how to find studies for your review, study coding, how to extract information from
studies, the assessment of research quality of the studies that are included in a review,
how to calculate effect sizes for systematic review, and then again, how to analyze those
and how to synthesize them across studies. We also provide consultation on the kinds
of research designs that would be appropriate for your question in a Campbell review, methods
on how to incorporate economic information in a review, and also methods and process
implementation and data. We also maintain a resource center as part of our purview as
the Methods Group. For example, we have tools there to help you complete a systematic review
such as the effect size calculator. We also provide review guidelines. We produce a number
of policy documents on specific issues that you might run into in a systematic review.
We also have useful links in our resource center to other resources that are useful
in thinking about completing a systematic review such as the Equator Network. We also
provide policy briefs, again, on specific issues that you might run into in a systematic
review. We also plan to update our resource center by March 2015 so that it becomes more
useful for everyone who is interested in completing a systematic review. I am now going to turn
it to Emily Tanner-Smith to talk about our methodological expectations. E. Tanner-Smith:
Thank you, Terri. So, I’d like to take this opportunity to now discuss briefly some recent
documents that we’ve published on The Campbell Collaboration website outlining some Methodological
Expectations for Campbell Collaboration Intervention Reviews. We’ll call these the MEC2IR Guidelines
for the Conduct and Reporting of Intervention Reviews. So really our MEC2IR Guidelines were
adapted from The Cochrane Collaboration’s MECIR standards for the expectations for The
Cochrane Collaboration Intervention Reviews. So Cochrane’s/MECIR standards really were
designed to provide clear and transparent standards for both the conduct and the reporting
of systematic review protocols, as well as completed manuscripts. So the goal of the
Cochrane/MECIR standards really is to provide a resource for both authors and editors to
make it clear what expectations are expected of them for the publishing of their protocols
and systematic reviews. So really the goal of these expectations is to help make the
publishing experience clear for authors, and also help streamline the editorial process.
So these methodological expectation guidelines really help to streamline the communications
between editorial teams, author review teams, as well as the methodologists who are going
to be reviewing the methods and the statistical analyses in the protocols and completed systematic
reviews. At The Campbell Collaboration we decided to adapt Cochrane’s/MECIR’s standards
to the MEC2IR standards to essentially take those existing standards and make some adaptations
to account for the different methods that often will occur in Campbell Collaboration’s
systematic reviews. So, essentially what we did here is again, we wanted to have clear
and transparent methods for our review teams and our author teams to make it clear to them
when their protocols and reviews come to the Methods Group for peer review that they know
what standards they will be evaluated on for their research. So, again, the goal is to
ensure consistency and transparency in the peer review process. So for the process for
developing these standards, we created a working group to work on the adaptation of the Cochrane/MECIR
standards and so we included representatives from all of The Campbell Collaboration Coordinating
Groups and they’re listed on the slide, and I want to take a minute just to thank
all of the participants in this working group who really worked tirelessly, and participated
on many calls to reach consensus on all of the adaptations that we ultimately made on
the MEC2IR Guidelines. Then finally, once the working group had reached consensus on
the adaptations, that proposal for those MEC2IR Guidelines were distributed to the co-chairs
of the Methods Coordinating Group, and then finally approved by The Campbell Collaboration’s
Steering Group. Some of you may be wondering what the primary differences are between the
MECIR and the MEC2IR Guidelines, and I think it’s important to first highlight that really
there are many similarities between the two documents, with the MECIR documents and the
MEC2IR documents, because really the methodological expectations for Campbell Collaboration reviews
are highly similar to those for Cochrane Collaboration reviews. So the few minor differences that
you will see really do focus around kind of three key issues. First are issues of risk
of bias assessments, as well as assessments of the methodological quality of the primary
studies included in the protocols and reviews, and essentially the adaptations that we made
here permit for a parallelity of methods for assessing risk of bias and methodological
quality given the types of research studies that may be more common in Campbell Collaboration’s
systematic reviews. Then finally you’ll notice some adaptations around the development
and reporting of summary of findings tables, again, recognizing that many Campbell Collaboration
reviews may include tens, if not hundreds, of studies, and so we have to account and
add some additional flexibility there in reporting standards. Then finally, the remainder of
the changes between the MECIR and the MEC2IR documents really are quite minor in terms
of changing each of the individual items and the conduct in reporting standards from perhaps
mandatory to highly desirable or vice-versa. But overall, the standards are very similar
and so we only made slight modifications to allow for the wide range of methods that we
see in Campbell Collaboration’s systematic reviews. So I will now hand it over to Josh
Polanin who can talk about how the MEC2IR standards will be developed and incorporated
into the editorial process. Josh Polanin: Thank you, Emily. I’m happy to be on the
call. As Emily mentioned, my name is Josh Polanin, and I am the Managing Editor of the
Methods Group. I’m going to talk briefly today about our expectations for Campbell
editors and relatedly what we expect from our authors as well. So, as we look at the
MEC2IR expectations, one of the most important points to mention to begin with, is that the
standards have been adapted as of October 1st and all the views moving forward and protocols
must comply with the reporting standards. These need to be completed for all protocols
that have not been accepted yet, and all reviews in publication. So if a team is working on
a review currently, and the protocol has been accepted but the review has not as of October
1st, 2014, the review team needs to take the time to go through the MEC2IR Standards and
Expectations, and make sure that the review complies with these expectations. Now, incorporating
these expectations into the review should be a fairly straightforward process. If you
look at the expectations – download it from the website – you will see that they are listed
fairly succinctly so that a review team can incorporate them. These are very helpful because
they help to align the Campbell community across the coordinating groups. Oftentimes
we’ll find that substantive backgrounds talk in a slightly different language even
for similar aspects of the review process, and these expectations help to align all of
us so that we’re all talking in a common language. This will also help not only across
the groups, but also within the coordinating groups between the editor and the authors.
If the editor and the authors communicate succinctly, the process will move much more
smoothly. The editors may turn to the MEC2IR Guidelines and Expectations and recommend
some changes in the reviews to the review team. The goal of the guidelines and expectations
are really to set forth a transparent set of expectations and guidelines for the review
team as Emily said. In practice, we think that this really should help review teams
know from the onset what they’re getting involved in. So we think that while the protocol
in the review is being written, a review team can read and incorporate these expectations
into the protocol. We’d also suggest that once the protocol and review has been written,
a member of the review team use the MEC2IR Guidelines and Expectations and use it as
a sort of checklist and go back to their written documents, and to make sure that everything
that’s included in the MEC2IR Guidelines is also included in the protocol and review.
This will help both the editorial process and the methodological reviews as we’ll
be following a similar practice in the Methods Group and, therefore, doing this at the onset,
will really help to speed the process and make sure that we’re all on the same page.
So now I think we’ll turn it over to Shawn, who will lead us through a bit of Q&A. Shawn
Grant: Thank you very much, Josh. My name is Shawn Grant, and I’m a coordinator for
the Campbell webcast series. After these three great presentations, what I would like to
do is make the session a bit interactive at the end, and we have a few questions for all
of our presenters about the standards and expectations they presented in this webcast
today. So my first question will be for Terri. I was wondering if Terri could let us know,
what is the advantage of having a Methods Group for The Campbell Collaboration? Terri
Pigott: Thanks, Shawn. The key advantage of having a Methods Group for Campbell is that
it ensures quality and consistency of reviews across The Campbell Collaboration. It is also
a central place where we can provide expertise on the methodology, the conduct and reporting
of a review, and we can also promote new methods for systematic reviews. Shawn Grant: Perfect!
Thank you, Terri. The second question, I would like to address to Josh. Josh, I was wondering
if you could let us know how one can get involved in the Campbell Methods Group, if they’re
interested in getting involved. Josh Polanin: Thanks, Shawn. Great question. There are a
couple of ways that you can actually get involved with the Methods Group. The first is you can
email myself or one of the other members, and request to be added as a member or an
affiliate. A member of the Methods Group has full voting rights in all of our upcoming
elections and is often called upon in different situations as a reviewer or to be part of
different subgroups. An affiliate is still aligned with the Methods Group but really
only receives different news bulletins or different updates about what is going on.
You can also request to be a peer reviewer. During the editorial process we ask for external
method peer reviews on both our protocols and our completed reviews. You can send the
request to me or to one of the other members of the Methods Group, and we’ll add you
to our list of methodological experts as a peer reviewer. Finally, there’s a couple
of other smaller ways that you can be involved. You can join a subgroup. As Terri mentioned
at the top of this podcast, there are a number of different subgroups that we have affiliated
with the Methods Group and we’re always looking for new members. You can also help
write a new series of methods whitepapers on different policy aspects that we’ll be
writing in the upcoming months. Finally, if you just like to hear about the news and information
coming out of the Methods Group, you can request to join the Methods Listserv where you will
receive our bi-annual newsletter, and this way you can keep up on everything that’s
happening in the Methods Group. Shawn Grant: Thanks, Josh. That’s perfect and we really
hope that those listening to this webcast do take the opportunity to get involved. The
next question I have is for Emily, and I was wondering if you could explain to us why Campbell
decided to adapt the MECIR guidelines from Cochrane, and why we didn’t create our own
from scratch. If you could address that, that’d be great. E. Tanner-Smith: Thanks, Shawn.
Yes. When the working group decided to begin adapting the Cochrane/MECIR guidelines, essentially
our ultimate goal was to highlight the fact that The Cochrane Collaboration and The Campbell
Collaboration are sister organizations, and both organizations are dedicated to promoting
and disseminating high quality systematic reviews and meta-analysis. So, really we share
many methodological expectations for both the conduct and reporting of intervention
reviews. Given that, and as I mentioned earlier in the webcast, really our methods overlap
substantially. So, we felt in the working group that it was much more realistic and
it made more sense to adapt the MECIR guidelines because again the majority of those guidelines
are going to be perfectly aligned with both the Cochrane expectations and the Campbell
expectations. So we felt it would also send to authors an important message that, again,
we share common goals and methodological expectations, and indeed that’s why we have many review
teams who seek to co-register their review products with both Cochrane and The Campbell
Collaboration. So because of that, and also there’s no need to reinvent the wheel as
they say, we wanted to adapt those guidelines that had already been developed and an extensive
amount of work had been developed and put into that by The Cochrane Collaboration folks.
So we’re really indebted to them for putting in all of that leg work at the beginning.
So really that’s why we decided it made more sense for us just to tweak those MECIR
Guidelines to make those slight modifications to change those expectations so that they
would apply to all of The Campbell Collaboration reviews that we have in our library. Shawn
Grant: Thanks, Emily. That’s really great to know about the beneficial relationship
that we have with Cochrane and the efficient use of resources within Campbell. The next
question going back around the horn is to Josh, and I think as Managing Editor of the
Methods Group you could best address these questions I’m sure we’d get from some
authors which are: ‘Why should I use these new expectations?’ and ‘Won’t it add
more time to the review process from the time it takes for my review to be published and
disseminated?’ Josh Polanin: Yes, I think this a very important question, Shawn. I’ll
address the first one – why should I use the MEC2IR Guidelines. I think the main thing
that these expectations set forth is in the title. They are expectations that we have
as a Methods Group and Campbell-wide for what the review teams should be putting forth in
their reviews. In the past, we’ve relied on review team expertise or a few of our policy
documents to help guide the review teams, but these expectations are listed in such
a way that they can be read and applied in a much more straightforward manner. So, picking
them up and downloading this document will really help with some of the more technical
big and small aspects of the review process. So, that segues into the second question nicely.
I think then at the onset while the expectations might look somewhat overwhelming to begin
with, we do think that this will have a significant impact on the amount of time in the review
process. That is because review teams will know what should be written in the protocol
and the completed reviews, and what the Methods Group will be looking for. There are some
great examples listed in the expectations, and we think that they’re written in such
a way that with just a little bit more work upfront, the review process in the end will
be significantly decreased because there won’t be a back and forth between the Methods Group
or the editorial team and the authors. Shawn Grant: Thanks, Josh. That’s certainly my
experience with the expectations and guidelines like these. It really helps to ensure the
rigor of the end-product and can even sometimes make the process more efficient because the
standards are clearer. So it’s very helpful to know. The next question is going around
the horn back to Terri. Say I am not familiar with one of the items in the new expectations,
whom should I contact with any questions I may have? Terri Pigott: Thanks, Shawn. Well,
your first point of contact should be someone on the editorial team, either the Managing
Editor, Josh Polanin, or one of the Co-Editors, Emily Tanner-Smith, or myself, Terri Pigott.
If we feel we need to get some other deeper expertise on some particular issue, we can
certainly send you to someone else we think that could help you out with figuring out
how to address that particular expectation. Shawn Grant: Great. Yes. We certainly have
a rich team with the Methods Group. There are many points of contact. If you go to their
website, you should be able to find contact info for all the people that Terri just mentioned.
I’m sure someone would be more than happy to help you out if you have any questions
about MEC2IR. Final question of the webinar, the softball question to Emily as the answer’s
on the slide, where can I find these new guidelines? E. Tanner-Smith: Alright, my favorite kind
of question. Thanks, Shawn. Yes, definitely check out the website. On The Campbell Collaboration
website, we have the MECIR Guidelines documents and so there you can find the two documents.
One, which outlines the expectations for the conduct of Campbell Collaboration Intervention
Reviews, and the second which will outline the expectations for the reporting of Campbell
Collaboration Intervention Reviews. So we encourage all of our author teams and anyone
interested in systematic reviewing or meta-analysis to download those documents and check them
out today. Shawn Grant: Great! Thanks, Emily, that’s very helpful, and thanks to all of
our participants in today’s webcast. Building off what Emily just said, we’d love for
you to get in touch not just about the new guidelines, but anything related to the collaboration
in general. You’ll see there is a link there for letting us know, giving us some feedback
on how this webcast was. If you’d like to get in touch about Campbell in general, there
is our contact email address and of course you can visit our website or follow us on
Twitter and Facebook for updates on what’s going on in The Collaboration. We thank you
for joining us for this webcast today and we look forward to seeing you for the next
one.

Leave a Reply

Your email address will not be published. Required fields are marked *