Study evaluations are performed on an endpoint/outcome-specific basis.
For each evaluation domain, core and prompting questions are provided to
guide the reviewer in assessing different aspects of study design and
conduct related to reporting, risk of bias and study sensitivity.
For some domains (see below), additional outcome- or chemical-specific
refinements to the criteria used to answer the questions should be developed
a priori by reviewers. Each domain receives a judgment of
Good, Adequate, Deficient, Not Reported
or Critically Deficient accompanied by the rationale and primary
study-specific information supporting the judgment. Once all domains
are evaluated, a confidence rating of High, Medium, or
Low confidence or Uninformative is assigned for each
endpoint/outcome from the study. The overall confidence rating
should, to the extent possible, reflect interpretations of the potential
influence on the results (including the direction and/or magnitude of
influence) across all domains. The rationale supporting the overall
confidence rating should be documented clearly and consistently, including
a brief description of any important strengths and/or limitations that were
identified and their potential impact on the overall confidence.
Domain judgments and overall ratings for all individual endpoints/outcomes
can be captured by a single (default) response, or you can create override
responses assigned to individual endpoints, outcomes, or results; provide a
descriptive label to describe which components the score refers to. Each response
must have a single default score; when selecting the default representative rating
for the domains and overall rating (i.e., the drop-down selection with the
associated color code), it is typically most appropriate to select the judgment
that best represents the study overall.
Requirements by Study Type
Domain | Metric | Bioassay | Epidemiology | In Vitro |
Risk of Bias100503381 |
Was administered dose or exposure level adequately randomized?100505547 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Was allocation to study groups adequately concealed?100505548 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were the comparison groups appropriate?100505549 |
- |
✔ |
✔ |
Risk of Bias100503381 |
Did the study design or analysis account for important confounding and modifying variables?100505550 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Did researchers adjust or control for other exposures that are anticipated to bias results?100505551 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were experimental conditions identical across study groups?100505552 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Did researchers adhere to the study protocol?100505553 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were the research personnel and human subjects blinded to the study group during the study?100505554 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were outcome data complete without attrition or exclusion from analysis?100505555 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were the outcome assessors blinded to study group or exposure level?100505556 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were confounding variables assessed consistently across groups using valid and reliable measures?100505557 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Can we be confident in the exposure characterization?100505558 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Can we be confident in the outcome assessment?100505559 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were all measured outcomes reported?100505560 |
✔ |
✔ |
✔ |
Risk of Bias100503381 |
Were there no other potential threats to internal validity (e.g., statistical methods were appropriate)?100505561 |
✔ |
✔ |
✔ |
Overall Study Confidence100503382 |
Overall Confidence Rating100505562 |
✔ |
✔ |
✔ |
Risk of Bias100503381
Was administered dose or exposure level adequately randomized?100505547
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?100505548
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Did the study design or analysis account for important confounding and modifying variables?100505550
Did researchers adjust or control for other exposures that are anticipated to bias results?100505551
Were experimental conditions identical across study groups?100505552
Did researchers adhere to the study protocol?100505553
Were the research personnel and human subjects blinded to the study group during the study?100505554
Were outcome data complete without attrition or exclusion from analysis?100505555
Were the outcome assessors blinded to study group or exposure level?100505556
Were confounding variables assessed consistently across groups using valid and reliable measures?100505557
Can we be confident in the exposure characterization?100505558
Can we be confident in the outcome assessment?100505559
Were all measured outcomes reported?100505560
Were there no other potential threats to internal validity (e.g., statistical methods were appropriate)?100505561
Overall Study Confidence100503382
Overall Confidence Rating100505562
Risk of Bias100503381
Was administered dose or exposure level adequately randomized?100505547
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?100505548
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Were the comparison groups appropriate?100505549
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Did the study design or analysis account for important confounding and modifying variables?100505550
Did researchers adjust or control for other exposures that are anticipated to bias results?100505551
Were experimental conditions identical across study groups?100505552
Did researchers adhere to the study protocol?100505553
Were the research personnel and human subjects blinded to the study group during the study?100505554
Were outcome data complete without attrition or exclusion from analysis?100505555
Were the outcome assessors blinded to study group or exposure level?100505556
Were confounding variables assessed consistently across groups using valid and reliable measures?100505557
Can we be confident in the exposure characterization?100505558
Can we be confident in the outcome assessment?100505559
Were all measured outcomes reported?100505560
Were there no other potential threats to internal validity (e.g., statistical methods were appropriate)?100505561
Overall Study Confidence100503382
Overall Confidence Rating100505562
Risk of Bias100503381
Was administered dose or exposure level adequately randomized?100505547
Randomization requires that each human subject or animal had an equal chance of being assigned to any study group including controls (e.g., use of random number table or computer generated randomization).
Was allocation to study groups adequately concealed?100505548
Allocation concealment requires that research personnel do not know which administered dose or exposure level is assigned at the start of a study. Human studies also require that allocation be concealed from human subjects prior to entering the study.
Note:
- a question under performance bias addresses blinding of personnel and human subjects to treatment during the study;
- a question under detection bias addresses blinding of outcome assessors.
Were the comparison groups appropriate?100505549
Comparison group appropriateness refers to having similar baseline characteristics between the groups aside from the exposures and outcomes under study.
Did the study design or analysis account for important confounding and modifying variables?100505550
Did researchers adjust or control for other exposures that are anticipated to bias results?100505551
Were experimental conditions identical across study groups?100505552
Did researchers adhere to the study protocol?100505553
Were the research personnel and human subjects blinded to the study group during the study?100505554
Were outcome data complete without attrition or exclusion from analysis?100505555
Were the outcome assessors blinded to study group or exposure level?100505556
Were confounding variables assessed consistently across groups using valid and reliable measures?100505557
Can we be confident in the exposure characterization?100505558
Can we be confident in the outcome assessment?100505559
Were all measured outcomes reported?100505560
Were there no other potential threats to internal validity (e.g., statistical methods were appropriate)?100505561
Overall Study Confidence100503382
Overall Confidence Rating100505562