In:
PLOS Biology, Public Library of Science (PLoS), Vol. 21, No. 5 ( 2023-5-1), p. e3002082-
Abstract:
The utility of mouse and rat studies critically depends on their replicability in other laboratories. A widely advocated approach to improving replicability is through the rigorous control of predefined animal or experimental conditions, known as standardization. However, this approach limits the generalizability of the findings to only to the standardized conditions and is a potential cause rather than solution to what has been called a replicability crisis. Alternative strategies include estimating the heterogeneity of effects across laboratories, either through designs that vary testing conditions, or by direct statistical analysis of laboratory variation. We previously evaluated our statistical approach for estimating the interlaboratory replicability of a single laboratory discovery. Those results, however, were from a well-coordinated, multi-lab phenotyping study and did not extend to the more realistic setting in which laboratories are operating independently of each other. Here, we sought to test our statistical approach as a realistic prospective experiment, in mice, using 152 results from 5 independent published studies deposited in the Mouse Phenome Database (MPD). In independent replication experiments at 3 laboratories, we found that 53 of the results were replicable, so the other 99 were considered non-replicable. Of the 99 non-replicable results, 59 were statistically significant (at 0.05) in their original single-lab analysis, putting the probability that a single-lab statistical discovery was made even though it is non-replicable, at 59.6%. We then introduced the dimensionless “Genotype-by-Laboratory” (GxL) factor—the ratio between the standard deviations of the GxL interaction and the standard deviation within groups. Using the GxL factor reduced the number of single-lab statistical discoveries and alongside reduced the probability of a non-replicable result to be discovered in the single lab to 12.1%. Such reduction naturally leads to reduced power to make replicable discoveries, but this reduction was small (from 87% to 66%), indicating the small price paid for the large improvement in replicability. Tools and data needed for the above GxL adjustment are publicly available at the MPD and will become increasingly useful as the range of assays and testing conditions in this resource increases.
Type of Medium:
Online Resource
ISSN:
1545-7885
DOI:
10.1371/journal.pbio.3002082
DOI:
10.1371/journal.pbio.3002082.g001
DOI:
10.1371/journal.pbio.3002082.g002
DOI:
10.1371/journal.pbio.3002082.g003
DOI:
10.1371/journal.pbio.3002082.g004
DOI:
10.1371/journal.pbio.3002082.g005
DOI:
10.1371/journal.pbio.3002082.t001
DOI:
10.1371/journal.pbio.3002082.t002
DOI:
10.1371/journal.pbio.3002082.t003
DOI:
10.1371/journal.pbio.3002082.t004
DOI:
10.1371/journal.pbio.3002082.t005
DOI:
10.1371/journal.pbio.3002082.s001
DOI:
10.1371/journal.pbio.3002082.s002
DOI:
10.1371/journal.pbio.3002082.s003
DOI:
10.1371/journal.pbio.3002082.s004
DOI:
10.1371/journal.pbio.3002082.s005
DOI:
10.1371/journal.pbio.3002082.s006
DOI:
10.1371/journal.pbio.3002082.s007
DOI:
10.1371/journal.pbio.3002082.s008
DOI:
10.1371/journal.pbio.3002082.s009
DOI:
10.1371/journal.pbio.3002082.s010
DOI:
10.1371/journal.pbio.3002082.s011
DOI:
10.1371/journal.pbio.3002082.s012
Language:
English
Publisher:
Public Library of Science (PLoS)
Publication Date:
2023
detail.hit.zdb_id:
2126773-X
Bookmarklink