The meta-analysis of MS peaks framework supports various methods for p-value combination, including Fisher’s, Edgington’s, Stouffer’s, Vote Count, Minimum P-value or Maximum P-value. The choice of statistical method depends on the goal of the meta-analysis as well as the data structure itself. Briefly, Fisher’s statistic, the Minimum P-value or Maximum P-value are traditionally used methods for combining p-values.
-
Fisher’s method is known to be more sensitive than other methods to very small (or very large) p-values which can result in a high false positive rate. For instance, a single very small p-value can lead to a significant combined p-value. “Fisher’s method employs the log product of individual P-values and thus, a single P-value of zero in one individual case will result in a combined P-value of zero regardless of the other P-values.” - (PMID: 26471455). This method should be followed when the data follows a Chi-squared distribution (positive values). It is also not recommended for use for meta-analysis with( >5 datasets).
-
Minimum P-value should be used to answer the question which metabolites are changed across at least one study? In this case, the minimum p-value among all studies is taken as the combined p-value.
-
Maximum P-value is the most restrictive method and should be used to answer the question of which genes are consistently changed across all studies? In this case, the maximum p-value among all studies is taken as the combined p-value.
-
Stouffer’s method attributes different weights to the p-values when combining them in a meta-analysis. This method should be applied when the data follows a Gaussian curve. This method is not as sensitive as Fisher’s to very small or very large p-values.
-
Edgington’s method uses the sum of the p-values and unlike Fisher’s method, is not sensitive to small p-values. It best fits circular data and it has been noted that using this method, a single large p-value can overwhelm small p-values (PMID: 11788962).
-
Vote counting method is limited to answering the question is there any evidence of an effect? One issue with this method is that it compares the number of “yes” studies to the number of “no” studies. The “yes” and “no” if often an arbitrary statistical cutoff that can bias the outcome. Secondly, it does not apply any weights to the studies, therefore the effect of a study with 1000 samples has the same weight as a study with 10 samples. While this method is simple to implement and interpret, this method should only be used when standard meta-analysis methods cannot be used. Meanwhile, at least 5 datasets are required for vote counting to reach the statistical confidence. To read more on vote counting here.