The metaanalysis of MS peaks framework supports various methods for pvalue combination, including Fisher’s, Edgington’s, Stouffer’s, Vote Count, Minimum Pvalue or Maximum Pvalue. The choice of statistical method depends on the goal of the metaanalysis as well as the data structure itself. Briefly, Fisher’s statistic, the Minimum Pvalue or Maximum Pvalue are traditionally used methods for combining pvalues.

Fisher’s method is known to be more sensitive than other methods to very small (or very large) pvalues which can result in a high false positive rate. For instance, a single very small pvalue can lead to a significant combined pvalue. “Fisher’s method employs the log product of individual Pvalues and thus, a single Pvalue of zero in one individual case will result in a combined Pvalue of zero regardless of the other Pvalues.”  (PMID: 26471455). This method should be followed when the data follows a Chisquared distribution (positive values). It is also not recommended for use for metaanalysis with( >5 datasets).

Minimum Pvalue should be used to answer the question which metabolites are changed across at least one study? In this case, the minimum pvalue among all studies is taken as the combined pvalue.

Maximum Pvalue is the most restrictive method and should be used to answer the question of which genes are consistently changed across all studies? In this case, the maximum pvalue among all studies is taken as the combined pvalue.

Stouffer’s method attributes different weights to the pvalues when combining them in a metaanalysis. This method should be applied when the data follows a Gaussian curve. This method is not as sensitive as Fisher’s to very small or very large pvalues.

Edgington’s method uses the sum of the pvalues and unlike Fisher’s method, is not sensitive to small pvalues. It best fits circular data and it has been noted that using this method, a single large pvalue can overwhelm small pvalues (PMID: 11788962).

Vote counting method is limited to answering the question is there any evidence of an effect? One issue with this method is that it compares the number of “yes” studies to the number of “no” studies. The “yes” and “no” if often an arbitrary statistical cutoff that can bias the outcome. Secondly, it does not apply any weights to the studies, therefore the effect of a study with 1000 samples has the same weight as a study with 10 samples. While this method is simple to implement and interpret, this method should only be used when standard metaanalysis methods cannot be used. Meanwhile, at least 5 datasets are required for vote counting to reach the statistical confidence. To read more on vote counting here.