Different results between MetaboAnalyst 5.0 and 6.0 using the same dataset and workflow

[b62gasar]

Dear all,

Some time ago I performed several analyses using MetaboAnalyst 5.0. We are currently running related experiments, so I wanted to check that I would obtain the same results using the new version of the software.

However, the analysis obtained with MetaboAnalyst 6.0 is different from the one previously obtained with version 5.0. As far as I can tell, I have followed exactly the same steps: log10 transformation and Pareto scaling.

I am not sure where the discrepancy might come from. I am attaching the analysis report generated previously with version 5.0 and the results obtained now with version 6.0, as well as the dataset used to generate the results.

I have also noticed that the commands shown at the end of the analysis history are slightly different in some aspects. However, as mentioned above, I have followed exactly the same workflow when performing the analysis. I should also mention that I am only using the online version of MetaboAnalyst, not the downloadable version.

Additionally, in the previous analysis performed with version 5.0 no data filtering was applied. However, in version 6.0 the report indicates “Feature filtering based on Interquartile Range (IQR) – removed 0 features based on the cutoff”. However, I did not explicitly select any filtering step, so I am unsure why this appears in the workflow.

In addition, this dataset had never produced any errors before. However, when uploading it now I sometimes receive a message indicating that some metabolites cannot be read or that some entries are duplicated. The message does not appear every time, so the behaviour seems somewhat random. I have attached the message as well.

Could this difference be due to changes in the default preprocessing or filtering settings between versions 5.0 and 6.0?

Thank you very much in advance for your help.

Analysis_Report 5.0.pdf (711.5 KB)

Analysis_Report 6.0.pdf (6.8 MB)

dataset.csv (93.8 KB)

[xia.lab]

MetaboAnalyst 5.0 was released in 2021. There have been significant changes in terms of underlying R version, methods, knowledge databases, as well as the workflows. There will be differences due to such changes. It is also natural how software evolves.

Quick suggestions:

1. Do not use report generation for this purpose (as it was deprecated since version 4.0);
2. Please ask speficific questions with reproducible steps

[b62gasar]

Thank you for your reply.

I understand that MetaboAnalyst 5.0 was released in 2021 and that many components of the software have evolved since then (R version, methods, databases, workflows, etc.). I also understand that some differences between versions are expected.

However, I am trying to understand the source of the differences in my case, because the preprocessing steps I apply are very simple: log10 transformation followed by Pareto scaling. Under these conditions, I would expect the statistical significance of many metabolites to remain relatively consistent.

In my previous analysis (performed about two years ago using MetaboAnalyst 5.0), several metabolites appeared as highly significant. When I run the same dataset now in MetaboAnalyst 6.0 using the same preprocessing steps, some of those metabolites are no longer significant, while others that were not significant before now appear significant.

This makes it difficult for me to properly compare my previous experiment with the new one, because I am unsure which version of the results should be considered more reliable.

I included the reports mainly to illustrate these differences. For example, the metabolites highlighted in the fold change analyses are not identical and do not show the same significance levels between versions. Another difference I noticed is in the distribution plot after data transformation, which also looks different between the two analyses.

Since the preprocessing steps are limited to log10 transformation and Pareto scaling, I am wondering whether the data might be handled differently internally in the new version (for example in terms of missing values, normalization defaults, or filtering steps).

Additionally, I would like to ask how results obtained with previous versions of MetaboAnalyst should be interpreted. If the newer version incorporates methodological improvements, should results generated with older versions be considered less reliable? In practice, would you recommend relying on the results from the current version when comparing experiments, or is there a way to reproduce analyses comparable to those obtained with MetaboAnalyst 5.0?

Thank you again for your help.

P.S. Attached steps of the process prior to analyzing data.

[xia.lab]

In general, our newer versions are backward compatible; and we test every release to ensure our existing tutorials remain relevant as much as possible.

It is possible that specific characteristics of your dataset are sensitive to minor perturbations in the newer version. We encourage you to explore our published tutorials—including our Nature Protocols—to help resolve the issue.

If you are a subscriber, you can also access version 2024-R1, a stable release, to see if it helps resolve the discrepancy.

[b62gasar]

Thank you for your suggestion.

I would like to try version 2024-R1 to better understand the differences. However, I am not sure whether I have subscriber access.

Could you please clarify what is required to access this version? Is it available through institutional access or any specific registration?

Thank you very much for your help.