How scores are calculated with random forests to be used in ROC analysis

[jc_Martin]

Random forests is used in the BIOMARKER ANALYSIS function in the “Tester module”, where we can chose appropriate compounds to be used simultaneously to build a ROC curve. I wonder how scores from the selected compounds can be built to be assessed in the ROC analysis using random forests. I tried to use the mean decrease accuracy as a weighted factor to apply to each compound value, but I did not reproduce the roc curve displayed while using the random forest key. The R commands are not usefull to me with this regard. Thank you for helping
jc

[jeff.xia]

Are you referring to reproducing the ROC in Tester mode vs Explorer mode? If so, you cannot. Explorer ROC and Tester ROC are for different purposes. Explorer ROC is more complex (double CV) in order to select features and to evaluate performance; while Tester just performs conventional CV (single CV) based on the given features. However, they will converge for very large sample (>500), in my experience.

Please see below for more details:

How are important features selected in multivariate ROC exploratory analysis?

The algorithm tries to identify important features through repeated random sub-sampling cross validation (CV). In each CV, two thirds (2/3) of the samples are used to evaluate the importance of each feature based on VIP scores (feature selection using PLSDA), Decreases in accuracy (feature selection using Random Forest), or Weighted coefficients (feature selection using Linear SVM) The top 2, 3, 5, 10 …100 (max) important features are used to build classification/regression models which are v…