Scaled False Discovery Proportion and Related Error Metrics

In multiple testing, a variety of control metrics of false positives have been introduced such as the Per Family Error Rate (PFER), Family-Wise Error Rate (FWER), the False Discovery Rate (FDR), the False Exceedence Rate (FER). In this talk, we present a comprehensive family of error rates together with a corresponding family of multiple testing procedures (MTP). Based on the needs of the problem at hand, the user can choose a particular member among these MTPs. The new error rate limits the number of false positives FP relative to an arbitrary non-decreasing function s of the number of rejections R. The quantity is called, the scaled false discovery proportion SFDP=FP/s(R). We present different procedures to control either the P(SFDP>q) or the E(SFDP) for any choice of the scaling function. An obvious choice is s(R)=min(R;k). As does FDR, this particular error rate FP/s(R) accepts a fixed percentage of false rejections among all rejections, but only up to R =k, then a stricter control takes over and for R > k, the number of false positives is limited to a percentage of the fixed value k, similar to PFER. The corresponding family of multiple testing procedures bridges the gap between the PFER (k =1) and the FDR (k = number of tests). A similar such bridge is obtained when s(R)=Rg with 0