Title:

Controlling the effect of multiple testing in Big Data

Group publication title:

Mathematical Economics

Creator:

Denkowska, Sabina

Subject and Keywords:

multiple testing ; FDR ; Big Data

Description:

Mathematical Economics, 2014, Nr 10 (17), s. 5-16

Abstrakt:

Big Data poses a new challenge to statistical data analysis. An enormous growthof available data and their multidimensionality challenge the usefulness of classical methodsof analysis. One of the most important stages in Big Data analysis is the verification ofhypotheses and conclusions. With the growth of the number of hypotheses, each of which istested at  significance level, the risk of erroneous rejections of true null hypotheses increases.Big Data analysts often deal with sets consisting of thousands, or even hundreds ofthousands of inferences. FWER-controlling procedures recommended by Tukey [1953], areeffective only for small families of inferences. In cases of numerous families of inferencesin Big Data analyses it is better to control FDR, that is the expected value of the fraction oferroneous rejections out of all rejections. The paper presents marginal procedures of multipletesting which allow for controlling FDR as well as their interesting alternative, that isthe joint procedure of multiple testing MTP based on resampling [...]

Publisher:

Wydawnictwo Uniwersytetu Ekonomicznego we Wrocławiu

Place of publication:

Wrocław

Date:

2014

Resource Type:

artykuł

Format:

application/pdf

Resource Identifier:

doi:10.15611/me.2014.10.01

Language:

eng

Relation:

Mathematical Economics, 2014, Nr 10 (17)

Rights:

Wszystkie prawa zastrzeżone (Copyright)

Access Rights:

Dla wszystkich w zakresie dozwolonego użytku

Location:

Uniwersytet Ekonomiczny we Wrocławiu