4. [10] Suppose we have two predictive models, M1 and M2 and run 10 rounds of 10

Question

4. [10] Suppose we have two predictive models, M1 and M2 and run 10 rounds of 10-fold cross-validation test using the two models. In each round, a training set and a testing set are determined based on the 10-fold cross-validation method, and the same training set (re- spectively testing set is used to build (respectively to test the two models. The error rates obtained from the test are given in the following table. Perform a t-test at a significance level of 0.001 to determine if one model is significantly better than the other. A t-distribution table is provided as an attachment to this assignment. 1 3 4 5 6 7 9 10 Models round 2 MMI 30.5 32.2 20.7 20.6 31.0 41.0 27.7 26.0 21.5 26.0 M. 22.4 14.5 22.4 19.6 20.7 20.4 22.1 19.4 16. 35.0

Explanation / Answer

here as subjects are same we will use paired t test

from above given information

as p value is greater then 0.001 we can not reject null hypothesis; and does not have sufficient evidence that one model is better then other

M1 M2 difference(d) 30.5 22.4 -8.1 32.2 14.5 -17.7 20.7 22.4 1.7 20.6 19.6 -1 31 20.7 -10.3 41 20.4 -20.6 27.7 22.1 -5.6 26 19.4 -6.6 21.5 16.2 -5.3 26 35 9 mean -6.450 std deviaition 8.7001 std error= std dev/(n)^(1/2) 2.7512 t stat= (d-0)/std error -2.3444 p value = 0.0205

Related Questions

Navigate

• Browse (All)
• Browse 4
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.