Explain how K-fold cross-validation is implemented on choosing a better classifi

Question

Explain how K-fold cross-validation is implemented on choosing a better classifier in the sense that it has less misclassification error rate. Please specify how to compute the cross-validation function.

Remark: You may write your algorithm for the following application. Suppose that we have N data: (x (1), y(1)), . . . ,(x (N) , y(N) ), where y takes value of 0 or 1, and two candidate logistic regression models:

Model 1 log(/(1 )) = 0 + 1x;

Model 2 log(/(1 )) = 0 + 1x + 2x 2 ,

where = P(y = 1|x).

Explanation / Answer

Let me first explain you the theory behind validation approach. LEts start with Leave-one-out cross-validation(LOOCV) LOOCV involves splitting the set of observations into two parts. However, instead of creating two subsets of comparable size, a single observation (x1,y1) is used for the validation set, and the remaining observations {(x2, y2), . . . , (xn, yn)} make up the training set. The statistical learning method is fit on the n 1 training observations, and a prediction yˆ is made for the excluded observation, using its value x1.

An alternative to LOOCV is k-fold CV. This approach involves randomly dividing the set of observations into k groups, or folds, of approximately equal size. The first fold is treated as a validation set, and the method is fit on the remaining k 1 folds.

## Below is the 2 line code to implement K fold validation in Loigistic Regression. You need to use it according to your ## need and check which model produces the least cross validation error.

library(boot)

glm.fit=glm(Y X, data = data_set, family = "binomial")

## Fitting the model 1. Incase of Model 2 you need to use Y ~X1 + X2

## For K Fold Cross Validation
cv.error = cv.glm(data_set, glm.fit, K=10)$delta[1]

The cv.glm() function produces a list with several components. The two numbers in the delta vector contain the cross-validation results.

One most important thing is you need to split your data_set between data_set_train and data_set_test. Only then you can check the acuracy of your model by evaluating the error in testing data set. General norm is to split 70% data for training and 30% for testing the model. You need to check your accuracy after running the cross validation in training data set and then tune hyper parameters of your given model accrodingly. Once you find the model with lowest cross validation parameters, then you can test the model on testing data set. This will give you unbiased result.

Hope it will help.

Related Questions

Navigate

• Browse (All)
• Browse E
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.