Source
r/AskStatistics•
WritingPretty

K-Nearest Neighbor without splitting data

I have an assignment for a data mining class where we are supposed to do classification on the mushroom dataset (if you're familiar), using Python, to determine which features determine if the mushroom is edible or poisonous.

Problem is, in this class we're are NEVER supposed to split the data into train/test.

Well I can't figure out how to do K-Nearest Neighbors without splitting the data and I can't find anything online that remotely shows how I would go about this. Anyone have any suggestions?

Posted Nov 12, 2022, 7:22 PM UTC

14
grandzooby•

Problem is, in this class we're are NEVER supposed to split the data into train/test.

Can you provide some clarity or a source on this? Is it in a exercise instruction, class syllabus?

A key part of any supervised ML is splitting off some test data to be used for validation. Otherwise, you're just "data dredging" and at high risk of over-fitting.

This data set is pretty small, but if you want to look at picking useful features, you may want to look at the Data Reduction section of this: https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm

2
WritingPrettyOP•

Thanks, I'll take a look at the Data Reduction section.

The never split instruction is part of the instructions for every assignment. I think the idea is that we're just supposed to be analyzing the dataset and not making any predictions but it makes no sense to me. It's been a frustrating quarter honestly.

1
grandzooby•

If you're only doing "exploratory data analysis" then that's... okay. But it means your results can't properly be generalized. That is, "this model, with this parameters fits this particular sample of data best" but then you can make no claims to how that model and parameters will work on new data.

1
blozenge•

Have you seen the knn.cv function provided by the {class} package in R? That uses Leave One Out Cross-validation - i.e. classification is repeated n times with the hold out being only the 1 sample being classified.

I think that that's the closest you can get knn to not having a train-test split.

If the probe datapoint were included in the training data then you would have direct data-leakage as each point is its own closest neighbour by definition. So knn with LOOCV would be the minimal way to avoid this.

2