Optimal ratio for data splitting

It is common to split a dataset into training and testing sets before fitting a statistical or machine learning model. However, there is no clear guidance on how much data should be used for training and testing. In this article, we show that the optimal training/testing splitting ratio is p:1$$ \sq...

Full description

Saved in:
Bibliographic Details
Published inStatistical analysis and data mining Vol. 15; no. 4; pp. 531 - 538
Main Author Joseph, V. Roshan
Format Journal Article
LanguageEnglish
Published Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.08.2022
Wiley Subscription Services, Inc
Subjects
Machine learning
Regression models
Splitting
Statistical analysis
testing
Training
validation
Online AccessGet full text
ISSN1932-1864
1932-1872
DOI10.1002/sam.11583

Cover

More Information
Summary:It is common to split a dataset into training and testing sets before fitting a statistical or machine learning model. However, there is no clear guidance on how much data should be used for training and testing. In this article, we show that the optimal training/testing splitting ratio is p:1$$ \sqrt{p}:1 $$, where p$$ p $$ is the number of parameters in a linear regression model that explains the data well.
Bibliography:Funding information
Division of Chemical, Bioengineering, Environmental, and Transport Systems, DMREF‐1921873; Division of Civil, Mechanical and Manufacturing Innovation, 1921646
ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:1932-1864
1932-1872
DOI:10.1002/sam.11583