DataSets

Get more data: can flip images, add random zooms and crops, and add distortion

Normalize:

Subtract mean: makes 0 mean

Normalize variance for every input

Normalize same for test and train

Train/Dev/Test Sets

Train with your training data

Dev to see which models perform bst

Test is to actually give you an accuracy rating for your chosen model rather than overfitting dev

With small data go 60/20/20

With huge data like 1,000,000 can leave 10,000 for dev and 10,000 for test giving a 98/1/1 split

Bias Vs. Variance

High Bias == underfitting

High Variance ==overfitting

Can simulteously have high bias and high variance

Train 1% | 15% | 15%

Dev 11% | 16% | 30%

​ high var high bi high var and bi

Changes based on optimal value, above assumes optimal is 0

High Bias?

  • Bigger Network

  • Train Longer

High Variance?

  • More data

  • Regularization

PreviousRegularizationNextEmbedding

Last updated