Paying attention to feature distribution alignment | Alex Shtoff
We discuss the meaning of weighted-orthogonality of function bases in feature engineering, and the relationship between the weight function and the feature distribution.| Alex Shtoff
Blog on optimization, machine learning, and software development.| Alex Shtoff
We discuss the meaning of weighted-orthogonality of function bases in feature engineering, and the relationship between the weight function and the feature distribution.| Alex Shtoff
Overparametrized Legendre polynomial regression with scikit-learn, with a few surprising properties!| Alex Shtoff
Overparametrized polynomial regression| Alex Shtoff
Advanced shape restrictions, such as combinations of monotonicity and convexity / concavity using polyhedral cones| Alex Shtoff
Fitting shape-restricted functions with ease using PyTorch.| Alex Shtoff
We develop an efficient alternative to PyTorch built-in dataloader class for the case of in-memory datasets, and lightweight models.| Alex Shtoff
We demonstrate how we can reduce model size by pruning un-needed neurons.| Alex Shtoff
We study a way to represent a tilted loss as an average of losses by lifting to a higher dimensional space, and employing regular SGD| Alex Shtoff
We study various polynomial bases from the bias-variance perspective, and the derivative-control properties of the Bernstein basis. This concludes our series on polynomial regression.| Alex Shtoff
We demonstrate an important use-case for Bernstein basis regularization in model calibration. We briefly discuss the use-cases of a well-calibrated machine learned classification model, and develop a simple calibrator that improves upon the ones provided by Scikit-Learn using regularization of the Bernstein basis.| Alex Shtoff
There is a well-known myth in the machine learning community - high degree polynomials are bad for modeling. In this post we debunk this myth.| Alex Shtoff
