From Scratch on OranLooney.com

ML From Scratch VI: Principal Component Analysis

Mon, 16 Sep 2019 00:00:00 +0000

In the previous article in this series we distinguished between two kinds of unsupervised learning (cluster analysis and dimensionality reduction) and discussed the former in some detail. In this installment we turn our attention to the later. In dimensionality reduction we seek a function $f : \mathbb{R}^n \mapsto \mathbb{R}^m$ where $n$ is the dimension of the original data $\mathbf{X}$ and $m$ is less than or equal to $n$.

ML From Scratch V: Gaussian Mixture Models

Wed, 05 Jun 2019 00:00:00 +0000

Consider the following motivating dataset: It is apparent that these data have some kind of structure; which is to say, they certainly are not drawn from a uniform or other simple distribution. In particular, there is at least one cluster of data in the lower right which is clearly separate from the rest.

ML From Scratch IV: Decision Trees

Fri, 01 Mar 2019 00:00:00 +0000

So far in this series we’ve followed one particular thread: linear regression -> logistic regression -> neural network. This is a very natural progression of ideas, but it really represents only one possible approach. Today we’ll switch gears and look at a model with completely different pedigree: the decision tree, sometimes also referred to as Classification and Regression Trees, or simply CART models.

ML From Scratch III: Backpropagation

Sun, 03 Feb 2019 00:00:00 +0000

In today’s installment of Machine Learning From Scratch we’ll build on the logistic regression from last time to create a classifier which is able to automatically represent non-linear relationships and interactions between features: the neural network. In particular I want to focus on one central algorithm which allows us to apply gradient descent to deep neural networks: the backpropagation algorithm.

ML From Scratch II: Logistic Regression

Thu, 27 Dec 2018 00:00:00 +0000

In this second installment of the machine learning from scratch we switch the point of view from regression to classification: instead of estimating a number, we will be trying to guess which of 2 possible classes a given input belongs to. A modern example is looking at a photo and deciding if its a cat or a dog.

ML From Scratch I: Linear Regression

Thu, 29 Nov 2018 00:00:00 +0000

To kick off this series, will start with something simple yet foundational: linear regression via ordinary least squares. While not particularly exciting, linear regression finds widespread use both as a standalone learning algorithm and as a building block in more advanced learning algorithms.

ML From Scratch: Introduction

Sun, 11 Nov 2018 00:00:00 +0000

Motivation As an apprentice, every new magician must prove to his own satisfaction, at least once, that there is truly great power in magic. —The Flying Sorcerers, by David Gerrold and Larry Niven How do you know if you really understand something? You could just rely on the subjective experience of feeling like you understand.