OranLooney.com
https://www.oranlooney.com/
Recent content on OranLooney.com
Hugo  gohugo.io
en
© Copyright {year} Oran Looney
Wed, 13 Sep 2023 00:00:00 +0000

Cracking Playfair Ciphers
https://www.oranlooney.com/post/playfair/
Wed, 13 Sep 2023 00:00:00 +0000
https://www.oranlooney.com/post/playfair/
In 2020, the Zodiac 340 cipher was finally cracked after more than 50 years of trying by amateur code breakers. While the effort to crack it was extremely impressive, the cipher itself was ultimately disappointing. A homophonic substitution cipher with a minor gimmick of writing diagonally, the main factor that prevented it from being solved much earlier was the several errors the Zodiac killer made when encoding it.
Substitution ciphers, which operate at the level of a single character, are children’s toys, the kind of thing you might get a decoder ring for from the back of a magazine.

My Dinner with ChatGPT
https://www.oranlooney.com/post/mydinnerwithchatgpt/
Sat, 10 Dec 2022 00:00:00 +0000
https://www.oranlooney.com/post/mydinnerwithchatgpt/
It's hard to talk about ChatGPT without cherrypicking. It's too easy to try a dozen different prompts, refresh each a handful of times, and report the most interesting or impressive thing from those sixty trials. While this problem plagues a lot of the public discourse around generative models, cherrypicking is particularly problematic for ChatGPT because it's actively using the chat history as context. (It might be using a $\mathcal{O}(n \log{} n)$ attention model like reformer or it might just be brute forcing it, but either it has an impressively long memory; about 2048 "

A History of Encabulation
https://www.oranlooney.com/post/encabulation/
Thu, 01 Dec 2022 00:00:00 +0000
https://www.oranlooney.com/post/encabulation/
To celebrate the 100th anniversary of the birth of encabulation  dated from Dr. Wolfgang Albrecht Klossner’s first successful run in that historic barn on the outskirts of Eisenhüttenstadt  this article* collects in one place a number of resources that provide, if not a comprehensive history, at least a catalogue of the major milestones and concepts.
The Original Turbo Encabulator For a number of years now, work has been proceeding in order to bring perfection to the crudely conceived idea of a transmission that would not only supply inverse reactive current for use in unilateral phase detractors, but would also be capable of automatically synchronizing cardinal grammeters.

Eight Billion People
https://www.oranlooney.com/post/eightbillion/
Mon, 14 Nov 2022 00:00:00 +0000
https://www.oranlooney.com/post/eightbillion/
Today is the last day when the number of people alive will start with a seven. Sometime late Tuesday afternoon, or perhaps early Wednesday morning, the population will cross the eight billion mark. When I was a kid, the number they taught us in school was five billion. By the time I was in college, it was up to six, and a decade ago it hit seven.
Now it’s eight.

ML From Scratch, Part 6: Principal Component Analysis
https://www.oranlooney.com/post/mlfromscratchpart6pca/
Mon, 16 Sep 2019 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart6pca/
In the previous article in this series we distinguished
between two kinds of unsupervised learning (cluster analysis and dimensionality
reduction) and discussed the former in some detail. In this installment we turn
our attention to the later.
In dimensionality reduction we seek a function \(f : \mathbb{R}^n \mapsto \mathbb{R}^m\) where \(n\) is the dimension of the original data \(\mathbf{X}\) and
\(m\) is less than or equal to \(n\). That is, we want to map some high dimensional
space into some lower dimensional space.

A Seriously Slow Fibonacci Function
https://www.oranlooney.com/post/slowfibonacci/
Sat, 06 Jul 2019 00:00:00 +0000
https://www.oranlooney.com/post/slowfibonacci/
I recently wrote an article which was ostensibly about the Fibonacci
series but was really about optimization techniques. I wanted to follow up on
its (extremely moderate) success by going in the exact opposite direction:
by writing a Fibonacci function which is as slow as possible.
This is not as easy as it sounds: any program can trivially be made slower,
but this is boring. How can we make it slow in a fair and interesting way?

ML From Scratch, Part 5: Gaussian Mixture Models
https://www.oranlooney.com/post/mlfromscratchpart5gmm/
Wed, 05 Jun 2019 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart5gmm/
Consider the following motivating dataset:
Unlabled Data
It is apparent that these data have some kind of structure; which is to say, they certainly are not drawn from a uniform or other simple distribution. In particular, there is at least one cluster of data in the lower right which is clearly separate from the rest. The question is: is it possible for a machine learning algorithm to automatically discover and model these kinds of structures without human assistance?

Adaptive Basis Functions
https://www.oranlooney.com/post/adaptivebasisfunctions/
Tue, 21 May 2019 00:00:00 +0000
https://www.oranlooney.com/post/adaptivebasisfunctions/
Today, let me be vague. No statistics, no algorithms, no proofs. Instead,
we’re going to go through a series of examples and eyeball a suggestive
series of charts, which will imply a certain conclusion, without actually
proving anything; but which will, I hope, provide useful intuition.
The premise is this:
For any given problem, there exists learned featured representations
which are better than any fixed/humanengineered set of features, even once
the cost of the added parameters necessary to also learn the new features into account.

ML From Scratch, Part 4: Decision Trees
https://www.oranlooney.com/post/mlfromscratchpart4decisiontree/
Fri, 01 Mar 2019 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart4decisiontree/
So far in this series we’ve followed one particular thread: linear regression
> logistic regression > neural network. This is a very natural progression of
ideas, but it really represents only one possible approach. Today we’ll switch
gears and look at a model with completely different pedigree: the decision
tree, sometimes also referred to as Classification and Regression
Trees, or simply CART models. In contrast to the earlier progression,
decision trees are designed from the start to represent nonlinear features and
interactions.

A Fairly Fast Fibonacci Function
https://www.oranlooney.com/post/fibonacci/
Tue, 19 Feb 2019 00:00:00 +0000
https://www.oranlooney.com/post/fibonacci/
A common example of recursion is the function to calculate the \(n\)th Fibonacci number:
def naive_fib(n):
if n < 2:
return n
else:
return naive_fib(n1) + naive_fib(n2)
This follows the mathematical definition very closely but it’s performance is
terrible: roughly \(\mathcal{O}(2^n)\). This is commonly patched up with dynamic
programming. Specifically, either the memoization:
from functools import lru_cache
@lru_cache(100)
def memoized_fib(n):
if n < 2:
return n
else:
return memoized_fib(n1) + memoized_fib(n2)
or tabulation:

ML From Scratch, Part 3: Backpropagation
https://www.oranlooney.com/post/mlfromscratchpart3backpropagation/
Sun, 03 Feb 2019 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart3backpropagation/
In today’s installment of Machine Learning From Scratch we’ll build on the logistic regression from last time to create a classifier which is able to automatically represent nonlinear relationships and interactions between features: the neural network. In particular I want to focus on one central algorithm which allows us to apply gradient descent to deep neural networks: the backpropagation algorithm. The history of this algorithm appears to be somewhat complex (as you can hear from Yann LeCun himself in this 2018 interview) but luckily for us the algorithm in its modern form is not difficult  although it does require a solid handle on linear algebra and calculus.

ML From Scratch, Part 2: Logistic Regression
https://www.oranlooney.com/post/mlfromscratchpart2logisticregression/
Thu, 27 Dec 2018 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart2logisticregression/
In this second installment of the machine learning from scratch
we switch the point of view from regression to classification: instead of
estimating a number, we will be trying to guess which of 2 possible classes a
given input belongs to. A modern example is looking at a photo and deciding if
its a cat or a dog.
In practice, its extremely common to need to decide between \(k\) classes where
\(k > 2\) but in this article we’ll limit ourselves to just two classes  the
socalled binary classification problem  because generalizations to many
classes are usually both tedious and straightforward.

ML From Scratch, Part 1: Linear Regression
https://www.oranlooney.com/post/mlfromscratchpart1linearregression/
Thu, 29 Nov 2018 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart1linearregression/
To kick off this series, will start with something simple yet foundational:
linear regression via ordinary least squares.
While not exciting, linear regression finds widespread use both as a standalone
learning algorithm and as a building block in more advanced learning
algorithms. The output layer of a deep neural network trained for regression
with MSE loss, simple AR time series models, and the “local regression” part of
LOWESS smoothing are all examples of linear regression being used as an
ingredient in a more sophisticated model.

ML From Scratch, Part 0: Introduction
https://www.oranlooney.com/post/mlfromscratchpart0introduction/
Sun, 11 Nov 2018 00:00:00 +0000
https://www.oranlooney.com/post/mlfromscratchpart0introduction/
Motivation
“As an apprentice, every new magician must prove to his own satisfaction, at
least once, that there is truly great power in magic.”  The Flying Sorcerers,
by David Gerrold and Larry Niven
How do you know if you really understand something? You could just rely on
the subjective experience of feeling like you understand. This sounds
plausible  surely you of all people should know, right? But this runs
headfirst into in the DunningKruger effect.

Visualizing Multiclass Classification Results
https://www.oranlooney.com/post/viztsne/
Thu, 23 Aug 2018 00:00:00 +0000
https://www.oranlooney.com/post/viztsne/
Introduction Visualizing the results of a binary classifier is already a challenge, but having more than two classes aggravates the matter considerably.
Let’s say we have $k$ classes. Then for each observation, there is one correct prediction and $k1$ possible incorrect prediction. Instead of a $2 \times 2$ confusion matrix, we have a $k^2$ possibilities. Instead of having two kinds of error, false positives and false negatives, we have $k(k1)$ kinds of errors.

Craps Variants
https://www.oranlooney.com/post/crapsgamevariants/
Wed, 11 Jul 2018 00:00:00 +0000
https://www.oranlooney.com/post/crapsgamevariants/
Craps is a suprisingly fair game. I remember calculating the probability of winning craps for the first time in an undergraduate discrete math class: I went back through my calculations several times, certain there was a mistake somewhere. How could it be closer than $\frac{1}{36}$?
(Spoiler Warning If you haven’t calculated these odds for yourself then you may want to do so before reading further. I’m about to spoil it for you rather thoroughly in the name of exploring a more general case.

Complex Numbers in R, Part II
https://www.oranlooney.com/post/complexrpart2/
Sat, 30 Jun 2018 00:00:00 +0000
https://www.oranlooney.com/post/complexrpart2/
This post is part of a series on complex number functionality in the
R programming language. You may want to read Part I before continuing if
you are not already comfortable with the basics.
In Part I of this series, we dipped our toes in the water by explicitly
creating some complex numbers and showing how they worked with the most basic
mathematical operators, functions, and plots.
In this second part, we’ll take a more indepth look at some scenarios where
complex numbers arise naturally – where they are less of a choice an more
of a necessity.

Complex Numbers in R, Part I
https://www.oranlooney.com/post/complexr/
Sun, 17 Jun 2018 00:00:00 +0000
https://www.oranlooney.com/post/complexr/
R, like many scientific programming languages, has firstclass support for
complex numbers. And, just as in most other programming languages, this
functionality is ignored by the vast majority of users.
Yet complex numbers can often offer surprisingly elegant formulations and
solutions to problems. I want to convince you that familiarizing yourself with
R’s excellent complex number functionality is well worth the effort and will
pay off in two different ways: first by showing you how they are so
amazingly useful you’ll want to go out of your way to use them, and then by
showing you how they are so common and fundamental to modern analysis that you
couldn’t avoid them if you wanted to.

So, Apparently I'm an iPad Developer Now
https://www.oranlooney.com/post/apparentlyipaddeveloper/
Wed, 28 Apr 2010 00:00:00 +0000
https://www.oranlooney.com/post/apparentlyipaddeveloper/
Last week my boss stopped by and dropped a brand spanking new iPad on my desk. "Make our application work on this," he commanded. "You have two days before we demo it at the trade show." Madness? No, these are web apps! You see, for the last couple years we've been working exclusively on AJAX applications: web pages stuffed with so much JavaScript they look and feel like desktop apps. It's harder than writing desktop software, but if you pull it off you get an application that can be run anywhere, instantly.

Deep Copy in JavaScript
https://www.oranlooney.com/post/deepcopyjavascript/
Wed, 25 Nov 2009 00:00:00 +0000
https://www.oranlooney.com/post/deepcopyjavascript/
Update 20171023: This article and code library have not kept up with the rapidly changing JavaScript landscape and are now hopelessly out of date. First came nonenumerable properties, and with ES2015 came the introduction of classes, proxies, symbols, and anonymous functions, all of which break the below logic. I'm afraid I no longer know how to fully copy the full menagerie of JavaScript objects while preserving relative references, and it's quite possible that no one else knows either.

Semantic Code
https://www.oranlooney.com/post/semanticcode/
Wed, 30 Apr 2008 00:00:00 +0000
https://www.oranlooney.com/post/semanticcode/
semantic (siman’tik) adj. 1. Of or relating to meaning, especially meaning in language.
Programming destroys meaning. When we program, we first replace concepts with symbols and then replace those symbols with arbitrary codes — that’s why it’s called coding.
At its worst programming is writeonly: the program accomplishes a task, but is incomprehensible to humans. See, for example, the story of Mel. Such a program is correct, yet at the same time meaningless.