Sign in

Oliver K. Ernst
Coding & AI — oliver-ernst.com

And no, it’s not in Python. Or C++.

Artistic visualization of neural network outputs with random weights (2D inputs, scalar output). Credit: Author. To reproduce this visualization, see Wolfram docs.

It might not even be in a language you already know, based on how relatively unpopular it is. But if you’ve studied math or physics at university you’ve probably at least heard of it.

I’m talking about Mathematica. And since version 11, it’s been possible to train neural networks directly in Mathematica.

Seriously? Mathematica?

If you’ve ever been interested in symbolic math, let’s get it out there: there’s really no parallel. Matlab just has basic symbolic capabilities, and is not really at all comparable (although both Mathematica and Matlab share…


How to serialize nested layers in TF 2.5.0.

A custom TF layer is one that subclasses from tf.keras.layers.Layer . This is powerful on its own, but a particularly desirable feature is to have nested layers. Serializing nested layers is a little bit of a headache, however, but necessary in order to save models with nested layers using model.save(...) .

Nested layers. Image credit: author.

Saving a custom layer

Let’s first make a custom layer:

This includes the get_config and from_config methods which are used to serialize the custom layer. Custom attributes like self.x are included by first calling the super class’s get_config() , and then using config.update({...})


Without amplifying the noise!

Noise, noise everywhere. Image by alexey.shikov (License: CC0 1.0)

That’s the real trick — how to differentiate a noisy signal, without amplifying the noise. It comes up all the time:

  • You’re taking data from an infrared sensor or some other displacement sensor, and want to compute velocity and acceleration.
  • You’re tracking the price of your favorite stock, option, or GME short and think your prediction will give you the edge to make money (spoiler: it won’t…).

The well known problem for a noisy signal is that:

  • Differentiation amplifies noise.
  • Integration introduces drift.

If you think you’re clever, you’ll come up with some sort of smoothing…


A tutorial on probabilistic PCA.

The Gaussian distributions in your probabilistic PCA model. Source: author.

There’s hardly a data scientist, scientist, programmer, or even marketing director who doesn’t about PCA (principal component analysis). It’s one of the most powerful tools for dimensionality reduction. If that marketing director is collecting survey data and looking to find target consumer groups for segmentation and analyzing the competition, PCA may very well be in play.

But you may have missed one of the simplest generative models that comes from an alternate view on PCA. If we derive PCA from a graphical model perspective, we arrive at probabilistic PCA. It allows us to:

  • Draw new…


Open source iOS app for referencing face mask data

The Masked Manual on the iOS app store: https://apps.apple.com/us/app/the-masked-manual/id1542536599

The problem:

Masks are important. Choosing the right type of mask and understanding its qualifications is critical. Currently, mask information is scattered between the openFDA database, FDA websites and CDC websites.

The solution:

“The Masked Manual” is a free iOS app that shows mask information compiled from openFDA, FDA websites and CDC websites. It lets you easily search for masks and see their qualifications, as well as instructions for wearing it. You can also use the camera to recognize and find your mask by scanning its packaging.


Your state’s voting power in the electoral college.

Screenshot from What’s my vote worth? Credit: Author original content.

With the recent 2020 election in the United States coming to a close (or should I say closed?), the arguments about the electoral college are flaring up again. Every four years, citizens in dense population areas argue in favor of either changing or abolishing the electoral college system, citing that their votes are being discounted relative to more rural areas.

What really is your voting power in different states? I made a little Flask application that you can find hosted on Heroku here:

It lets you explore what your vote is really…


And where are they in machine learning?

Image by hans-johnsonsource and license (CC BY-ND 2.0).

You can find the complete code for this tutorial on GitHub here.

We will review the theory for line search methods in optimization, and end with a practical implementation.

Motivation

In all optimization problems, we are ultimately interested in using a computer to find the parameters x that minimize some function f(x) (or -f(x) , if it is a maximization problem). Starting from an initial starting guess x_0, it is common to proceed in one of three ways:

  • Gradient-free optimization — don’t laugh! Everyone does this. Here we are just guessing the next parameters:


From UIViewController to screenshot to social media.

Send your shares into space. Source: Oliver K. Ernst (author). License: Attribution-NonCommercial-NoDerivatives 4.0 International

The problem:

Caveats:

  • You want to create a custom image by creating a custom UIViewController.
  • You don’t want the UIViewController to show.
  • You want to show a nice preview of the image that you’ll share.

You cant find the complete project on GitHub here.

Let’s get started with a clean project.

Project setup

Let’s start with a clean project. Add a share button to the default view controller. For the button image, you can choose square.and.arrow.up


Back to the bulk of the cake — unsupervised learning — with the latest tools.

Image source: Oliver K. Ernst (author). Attribution-NonCommercial-NoDerivatives 4.0 International

Yann LeCunn stated at NIPS 2016:

If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL).

RBMs are at the heart of unsupervised learning — they’re about finding good latent representations of the data, which can then be used for some supervised task such as classification. They’re also more flexible than other generative frameworks — in contrast to VAEs, for example, the distribution over…

Oliver K. Ernst

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store