Making Sense of PyTorch

What I ended up discovering:

What I thought neural networks were like: Round #1

What I thought neural networks were like: Round #2

What I think neural networks are like today: Round #3

  • There a network of nodes and connections in there but it doesn’t appear in the code, similar to how there are packets and TCP/IP but you won’t see much of that in frontend JavaScript.
  • A great deal of work can be encapsulated in pre-trained networks and weights (such as ResNet). For a basic image labeler around that module I need to know only a linear transform and something like softmax or argsort.

Understanding inputs as tensors

Using a GPU

AutoML and Pre-Trained Models vs. From Scratch

Which config and options should I choose?

  • Including the optimizer in the learned parameters:
  • Removing BatchNorm without sacrificing quality:


import matplotlib.pyplot as plt
from sklearn.metrics import ConfusionMatrixDisplay

Stuff that still is kinda sketchy (for researchers, not just newbies)

Transformers vs. Doing ResNet Better

Normalizing and Augmenting

Adversarial Examples

Understanding vector-space

Robustness and drift




Web->ML developer and mapmaker.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Featureform: The ML Feature Store

Using BERT For Classifying Documents with Long Texts

The Game of Life vs. Convolutions

Classifying Text Reviews of Amazon Products Using Naive Bayes

Introducing Generalized Integrated Gradients (GIG)

We’re Releasing a Free, Hands-On Quantum Machine Learning Course Online

SIGIR 2019: Where did all the Information Retrieval go?

Machine Learning — The Self-Starter Way!!!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nick Doiron

Nick Doiron

Web->ML developer and mapmaker.

More from Medium

ML Arxiv Haul #2

Great Expectations: The Future of Health Tech

Slaying the ML Dragon

Artist Spotlight #1: Jesse Woolston