A series of tutorials that dive deep into the fundamental concepts underlying modern artificial intelligence with the theory and implementations in Go and Rust.

These tutorials should be accessible to anyone and only a basic understanding of mathematics is assumed. All the prerequisites and foundational concepts are introduced to allow an understanding from the basic to the advanced.

Prerequisites and Foundations

Probability & Statistics: An introduction to probability and statistics and the fundamental concepts that will be important in many of the tutorials to follow. Starting with the basic concepts of probability of an event occurring to probability density functions, odds, log odds and the odds ratios among other things.

Differential Calculus: A deep dive into the rate of change of a quantity for a given input. This tutorial on differential calculus walks through the concepts of slope of a line and the techniques used to calculate the slope of a curve at a given point on the curve. Derivatives of several important functions are introduced along with proofs as well as the chain rule and product rule of derivatives which will be important in tutorials to follow. Finally partial derivatives are explained with an eye to understanding the derivative of a function with respect to a given parameter.

Vectors & Matrices: Vectors, Matrices and Matrix algebra are foundational to understanding the concepts behind AI and this tutorial walks through the building blocks and fundamental concepts of matrices and vectors and the key axioms of linear algebra using matrices.

Linear Regression

Simple Linear Regression: Simple linear regression is a statistical technique of making predictions from data. The tutorial introduces a linear model in two dimensions that examines the relation ship between one dependent variable and one independent variable and then finds line that best fits the data. Once the line of best fit is determined it can be used to predict values of y (the dependent variable) for any value of x (the independent variable)

Simple Linear Regression - Proof of the Closed Form: This tutorial walks through a detailed proof of the the closed form solution of simple linear regression introduced above. This proof walks through solving two partial differential equations to compute the values of the two parameters. While this proof provides a detailed explanation (and requires an understanding of differential calculus), for the curious reader of how the closed form solution is obtained, it isn't strictly necessary for the material ahead and can be skipped if needed.

Multiple Regression: Multiple regression extends the two dimensional linear model introduced in Simple Linear Regression to k + 1 dimensions with one dependent variable, k independent variables and k+1 parameters. The general matrix form of the model is introduced along with a closed form solution that depends on the feature matrix being invertible (which is not always possible) and sets the stage for using gradient descent to solve linear regression.

Multiple Regression - Deriving the Matrix Form: The closed matrix form for multiple regression is a solution of the partial differential equation w.r.t to the parameter vector (B). This tutorial walks through the steps of solving the matrix partial differential equation to derive the closed form solution which depends on the feature matrix being invertible (which is not always possible).

Gradient Descent

Gradient Descent for Simple Linear Regression: An introduction to gradient descent and its application to simple linear regression. Gradient optimizes the model parameters by successively computing the slope of the cost function (the gradient) and then iteratively walking down the cost curve till the gradient is close to zero or a specific number of iterations.

Gradient Descent for Multiple Regression: Gradient descent is applied to the general matrix form of multiple regression and a model with k+1 parameters is optimized by successively computing the gradient vector and then walking down the cost function. The matrix closed form solution for multiple regression requires inverting a matrix which is computationally expensive in most cases (and impossible in many cases). Gradient descent is computationally more efficient and is preferred especially when working with large datasets and models.

Logistic Regression

Logistic Regression - Fundamentals: An introduction to binary classification using Logistic Regression. Logistic Regression uses the Logistic Function to convert a linear combination of input features into probabilities along with a threshold to classify the predictions into one of two values (binary classification). The logistic function and the general matrix form for Logistic Regression is introduced with a model with parameters B and W.

Cost Function & Gradient Descent for Logistic Regression: An introduction to the Cost function for Logistic Regression long with its partial derivative (the gradient vector). The model parameters (B & W) are then optimized using Maximum Likelihood Estimation and Gradient Descent.

Logistic Regression - Derivative of the Cost Function: This tutorial walks through the partial derivative of the cost function for logistic regression. This cost function can then be used to optimize the model parameters using Maximum Likelihood Estimation & Gradient Descent.

Neural Networks

(coming soon)

Naive Bayes Classifiers

(coming soon)

Decision Trees

(coming soon)

Markov Decision Processes

(coming soon)

Reinforcement Learning

(coming soon)