💡 Neural Networks

Neural Networks#

Layered models of interconnected units that learn complex functions from data.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

A neural network is a model of interconnected neurons arranged in layers — an input layer, one or more hidden layers, and an output layer. Each neuron computes a weighted sum of its inputs, adds a bias, and passes the result through a non-linear activation function, letting the network model complex relationships.

Weights, biases, activations#

Weights set the strength of each connection and biases shift the activation; the non-linearity is what lets stacked layers represent functions a linear model cannot. Common activations are ReLU, sigmoid and tanh. A single unit computes

\[a = \phi\!\left( \sum_i w_i x_i + b \right),\]

where \(\phi\) is the activation function.

Forward pass and backpropagation#

In the forward pass, inputs flow layer by layer to an output, and a loss function (MSE for regression, cross-entropy for classification) measures the error. Backpropagation then applies the chain rule to compute the gradient of that loss with respect to every weight, propagating the error backward from output to input; gradient descent updates the weights, with the learning rate \(\eta\) setting the step size:

\[w \leftarrow w - \eta \, \frac{\partial \mathcal{L}}{\partial w}.\]

In practice#

Depth and width are chosen for the task, validation data guards against overfitting, and frameworks like PyTorch, TensorFlow and Keras implement backpropagation automatically.

from sklearn.neural_network import MLPClassifier

clf = MLPClassifier(hidden_layer_sizes=(64, 32), activation="relu", max_iter=500)
clf.fit(X_train, y_train)

Theme: AI & ML Concepts · All terminology

Hint

Mind map — connected ideas

Support Vector Machines (SVMs) · Logistic Regression · Decision Trees · LSTM — Long Short-Term Memory Networks · Deep Ensembles · Autoencoder

Hint

More in AI & ML Concepts

AI (Artificial Intelligence) · Classification Models · Computer Vision (CV) · Decision Trees · Linear Models · LLMs (Large Language Models) · Logistic Regression · Machine Learning (ML) · Medical AI · Natural Language Processing (NLP) · Regression Models · Support Vector Machines (SVMs) · Target Variable