How Neural Networks Learn

What Is a Neural Network?

A neural network is a computer model that learns a relationship between inputs and an output. It contains small calculation units called neurons, arranged in layers and joined by numbered connections called weights.

During training, the network sees examples with known answers. It makes guesses, checks its mistakes, and slowly changes its weights. After enough useful examples, it can make a prediction for an example it has not seen before.

Biological neuron vs artificial neuron

The biological comparison is useful as a memory aid: dendrites receive signals, synapses affect their strength, the cell body combines them, and the axon carries a response. An artificial neuron borrows that rough story using inputs, weights, a calculation, an activation function, and an output.

From one neuron to a complete network

A network connects many artificial neurons in layers. The following diagram shows those parts in our student-result problem.

The Five Key Components of a Neural Network

Before solving an example, first see what a neural network is made of. These five ideas appear in every later section.

Learning happens in three repeating stages

Beginner Word Bank: Know These Terms Before We Use Them

You do not have to memorize this list now. Use it as a map: when a word appears later, you will already know its plain meaning.

Start With a Real-Life Problem

We could write fixed rules such as “if study time is above six hours, predict pass.” But real life is rarely one rule. A student may study less and have excellent attendance, or study more but sleep very little. A neural network can learn how several clues work together from examples.

What Happens Inside One Neuron?

A neuron has three small jobs: multiply each input by its weight, add those results and the bias, then activate the result. The diagram follows one imaginary student. To keep the arithmetic readable, we manually scale each teaching input against a stated reference: 8 study hours out of 10 gives 8÷10=0.8; 90% attendance gives 90÷100=0.9; and 7 sleep hours against a 10-hour reference gives 7÷10=0.7. These are teaching choices, not a recommended school-scoring system.

The same calculation, one line at a time

These weights were chosen only to show the arithmetic. In a real network, training discovers useful weight and bias values from data.

Why Do We Need More Than One Neuron?

One neuron can form one simple decision. Many neurons can notice different combinations. One hidden neuron may respond strongly to “high attendance and moderate study,” while another may respond to “long study but little sleep.” The next layer can combine those smaller patterns into a better prediction.

How the Network Makes One Prediction

Moving information from input to output is called a forward pass (or forward propagation). Follow our student once through the network:

1. 1

   Prepare the inputs

   Study, attendance, and sleep are converted to numbers on similar scales. Text or images would also need to be converted to numbers.

2. 2

   Send values into the hidden layer

   Every hidden neuron receives the inputs, applies its own weights and bias, and uses an activation function.

3. 3

   Pass hidden outputs forward

   Those outputs become the inputs for the next layer. The same simple neuron calculation happens again.

4. 4

   Produce a final score

   The output neuron returns 0.67. For this teaching task we read that as an estimated 67% chance of passing.

5. 5

   Apply a decision rule if needed

   If this teaching example uses 0.50 as its manually chosen threshold, 0.67 becomes “likely to pass.” The probability and threshold are different things.

How Does a Neural Network Learn?

At the beginning, weights are usually small, unhelpful numbers, so the guesses are poor. Training improves them using examples whose correct answers are known.

One tiny learning moment

One pass through all training examples is called an epoch. A small group processed together is a batch. The learning rate controls how large each update is. Too large can jump past a good solution; too small can make learning very slow.

Activation Functions, Without the Mystery

An activation function decides what signal a neuron sends onward. Without non-linear activations, many stacked layers would still behave like one simple straight-line rule.

Different Neural Networks for Different Data

“Neural network” is a family name. The shape of the network changes to suit the kind of information it receives. As a beginner, remember these main choices:

There are more specialized architectures, but these six give a beginner the most useful map. Later lessons on this site explain CNNs, recurrent networks, transformers, autoencoders, and GANs separately.

A Complete Two-Layer Forward Pass in NumPy

The single-neuron calculation above explains the basic operation. A layer performs several such calculations together. This executable example uses one student row, two hidden neurons, and one output neuron. Every array is written explicitly so the printed intermediate values can be checked.

import numpy as np

X = np.array([[0.8, 0.9, 0.7]])
W1 = np.array([[ 0.5, -0.4],
               [ 0.3,  0.6],
               [-0.2,  0.1]])
b1 = np.array([0.1, -0.2])

Z1 = X @ W1 + b1
A1 = np.maximum(0, Z1)       # ReLU

W2 = np.array([[0.8],
               [0.4]])
b2 = np.array([-0.1])

Z2 = A1 @ W2 + b2
A2 = 1 / (1 + np.exp(-Z2))   # sigmoid

print("Z1:", Z1)  # [[0.63 0.09]]
print("A1:", A1)  # [[0.63 0.09]]
print("Z2:", Z2)  # [[0.44]]
print("A2:", A2)  # [[0.60825903]]

Try the Neuron in Python

This short program repeats the hand calculation from Figure 4. It does not train a full network yet; it shows exactly how one neuron creates an output.

import math

# One student's input values (scaled between 0 and 1)
study = 0.8
attendance = 0.9
sleep = 0.7

# Example weights and bias
weighted_sum = (
    study * 0.7
    + attendance * 0.8
    + sleep * 0.3
    - 0.8
)

# Sigmoid changes the raw score into a value from 0 to 1
probability = 1 / (1 + math.exp(-weighted_sum))

print(round(weighted_sum, 2))  # 0.69
print(round(probability, 2))   # 0.67
print(f"Estimated pass chance: {probability:.0%}")
# Estimated pass chance: 67%

Now connect neurons with Keras

Libraries create and train all those connected neurons for us. This is the smallest readable model for three student inputs and one yes/no output:

import numpy as np
from tensorflow import keras

keras.utils.set_random_seed(7)

# Eight labelled teaching examples. Each row is
# [study, attendance, sleep]; each target is pass=1 or fail=0.
X_train = np.array([
    [0.9, 0.9, 0.8], [0.8, 0.7, 0.7], [0.7, 0.9, 0.6], [0.6, 0.8, 0.8],
    [0.2, 0.4, 0.5], [0.3, 0.3, 0.6], [0.4, 0.5, 0.4], [0.1, 0.6, 0.5],
], dtype=np.float32)
y_train = np.array([1, 1, 1, 1, 0, 0, 0, 0], dtype=np.float32)

model = keras.Sequential([
    keras.layers.Input(shape=(3,)),       # 3 inputs
    keras.layers.Dense(4, activation="relu"),  # 4 hidden neurons
    keras.layers.Dense(1, activation="sigmoid") # pass chance
])

model.compile(
    optimizer="adam",              # how weights are adjusted
    loss="binary_crossentropy",     # how wrong each guess is
    metrics=["accuracy"]
)

model.fit(X_train, y_train, epochs=50, batch_size=4, verbose=0)

chance = model.predict([[0.8, 0.9, 0.7]])[0][0]
print(f"Estimated pass chance: {chance:.0%}")
# The seed makes this teaching run repeatable on a given setup, but minor
# numerical differences can still occur across TensorFlow versions/hardware.

What Can Go Wrong?

Check Your Understanding

Summary: The Whole Idea in Seven Lines

Continue Learning