Computer Science
Grade 12
20 min
Introduction to Neural Networks: Perceptrons and Activation Functions
Understand the basic building blocks of neural networks, including perceptrons, activation functions (sigmoid, ReLU, tanh), and their role in learning.
Tutorial Preview
1
Introduction & Learning Objectives
Learning Objectives
Define a perceptron and identify its core components: inputs, weights, and bias.
Calculate the weighted sum (net input) for a given set of inputs, weights, and a bias term.
Explain the purpose of an activation function in a neural network.
Apply different activation functions (Heaviside Step, Sigmoid, ReLU) to a weighted sum to determine a neuron's output.
Model a simple, linearly separable problem, such as a logical AND gate, using a single perceptron.
Analyze how adjusting weights and bias affects the perceptron's output and its decision boundary.
Describe the limitation of a single perceptron in solving non-linearly separable problems like XOR.
How does your phone instantly recognize your face or a photo service automatically tag your frien...
2
Key Concepts & Vocabulary
TermDefinitionExample
PerceptronThe simplest form of an artificial neuron, a computational unit that takes multiple binary inputs, processes them, and returns a single binary output. It's the foundational element of a neural network.A perceptron designed to model an OR gate. It takes two inputs (0 or 1) and outputs 1 if at least one input is 1, otherwise it outputs 0.
Weight (w)A numerical value associated with each input to a perceptron. It represents the strength or importance of that input in the neuron's decision-making process.In a spam detector, the weight for the input 'contains the word free' might be a high positive value like 0.8, while the weight for 'sender is in contacts' might be a high negative value like -1.2.
Bias (b)A constant value added to...
3
Core Syntax & Patterns
Weighted Sum Formula
z = (x₁ * w₁) + (x₂ * w₂) + ... + (xₙ * wₙ) + b OR z = Σ(xᵢ * wᵢ) + b
This formula is the first step in a perceptron's calculation. Multiply each input value (xᵢ) by its corresponding weight (wᵢ), sum all the results, and then add the bias term (b).
Perceptron Output Formula
y = f(z)
The final output (y) of the perceptron is determined by passing the weighted sum (z) through an activation function (f). This step decides whether the neuron 'fires' or not.
Heaviside Step Activation Function
f(z) = 1 if z ≥ 0, else 0
A simple, non-differentiable activation function used in the original perceptron model. It provides a binary output, making it suitable for classification tasks where a hard decision is needed.
Sigmoid Activation Fu...
4 more steps in this tutorial
Sign up free to access the complete tutorial with worked examples and practice.
Sign Up Free to ContinueSample Practice Questions
Challenging
Why is a single perceptron fundamentally incapable of solving the XOR (exclusive OR) problem?
A.The XOR problem requires more than two inputs, which a single perceptron cannot handle.
B.The XOR problem requires a probabilistic output, which is only possible with a Sigmoid function.
C.The XOR problem requires negative weights, which are not allowed in a perceptron.
D.The XOR problem is not linearly separable, meaning its classes cannot be separated by a single straight line.
Challenging
A perceptron with a Heaviside step function needs to output 1 for the inputs x1=0.5, x2=0.5. The weights are fixed at w1=1.0 and w2=1.0. What is the minimum integer value the bias 'b' could have to achieve this output?
A.-2
B.0
C.-1
D.1
Challenging
You are tasked with designing a perceptron to model a logical OR gate using a Heaviside step function. Which set of weights (w1, w2) and bias (b) would work?
A.w1=1, w2=1, b=-0.5
B.w1=1, w2=1, b=-1.5
C.w1=-1, w2=-1, b=1.5
D.w1=0.5, w2=0.5, b=-1.5
Want to practice and check your answers?
Sign up to access all questions with instant feedback, explanations, and progress tracking.
Start Practicing FreeMore from Artificial Intelligence: Deep Learning Fundamentals and Applications
Multi-Layer Perceptrons (MLPs): Architecture and Backpropagation
Convolutional Neural Networks (CNNs): Image Recognition
Recurrent Neural Networks (RNNs): Sequence Modeling
Long Short-Term Memory (LSTM) Networks: Overcoming Vanishing Gradients
Word Embeddings: Representing Words as Vectors (Word2Vec, GloVe)