Neural Network Architecture

Learning Objectives Define and differentiate between key components of a neural network: neurons, layers, weights, and biases. Explain the flow of data through a feedforward neural network, from the input to the output layer. Compare and contrast different types of layers: input, hidden, and output. Describe the role of an activation function and provide examples like ReLU and Sigmoid. Design a simple neural network architecture for a given classification problem. Calculate the total number of trainable parameters (weights and biases) in a given network architecture. Explain how network 'depth' and 'width' impact model complexity. Ever wonder how your phone instantly recognizes your face or how a translation app works? 🤔 It's all about the bluepri...

TermDefinitionExample Neuron (or Node)The fundamental processing unit of a neural network. It receives one or more weighted inputs, applies an activation function, and produces an output.In a network classifying images, a neuron in the first hidden layer might activate only when it detects a horizontal edge in a small part of the image. LayerA collection of neurons that operate together at a specific depth within the network. Data is processed one layer at a time.A network has an Input Layer for raw pixel data, several Hidden Layers for feature extraction (like finding edges, then shapes, then objects), and an Output Layer for the final classification (e.g., 'cat' or 'dog'). Weights and BiasesWeights are parameters that determine the strength of the connection between...

Input Layer Sizing Rule Number of input neurons = Number of features in the input data. This rule defines the entry point of your network. For a 28x28 pixel grayscale image, the input data is 784 pixels, so you need 784 input neurons. For a dataset with 10 features (e.g., age, income, etc.), you need 10 input neurons. Output Layer Design Rule Binary Classification: 1 output neuron (Sigmoid activation). Multi-Class Classification (N classes): N output neurons (Softmax activation). This rule determines the network's final output structure. Use Sigmoid to get a single probability between 0 and 1 (e.g., cat vs. not-cat). Use Softmax to get a probability distribution across multiple classes (e.g., cat, dog, or bird), where all probabilities sum to 1. Parameter Calculatio...