Deep Learning Frameworks: TensorFlow and PyTorch

Learning Objectives Compare and contrast the core philosophies of TensorFlow (define-and-run) and PyTorch (define-by-run). Define a Tensor and explain its role as the fundamental data structure in deep learning frameworks. Construct a simple sequential neural network model using both TensorFlow's Keras API and PyTorch's nn.Module. Implement a basic training loop, including forward pass, loss calculation, backpropagation, and optimizer step. Differentiate between eager execution (PyTorch default) and graph execution (TensorFlow default). Explain the role of an optimizer and a loss function in the model training process. Ever wonder how your phone recognizes your face or how Netflix recommends the perfect movie? 🤖 The magic behind it is powered by powerful tools cal...

TermDefinitionExample TensorThe fundamental data structure in both TensorFlow and PyTorch. It is a multi-dimensional array, similar to a NumPy array, that can run on GPUs for accelerated computing.A 256x256 color image can be represented as a 3D tensor of shape (256, 256, 3), where the dimensions are height, width, and color channels (Red, Green, Blue). Computational GraphA directed acyclic graph (DAG) where nodes represent mathematical operations and edges represent the tensors that flow between them. Frameworks use this graph to calculate gradients during backpropagation.For the expression `c = (a * b) + 1`, the graph has input nodes `a` and `b`, a multiplication node, an addition node, and an output node `c`. Eager Execution (Define-by-Run)An imperative programming style where operatio...

TensorFlow (Keras) Sequential Model import tensorflow as tf model = tf.keras.Sequential([ tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10, activation='softmax') ]) Use the `tf.keras.Sequential` API for building simple models that are a linear stack of layers. You define the model by passing a list of layer instances. It's the quickest way to get started in TensorFlow. PyTorch nn.Module Class import torch.nn as nn class MyModel(nn.Module): def __init__(self): super().__init__() self.layer1 = nn.Linear(784, 128) self.layer2 = nn.Linear(128, 10) def forward(self, x): x = nn.functional.relu(self.layer1(x)) return self.layer2(x) Use a custom class that inherits from `torch.nn.Module` for building any...