Few edits on autodiff for clarity

Author: ansantam

tutorial.ipynb

@@ -240,7 +240,7 @@
     "  <img src=\"fig/compgraph.png\" alt=\"test\" style=\"width:60%; margin:auto; display:block;\"/>\n",
     "</a>\n",
     "\n",
-    "| $\\frac{dz}{dx}$       | $\\frac{dz}{dy}$ |\n",
+    "| $\\frac{\\partial z}{\\partial x}$       | $\\frac{\\partial z}{\\partial y}$ |\n",
     "|--------------|---------|\n",
     "| $ \\frac{\\partial z}{\\partial x} = \\frac{\\partial u_1}{\\partial x} + \\frac{\\partial u_2}{\\partial x}$ | $\\frac{\\partial z}{\\partial y} = \\frac{\\partial u_1}{\\partial y} + \\frac{\\partial u_2}{\\partial y}$ | \n",
     "| $\\frac{\\partial u_1}{\\partial x} = \\frac{\\partial (x^2)}{\\partial x} = 2x $ | $\\frac{\\partial u_1}{\\partial y} = \\frac{\\partial (x^2)}{\\partial y} = 0$ | \n",
@@ -260,19 +260,42 @@
    "source": [
     "<h2 style=\"color: #b51f2a\">Forward mode and reverse mode autodiff</h2>\n",
     "\n",
+    "$z = x^2 + 3xy +1 \\ ; \\ u_1 = x^2 \\ ; \\ u_2 = 3xy \\ ; \\ z = u_1 + u_2 + 1$\n",
+    "\n",
     "### Forward mode\n",
-    "- Computes derivatives in a single forward pass in the computational graph, computing the derivatives alongside function values.\n",
-    "- Computes derivatives **one input at a time**.\n",
+    "- Propagates derivatives from inputs to outputs in a single forward pass in the computational graph by computing function values and their derivatives simultaneously.\n",
+    "    - the calculation is basically what we saw in the previous slide\n",
     "\n",
     "### Reverse mode\n",
-    "-  Computes gradients backward in the computational graph using the chain rule (used in <u>backpropagation</u>).\n",
-    "- Computes derivatives **one output at a time** but for all inputs.\n",
-    "\n",
+    "- Computes gradients backward in the computational graph using the chain rule (used in <u>backpropagation</u>)\n",
+    "    - Start with $ \\frac{\\partial z}{\\partial z} = 1 $\n",
+    "    - Compute contributions to intermediate variables:\n",
+    "    - $ \\frac{\\partial z}{\\partial u_1} = 1, \\quad \\frac{\\partial z}{\\partial u_2} = 1 $\n",
+    "    - $ \\frac{\\partial u_1}{\\partial x} = 2x $\n",
+    "    - $ \\frac{\\partial u_2}{\\partial x} = 3y, \\quad \\frac{\\partial u_2}{\\partial y} = 3x $\n",
+    "    - Apply the chain rule:\n",
+    "    - $ \\frac{\\partial z}{\\partial x} = \\frac{\\partial z}{\\partial u_1} \\cdot \\frac{\\partial u_1}{\\partial x} + \\frac{\\partial z}{\\partial u_2} \\cdot \\frac{\\partial u_2}{\\partial x} = 2x + 3y $\n",
+    "    - $ \\frac{\\partial z}{\\partial y} = \\frac{\\partial z}{\\partial u_2} \\cdot \\frac{\\partial u_2}{\\partial y} = 3x $\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "<h2 style=\"color: #b51f2a\">Forward mode and reverse mode autodiff</h2>\n",
     "\n",
     "| Mode          | Best for | Complexity | Common Use Cases |\n",
     "|--------------|---------|------------|------------------|\n",
     "| Forward Mode | Few inputs, many outputs | O(n) per input | Physics simulations, sensitivity analysis |\n",
-    "| Reverse Mode | Many inputs, few outputs | O(n) per output | Deep learning, optimization problems |\n"
+    "| Reverse Mode | Many inputs, few outputs | O(n) per output | Deep learning, optimization problems |\n",
+    "\n",
+    "<a href=https://e-dorigatti.github.io/math/deep%20learning/2020/04/07/autodiff.html>\n",
+    "  <img src=\"fig/compgraph.png\" alt=\"test\" style=\"width:60%; margin:auto; display:block;\"/>\n",
+    "</a>"
    ]
   },
   {