Analysis on Manifolds — Fall 2025

Lecture 2: Derivatives in Several Variables

KSM3E04: Analysis on Manifolds — Fall 2025 | August 13, 3:00–4:00 PM

From single-variable calculus, we know that the derivative captures the “best linear approximation” of a function at a point. But what happens when the inputs and outputs are no longer numbers, but vectors? How should we extend the notion of the derivative to higher dimensions?

The obstacle is clear: we cannot divide vectors as we did with real numbers. The natural idea, then, is to keep the spirit of the one-variable definition: near a point, a differentiable function should behave almost like a linear map.

This reasoning leads us to the total derivative, a precise formulation of differentiability for functions $f:\mathbf{R}^n \to \mathbf{R}^m$. In this lecture, we trace this line of thought: starting from the familiar case of one variable, asking what carries over, and step by step arriving at the higher-dimensional definition. We will also work out an example and see why the derivative, when it exists, is unique.

1. Rethinking the Derivative in One Variable

For $f:U\subset \mathbf{R}\to\mathbf{R}$, with $U$ open and $x_0\in U$, the usual derivative is

$$f'(x_0) := \lim_{x\to x_0;\,x\in U\setminus\{x_0\}} \frac{f(x)-f(x_0)}{x-x_0}.$$

To avoid dividing vectors in higher dimensions, we reinterpret the derivative as a linear approximation.

Lemma 1 (One-variable reformulation).

Let $U\subset\mathbf{R}$ be open, $f:U\to\mathbf{R}$, $x_0\in U$, and $L\in\mathbf{R}$. The following are equivalent:

$f$ is differentiable at $x_0$ and $f'(x_0)=L$.
$$\lim_{x\to x_0;\,x\in U\setminus\{x_0\}} \frac{\,\big|f(x) - \big(f(x_0) + L(x-x_0)\big)\big|\,}{|x-x_0|} = 0.$$

In words: $f(x) - f(x_0) \approx L(x-x_0)$, with an error that is small compared to $|x-x_0|$.

Proof

(1) $\Rightarrow$ (2): Suppose $f$ is differentiable at $x_0$ with $f'(x_0)=L$. Then

\[ \begin{align*} \lim_{x\to x_0} \frac{f(x)-f(x_0)}{x-x_0} &= L, \\ \lim_{x\to x_0} \left(\frac{f(x)-f(x_0)}{x-x_0} - L\right) &= 0, \\ \lim_{x\to x_0} \frac{f(x)-f(x_0) - L(x-x_0)}{x-x_0} &= 0, \\ \lim_{x\to x_0} \frac{|f(x)-f(x_0) - L(x-x_0)|}{|x-x_0|} &= 0. \end{align*} \]

which is exactly condition (2).

(2) $\Rightarrow$ (1): Assume condition (2). Then

\[ \begin{align*} \lim_{x\to x_0} \frac{f(x)-f(x_0) - L(x-x_0)}{x-x_0} &= 0, \\ \lim_{x\to x_0} \frac{f(x)-f(x_0)}{x-x_0} &= L. \end{align*} \]

Hence $f$ is differentiable at $x_0$ with derivative $L$.

Therefore, the two formulations are equivalent.

This perspective scales to $\mathbf{R}^n\to\mathbf{R}^m$ by replacing the scalar $L$ with a linear map. Importantly, this linear transformation depends on the chosen point $x_0$ and may change from one point to another.

2. Differentiability in $\mathbf{R}^n\to\mathbf{R}^m$

Definition 1 (Total derivative).

Let $U\subseteq \mathbf{R}^n$ be an open set, $f:U \to \mathbf{R}^m$, and $x_0 \in U$. A linear map $L:\mathbf{R}^n \to \mathbf{R}^m$ is called the derivative of $f$ at $x_0$ if

$$\lim_{x\to x_0;\,x\in U\setminus\{x_0\}} \frac{\| f(x) - (f(x_0) + L(x-x_0)) \|}{\|x-x_0\|} = 0,$$

where $\|\cdot\|$ denotes the Euclidean norm. In other words, near $x_0$, the function $f$ is well-approximated by the linear transformation $L$, and the error vanishes faster than $\|x-x_0\|$ as $x \to x_0$.

Notation: When the derivative exists, it is a linear map that depends on the point $x_0$. It is denoted by $Df(x_0)$, $df_{x_0}$, or sometimes $f'(x_0)$. The notation $df_{x_0}$ is often used to emphasize the connection to the differential of $f$. The uniqueness of this map will be established later in the lecture.

Intuitively, we can think of the total derivative as a higher-dimensional generalization of the tangent line in one-variable calculus: it provides the best linear approximation of $f$ near the point $x_0$.

3. Worked Example & Uniqueness of the Derivative

Example 2. Let $U \subset \mathbf{R}^2$ be an open set, and consider $f:U\to\mathbf{R}^2$ given by $$f(x,y) = (x^2 + y, \, x y^2 + y^3)$$ at the point $x_0=(1,2)$. We want to understand what the derivative $Df(x_0)$ looks like. Intuitively, it should be a linear map $L:\mathbf{R}^2 \to \mathbf{R}^2$ such that

$$\lim_{(x,y)\to(1,2)} \frac{\| f(x,y) - (f(1,2) + L((x,y)-(1,2))) \|}{\|(x,y)-(1,2)\|} = 0.$$

Setting $(x,y) = (1,2) + (a,b)$, we expand

$$ \begin{align*} f(1+a,2+b) - f(1,2) &= ((1+a)^2 + (2+b) - (1^2+2), \, (1+a)(2+b)^2 + (2+b)^3 - (1\cdot 2^2 + 2^3)) \\ &= (2a + b + a^2, \, 4a + 12b + \text{higher order terms in } a,b) \end{align*} $$

The derivative $Df(1,2)$ is the linear part of this expansion: $$Df(1,2)(a,b) = (2a + b, \, 4a + 12b).$$ We will see systematic ways to compute such derivatives explicitly in the next lecture.

It is natural to ask whether this derivative is unique. Indeed, it is:

Lemma 2 (Uniqueness of the derivative)

Suppose $x_0$ is an interior point of $U \subset \mathbf{R}^n$ (assumed open) and $f:U \to \mathbf{R}^m$ is differentiable at $x_0$ with derivatives $L_1$ and $L_2$. Then $L_1 = L_2$.

Proof

By definition, differentiability at $x_0$ with $L_1$ and $L_2$ gives

$$ \begin{align*} \lim_{x \to x_0} \frac{\|f(x) - (f(x_0) + L_1(x-x_0))\|}{\|x-x_0\|} &= 0, \\ \lim_{x \to x_0} \frac{\|f(x) - (f(x_0) + L_2(x-x_0))\|}{\|x-x_0\|} &= 0. \end{align*} $$

Subtracting the two expressions and using the triangle inequality,

$$ \begin{align*} \frac{\|(L_2 - L_1)(x-x_0)\|}{\|x-x_0\|} &\le \frac{\|f(x) - (f(x_0) + L_1(x-x_0))\|}{\|x-x_0\|} + \frac{\|(f(x_0) + L_2(x-x_0)) - f(x)\|}{\|x-x_0\|} \\ &\to 0 \quad \text{as } x \to x_0. \end{align*} $$

Let $L = L_2 - L_1$. We have shown that $\lim_{h \to 0} \frac{\|L(h)\|}{\|h\|} = 0$. To show $L=0$, consider any non-zero vector $v \in \mathbf{R}^n$. Let $h = tv$ for $t \in \mathbf{R}, t \to 0$. Then

$$ \begin{align*} \lim_{t \to 0} \frac{\|L(tv)\|}{\|tv\|} &= \lim_{t \to 0} \frac{|t| \|L(v)\|}{|t| \|v\|} = \frac{\|L(v)\|}{\|v\|}. \end{align*} $$

Since this limit must be 0, we have $\|L(v)\|/\|v\| = 0$, which implies $L(v)=0$ for any $v \neq 0$. Since $L$ is linear, $L(0)=0$, so $L(v)=0$ for all $v \in \mathbf{R}^n$. Thus $L=0$, and $L_1 = L_2$.

Consequently, we may unambiguously write the Newton approximation

$$f(x) \approx f(x_0) + Df(x_0)(x-x_0) \quad (x\to x_0).$$

This is the higher-dimensional analogue of the tangent-line approximation. Later lectures will formalize the derivative as a Jacobian matrix for practical computation.

Remarks

If $f=g$ on $U$ and both are differentiable at $x_0$, then $Df(x_0)=Dg(x_0)$.
Verifying differentiability from first principles can be tedious; Jacobian matrices provide a systematic alternative.

4. Partial and Directional Derivatives

We now connect the abstract notion of the total derivative with the more concrete tools of partial and directional derivatives.

Definition 3 (Directional derivative).

Let $U \subset \mathbf{R}^n$ be an open set, $f:U\to\mathbf{R}^m$, $x_0\in U$, and $v\in\mathbf{R}^n$. If the limit

$$D_v f(x_0) := \lim_{t\to 0} \frac{f(x_0+tv)-f(x_0)}{t}$$

exists, we say $f$ is differentiable in the direction $v$ at $x_0$.

Lemma 3.

If $f$ is differentiable at $x_0$, then for any $v\in\mathbf{R}^n$, the directional derivative $D_v f(x_0)$ exists, and it is given by the action of the total derivative on the vector $v$:

$$D_v f(x_0) = Df(x_0)(v).$$

Proof

By definition of differentiability at $x_0$, there exists a function $\varepsilon(h)$ with $\lim_{h\to 0} \varepsilon(h) = 0$ such that

$$f(x_0 + h) - f(x_0) = Df(x_0)(h) + \|h\| \, \varepsilon(h).$$

Set $h = t v$ for a scalar $t\neq 0$. Then

$$ \begin{align*} \frac{f(x_0 + t v) - f(x_0)}{t} &= \frac{Df(x_0)(tv) + \|tv\|\varepsilon(tv)}{t} \\ &= \frac{t Df(x_0)(v) + |t|\|v\|\varepsilon(tv)}{t} \\ &= Df(x_0)(v) + \frac{|t|}{t}\|v\|\varepsilon(tv). \end{align*} $$

As $t \to 0$, $\varepsilon(tv) \to 0$. Since $|t|/t$ is bounded by 1, the second term vanishes. Taking the limit gives

$$D_v f(x_0) = Df(x_0)(v).$$

Hence, the lemma is proved.

Definition 4 (Partial derivative).

Let $\{e_1, \dots, e_n\}$ be the standard basis for $\mathbf{R}^n$. The partial derivative of $f$ with respect to the $j$-th coordinate $x^j$ at $x_0$ is the directional derivative in the direction $e_j$:

$$\frac{\partial f}{\partial x^j}(x_0) := D_{e_j}f(x_0) = \lim_{t\to 0} \frac{f(x_0 + t e_j) - f(x_0)}{t},$$

provided the limit exists.

This lemma provides the crucial link between the abstract linear map $Df(x_0)$ and a concrete matrix representation. Since $Df(x_0)$ is a linear map, its action on any vector $v = \sum v^j e_j$ is determined by its action on the basis vectors $e_j$. This leads us to the Jacobian matrix.

Definition 5 (The Jacobian Matrix).

If $f: U \subset \mathbf{R}^n \to \mathbf{R}^m$ is differentiable at $x_0$, its derivative is the linear map $Df(x_0): \mathbf{R}^n \to \mathbf{R}^m$. The matrix representation of this linear map with respect to the standard bases of $\mathbf{R}^n$ and $\mathbf{R}^m$ is called the Jacobian matrix of $f$ at $x_0$. It is denoted by $J_f(x_0)$ or simply by $Df(x_0)$, and its columns are the vectors $\frac{\partial f}{\partial x^j}(x_0)$:

$$ J_f(x_0) = Df(x_0) = \begin{pmatrix} \frac{\partial f^1}{\partial x^1}(x_0) & \cdots & \frac{\partial f^1}{\partial x^n}(x_0) \\ \vdots & \ddots & \vdots \\ \frac{\partial f^m}{\partial x^1}(x_0) & \cdots & \frac{\partial f^m}{\partial x^n}(x_0) \end{pmatrix}. $$

The $(i,j)$-th entry of the matrix is $\frac{\partial f^i}{\partial x^j}(x_0)$, where $f^i$ is the $i$-th component function of $f$.

Note that the existence of partial derivatives alone does not guarantee differentiability. However, if all partial derivatives $\frac{\partial f^i}{\partial x^j}$ exist and are continuous in a neighborhood of $x_0$, then $f$ is differentiable at $x_0$.

5. Problems for Practice

Exercise 1: Differentiability of a Matrix Map

Let $A$ be an $m \times n$ real matrix, and define $T:\mathbf{R}^n \to \mathbf{R}^m$ by $T(x)=Ax$.

Prove directly from the definition that $T$ is differentiable at every $x_0 \in \mathbf{R}^n$.
Show that the derivative $DT(x_0)$ is the linear map represented by $A$ itself, independent of $x_0$.
Observe that this illustrates the general fact that linear maps are their own derivatives.

Exercise 2: Using the Jacobian Matrix

Let $f:\mathbf{R}^3\to\mathbf{R}^4$ be given by $f(x,y,z)=(xy, yz, xz, xyz)$. Fix $x_0=(1,1,1)$. Compute the Jacobian matrix $J_f(x_0)$ by calculating the partial derivatives.

Exercise 3: A Two-Variable Example

For $f:\mathbf{R}^2\to\mathbf{R}^2$, $f(x,y)=(x^2,y^2)$ and $x_0=(1,2)$, let $h=(a,b)$. Show that the error term $E(h) = f(x_0+h) - f(x_0) - L(h)$ for the correct derivative $L$ satisfies $\lim_{h\to 0} \|E(h)\|/\|h\| = 0$.

Exercise 4: Constant Maps and the Converse

Let $f:\mathbf{R}^n \to \mathbf{R}^m$ be the constant map $f(x)=c$, where $c \in \mathbf{R}^m$ is fixed.

Show directly from the definition that $f$ is differentiable at every point $x_0 \in \mathbf{R}^n$.
Prove that $Df(x_0)$ is the zero linear map.
Prove the converse: if $U \subset \mathbf{R}^n$ is a connected open set and $Df(x)=0$ for all $x \in U$, then $f$ must be constant on $U$.
Compare this with Exercise 1 and note the contrast: linear maps are their own derivatives, whereas constant maps have zero derivative.

Exercise 5: Vanishing Derivative at Local Extrema

Let $f:\mathbf{R}^n \to \mathbf{R}$ be differentiable at $x_0$.

Prove that if $x_0$ is a point of local maximum or local minimum, then $Df(x_0)=0$.
Now consider the converse: does $Df(x_0)=0$ necessarily imply that $x_0$ is a local extremum? Examine the example $f(x,y)=x^2 - y^2$. Here $Df(0,0)=0$, yet $(0,0)$ is not a maximum or minimum.
Think carefully about what happens along different directions:
- Along the $x$–axis ($y=0$), $f(x,0)=x^2$ has a local minimum at $x=0$.
- Along the $y$–axis ($x=0$), $f(0,y)=-y^2$ has a local maximum at $y=0$.
Because the function behaves like a maximum in one direction and a minimum in another, the point $(0,0)$ is called a saddle point. This illustrates that vanishing derivative alone does not guarantee an extremum.

Exercise 6: Differentiability via Components

Let $f:\mathbf{R}^n \to \mathbf{R}^m$ be given by $f(x)=(f^1(x),\dots,f^m(x))$, where each $f^i:\mathbf{R}^n \to \mathbf{R}$ is a real-valued function.

Prove that $f$ is differentiable at $x_0 \in \mathbf{R}^n$ if and only if each component $f^i$ is differentiable at $x_0$.
Show that in this case, the Jacobian matrix $Df(x_0)$ is obtained by stacking the gradients $\nabla f^i(x_0)$ as rows.

Exercise 7: The Derivative as a Linear Operator

Let $U \subset \mathbf{R}^n$ be an open set, and let $f, g: U \to \mathbf{R}^m$ be differentiable at a point $x_0 \in U$. Let $c \in \mathbf{R}$ be a scalar.

Prove that the function $f+g$ is differentiable at $x_0$ and that $D(f+g)(x_0) = Df(x_0) + Dg(x_0)$.
Prove that the function $cf$ is differentiable at $x_0$ and that $D(cf)(x_0) = c \cdot Df(x_0)$.

These two properties show that for a fixed point $x_0$, the act of differentiation is a linear transformation from the vector space of functions differentiable at $x_0$ to the vector space of linear maps from $\mathbf{R}^n$ to $\mathbf{R}^m$.

← Lecture 1 Lecture 3 →