Linear Approximation and Differentiability

Directional Derivatives

Def: Let ${\mathbb{S}}^n\subseteq {\mathbb{R}}^{n+1}$ is called the $n-$sphere, and it is defined as ${\mathbb{S}}^n=\{x\in {\mathbb{R}}^{n+1}\mid \parallel x\parallel =1\}$ , and an $n-$sphere of radius $r$ is defined as: ${\mathbb{S}}^n(r)=\{x\in {\mathbb{R}}^{n+1}\mid \parallel x\parallel =r\}$. If $x\in {\mathbb{S}}^n$ are gonna be denoted as $\hat{x}$.

Def: Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R}$, $x\in S$ and and $\hat{u}\in {\mathbb{S}}^{n-1}$. We say $f$ is differentiable at $\mathbf{x}$ and in the direction $\hat{u}$ if, the following limit exists:

in that case we say that the directional derivative of $f$ at $x$ in the direction $\hat{\mathbf{u}}$ is denoted as ${\mathrm{\nabla }}_{\hat{u}}f(x),f_{\hat{u}}^{\prime }(x),D_{\hat{u}}f(x),{\partial }_{\hat{u}}f(x),\frac{\partial f}{\partial \hat{u}}$ with their value corresponding to the limit above.

Th: Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R}$, $x\in S$ and $\hat{u}\in {\mathbb{S}}^{n-1}$ if ${\partial }_{\hat{u}}f(x)$ exists then:

Th: Let $f,g:S\subseteq {\mathbb{R}}^n\to \mathbb{R},x\in S$ and $\hat{u}\in {\mathbb{S}}^{n-1}$ if both ${\partial }_{\hat{u}}f(x)$ and ${\partial }_{\hat{u}}g(x)$ exists then:

• ${\partial }_{\hat{u}}(f+g)(x)$ exists and ${\partial }_{\hat{u}}(f+g)(x)={\partial }_{\hat{u}}f(x)+{\partial }_{\hat{u}}g(x)$
• Given $\alpha \in \mathbb{R}$, then ${\partial }_{\hat{u}}(\alpha f)(x)$ exists and ${\partial }_{\hat{u}}(\alpha f)(x)=\alpha {\partial }_{\hat{u}}f(x)$
• ${\partial }_{\hat{u}}(fg)(x)$ exists and ${\partial }_{\hat{u}}(fg)(x)=f(x){\partial }_{\hat{u}}g(x)+{\partial }_{\hat{u}}f(x)g(x)$
• If $g$ is continuous at $x$ and $g(x)\ne 0$, then ${\partial }_{\hat{u}}(f/g)(x)$ exists and:

Chain Rule: Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R},x\in S$, $\hat{u}\in {\mathbb{S}}^{n-1}$ and $f(S)\subseteq (a,b)$. If $g:(a,b)\subseteq \mathbb{R}\to \mathbb{R}$ is differentiable at $f(x)$ and ${\partial }_{\hat{u}}f(x)$ exists, then ${\partial }_{\hat{u}}(g\circ f)((x)$ exists and:

Def: Let $a,b\in {\mathbb{R}}^n$, then the set $[a,b]\subseteq {\mathbb{R}}^n$ is defined as:

MVT for Directional Derivatives:

Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R}$, and $a,b\in S$ where $a\ne b$, $\hat{u}=(b-a)/\parallel b-a\parallel \in {\mathbb{S}}^{n-1}$. If ${\partial }_{\hat{u}}f(x)$ exists for all $x\in [a,b]$, then there’s an $t^{\prime }\in (0,\parallel b-a\parallel )$ such that $c=a+t^{\prime }\hat{u}$,

Def: Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R}$, considering the special case of the directional derivative, such that $\hat{u}\in \{e_i{\}}_{i=1}^n$ then ${\partial }_{e_i}f$ is called the ********************************partial derivative with respect to $x_i$ at $x_0$ usually denoted as: ${\partial }_{x_i}f,{\partial }_{i}f,\frac{\partial f}{\partial x_i},f_x,f_x^{\prime },\frac{\partial }{\partial x_i}f$ .

MVT for Partial Derivatives:

Let $f:S\subseteq {\mathbb{R}}^n\to \mathbb{R}$, and $\mathbf{a}\mathbf{,}\mathbf{b}\in S$ where $\mathbf{a}\ne \mathbf{b}$, and $\mathbf{b}\mathbf{-}\mathbf{a}=(b-a)e_i$. For some $1\le i\le n$, ${\partial }_{i}f(x)$ exists for all $x\in \mathbf{[}\mathbf{a}\mathbf{,}\mathbf{b}\mathbf{]}$, then there’s a $t^{\prime }\in (0,|b-a|)$ such that $\mathbf{c}\mathbf{=}\mathbf{a}+t^{\prime }{e}_i\in \mathbf{[}\mathbf{a}\mathbf{,}\mathbf{b}\mathbf{]}$ that

Total Derivative

Def: For a set $D\subseteq {\mathbb{R}}^n$ with $x_0\in D$, let $D_{x_0}=\{h\in {\mathbb{R}}^n\mid x_0+h\in D\}$.

We will consider the set $D$ as open subset of ${\mathbb{R}}^n$, to simplify things.

Def: Consider a function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$, where $D$ is an open subset of ${\mathbb{R}}^n$, and let $x\in D$. The *************difference function $\delta f_x:D_x\subseteq {\mathbb{R}}^n\to {\mathbb{R}}^n$ is defined by:

Def: Let $g:N\subseteq {\mathbb{R}}^n\to \mathbb{R}$ and $g^{\ast }:M\subseteq {\mathbb{R}}^n\to \mathbb{R}$ be two functions defined on open domains $N$ and $M$ such that $\mathbf{0}\in N\cap M$. We say that $g$ and $g^{\ast }$ *********************************closely approximate each other near $\mathbf{0}\in \mathbb{R}$ if

Equivalently, $g$ and $g^{\ast }$ closely approximate each other near $\mathbf{0}$ if there exists a function $\eta :N\cap M\subseteq {\mathbb{R}}^n\to \mathbb{R}$ such that:

• $g(h)=g^{\ast }(h)+\parallel h\parallel \eta (h)$
• $\underset{h\to \mathbf{0}}{lim}\eta (h)=0$

Def: A function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ which is defined on an open subset $D\in {\mathbb{R}}^n$ is differentiable at $x\in D$ if the difference function $\delta f_x:D_x\subseteq {\mathbb{R}}^n\to \mathbb{R}$ can be closely approximated by a linear function near $\mathbf{0}$.

Equivalently, $f$ is differentiable at $\mathbf{p}$ if there exists a linear function $L:{\mathbb{R}}^n\to \mathbb{R}$ and a function $\eta :D_p\subseteq {\mathbb{R}}^n\to \mathbb{R}$ such that, given $\mathbf{h}\in D_p$:

and

Th: A function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ which is differentiable at $x\in D$ is also continuous there.

Th: Let $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ be differentiable at $x\in D$. Then there sis only one close linear approximation to $\delta f_x$ near $\mathbf{0}$.

Def: If $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ is differentiable at $x\in D$, then the unique close linear approximation to $\delta f_x$ is denoted by $d{f}_x$ and it is called the differential of $f$ at $x$.

Th: If $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ and $x\in D$, if $f$ is differentiable at $x$, then for any $\hat{u}\in {\mathbb{S}}^{n-1}$ ${\partial }_{\hat{u}}f(x)$ exists, and it is equal to $d{f}_x(\hat{u})$.

• Properties

  Let $f,g:D\subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ , $x\in D$ if $f$ and $g$ are differentiable at $x$. Then

  $f+g$ is differentiable at $x$

  $$d(f+g{)}_x=d{f}_x+d{g}_x$$

  Let $\lambda \in \mathbb{R}$, then $\lambda f$ is differentiable at $x$

  $$d(\lambda f{)}_x=\lambda d{f}_x$$

  $fg$ is differetiable at $x$

  $$d(fg{)}_x=f(x)d{g}_x+g(x)d{f}_x$$

  if $g(x)\ne 0$, then $f/g$ is differentiable at $x$

  $$d{\left(\frac{f}{g}\right)}_x=\frac{1}{g^2(x)}(g(x)d{f}_x-f(x)d{g}_x)$$

Cor: Let $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ be differentiable at $x\in D$. Then:

• the partial derivative ${\partial }_{i}f(x)$ exists and it is equal to $d{f}_x(e_i)$, for each $1\le i\le n$.

• for all $h\in {\mathbb{R}}^n$,

  $$d{f}_{\mathbf{p}}(h)=\sum _{i=1}^n(h\cdot e_i){\partial }_{i}f(x)$$

• the $1\times n$ matrix representing $d{f}_x$, with respect to the canonical basis is:

  $$J_f(x)=[{\partial }_{1}f(x)\cdots {\partial }_{n}f(x)]$$

  The matrix $J_f(x)$ representing the differential $d{f}_x$ is called the Jacobian matrix or $f$at $x$. Other notation regarding the Jacobian is

  $$J_f(p)=\frac{\partial f}{\partial \mathbf{x}}(x)$$

Th: The function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ is differentiable at $x\in D$ iff:

• all partial derivatives of $f$ exist at $x$

• there exists a function $\eta :D_x\subseteq {\mathbb{R}}^n\to \mathbb{R}$ such that for all $h\in D_x$

  $$f(x+h)-f(x)=\sum _{k=1}^n(h\cdot e_k){\partial }_{k}f(x)+\parallel h\parallel \eta (h)$$

  where

  $$\underset{h\to \mathbf{0}}{lim}\eta (h)=0$$

Th: Let $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$, $\mathbf{x}\in D$, $r>0$ such that $B_r(\mathbf{x})\subseteq U$. If for any $1\le i\le n$, and all $\mathbf{y}\in B_r(\mathbf{x})$, ${\partial }_{i}f(\mathbf{y})$ exists and is continuous at $\mathbf{x}$. Then, $f$ is differentiable at $\mathbf{x}$.

Th of Pain: Let $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$, $\mathbf{x}\in D$, $r>0$ such that $B_r(\mathbf{x})\subseteq U$. If for any $1\le i\le n$, and all $\mathbf{y}\in B_r(\mathbf{x})$, ${\partial }_{i}f(\mathbf{y})$ exists and for $2\le i\le n$ , ${\partial }_{i}f$ is continuous at $\mathbf{x}$. Then, $f$ is differentiable at $\mathbf{x}$.

Def: A function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$, whose partial derivatives exists throughout a neighbourhood of $\mathbf{x}\in D$ and are continuous at $\mathbf{x}$, is said to be continuously differentiable at $\mathbf{x}$.

Def: A function $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ is said to be continuously differentiable or a ${\mathcal{C}}^{\mathcal{1}}$ function, if it is continuously differentiable at each point $\mathbf{x}\in D$

Euler's Homogeneous Function Theorem

Let $f$ be a homogeneous function of order $n$, meaning

Then we get that

Tangent Space

Let $G\subseteq {\mathbb{R}}^n$ be the graph a differentiable function $f:D\subseteq {\mathbb{R}}^{n-1}\to \mathbb{R}$. For any ${\mathbf{x}}_0\in D$, the set $T\in {\mathbb{R}}^n$ with the equation

where ${\mathbf{x}}_0=(x_1,\dots ,x_{n-1})\in {\mathbb{R}}^{n-1}$. $T$ is called the **************tangent space to $G$ at $({\mathbf{x}}_0,f({\mathbf{x}}_0))$

Chain Rule

Let $g:E\subseteq \mathbb{R}\to {\mathbb{R}}^n$ be defined on an open interval $E$ and let $f:D\subseteq {\mathbb{R}}^n\to \mathbb{R}$ be defined on an open set $D$ such that $g(E)\subseteq D$. Define $F:E\subseteq \mathbb{R}\to \mathbb{R}$ to be the composite function given by $F(t)=(f\circ g)(t)$

Suppose that $g$ is differentiable at $a\in E$ and that $f$ is differentiable at $g(a)\in D$. Then $F$ is differentiable at a $a$ and its differential is given by:

Thus:

Equivalently:

where the partials are evaluated at $g(a)$ and $x_k^{\prime }$ is evaluated at $a$.