We will start from the one-variable definition of the derivative at a point p, namely
Let's change above to equivalent form of
which achieved after pulling f'(p) inside and putting it over a common denominator.
We can't divide by vectors, so this definition can't be immediately extended to the multiple variable case. Nonetheless, we don't have to: the thing we took interest in was the quotient of two small distances (magnitudes), not their other properties (like sign). It's worth noting that 'other' property of vector neglected is its direction. Now we can divide by the absolute value of a vector, so lets rewrite this definition in terms of absolute values
Another form of formula above is obtained by letting we have and if , the , so
where can be thought of as a 'small change'.
So, how can we use this for the several-variable case?
If we switch all the variables over to vectors and replace the constant (which performs a linear map in one dimension) with a matrix (which denotes also a linear map), we have
If this limit exists for some f : Rm → Rn, and there is a linear map A : Rm → Rn (denoted by matrix A which is m×n), we refer to this map as being the derivative and we write it as Dp f.
A point on terminology - in referring to the action of taking the derivative (giving the linear map A), we write Dp f, but in referring to the matrix A itself, it is known as the Jacobian matrix and is also written Jp f. More on the Jacobian later.
There are a number of important properties of this formulation of the derivative.
If f is differentiable at p for x close to p, |f(x)-(f(p)+A(x-p))| is small compared to |x-p|, which means that f(x) is approximately equal to f(p)+A(x-p).
We call an expression of the form g(x)+c affine, when g(x) is linear and c is a constant. f(p)+A(x-p) is an affine approximation to f(x).
Jacobian matrix and partial derivativesEdit
The Jacobian matrix of a function is in the form
for a f : Rm → Rn, Jp f is a n×m matrix.
The consequence of this is that if f is differentiable at p, all the partial derivatives of f exist at p.
However, it is possible that all the partial derivatives of a function exist at some point yet that function is not differentiable there, so it is very important not to mix derivative (linear map) with the Jacobian (matrix) especially in situations akin to the one cited.
Continuity and differentiabilityEdit
Furthermore, if all the partial derivatives exist, and are continuous in some neighbourhood of a point p, then f is differentiable at p. This has the consequence that for a function f which has its component functions built from continuous functions (such as rational functions, differentiable functions or otherwise), f is differentiable everywhere f is defined.
We use the terminology continuously differentiable for a function differentiable at p which has all its partial derivatives existing and are continuous in some neighbourhood at p.