Calculus —Derivatives that does the magic
Let us see some terminologies that are important to understand calculus in easy way
What is Derivative?
Derivative is the slope of the tangent line at the instant where it is measured. This is measured as dx/dt where x is the measure of x-axis and t is the measure of y-axis.
Let us take an example of velocity which is change in distance over change in time
So the slope is change in distance / change in time which is also represented as dx/dt
dx/dt is nothing but the change in x ( ∆x) over change in t ( ∆t) is made minimum such that the value is slope of the tangent line which is dx/dt.
Let us represent y-axis with y and x-axis with x so the slope would be dy/dx.
Derivative of f is expressed as f’(x) or dy/dx
Derivative of a constant is zero because slope is always zero as it is constant line there is no change in y
Derivative of a line is ∆y/∆x = a
In the above plot, we can say that f(x) = ax + b
so from the above calculation, df/dx would give us a which is the slope.
Derivative of Quadratic Functions
Let us consider the quadratic function which is f(x) = x²
The Slope is 2x + ∆x. If the ∆x → 0 which means the tangent of the quadratic equation, the slope is 2x
Derivative of Cubic Functions
Consider the simplest form of cubic equation which is y = f(x) = x³
Substituting in the formula for slope we get slope = 3x∆x + 3x² + ∆x²
Now at the tangent of the line or ∆x → 0 then the value of slope is 3x²
Derivative of Power functions
Consider the derivative of simplest form which is 1/x
This is the hyperbola and let us look at the tangent lines.
In the above calculation, the value of slope can be obtained as -(1/(x² + x∆x ))
As ∆x → 0 then the value would be -1/x²
From all the above derivatives we can conclude that the derivatives can be defined as
This holds good for any values of n
Derivative of Inverse Function
Inverse of a function does exactly opposite of what ‘f’ does and it can be represented as ‘g’
For example consider the plot of f(x) = x²
What about the derivatives—> g’(y) = 1 / f’(x)
Derivatives of Trigonometric Functions
Consider the derivative of Trigonometric function which is sin(x)
Th derivative of f(x) = sin(x) is f’(x) = cos(x)
Similarly, for f(x) = cos(x) is f’(x) = -sin(x)
Derivative of Exponential Function
The exponential function is represented by e^x . where e is the euler number. The numerical value of e is 2.71828182.. The decimal never terminates because a number is irrational, which means it cannot be expressed as a ratio between two integers.
As you notice, the more we increase n, the more these converges to a certain number. The actual number that this gets closer and closer is 2.71828182..
Let us calculate the derivative of e^x
When we calculate the slope of the tangent line we get the value as e^x
Derivative of Logarithm
Let us find the derivative of log(x). first let us see the plot of log(x)
As you observe, the log(x) is just opposite to the e^x which you can clearly say
So the derivative of log(y) is 1/y
Non Differential Functions
Functions for which the derivative cannot be found at every point is called non differentiable functions.
If we are able to find the tangent at the point then we have derivative of the functions. Consider a function for which the tangent cannot be drawn for example,
In this case, it is hard to draw tangent at 0. The entire functions is non-differentiable because a derivative does not exist for all the points in the domain.
The non-differential functions are
- The functions that are Corners/Cusps
- Step functions which do not have continuous cannot be differentiable.
- Consider the example f(x) = x^ 1/3
At the point 0, the tangent is vertical and is parallel to the Y-axis and does not have slope.
Properties of Derivative
In order to find derivatives of more complex functions, we have certain rules
Multiplication by a Scalar
If the function g(x) has the function multiplied by scalar factor then the derivative of that function would be multiplied by scalar. For Example, if the function f(x) = x² and we have function f(x) = 4x² which is multiplied by 4, then the derivative of second function is multiplied by 4 of first function which is 8x
Sum Rule
If the function f = g + h, which is the sum of other 2 functions then f’ = g’ + h’
Product Rule
If the function f = gh, which is the product of 2 functions then f’ = g’h + gh’
Suppose we have f(x) = xe^x then f’(x) = xe^x + e^x
Chain Rule
If we have a function with respect to t which is dh/dt and the derivative of g with respect to h then the consider one more function f which is derivative with respect to g then it looks something like this
The chain rule is
But how is this technique helpful in Machine learning
How can I connect derivatives to Machine Learning
In Machine Learning, we use derivatives for optimization. Optimizations is when you want to find the maximum or minimum value of a function. This is very important in machine learning because in machine learning, you want to find the model that best fits your dataset, and in order to find this model what you do is you calculate an error function that tells you how far are you from an ideal model. When you are able to minimize this error function then you have the best model. As you mode to complex problems, there can be more than one minima or maxima where the slope is zero. When you want to optimize a function whether maximizing or minimizing it and the function is differentiable at every point, then you know one thing, it’s that the candidates for maximum and minimum are those points for which the derivative is zero.
