Mathematical formula in calculus you must know for machine learning①

As you may know, for the sake of appropriate understanding towards "machine learning", mathematical knowledge is imperative. In this article, I will share with you some mathematical formula in calculus you must know.

1. Product rule of differentiation¶

"Product rule of differentiation" is one of ways of evaluating derivatives.
$$(f(x)g(x))' = f(x)g'(x) + f'(x)g(x)$$ You can derive this formula by followings with comparative ease.
From definitin of differentiation, $$ \begin{eqnarray} (f(x)g(x))' &=& \lim_{h\rightarrow 0}\frac{f(x+h)g(x+h) - f(x)g(x)}{h} \\ &=& \lim_{h\rightarrow 0}\frac{f(x+h)g(x+h) + f(x+h)g(x) - f(x+h)g(x) - f(x)g(x)}{h} \\ &=& \lim_{h\rightarrow 0}\frac{(g(x+h) - g(x))f(x+h)}{h} + \lim_{h\rightarrow 0}\frac{(f(x+h) -f(x))g(x)}{h}\end{eqnarray}$$ Now we assume $f(x)$ is differentiable, therefore, let alone, $f(x)$ is continuous.　Consequetnly, $\lim_{h\rightarrow 0} f(x+h) = f(x)$. Hence,
$$ \begin{equation} (f(x)g(x))' = g'(x)f(x) + g(x)f'(x) \end{equation}$$

2. Quotient rule of differentiation¶

In calculus, quotient rule of differentiation is a method to find the derivative of a function that is the ratio of two different functions. source : Quotient rule
$$ \begin{equation} \left\{\frac{g(x)}{f(x)}\right\}' = \frac{g'(x) - f'(x)}{\{f(x)\}^2} \end{equation}$$ You can derive this formula as bellow with comparative ease. $$ \begin{eqnarray} (\frac{g(x)}{f(x)})' &=& \lim_{h\rightarrow 0}\frac{\frac{g(x+h)}{f(x+h)} - \frac{g(x)}{f(x)}}{h} \\ &=& \lim_{h\rightarrow 0}\frac{g(x+h)f(x) - g(x)f(x+h)}{f(x)f(x+h)h} \\ &=& \lim_{h\rightarrow 0}\frac{g(x+h)f(x) - f(x)g(x) + f(x)g(x) - g(x)f(x+h)}{f(x)f(x+h)h} \\ &=& \lim_{h\rightarrow 0}\left\{\frac{\left\{g(x+h) - g(x)\right\}f(x)}{h} - \frac{\left\{f(x+h) - f(x)\right\}g(x)}{h}\right\}\frac{1}{f(x)f(x+h)h}\\\end{eqnarray}$$
Since $f(x)$ is differentiable let alone continuous, $$ \begin{equation} \left\{\frac{g(x)}{f(x)}\right\}' = \frac{g'(x) - f'(x)}{\{f(x)\}^2} \end{equation}$$

3. Integration by parts¶

"Integration by parts" or "partial integration" is a process to find the integral of a product of function in terms of the intergral of their derivative and anti derivative. source : Integration by parts $$(f(x)g(x))' = f(x)G(x) - \int f'(x)G(x)dx \ \ \ \ where\ G(x) = \int g(x)dx$$
You can derive this formula by following with comparative ease.
From "product rule of calculus",
$$\begin{eqnarray} (f(x)G(x))' &=& f'(x)G(x) + f(x)G'(x) \\ &=& f'(x)G(x) + f(x)g(x) \end{eqnarray}$$ Therefore,　 　 $$f(x)g(x) = (f(x)G(x))' - f'(x)G(x)$$ Take Integral on both sides, $$\begin{eqnarray} \int f(x)g(x)dx &=& \int(f(x)G(x))' - f'(x)G(x)dx \\ &=& \int(f(x)G(x))'dx - \int f(x)'G(x)dx \\ &=& f(x)G(x) - \int f(x)'G(x) dx\end{eqnarray} $$

4. L'Hôpital's rule¶

"L'Hôpital's rule" uses derivative to help evaluate limits involving indeterminant form. source : L'Hôpital's rule
If $\lim_{x \rightarrow c}f(x) = lim_{x \rightarrow c}g(x) = 0\ or\ \infty$,
$$\lim_{x \rightarrow c} \frac{g(x)}{f(x)} = \lim_{x \rightarrow c}\frac{g'(x)}{f'(x)}$$