I searched on the youtube to find the reasonings to the learning machines because I wanted to understand the theoretical background. I found this biological explanation which shows where the summation Σ function and Sigmoid function came from.
Equation of Sigmoid:
Derivative of Sigmoid:
return 1.0 / ( 1.0 + np.exp( -x ) )
y = sigmoid_fn( x )
return y * ( 1.0 – y )
Derivative of the Hyperbolic Tangent Function.
[Matrix 1 X 4 ] * [Matrix 1 X 4]
[Matrix 1 X 4 ] * [Matrix 4 X 1]