massOfai

Activation Function

Neural Networks

Non-linear function determining neuron output

What is Activation Function?

Introduces non-linearity to neural networks, allowing them to learn complex patterns. Common functions: ReLU, sigmoid, tanh.

Real-World Examples

  • •ReLU: max(0, x)
  • •Sigmoid: 1/(1+e^-x)
  • •Tanh: (e^x - e^-x)/(e^x + e^-x)

When to Use This

Every neuron in a neural network needs an activation function

Related Terms

Learn more about concepts related to Activation Function

Neural Network

Brain-inspired computing model with interconnected nodes

ReLU (Rectified Linear Unit)

Activation function: max(0, x)

Sigmoid Function

S-shaped activation: 1/(1+e^-x)

Backpropagation

Algorithm for training neural networks