What functions let the network reduce the impact of a feature or combination of features?

• A. Activation functions
• B. Loss functions
• C. Optimizers
• D. Regularization

Answer: (A)

Activation functions determine how a neuron translates its input into its output, thereby shaping how much a feature can influence the network’s result. They can reduce the impact of a feature by saturating or clipping the signal: for example, sigmoid or tanh compresses large inputs into a limited range, so extreme values contribute less to the output, while ReLU zeros out negative inputs, effectively removing their influence in that neuron. This non-linear transformation lets the network model complex patterns while keeping individual feature contributions from becoming too dominant. By contrast, loss functions define what the network is trying to optimize, optimizers adjust weights during training, and regularization constrains weights to prevent overfitting—each influences learning in a different way, but activation functions are the mechanism that directly modulates how input features are transformed into outputs.