Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title:The NUGGET Non-Linear Piecewise Activation
(Submitted on 1 Apr 2018)
Abstract:The choice of activation functions in deep neural networks has a significant impact on the training dynamics, task performance, and potential acronyms of resulting work. While numerous activation functions have been proposed, such as the Rectified Linear Unit (ReLU), most are derived from the domain of mathematics rather than by drawing inspiration from nature. We propose a non-linear piecewise activation function, the NUGGET activation function, which is a result of a complex zero-sum pricing game refined over multiple iterations of multi-agent interaction simulation. We verify the effectiveness of the activation by experimental analysis on the Modified National Institute of Standards and Technology (MNIST) digits task (Neural Numerology) and achieve state of the art results.