We gratefully acknowledge support from
the Smerity Research institute
and member institutions
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title:The NUGGET Non-Linear Piecewise Activation

Abstract:The choice of activation functions in deep neural networks has a significant impact on the training dynamics, task performance, and potential acronyms of resulting work. While numerous activation functions have been proposed, such as the Rectified Linear Unit (ReLU), most are derived from the domain of mathematics rather than by drawing inspiration from nature. We propose a non-linear piecewise activation function, the NUGGET activation function, which is a result of a complex zero-sum pricing game refined over multiple iterations of multi-agent interaction simulation. We verify the effectiveness of the activation by experimental analysis on the Modified National Institute of Standards and Technology (MNIST) digits task (Neural Numerology) and achieve state of the art results.
Comments: 3 pages
Subjects: Computation and Language (cs.CL); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Cite as: arXiv:1804.404 [cs.CL]
 

Submission history

From: Stephen Merity [view email]
[v1] Fri, 23 Feb 2018 08:20:23 GMT (804kb,D)