Proceedings of the Second Annual IEEE Conference on Neural Networks,
New York: IEEE Press, I:657-I:664, July 1988

THERE EXISTS A NEURAL NETWORK THAT
DOES NOT MAKE AVOIDABLE MISTAKES

A. Ronald Gallant and Halbert White

Department of Statistics, North Carolina State University
Department of Economics, University of California San Diego

ABSTRACT

We show that a multiple input, single output, single hidden layer feedforward network with (known) hardwired connections from input to hidden layer, monotone squashing at the hidden layer and no squashing at the output embeds as a special case a "Fourier network" which, yields a Fourier series approximation to a given function as its output. Thus, such networks possess all the approximation properties of Fourier series representations. In particular, approximation to any desired accuracy of any square integrable function can be achieved by such a network, using sufficiently many hidden units. In this sense, such networks do not make avoidable mistakes.