Deakin University
Browse

File(s) under permanent embargo

Determining regularization parameters for derivative free neural learning

conference contribution
posted on 2005-12-01, 00:00 authored by R Ghosh, M Ghosh, John YearwoodJohn Yearwood, A Bagirov
Derivative free optimisation methods have recently gained a loi of attractions for neural learning. The curse of dimensionality for the neural learning problem makes local optimization methods very attractive; however the error surface contains many local minima. Discrete gradient method is a special case of derivative free methods based on bundle methods and has the ability lo jump over many local minima. There are two types of problems that are associated with this when local optimization methods are used for neural learning. The first type of problems is initial sensitivity dependence problem - that is commonly solved by using a hybrid model. Our early research has shown that discrete gradient method combining with other global methods such as evolutionary algorithm makes them even more attractive. These types of hybrid models have been studied by other researchers also. Another less mentioned problem is the problem of large weight values for the synaptic connections of the network. Large synaptic weight values often lead to the problem of paralysis and convergence problem especially when a hybrid model is used for fine tuning the learning task. In this paper we study and analyse the effect of different regularization parameters for our objective function to restrict the weight values without compromising the classification accuracy. © Springer-Verlag Berlin Heidelberg 2005.

History

Volume

3587 LNAI

Pagination

71 - 79

Publisher

Springer

Location

Leipzig, Germany

Place of publication

Berlin, Germany

Start date

2005-07-09

End date

2005-07-11

ISSN

0302-9743

eISSN

1611-3349

ISBN-10

3540269231

Publication classification

EN.1 Other conference paper

Title of proceedings

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Usage metrics

    Research Publications

    Categories

    No categories selected

    Keywords

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC