lemga::FeedForwardNN

FeedForwardNN Class Reference

#include <feedforwardnn.h>

Inheritance diagram for FeedForwardNN:

Inheritance graph
[legend]
Collaboration diagram for FeedForwardNN:

Collaboration graph
[legend]
List of all members.

Public Types

typedef std::vector< NNLayer::WVEC > WEIGHT
enum  TRAIN_METHOD {
  GRADIENT_DESCENT, LINE_SEARCH, CONJUGATE_GRADIENT, WEIGHT_DECAY,
  ADAPTIVE_LEARNING_RATE
}

Public Member Functions

 FeedForwardNN ()
 FeedForwardNN (const FeedForwardNN &)
 FeedForwardNN (std::istream &is)
virtual ~FeedForwardNN ()
const FeedForwardNN & operator= (const FeedForwardNN &)
virtual const id_t & id () const
virtual FeedForwardNN * create () const
 Create a new object using the default constructor.
virtual FeedForwardNN * clone () const
 Create a new object by replicating itself.
UINT size () const
const NNLayer & operator[] (UINT n) const
void add_top (const NNLayer &)
void add_bottom (const NNLayer &)
void set_batch_mode (bool b=true)
void set_train_method (TRAIN_METHOD m)
void set_parameter (REAL lr, REAL mincst, UINT maxrun)
virtual bool support_weighted_data () const
 Whether the learning model/algorithm supports unequally weighted data.
virtual void initialize ()
 Initialize the model for training.
virtual REAL train ()
 Train with preset data set and sample weight.
virtual Output operator() (const Input &) const
WEIGHT weight () const
void set_weight (const WEIGHT &)
REAL cost (UINT idx) const
REAL cost () const
WEIGHT gradient (UINT idx) const
WEIGHT gradient () const
void clear_gradient () const
bool stop_opt (UINT step, REAL cst)

Protected Member Functions

virtual bool serialize (std::ostream &, ver_list &) const
virtual bool unserialize (std::istream &, ver_list &, const id_t &=empty_id)
virtual REAL _cost (const Output &F, const Output &y) const
virtual Output _cost_deriv (const Output &F, const Output &y) const
virtual void log_cost (UINT epoch, REAL err)

Protected Attributes

UINT n_layer
 # of layers == layer.size()-1.
std::vector< NNLayer * > layer
 layer pointers (layer[0] == 0).
std::vector< Output > _y
 buffer for outputs.
std::vector< Output > _dy
 buffer for derivatives.
bool online_learn
TRAIN_METHOD train_method
REAL learn_rate
REAL min_cst
UINT max_run

Detailed Description

Todo:
documentation

Definition at line 18 of file feedforwardnn.h.

Member Typedef Documentation

typedef std::vector<NNLayer::WVEC> WEIGHT
 

Definition at line 28 of file feedforwardnn.h.

Member Enumeration Documentation

enum TRAIN_METHOD
 

Enumerator:
GRADIENT_DESCENT 
LINE_SEARCH 
CONJUGATE_GRADIENT 
WEIGHT_DECAY 
ADAPTIVE_LEARNING_RATE 

Definition at line 29 of file feedforwardnn.h.

Constructor & Destructor Documentation

FeedForwardNN (   ) 
 

Todo:
Online learning is not implemented

Definition at line 22 of file feedforwardnn.cpp.

References FeedForwardNN::layer.

Referenced by FeedForwardNN::clone(), and FeedForwardNN::create().

FeedForwardNN (  const FeedForwardNN &   ) 
 

Definition at line 31 of file feedforwardnn.cpp.

References FeedForwardNN::layer, and FeedForwardNN::n_layer.

FeedForwardNN (  std::istream &  is  )  [inline, explicit]
 

Definition at line 51 of file feedforwardnn.h.

~FeedForwardNN (   )  [virtual]
 

Definition at line 42 of file feedforwardnn.cpp.

Member Function Documentation

virtual REAL _cost (  const Output &  F,
const Output &  y
)  const [inline, protected, virtual]
 

Definition at line 85 of file feedforwardnn.h.

References LearnModel::r_error().

Referenced by FeedForwardNN::cost().

Output _cost_deriv (  const Output &  F,
const Output &  y
)  const [protected, virtual]
 

Definition at line 213 of file feedforwardnn.cpp.

References LearnModel::_n_out, and LearnModel::n_output().

Referenced by FeedForwardNN::gradient().

void add_bottom (  const NNLayer &   ) 
 

void add_top (  const NNLayer &   ) 
 

Definition at line 129 of file feedforwardnn.cpp.

References FeedForwardNN::_dy, LearnModel::_n_in, LearnModel::_n_out, FeedForwardNN::_y, NNLayer::clone(), FeedForwardNN::layer, LearnModel::n_input(), FeedForwardNN::n_layer, and LearnModel::n_output().

void clear_gradient (   )  const
 

Definition at line 278 of file feedforwardnn.cpp.

References FeedForwardNN::layer, and FeedForwardNN::n_layer.

Referenced by FeedForwardNN::gradient().

virtual FeedForwardNN* clone (   )  const [inline, virtual]
 

Create a new object by replicating itself.

Returns:
A pointer to the new copy.
The code for a derived class Derived is always 
 return new Derived(*this);
Though seemingly redundant, it helps to copy an object without knowing the real type of the object.
See also:
C++ FAQ Lite 20.6

Implements LearnModel.

Definition at line 57 of file feedforwardnn.h.

References FeedForwardNN::FeedForwardNN().

REAL cost (   )  const
 

Definition at line 226 of file feedforwardnn.cpp.

References LearnModel::ptd, and LearnModel::ptw.

Referenced by FeedForwardNN::train().

REAL cost (  UINT  idx  )  const
 

Definition at line 222 of file feedforwardnn.cpp.

References FeedForwardNN::_cost(), LearnModel::get_output(), and LearnModel::ptd.

virtual FeedForwardNN* create (   )  const [inline, virtual]
 

Create a new object using the default constructor.

The code for a derived class Derived is always 

 return new Derived();

Implements LearnModel.

Definition at line 56 of file feedforwardnn.h.

References FeedForwardNN::FeedForwardNN().

FeedForwardNN::WEIGHT gradient (   )  const
 

Definition at line 252 of file feedforwardnn.cpp.

References FeedForwardNN::_cost_deriv(), FeedForwardNN::_dy, LearnModel::_n_out, FeedForwardNN::_y, FeedForwardNN::clear_gradient(), FeedForwardNN::layer, FeedForwardNN::n_layer, LearnModel::ptd, LearnModel::ptw, and FeedForwardNN::size().

FeedForwardNN::WEIGHT gradient (  UINT  idx  )  const
 

Definition at line 235 of file feedforwardnn.cpp.

References FeedForwardNN::_cost_deriv(), FeedForwardNN::_dy, FeedForwardNN::_y, FeedForwardNN::clear_gradient(), FeedForwardNN::layer, FeedForwardNN::n_layer, and LearnModel::ptd.

virtual const id_t& id (   )  const [virtual]
 

Returns:
Class ID string (class name)

Implements Object.

void initialize (   )  [virtual]
 

Initialize the model for training.

Todo:
whether or when to initialize is a big problem. If we allow "continue_on_learning", then sometimes the initialization is not needed
Note:
this was a virtual function, I changed it to be empty

Reimplemented from LearnModel.

Definition at line 146 of file feedforwardnn.cpp.

References FeedForwardNN::layer, and FeedForwardNN::n_layer.

void log_cost (  UINT  epoch,
REAL  err
)  [protected, virtual]
 

Definition at line 184 of file feedforwardnn.cpp.

References FeedForwardNN::learn_rate, and LearnModel::logf.

Referenced by FeedForwardNN::stop_opt().

Output operator() (  const Input &   )  const [virtual]
 

Implements LearnModel.

Definition at line 192 of file feedforwardnn.cpp.

References FeedForwardNN::_y, LearnModel::n_input(), and FeedForwardNN::n_layer.

const FeedForwardNN & operator= (  const FeedForwardNN &   ) 
 

Definition at line 46 of file feedforwardnn.cpp.

References FeedForwardNN::_dy, FeedForwardNN::_y, FeedForwardNN::layer, FeedForwardNN::learn_rate, FeedForwardNN::max_run, FeedForwardNN::min_cst, FeedForwardNN::n_layer, FeedForwardNN::online_learn, and FeedForwardNN::train_method.

const NNLayer& operator[] (  UINT  n  )  const [inline]
 

Definition at line 61 of file feedforwardnn.h.

References FeedForwardNN::layer.

bool serialize (  std::ostream &  ,
ver_list & 
)  const [protected, virtual]
 

Reimplemented from LearnModel.

Definition at line 67 of file feedforwardnn.cpp.

References FeedForwardNN::layer, FeedForwardNN::learn_rate, FeedForwardNN::max_run, FeedForwardNN::min_cst, FeedForwardNN::n_layer, FeedForwardNN::online_learn, and SERIALIZE_PARENT.

void set_batch_mode (  bool  b = true  )  [inline]
 

Definition at line 65 of file feedforwardnn.h.

References FeedForwardNN::online_learn.

void set_parameter (  REAL  lr,
REAL  mincst,
UINT  maxrun
)  [inline]
 

Parameters:
lr learning rate.
mincst minimal cost (error) need to be achieved during training.
maxrun maximal # of epochs the training should take.

Definition at line 72 of file feedforwardnn.h.

References FeedForwardNN::learn_rate, FeedForwardNN::max_run, and FeedForwardNN::min_cst.

void set_train_method (  TRAIN_METHOD  m  )  [inline]
 

Definition at line 66 of file feedforwardnn.h.

References FeedForwardNN::train_method.

void set_weight (  const WEIGHT &   ) 
 

Definition at line 207 of file feedforwardnn.cpp.

References FeedForwardNN::layer, and FeedForwardNN::n_layer.

UINT size (   )  const [inline]
 

Definition at line 60 of file feedforwardnn.h.

References FeedForwardNN::n_layer.

Referenced by FeedForwardNN::gradient().

bool stop_opt (  UINT  step,
REAL  cst
) 
 

Definition at line 283 of file feedforwardnn.cpp.

References FeedForwardNN::log_cost(), FeedForwardNN::max_run, and FeedForwardNN::min_cst.

virtual bool support_weighted_data (   )  const [inline, virtual]
 

Whether the learning model/algorithm supports unequally weighted data.

Returns:
true if supporting; false otherwise. The default is false, just for safety.
See also:
set_train_data()

Reimplemented from LearnModel.

Definition at line 75 of file feedforwardnn.h.

REAL train (   )  [virtual]
 

Train with preset data set and sample weight.

Returns:
Probably the training error.
Todo:
Make the return type void

Implements LearnModel.

Definition at line 151 of file feedforwardnn.cpp.

References FeedForwardNN::ADAPTIVE_LEARNING_RATE, FeedForwardNN::CONJUGATE_GRADIENT, FeedForwardNN::cost(), FeedForwardNN::GRADIENT_DESCENT, lemga::iterative_optimize(), FeedForwardNN::learn_rate, FeedForwardNN::LINE_SEARCH, FeedForwardNN::n_layer, LearnModel::ptd, LearnModel::ptw, FeedForwardNN::train_method, and FeedForwardNN::WEIGHT_DECAY.

bool unserialize (  std::istream &  ,
ver_list &  ,
const id_t &  = empty_id
)  [protected, virtual]
 

Reimplemented from LearnModel.

Definition at line 79 of file feedforwardnn.cpp.

References FeedForwardNN::_dy, LearnModel::_n_in, LearnModel::_n_out, FeedForwardNN::_y, Object::create(), Object::empty_id, FeedForwardNN::learn_rate, FeedForwardNN::max_run, FeedForwardNN::min_cst, FeedForwardNN::online_learn, and UNSERIALIZE_PARENT.

FeedForwardNN::WEIGHT weight (   )  const
 

Definition at line 200 of file feedforwardnn.cpp.

References FeedForwardNN::layer, and FeedForwardNN::n_layer.

Member Data Documentation

std::vector<Output> _dy [mutable, protected]
 

buffer for derivatives.

Definition at line 41 of file feedforwardnn.h.

Referenced by FeedForwardNN::add_top(), FeedForwardNN::gradient(), FeedForwardNN::operator=(), and FeedForwardNN::unserialize().

std::vector<Output> _y [mutable, protected]
 

buffer for outputs.

Definition at line 40 of file feedforwardnn.h.

Referenced by FeedForwardNN::add_top(), FeedForwardNN::gradient(), FeedForwardNN::operator()(), FeedForwardNN::operator=(), and FeedForwardNN::unserialize().

std::vector<NNLayer*> layer [protected]
 

layer pointers (layer[0] == 0).

Definition at line 39 of file feedforwardnn.h.

Referenced by FeedForwardNN::add_top(), FeedForwardNN::clear_gradient(), FeedForwardNN::FeedForwardNN(), FeedForwardNN::gradient(), FeedForwardNN::initialize(), FeedForwardNN::operator=(), FeedForwardNN::operator[](), FeedForwardNN::serialize(), FeedForwardNN::set_weight(), and FeedForwardNN::weight().

REAL learn_rate [protected]
 

Definition at line 45 of file feedforwardnn.h.

Referenced by FeedForwardNN::log_cost(), FeedForwardNN::operator=(), FeedForwardNN::serialize(), FeedForwardNN::set_parameter(), FeedForwardNN::train(), and FeedForwardNN::unserialize().

UINT max_run [protected]
 

Definition at line 46 of file feedforwardnn.h.

Referenced by FeedForwardNN::operator=(), FeedForwardNN::serialize(), FeedForwardNN::set_parameter(), FeedForwardNN::stop_opt(), and FeedForwardNN::unserialize().

REAL min_cst [protected]
 

Definition at line 45 of file feedforwardnn.h.

Referenced by FeedForwardNN::operator=(), FeedForwardNN::serialize(), FeedForwardNN::set_parameter(), FeedForwardNN::stop_opt(), and FeedForwardNN::unserialize().

UINT n_layer [protected]
 

# of layers == layer.size()-1.

Definition at line 38 of file feedforwardnn.h.

Referenced by FeedForwardNN::add_top(), FeedForwardNN::clear_gradient(), FeedForwardNN::FeedForwardNN(), FeedForwardNN::gradient(), FeedForwardNN::initialize(), FeedForwardNN::operator()(), FeedForwardNN::operator=(), FeedForwardNN::serialize(), FeedForwardNN::set_weight(), FeedForwardNN::size(), FeedForwardNN::train(), and FeedForwardNN::weight().

bool online_learn [protected]
 

Definition at line 43 of file feedforwardnn.h.

Referenced by FeedForwardNN::operator=(), FeedForwardNN::serialize(), FeedForwardNN::set_batch_mode(), and FeedForwardNN::unserialize().

TRAIN_METHOD train_method [protected]
 

Definition at line 44 of file feedforwardnn.h.

Referenced by FeedForwardNN::operator=(), FeedForwardNN::set_train_method(), and FeedForwardNN::train().

The documentation for this class was generated from the following files:
Generated on Mon Jan 9 23:44:47 2006 for LEMGA by  doxygen 1.4.6