Public Member Functions | |
| Mlp () | |
| Construct uninitialized; only for reading content from a stream very soon. | |
| Mlp (const vector< size_t > &inneur) | |
| Construct w. | |
| Mlp (const vector< size_t > &inneur, const vector< ActF > &iactf, const vector< double > &iweights) | |
| Construct with given weights and actfs. | |
| Mlp (const Mlp &other) | |
| Construct a deep-copy of another similar dude. | |
| Mlp (const vector< size_t > &inneur, const vector< ActF > &iactf, SynapticRandomizer &sr) | |
| Construct with given weights and actfs. | |
| size_t | getWorkspaceSize () |
| Return the number of doubles needed in the external workspace storage that is always required for the layerwise computations. | |
| size_t | getNLayers () const |
| Return number of layers as input-hidden-. | |
| size_t | getNWeights () const |
| Return the total number of weights. | |
| size_t | getNNeurons (int layer) const |
| Return the total number of nodes/neurons on a layer. | |
| size_t | getNHiddenNeurons () const |
| Return the total number of neurons on hidden layers. | |
| size_t | getNumNonzeroWeights () const |
| Return the number of non-zero weights; the count is for strict zeros without numerical tolerance, so small weights should be pruned before this measure is useful. | |
| size_t | getNumConnectedInputs () const |
| Return the number of connected input nodes, i.e., those which feed into at least one neuron on the next layer. | |
| size_t | getNumInputs () const |
| Return the number of input nodes. | |
| size_t | getNumOutputs () const |
| Return the number of output nodes. | |
| void | forward (const vector< double > &input, double *workspace) const |
| Feed an input vector to produce the outputs of each layer; the outputs are stored in the external workspace memory array for future examination or backpropagation computations. | |
| void | errorVec (const vector< double > &target, double *workspace) const |
| Evaluate the error vector (N(x)-t) for a target vector; call this after forward() if the error was to be computed instead of only the network outputs; updates the result in the workspace. | |
| vector< double > | copyOutputVec (double *workspace) const |
| Create and return a copy of the current workspace output (network output or error vector, depending on workspace state). | |
| size_t | outputVecAsClassIndex (const double *workspace) const |
| Return a base-1 index to the class (1,2,...C) represented by the output (must be called directly after the forward pass). | |
| void | backwardEucSq (double coeff, double *destE, double *destG, double *workspace, ErrT errortype) const |
| Add the weighted error to the destination value and optionally its gradient by backpropagating through the workspace; must be called after forward() and errorVec() because the error vector must be computed in the workspace in order to evaluate the error contribution or backpropagate. | |
| void | weightDecaySq (double coeff, double *destE, double *destG, bool excludeOutputBias) const |
| Add a weight decay term of the form coeff/2 * sum w_i^2 - sum of either all weights, or weights except the biases of the output layer. | |
| void | weightDecayAbs (double coeff, double *destE, double *destG, bool excludeOutputBias) const |
| Add a weight decay term of the form coeff * sum abs(w_i) - sum of either all weights, or weights except the biases of the output layer. | |
| void | update (double coeff, const double *wupd) |
| Update the weights by adding values multiplied by a coefficient to the current weight values. | |
| void | addErrorAndGradient (const vector< double > &input, const vector< double > &target, double coefficient, double *sqerror, double *error, vector< double > *mseGrad, vector< double > *meeGrad) const |
| Evaluate and add to the referred outputs the squared or non-squared error and partial (sub)derivatives wrt weights, multiplied by a coefficient, for a given input and a given target vector. | |
| void | toStream (ostream &o) |
| Stream to an ASCII version, easily readable in simple scripts, e.g., in Matlab or Octave. | |
| void | fromStream (istream &ins) |
| Set new values from an ASCII stream, as created by toStream(). | |
| string | prettyPrintLayerSizes () |
| Pretty-print layer sizes, ex. | |
| string | prettyPrintLayerActivations () |
| Pretty-print layer activations, ex. | |
| string | prettyPrintWeights () const |
| Pretty-print weight matrices. | |
| string | prettyPrintGradient (const vector< double > &grad) const |
| Pretty-print gradient matrices. | |
| string | prettyPrint () |
| Pretty-print everything. | |
Protected Member Functions | |
| void | initRnd (double a, SynapticRandomizer &sr) |
| Re-initialize weights using random values from U(-a,a). | |
Protected Attributes | |
| vector< double > | weights |
| Linear storage of weights. | |
| vector< size_t > | nneur |
| Linear storage of neuron count in-hid-out. | |
| vector< ActF > | actf |
| Linear storage of activation types. | |
| Mlp::Mlp | ( | const vector< size_t > & | inneur | ) |
Construct w.
all-zero weights and tanh-tanh-lin activ (default).
| Mlp::Mlp | ( | const vector< size_t > & | inneur, | |
| const vector< ActF > & | iactf, | |||
| const vector< double > & | iweights | |||
| ) |
Construct with given weights and actfs.
Deep copies are made.
| Mlp::Mlp | ( | const Mlp & | other | ) |
Construct a deep-copy of another similar dude.
| Mlp::Mlp | ( | const vector< size_t > & | inneur, | |
| const vector< ActF > & | iactf, | |||
| SynapticRandomizer & | sr | |||
| ) |
Construct with given weights and actfs.
Initialize with weights with a given SynapticRandomizer. If you use threads, initialize the randomizers properly for each thread!
| void jymlp::Mlp::addErrorAndGradient | ( | const vector< double > & | input, | |
| const vector< double > & | target, | |||
| double | coefficient, | |||
| double * | sqerror, | |||
| double * | error, | |||
| vector< double > * | mseGrad, | |||
| vector< double > * | meeGrad | |||
| ) | const |
Evaluate and add to the referred outputs the squared or non-squared error and partial (sub)derivatives wrt weights, multiplied by a coefficient, for a given input and a given target vector.
This is likely to be the innermost computation in algorithmic iteration, and thus the function that dominates the computational cost. Therefore, exceptionally, the output parameters are raw pointers to numerical vectors, and there may be a nullptr for any value that is not necessary for the current purpose.
Some profiling should be done to make sure this really is the place to optimize and if useful optimizations can be done.
FIXME: Split this to (1) forward eval, (2) error computation, (3) backward loop.
| void Mlp::backwardEucSq | ( | double | coeff, | |
| double * | destE, | |||
| double * | destG, | |||
| double * | workspace, | |||
| ErrT | errortype | |||
| ) | const |
Add the weighted error to the destination value and optionally its gradient by backpropagating through the workspace; must be called after forward() and errorVec() because the error vector must be computed in the workspace in order to evaluate the error contribution or backpropagate.
Passing nullptr as destG bypasses the backpropagation step, in which case no changes are made to the workspace.
FIXME: That means that the function could be overloaded with a const* version for workspace!!
| void Mlp::forward | ( | const vector< double > & | input, | |
| double * | workspace | |||
| ) | const |
Feed an input vector to produce the outputs of each layer; the outputs are stored in the external workspace memory array for future examination or backpropagation computations.
Size of the workspace must be at least numberOfLayers * max(nneur).
| size_t Mlp::getNHiddenNeurons | ( | ) | const |
Return the total number of neurons on hidden layers.
| size_t jymlp::Mlp::getNLayers | ( | ) | const [inline] |
Return number of layers as input-hidden-.
..-hidden-output
| size_t jymlp::Mlp::getNNeurons | ( | int | layer | ) | const [inline] |
Return the total number of nodes/neurons on a layer.
| size_t jymlp::Mlp::getNumInputs | ( | ) | const [inline] |
Return the number of input nodes.
| size_t jymlp::Mlp::getNumOutputs | ( | ) | const [inline] |
Return the number of output nodes.
| size_t jymlp::Mlp::getNWeights | ( | ) | const [inline] |
Return the total number of weights.
| void Mlp::initRnd | ( | double | a, | |
| SynapticRandomizer & | sr | |||
| ) | [protected] |
Re-initialize weights using random values from U(-a,a).
| string Mlp::prettyPrint | ( | ) |
Pretty-print everything.
| string Mlp::prettyPrintGradient | ( | const vector< double > & | grad | ) | const |
Pretty-print gradient matrices.
| string Mlp::prettyPrintLayerActivations | ( | ) |
Pretty-print layer activations, ex.
"in-tanh-tanh-lin"
| string Mlp::prettyPrintLayerSizes | ( | ) |
Pretty-print layer sizes, ex.
"3-4-3"
| string Mlp::prettyPrintWeights | ( | ) | const |
Pretty-print weight matrices.
| void Mlp::toStream | ( | ostream & | o | ) |
Stream to an ASCII version, easily readable in simple scripts, e.g., in Matlab or Octave.
Format: Output always begins with format version. Currently only "version 1" is implemented.
Version 1 stream (values packed & separated with a single space):
int version, int[nlay] number of layers, int[nlay] number of neurons on each layer, ActF[nlay] activation functions (input activation must be always Unset == 0), double[nneurTot] synaptic weights as a row-major linear array.
| void Mlp::update | ( | double | coeff, | |
| const double * | wupd | |||
| ) |
Update the weights by adding values multiplied by a coefficient to the current weight values.
For example update(-.001, grad) for the simplest kind of steepest-descent backprop. The same function can be used for random "jogging", for example update(.1, gaussian).
1.6.1
