Scilab Function
Last update : 26/6/2006
ANN_REPET - Function to train different repetitions of a feed-forward artificial neural network with a split sample test procedure.
Calling Sequence
-
[W,OUT,C,RMSE,[IN_Stat,TARG_Stat,RMSErepet,Wrepet,OUTrepet]]=ANN_REPET(IN,TARG,Nhid,ChemRAND,[NbRepet,EPOCH])
Parameters
-
IN
: Input data (matrix [PxN] where P is the number of input neurons and N the number of input patterns)
-
TARG
: Target data (matrix [1xP]). CAUTION ! This function only works with one output !!
-
Nhid
: Number of neurons in the hidden layer
-
ChemRAND
: Complete path of the text file containing the random numbers matrix. CAUTION ! These numbers should be in the [0..1] interval !
-
NbRepet
: Number of repetitions (should be >2). Default = 30
-
EPOCH
: Number of epochs (should be >2). Default = 30
-
W
: Final weight and biais values (Matrix [NPx3] where NP is the number of network parameters stored in vector format, see ANN_CONV_W). W(:,1) = parameters resulting from a calibration on the first half of the dataset (P1).
W(:,2) = parameters resulting from a calibration on the second half of the dataset (P2).
W(:,3) = parameters resulting from a calibration on the whole dataset (P3).
-
OUT
: Final network outputs (Matrix [Nx4] where N is the number of patterns). OUT(:,1) = output resulting from a calibration on P1.
OUT(:,2) = output resulting from a calibration on P2.
OUT(:,3) = output resulting from a calibration on P3.
OUT(:,4) = output in validation mode (parameters from P2 are applied on P1 and vice-versa).
-
C
: Quality criteria of the simulation in validation mode (Matrix [4x1] see CRIT function).
-
RMSE
: Root Mean square error of calibrated output on the 3 sub-periods (Matrix [3x1]).
RMSE(1) = RMSE resulting from a calibration on P1.
RMSE(2) = RMSE resulting from a calibration on P2.
RMSE(3) = RMSE resulting from a calibration on P3.
-
IN_Stat
: Values used for input normalisation (Matrix [2P x 3]). The first P lines are the means and the following P lines are the standard deviations. Each columns is dedicated to a sub-period (P1, P2 and P3).
-
TARG_Stat
: Values used for target normalisation (Matrix [2 x 3]). The first line is the mean and the second line is the standard deviation. Each columns is dedicated to a sub-period (P1, P2 and P3).
-
RMSErepet
: Root Mean square error of calibrated output on the 3 sub-periods for all the repetitions (Matrix [NbRepet x 3]).
-
Wrepet
: Neural network weights and biases for the 3 sub-periods and all the repetitions (Matrix [NP x NbRepet x 3]).
-
OUTrepet
: Neural network outputs for the 3 sub-periods and all the repetitions (Matrix [N x NbRepet x 3]).
Description
-
The function performs the following actions:
(0) Inputs AND outputs are normalised by their mean and 2 x standard deviation (to keep generalisation capacity on extremas).
-
(1) The dataset is split into 2 equal parts, P1 and P2. P3 represents the whole dataset.
-
(2) On each sub-period, a neural network is calibrated with a Levenberg-Marquardt algorithm (with bayesian regulation, see ANN_LMBR).
-
(3) This calibration is repeated NR times with NR the number of repetitions. Each calibration starts from a different and randomly defined initial parameter set (weight only, all biaises are set to 0). This leads to 3*NR parameters sets.
-
(4) For each sub-period, a median simulation is calculated from the NR outputs generated by the NR calibrations realised in step 3 (Median simulation = simulation constituted with median values of the NR simulation at each points of the dataset). This leads to 3 simulations (one for each sub-period)
-
(5) For each sub-period, The simulation showing the highest similarity with the median simulation is selected as the "best repetition" (similarity on the basis of smallest RMSE). This leads to 3 parameters sets (one for each sub-period).
-
(6) Parameters of this "best simulation" are returned and validation criteria are calculated (parameters from P2 are applied on P1 and vice-versa to get a complete simulation).
Examples
// Calibration of a network with 6 input nodes, 4 nodes in the hidden layer and 1 output node
IN = [(1:0.05:10);sin(1:0.05:10)]+rand(2,181);
TARG = abs((1:0.05:10).^0.01+(sin(1:0.05:10)).^0.5)+rand(1,181);
Nhid = 4;
ChemRAND = 'C:\_JUL\!UTILS\sources\RAND.txt'; // To be modified
[W,OUT,C,RMSE] = ANN_REPET(IN,TARG,Nhid,ChemRAND);
// Graph >> goog calibration but validation problems on this example !
plot2d([TARG' OUT]);
legends(['Observed data' 'Calibration on P1' 'Calibration on P2' 'Calibration on P3' 'Validation'],1:5,'lr')
See Also
ANN_CONV_W
,
ANN_LMBR
,
ANN_NORM
,
ANN_SIM
,
CRIT
,
Authors
-
Julien Lerat
CEMAGREF Antony, HBAN Unit, julien.lerat@cemagref.fr
Bibliography
Klemes, V. (1986). "Operational testing of hydrological simulation models." Hydrological Sciences Journal/Journal des Sciences Hydrologiques 31(1): 13-24.
Foresee and Hagan, Proceedings of the International Joint Conference on Neural Networks, June, 1997.
Used Function
ANN_CONV_W, ANN_LMBR, ANN_NORM,CRIT