How to evaluate the pattern recognition peformance of newrb RBFNN

I want to use a RBFNN for character recognition. I have 40 images stored in a input vector N and 40 targets stored in a target vector T which I supplied as the first two arguments of the function newrb (net = newrb(N,T) and now I want to evaluate the performance of the RBFNNs ability to recognise patterns using a train, validation and test but I do not know how.

I am new to matlab and have only used neural networks in Python before now. I wanted to divide up my data into these three sets using the dividerand function. In Python there is a function that divides up input and target vectors at the same time so that the input and target vectors still correspond but dividerand only lets you split one vector at a time into train, validation and test sets so if I used this function twice (once on the input vector and once on the target vector) the input and target vectors will not correspond. Is there any way to divide up input and target vectors at the same time so target and input vectors still match up after data has been divided? And if so which function is the validation set used in if it's not newrb?

I first tested the RBF actually does something by doing Y = sim(N) and it did as 100% accuracy is returned but when creating the RBF using newrb(N,T) it says there are zero neurons in the network, is this normal? When I test the network using sim on unseen data Y = sim(unseen) data, the results of Y is 25% recognition accuracy which is very poor in comparison to the multilayer perceptron I created in Python where I would get 90+% so am I doing something wrong as in is sim not the correct function to test the performance of newrb?

Sorry for the long question, I am very new to matlab. Thank you in advance to anyone who can help me out.

Answer 1

[Greg Heath]

>I want to use a RBFNN for character recognition. I have 40 images >stored in a input vector N and 40 targets stored in a target vector T >which I supplied as the first two arguments of the function newrb >(net = newrb(N,T)

1. You erroneously use the term "vector" instead of "matrix".

2: Please use N for the number of input/target vector pairs

3. NEWRB, NEWRBE & NEWGRNN are ancient RBFNN functions that are inferior in flexibility to the current curvefitting and pattern recognition functions FITNET and PATTERNNET, respectively.

   a. In particular, THE FORMER ARE NOT EQUIPPED TO 
       AUTOMATICALLY DEAL WITH DATA DIVISION AND 
       VALIDATION STOPPING.

   b. Although the latter 2 automatically deal with 
data division, when used with RADBAS functions, the 
results are ELLIPTICAL bASIS FUNCTIONS (EBFNNs)!

4. Input data to MATLAB NN functions should be stored in two separate 2-D MATRICES.

5. Each of the N 2-D input images should be either a. vectorized into an I-dimensional vector (use colon operator) or b. represented by extracted features which are stored in a single I-dimensional vector

6. Each of the N I-dimensional "I"nput vectors corresponds to an O-dimensional "O"utput target vector that, for pattern-recognition and classification, is a unit vector with a single one ("1") and O-1 zeros ("0").

7. The input and target matrices have the dimensions [ I N ] = size(input) [ O N ] = size(target)

8. For pattern-recognition, the target vectors are columns of a (0,1) unit vector . The input vector is assigned to the class corresponding to the row of the output vector that has the highest value.

> and now I want to evaluate the performance of the RBFNNs ability > to recognise patterns using a train, validation and test but I do not > know how ... I wanted to divide up my data into these three sets > using the dividerand function.

9. Again, there is no efficient way to use validation stopping datadivision with the above RBFNNs. I suggest using PATTERNNET EBFNNs.

PATTERNNET allows several automated data division functions. DIVIDERAND is the default and does not have to be explicitly specified.Specifying RADBAS functions in the hidden layer will result in EBFNNs.

References

http://www.mathworks.com/matlabcentral/newsreader/view_thread/304783#948573 http://www.mathworks.com/matlabcentral/newsreader/view_thread/315697#862831 http://www.mathworks.com/matlabcentral/newsreader/view_thread/151286#380974

HOPE THIS HELPS.

Thank you for formally accepting my answer

GREG

[nkbapt]

Thank you very much for your corrections and your detailed answer. It has been very helpful and I am deeply grateful. I have implemented your suggestion of implementing patternet with the radbas function and this has solved a lot of my issues

One last question about the subject. If I wanted to also continue with using a RBF using newrb for pattern recognition without worrying about the division of the dataset, how can the performance of newrb on unseen data be evaluated? Is using sim(newrbNet,unseendata) the only way to do this? Right not using sim(newrbNet,unseendata) has very poor recognition results so I am wondering if there is another way to evaluate the data which I have missed or whether newrb is not good for pattern classfication.

Thank you for your time