BasicCSVClassifier does not classify at all

[dkunzinfa]

Issue Description

Running BasicCSVClassifier results in a bad classifcation stats:

o.n.l.f.Nd4jBackend - Loaded [CpuBackend] backend
o.n.n.NativeOpsHolder - Number of threads used for linear algebra: 4
o.n.l.c.n.CpuNDArrayFactory - *********************************** CPU Feature Check Warning ***********************************
o.n.l.c.n.CpuNDArrayFactory - Warning: Initializing ND4J with Generic x86 binary on a CPU with AVX/AVX2 support
o.n.l.c.n.CpuNDArrayFactory - Using ND4J with AVX/AVX2 will improve performance. See deeplearning4j.org/cpu for more details
o.n.l.c.n.CpuNDArrayFactory - Or set environment variable ND4J_IGNORE_AVX=true to suppress this warning
o.n.l.c.n.CpuNDArrayFactory - *************************************************************************************************
o.n.n.Nd4jBlas - Number of threads used for OpenMP BLAS: 4
o.n.l.a.o.e.DefaultOpExecutioner - Backend used: [CPU]; OS: [Windows 10]
o.n.l.a.o.e.DefaultOpExecutioner - Cores: [8]; Memory: [14,2GB];
o.n.l.a.o.e.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
o.d.e.d.BasicCSVClassifier - Build model....
o.d.n.m.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
o.d.o.l.ScoreIterationListener - Score at iteration 0 is 1.2347359410511736
o.d.o.l.ScoreIterationListener - Score at iteration 100 is 0.1004844711532599
o.d.o.l.ScoreIterationListener - Score at iteration 200 is 0.039669264507044
o.d.o.l.ScoreIterationListener - Score at iteration 300 is 0.024435827880011744
o.d.o.l.ScoreIterationListener - Score at iteration 400 is 0.017831637223516515
o.d.o.l.ScoreIterationListener - Score at iteration 500 is 0.014205604188074546
o.d.o.l.ScoreIterationListener - Score at iteration 600 is 0.011934892657567751
o.d.o.l.ScoreIterationListener - Score at iteration 700 is 0.010388604478515275
o.d.o.l.ScoreIterationListener - Score at iteration 800 is 0.009272695178875455
o.d.o.l.ScoreIterationListener - Score at iteration 900 is 0.00843236932311402
o.d.e.d.BasicCSVClassifier - 

========================Evaluation Metrics========================
 # of classes:    3
 Accuracy:        0,1818
 Precision:       0,3333
 Recall:          0,1818	(2 classes excluded from average)
 F1 Score:        0,3077	(2 classes excluded from average)
Precision, recall & F1: macro-averaged (equally weighted avg. of 3 classes)

Warning: 2 classes were never predicted by the model and were excluded from average recall
Classes excluded from average recall: [1, 2]

=========================Confusion Matrix=========================
  0  1  2
----------
  8 13 23 | 0 = 0
  0  0  0 | 1 = 1
  0  0  0 | 2 = 2

Confusion matrix format: Actual (rowClass) predicted as (columnClass) N times
==================================================================

Version Information

• 1.0.0-beta6
• WIN10

Question

Is it a configuration error in any form? Or is the example just not working with the newest version?

[eraly]

Hi @dkunzinfa Thank you for filing this issue. The test data labels are incorrect...They are all incorrectly marked. If you look at the print outs of the loss you will see it decreasing indicating the nn is quite easily fitting this dataset. Also the output prints which I have pasted below also shows that the neural net has learned the problem quite successfully. The labels that the evaluation are carried out against are corrupt. We will fix this as it is plain wrong...

o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=10.0, yearsLived=19}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=60.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=12.0, yearsLived=17}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=50.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=16.0, yearsLived=15}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=7}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=40.0, yearsLived=10}
o.d.q.d.BasicCSVClassifier - {eats=Ice Cream, sounds=Bark, classifier=Dog, weight=100.0, yearsLived=11}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=10.0, yearsLived=19}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=60.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=12.0, yearsLived=17}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=50.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=16.0, yearsLived=15}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=7}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=40.0, yearsLived=10}
o.d.q.d.BasicCSVClassifier - {eats=Ice Cream, sounds=Bark, classifier=Dog, weight=100.0, yearsLived=11}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=10.0, yearsLived=19}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=60.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=12.0, yearsLived=17}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=50.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=16.0, yearsLived=15}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=7}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=40.0, yearsLived=10}
o.d.q.d.BasicCSVClassifier - {eats=Ice Cream, sounds=Bark, classifier=Dog, weight=100.0, yearsLived=11}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=200.0, yearsLived=80}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=170.0, yearsLived=65}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=170.0, yearsLived=65}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=7}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=170.0, yearsLived=65}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=62.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=70.0, yearsLived=7}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=170.0, yearsLived=44}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=12.0, yearsLived=17}
o.d.q.d.BasicCSVClassifier - {eats=Ice Cream, sounds=Singing, classifier=Human, weight=160.0, yearsLived=55}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Bark, classifier=Dog, weight=65.0, yearsLived=8}
o.d.q.d.BasicCSVClassifier - {eats=Cats, sounds=Bark, classifier=Dog, weight=14.0, yearsLived=9}
o.d.q.d.BasicCSVClassifier - {eats=Hotdogs, sounds=Talking, classifier=Human, weight=150.0, yearsLived=72}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=12.0, yearsLived=17}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=16.0, yearsLived=20}
o.d.q.d.BasicCSVClassifier - {eats=Mice, sounds=Meow, classifier=Cat, weight=23.0, yearsLived=19}
o.d.q.d.BasicCSVClassifier - {eats=Ice Cream, sounds=Singing, classifier=Human, weight=300.0, yearsLived=80}

[dkunzinfa]

Thanks for taking care of the issue! I think it is not a big deal as the CSVExample class is also providing an example on the Iris data set.

So btw. thanks for all those great examples! It was really helpful to get the library working for some of my ideas in a really quick timeframe!

[eraly]

Sure thing. We are working on updating things and redoing/reorganizing docs examples etc.