# Training accuracy is ~97% but validation accuracy is stuck at ~40%

Data Science Asked by Punit Singh on October 30, 2020

I am trying to classify images into 27 classes using a Conv2D network. The training accuracy rises through epochs as expected but the val_accuracy and val_loss values fluctuate severely and are not good enough.

I am using separate datasets for training and validation. The images are 256 x 256 in size and are binary threshold images.

There are 22127 images in the training set (~800 for each class) and
11346 images in the validation set (~400 for each class) so I believe there is no class imbalance.

Here is my architecture:

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 126, 126, 32)      320
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 63, 63, 32)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 61, 61, 32)        9248
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 30, 30, 32)        0
_________________________________________________________________
flatten_1 (Flatten)          (None, 28800)             0
_________________________________________________________________
dense_1 (Dense)              (None, 128)               3686528
_________________________________________________________________
dropout_1 (Dropout)          (None, 128)               0
_________________________________________________________________
dense_2 (Dense)              (None, 96)                12384
_________________________________________________________________
dropout_2 (Dropout)          (None, 96)                0
_________________________________________________________________
dense_3 (Dense)              (None, 64)                6208
_________________________________________________________________
dense_4 (Dense)              (None, 27)                1755
=================================================================

Total params: 3,716,443
Trainable params: 3,716,443
Non-trainable params: 0

Found 22127 images belonging to 27 classes.
Found 11346 images belonging to 27 classes.


here is the code I used to make this model:

classifier = Sequential()

classifier.add(Convolution2D(32, (3, 3), input_shape=(sz, sz, 1), activation='relu'))

activation='relu'))

activation='relu'))

activation='relu'))



optimizer used is adam and loss function I used is categorical_crossentropy.

from keras import optimizers

classifier.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

classifier.summary()


This is the code for flowing the data from the directories to the model

train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)

training_set = train_datagen.flow_from_directory('Dataset2/Training Data',
target_size=(sz, sz),
batch_size=10,
color_mode='grayscale',
class_mode='categorical',
shuffle = True)

test_set = test_datagen.flow_from_directory('Dataset2/Test Data',
target_size=(sz , sz),
batch_size=10,
color_mode='grayscale',
class_mode='categorical',
#shuffle = True
)

classifier_obj = classifier.fit_generator(
training_set,
#steps_per_epoch=22127,
epochs=10,
validation_data=test_set,
#validation_steps=11346
verbose = 1)


the output this code produced was:

Epoch 1/10
2213/2213 [==============================] - 302s 136ms/step - loss: 0.0130 - accuracy: 0.7108 - val_loss: 0.0604 - val_accuracy: 0.3293
Epoch 2/10
2213/2213 [==============================] - 312s 141ms/step - loss: 0.0035 - accuracy: 0.9371 - val_loss: 0.0279 - val_accuracy: 0.4069
Epoch 3/10
2213/2213 [==============================] - 305s 138ms/step - loss: 0.0024 - accuracy: 0.9558 - val_loss: 0.0260 - val_accuracy: 0.4266
Epoch 4/10
2213/2213 [==============================] - 336s 152ms/step - loss: 0.0018 - accuracy: 0.9680 - val_loss: 0.0452 - val_accuracy: 0.4323
Epoch 5/10
2213/2213 [==============================] - 310s 140ms/step - loss: 0.0015 - accuracy: 0.9731 - val_loss: 0.0339 - val_accuracy: 0.3659
Epoch 6/10
2213/2213 [==============================] - 317s 143ms/step - loss: 0.0014 - accuracy: 0.9746 - val_loss: 0.0415 - val_accuracy: 0.4496
Epoch 7/10
2213/2213 [==============================] - 297s 134ms/step - loss: 0.0012 - accuracy: 0.9777 - val_loss: 0.0379 - val_accuracy: 0.4512
Epoch 8/10
2213/2213 [==============================] - 285s 129ms/step - loss: 0.0012 - accuracy: 0.9782 - val_loss: 0.0157 - val_accuracy: 0.4603
Epoch 9/10
2213/2213 [==============================] - 274s 124ms/step - loss: 0.0012 - accuracy: 0.9795 - val_loss: 0.0289 - val_accuracy: 0.4430
Epoch 10/10
2213/2213 [==============================] - 274s 124ms/step - loss: 9.1970e-04 - accuracy: 0.9837 - val_loss: 0.0459 - val_accuracy: 0.4800
Model Saved
Weights saved


Seeing promising results on this, I increased the number of epochs in hopes of getting a higher val_Accuracy but the result was that the val_accuracy was lower than the previous one.

I changed some parameters and ran the code with 100 epochs overnight. This was the output:

Some of the things that I found in other answers and have tried but failed:

1. used SGD optimizer
2. used regularizers
3. played around with the number of units in Dense layers.

One of the major issue is the amount of records per categories. Definitely using data augmentation is always helpful to get good accuracy but much better approach is to use transfer learning techniques with data augmentation.

In your approach model is started overfitting because of that it's started having good training accuracy but bad validation accuracy. Let's have look into implementation of transfer learning. To understand more on transfer learning check this link Article Link

import tensorflow.keras as keras

model = keras.models.Sequential()

model.layers[0].trainable=False

model.summary()

model_obj = model.fit_generator(training_set, epochs=1, validation_data=test_set, verbose = 1)


Don't forgot to pass data augmented generator as a input data into fit_generator. There is no effect if you are using data augmentation on validation/test. You can use cross validation techniques or cnn based ensemble techniques for more good results.

Answered by Swapnil Pote on October 30, 2020

## Related Questions

### How do I calculate the delta term of a Convolutional Layer, given the delta terms and weights of the previous Convolutional Layer?

1  Asked on February 2, 2021 by cdwoelk

### Is there any package in python that can identify similarity between alphanumeric alias names of a parameter?

2  Asked on February 2, 2021 by rb173

### Pandas Data Frame: Calculating custom moving average

1  Asked on February 2, 2021

### What does this formula in Glorot & Bengio mean?

1  Asked on February 2, 2021

### Clustering data with a constraint

0  Asked on February 2, 2021 by aibek

### How does TF-IDF classify a document based on “Score” alloted to each word

1  Asked on February 1, 2021 by hakiki_makato

### How to do backward features elimination when considering interactions between them

1  Asked on February 1, 2021 by oussama-jabri

### Suspiciously good accuracy using neural network

1  Asked on February 1, 2021 by po-chen-liu

### Loss function in order to find the most certain samples

0  Asked on February 1, 2021 by user4550050

### split flattened tensor to perform class wise loss

0  Asked on February 1, 2021

### scale_pos_weight using XGBoost’s Learning API

1  Asked on February 1, 2021 by yatu

### what is correct way to perform normalization on data in Auto encoder?

1  Asked on February 1, 2021 by milan_harkhani

### Whether Interaction terms should be included in Linear Regression analysis?

3  Asked on February 1, 2021 by ali-shanaa

### Difference between AlphaGo’s policy network and value network

6  Asked on January 31, 2021

### Adding high p-value and low R square features in linear regression model to improve result

1  Asked on January 31, 2021 by shahnawaz-khan

### Is there any inductive Graph Variational Auto Encoder?

0  Asked on January 31, 2021 by amogh-mishra

### Question about reshaping array size for KNN Classifiers

1  Asked on January 31, 2021 by leeann-capistran

### What statistical method should i use to find Correlation between number of days and AmountEarned

1  Asked on January 31, 2021 by lordoftheringyvr