Nsryjdtyk

Background

I am training a model that takes as inputs two images. As the data is too large to fit into my machines RAM, I am using flow_from_dataframe to create the generators for training and validation - two training generators which each provide one of the respective images (front view & back view as indicated by the x_col parameter) and two generators for validation respectively.

Like this:

datagen=ImageDataGenerator(rescale=1. / 255)



X1_train_generator =datagen.flow_from_dataframe(dataframe=train, directory=data_dir, x_col="front", y_col=target, has_ext=True, class_mode="categorical", target_size=(224,224), batch_size=batch_size,seed = 1)

X2_train_generator=datagen.flow_from_dataframe(dataframe=train, directory=data_dir, x_col="back", y_col=target, has_ext=True, class_mode="categorical", target_size=(224,224), batch_size=batch_size,seed = 1)



X1_validation_generator =datagen.flow_from_dataframe(dataframe=test, directory=data_dir, x_col="front", y_col=target, has_ext=True, class_mode="categorical", target_size=(224,224), batch_size=batch_size,seed = 1)

X2_validation_generator=datagen.flow_from_dataframe(dataframe=test, directory=data_dir, x_col="back", y_col=target, has_ext=True, class_mode="categorical", target_size=(224,224), batch_size=batch_size,seed = 1)

In order to combine the train and validation generator for training I am using:

def format_gen_outputs(gen1,gen2):

    x1 = gen1[0]

    x2 = gen2[0]

    y1 = gen1[1]

    return [x1, x2], y1



train_combo_gen= map(format_gen_outputs, X1_train_generator , X2_train_generator )

validation_combo_gen= map(format_gen_outputs, X1_validation_generator , X2_validation_generator )

Now I use fit_generator to train my model passing train_combo_gen for training purpose and validation_combo_gen to the validation_data parameter for validation purpose

Problem

However, I realize that both my X1_train_generator and X2_train_generator show a different .class_indices mapping than my other two validation generators X1_validation_generator and X2_validation_generator.

Like this (notice how cat & dog are assigned to different classes):

X1_train_generator.class_indices

>> {'cat': 0, 'dog': 1, 'car': 2, 'bike': 3}



X1_validation_generato.class_indices

>> {'dog': 0, 'cat': 1, 'car': 2, 'bike': 3}

Question

Hence, I don't trust my val_loss and val_acc during training. Is there any way to fix this i.e. remap the classes within the generators?

When you don't explicitly set the classes using classes argument, the flow_from_dataframe internally uses pandas Series unique method on the y_col column to find the classes:

if not classes:

    classes = 

    if class_mode not in ["other", "input", None]:

        classes = list(self.df[y_col].unique())

The unique method would return the unique values in order of appearance in the column. Since the order of appearance of labels in your train and test dataframe are different from each other, you would get different indices for classes.

One workaround is to explicitly set the classes argument for all the flow_from_dataframe calls to guarantee the same class indices mapping in train and validation generators:

X1_train_generator =datagen.flow_from_dataframe(dataframe=train, 

                                                classes=['cat', 'dog', 'car', 'bike'], ...)



# do the same for `X2_train_generator`, `X1_validation_generator` and `X2_validation_generator`