• Certain APIs, like tf.function, tf.keras, etc. are designed to use Graph execution, for performance and portability. When debugging, use tf.config.experimental_run_functions_eagerly(True) to use Eager execution inside this code.

@tf.function
def f(x):
  if x > 0:
    import pdb
    pdb.set_trace()
    x = x + 1
  return x

tf.config.experimental_run_functions_eagerly(True)
f(tf.constant(1))

• Since GlobalAveragePooling2D transforms the "images" into their average (a single number), preserving only the number of filter or dimensions, it allows CNN models to be flexible regarding the input image sizes.

base_model_out = base_model(inputs)
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()(base_model_out)
prediction_layer = tf.keras.layers.Dense(45, activation='softmax')(global_average_layer)

• The randomized data augmentation layers of tf.keras.layers.experimental.preprocessing, like tf.keras.layers.experimental.preprocessing.RandomFlip, are only applied during training, as one usually desires.

• It seems like all of TensorFlow’s pre-trained models, including the ones from TensorFlow Hub, expect a data range of [0, 1]. You can see this in the TensorFlow Hub guidelines, in BiT’s example notebook and in this EfficientNetB0 saved model in TensorFlow Hub.

• Dataset.padded_batch transformation enables you to batch tensors of different shape by specifying one or more dimensions in which they may be padded.

dataset = tf.data.Dataset.range(100)
dataset = dataset.map(
	lambda x: tf.fill(
		[tf.cast(x, tf.int32)],
		x
	)
)
dataset = dataset.padded_batch(
	batch_size=4,
	padded_shapes=(None,),
	padding_values=0
)

• Using repeat before shuffle and batch methods mixes the boundaries of epochs (i.e., epochs don't have data clearly separated between them; they don't go through the data in the same pattern).