ResNet18-2DD

A fully serializable 2D implementation of ResNet18, incorporating improvements from the paper "Bag of Tricks for Image Classification with Convolutional Neural Networks" along with additional personal optimizations and modifications.

This repository also includes implementations of the Hardswish and Mish activation functions:

• Searching for MobileNetV3
• Mish: A Self Regularized Non-Monotonic Neural Activation Function

The codebase is fully integratable inside the TensorFlow and Keras code pipelines.

Key Enhancements

• Modified Stem: Utilizes three convolutional layers instead of a single one.
• ResNet-B Inspired Strides: Moved the stride placement in the residual blocks from the first convolution to the second.
• ResNet-D Inspired Shortcut: Introduces an average pooling layer before the 1x1 convolution in the shortcut connection.

Note: The ResNet-C image is sourced from the referenced paper, while the shortcut image is created by the author.

Installation & Usage

This code is compatible with Python 3.12.8 and TensorFlow 2.18.0.

from ResNet182DD import ResNet182DD


model = ResNet182DD()
model.build((None, 256, 256, 3))
model.summary()

Model Summary Example

Model: "res_net182dd"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d_layer (Conv2DLayer)           │ (None, 128, 128, 32)        │             864 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_layer_1 (Conv2DLayer)         │ (None, 128, 128, 32)        │           9,216 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_layer_2 (Conv2DLayer)         │ (None, 128, 128, 64)        │          18,432 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D)         │ (None, 64, 64, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd (Residual2DD)            │ (None, 64, 64, 64)          │          73,728 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_1 (Residual2DD)          │ (None, 32, 32, 128)         │         229,376 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_2 (Residual2DD)          │ (None, 32, 32, 128)         │         294,912 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_3 (Residual2DD)          │ (None, 16, 16, 256)         │         917,504 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_4 (Residual2DD)          │ (None, 16, 16, 256)         │       1,179,648 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_5 (Residual2DD)          │ (None, 8, 8, 512)           │       3,670,016 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ residual2dd_6 (Residual2DD)          │ (None, 8, 8, 512)           │       4,718,592 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ global_average_pooling2d             │ (None, 512)                 │               0 │
│ (GlobalAveragePooling2D)             │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense)                        │ (None, 256)                 │         131,328 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
 Total params: 11,243,616 (42.89 MB)
 Trainable params: 11,243,616 (42.89 MB)
 Non-trainable params: 0 (0.00 B)

License

This work is under an MIT License.