Multiclass Classification with Softmax Function

This project demonstrates a multiclass classification problem using a neural network with the softmax activation function. The model is implemented using TensorFlow and Keras, and trained on a synthetic dataset generated with make_blobs.

Project Structure

• model.ipynb: Jupyter notebook containing the model definition, training, and prediction code.
• README.md: Project documentation.

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/your-repo-name.git
   cd your-repo-name

2. Create and activate a virtual environment:

   python -m venv env
   .\env\Scripts\activate  # On Windows
   source env/bin/activate  # On macOS/Linux

3. Install the required packages:

   pip install -r requirements.txt

Usage

1. Open the Jupyter notebook:

   jupyter notebook model.ipynb

2. Run the cells to train the model and make predictions.

Model Description

Imports and Setup

The notebook starts by importing necessary libraries such as numpy, matplotlib, tensorflow, and sklearn. It also sets up logging to suppress TensorFlow warnings.

Softmax Function

A custom softmax function my_softmax is defined to convert logits to probabilities.

Dataset Creation

A synthetic dataset is created using make_blobs with 4 centers.

Model Definition and Training

Two models are defined and trained:

1. Model with Softmax Activation:

   • Dense layer with 25 units and ReLU activation.
   • Dense layer with 15 units and ReLU activation.
   • Dense layer with 4 units and softmax activation.
   • Compiled using SparseCategoricalCrossentropy loss and Adam optimizer.
   • Trained on the synthetic dataset for 10 epochs.

2. Model with Linear Activation:

   • Dense layer with 25 units and ReLU activation.
   • Dense layer with 15 units and ReLU activation.
   • Dense layer with 4 units and linear activation.
   • Compiled using SparseCategoricalCrossentropy loss with from_logits=True and Adam optimizer.
   • Trained on the synthetic dataset for 10 epochs.

Predictions and Accuracy Calculation

Predictions are made on the training data using the trained models. The predicted class probabilities and the predicted classes are printed. The largest and smallest values in the predicted probabilities are also printed. The accuracy of the model is calculated by comparing the predicted classes with the actual labels.

Example Code

Here is an example of how to define and train the model with softmax activation:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
import tensorflow as tf
import numpy as np
from sklearn.datasets import make_blobs

# Create synthetic dataset
centers = [[-5, 2], [-2, -2], [1, 2], [5, -2]]
X_train, y_train = make_blobs(n_samples=2000, centers=centers, cluster_std=1.0, random_state=30)

# Define the model with softmax activation
model = Sequential([
    Dense(25, activation='relu', input_shape=(2,)),
    Dense(15, activation='relu'),
    Dense(4, activation='softmax')
])

# Compile the model
model.compile(
    loss=tf.keras.losses.SparseCategoricalCrossentropy(),
    optimizer=tf.keras.optimizers.Adam(0.001),
    metrics=['accuracy']
)

# Train the model
model.fit(X_train, y_train, epochs=10)

# Predict the class probabilities
p_preferred = model.predict(X_train)
print(f"two example output vectors:\n {p_preferred[:2]}")
print("largest value", np.max(p_preferred), "smallest value", np.min(p_preferred))