This project demonstrates a multiclass classification problem using a neural network with the softmax activation function. The model is implemented using TensorFlow and Keras, and trained on a synthetic dataset generated with make_blobs
.
model.ipynb
: Jupyter notebook containing the model definition, training, and prediction code.README.md
: Project documentation.
-
Clone the repository:
git clone https://github.com/yourusername/your-repo-name.git cd your-repo-name
-
Create and activate a virtual environment:
python -m venv env .\env\Scripts\activate # On Windows source env/bin/activate # On macOS/Linux
-
Install the required packages:
pip install -r requirements.txt
-
Open the Jupyter notebook:
jupyter notebook model.ipynb
-
Run the cells to train the model and make predictions.
The notebook starts by importing necessary libraries such as numpy
, matplotlib
, tensorflow
, and sklearn
. It also sets up logging to suppress TensorFlow warnings.
A custom softmax function my_softmax
is defined to convert logits to probabilities.
A synthetic dataset is created using make_blobs
with 4 centers.
Two models are defined and trained:
-
Model with Softmax Activation:
- Dense layer with 25 units and ReLU activation.
- Dense layer with 15 units and ReLU activation.
- Dense layer with 4 units and softmax activation.
- Compiled using
SparseCategoricalCrossentropy
loss andAdam
optimizer. - Trained on the synthetic dataset for 10 epochs.
-
Model with Linear Activation:
- Dense layer with 25 units and ReLU activation.
- Dense layer with 15 units and ReLU activation.
- Dense layer with 4 units and linear activation.
- Compiled using
SparseCategoricalCrossentropy
loss withfrom_logits=True
andAdam
optimizer. - Trained on the synthetic dataset for 10 epochs.
Predictions are made on the training data using the trained models. The predicted class probabilities and the predicted classes are printed. The largest and smallest values in the predicted probabilities are also printed. The accuracy of the model is calculated by comparing the predicted classes with the actual labels.
Here is an example of how to define and train the model with softmax activation:
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
import tensorflow as tf
import numpy as np
from sklearn.datasets import make_blobs
# Create synthetic dataset
centers = [[-5, 2], [-2, -2], [1, 2], [5, -2]]
X_train, y_train = make_blobs(n_samples=2000, centers=centers, cluster_std=1.0, random_state=30)
# Define the model with softmax activation
model = Sequential([
Dense(25, activation='relu', input_shape=(2,)),
Dense(15, activation='relu'),
Dense(4, activation='softmax')
])
# Compile the model
model.compile(
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
optimizer=tf.keras.optimizers.Adam(0.001),
metrics=['accuracy']
)
# Train the model
model.fit(X_train, y_train, epochs=10)
# Predict the class probabilities
p_preferred = model.predict(X_train)
print(f"two example output vectors:\n {p_preferred[:2]}")
print("largest value", np.max(p_preferred), "smallest value", np.min(p_preferred))