This experiment investigates potential bias in several machine learning models applied to sentiment analysis of patient reviews. The models used include zero-R,
- Preprocessing only: In the initial phase, the models struggle with inherent biases present in the raw data, primarily due to class imbalance, where positive comments dominate.
- Balancing genders: This phase addresses gender imbalance in the data. While gender balancing improves fairness between male and female groups, it does not resolve the class imbalance issue.
- Balancing classes: In the final phase, the class distribution between positive and negative reviews is balanced. This significantly enhances model performance, particularly for negative sentiment classification. The analysis concludes that the main source of bias in the model is the overrepresentation of positive comments, not gender.
.
├── README.md
├── data
│ ├── README.txt
│ ├── TFIDF_TRAIN.csv # training data in TFIDF representation
│ ├── TFIDF_VALIDATION.csv # validation data in TFIDF representation
│ ├── TRAIN.csv # raw training data with labels
│ └── VALIDATION.csv # raw validation data with labels
├── main.ipynb
└── requirements.txt
The validation set is used as a test set in this project because the original test set is unlabeled.
It is recommended to run the code for this project within a development container in VS Code.
git clone https://github.com/wille-wang/ml-sentiment-bias.git
cd ml-sentiment-bias
code .
devcontainer open . # Install `ms-vscode-remote.remote-containers` in VS Code firstThis project is modified from Assignment 3 of Introduction to Machine Learning (COMP90049) at the University of Melbourne, Semester 2, 2023.