Skip to content

Latest commit

 

History

History
69 lines (59 loc) · 3.84 KB

README.md

File metadata and controls

69 lines (59 loc) · 3.84 KB

Relational Proxies: Emergent Relationships as Fine-Grained Discriminators

Official implementation of "Relational Proxies: Emergent Relationships as Fine-Grained Discriminators", NeurIPS 2022 (Spotlight).
Additional links: Spotlight Presentation | arXiv | Video & Poster

Our framework helps learn a cross-view representation by modelling local-to-global emergent relationships for Fine-Grained Visual Categorization (FGVC).

Model Diagram

Environment Setup

This project is implemented using PyTorch. A conda environment with all related dependencies can be created as follows:

  1. Clone the project repository:
git clone https://github.com/abhrac/relational-proxies.git
cd relational-proxies
  1. Create and activate conda environment:
conda env create -f environment.yml
conda activate relational-proxies
  1. Download the .pth file from here and place it in the ./view_extractor/ folder under the project root.

Training

To train the model from scratch, run the following:

python3 src/main.py --data_root='RootDirOfAllDatasets' --dataset='DatasetName'

The run_expt.sh file contains sample training commands.

Evaluation

To evaluate on a dataset using pretrained weights, first download the model for the corresponding dataset from here and place it under the folder ./checkpoint/$DataSetName/, where ./checkpoint is under the project root, but could optionally be elsewhere too (see src/options.py). Then, run the following command:

python3 src/main.py --data_root='RootDirForAllDatasets' --dataset='DatasetName' --pretrained --eval_only

Results

FGVC Aircraft Stanford Cars CUB NABirds iNaturalist Cotton Cultivar Soy Cultivar
MaxEnt, NeurIPS'18 89.76 93.85 86.54 - - - -
DBTNet, NeurIPS'19 91.60 94.50 88.10 - - - -
StochNorm, NeurIPS'20 81.79 87.57 79.71 74.94 60.75 45.41 38.50
MMAL, MMM'21 94.70 95.00 89.60 87.10 69.85 65.00 47.00
FFVT, BMVC'21 79.80 91.25 91.65 89.42 70.30 57.92 44.17
CAP, AAAI'21 94.90 95.70 91.80 91.00 - - -
TransFG, AAAI'22 80.59 94.80 91.70 90.80 71.70 45.84 38.67
Ours (Relational Proxies) 95.25 $\pm$ 0.02 96.30 $\pm$ 0.04 92.00 $\pm$ 0.01 91.20 $\pm$ 0.02 72.15 $\pm$ 0.03 69.81 $\pm$ 0.04 51.20 $\pm$ 0.02

Disclaimer

The pretrained weights provided can be used to reproduce the results in the paper. However, since the training of the models were done using pretrained weights from prior works ([48] in the main paper), and it consisted of several phases with slightly different hyperparameters for each phase, not all of which were always kept track of, running the training scripts from scratch with the default hyperparameters provided here is unlikely to produce expected results. The purpose of the training scripts provided here is to illustrate how the end-to-end pipelines were implemented. The purpose of the pretrained weights provided here is to reproduce the actual results.

Citation

@inproceedings{Chaudhuri2022RelationalProxies,
 author = {Abhra Chaudhuri and Massimiliano Mancini and Zeynep Akata and Anjan Dutta},
 booktitle = {Proceedings of Advances in Neural Information Processing Systems (NeurIPS)},
 title = {Relational Proxies: Emergent Relationships as Fine-Grained Discriminators},
 year = {2022}
}