Skip to content

Visual Perturbation Metrics #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
3 of 5 tasks
someshsingh22 opened this issue Aug 22, 2021 · 5 comments · May be fixed by #63
Open
3 of 5 tasks

Visual Perturbation Metrics #61

someshsingh22 opened this issue Aug 22, 2021 · 5 comments · May be fixed by #63
Assignees
@devaletanmay @ShreeyashGo
Labels
good first issue Good for newcomers Implementation For implementation of a new algorithm/tutorial Priority:High High Priority
Milestone
Unit-2: Computer ...

Comments

@someshsingh22
Copy link
Member

someshsingh22 commented Aug 22, 2021 •
edited
Loading

For evasive whitebox or blackbox attacks, the objective of each attack is to fool the model to predict a different class but making it deceptive by making small changes, these changes are measured in distances for Example the L1/L2 Norm of difference.

Implement these metrics

  • L1, L2 ... Lk Norm
  • ISSM
  • PSNR
  • SAM
  • SRE

You can find numpy and cv2 implementation at https://github.com/up42/image-similarity-measures/blob/master/image_similarity_measures/quality_metrics.py
@someshsingh22 someshsingh22 added good first issue Good for newcomers Priority:High High Priority Implementation For implementation of a new algorithm/tutorial labels Aug 22, 2021
@ShreeyashGo
Copy link

I would like to take this up. I plan on creating a function that will take the original and the image/feature vector after the adversarial attack and output the L1, L2, L(infinity) norms. Should I also be implementing L0 norms and other less conventional norms like L3, L4.... ?

@devaletanmay
Copy link

I would like to take up this issue

@someshsingh22
Copy link
Member Author

@ShreeyashGo @devaletanmay we are not using separate classes as the code doesn't look very clean with many metrics, also I am adding more metrics you can split the implementation

@someshsingh22 someshsingh22 linked a pull request Aug 24, 2021 that will close this issue
Visual Perturbations #63
Open
@ShreeyashGo
Copy link

After implementing the SRE and SAM, these are my observations
Test SRE: 41.36633261587073
SRE from implementation: 39.9395

Test SAM: 89.34839413786915 (with default dtype of the inputted numpy array as integer)
Test SAM: 34.38530383960234 (with dtpye of the inputted numpy array changed to float)
SAM from implementation: 34.385303497314453

@someshsingh22
Copy link
Member Author

Ok we will change the test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@devaletanmay devaletanmay

@ShreeyashGo ShreeyashGo

Labels
good first issue Good for newcomers Implementation For implementation of a new algorithm/tutorial Priority:High High Priority
Projects
None yet
Milestone
Unit-2: Computer Vision
Development

Successfully merging a pull request may close this issue.

Visual Perturbations Adversarial-Deep-Learning/code-soup
3 participants
@someshsingh22 @devaletanmay @ShreeyashGo