Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report function alignment accuracy #59

Merged
merged 3 commits into from
Jan 5, 2025

Conversation

disinvite
Copy link
Collaborator

reccmp-reccmp now reports how many functions have the same virtual address in both the original and recompiled binary.

For example:

Total effective accuracy 97.83% across 4362 functions (97.03% actual accuracy)
68 functions are aligned (1.56%)

Should we add this to SVG output also?

Copy link
Collaborator

@jonschz jonschz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@@ -322,15 +332,20 @@ def main():

implemented_funcs = function_count

# Substitute an alternate value for the total number of functions in the file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that you added multiple explanations! How about Substitute an alternate value for the total number of functions in the target (e.g. to account for functions that have not been stubbed yet but have been counted by some analysis tool like Ghidra)? Or have I misunderstood the purpose of this argument?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that's correct. I'll expand the comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're here, should this really be function_count = max(function_count, int(args.total)) to prevent exceeding 100% accuracy?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes 👍

if args.total:
function_count = int(args.total)

if function_count > 0:
effective_accuracy = total_effective_accuracy / function_count * 100
actual_accuracy = total_accuracy / function_count * 100
alignment_accuracy = functions_aligned_count / function_count * 100
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe alignment_percentage? I think accuracy is fine for the other variables since the code can be matched to a fractional degree, but alignment is binary

@disinvite disinvite marked this pull request as ready for review January 5, 2025 18:40
@disinvite disinvite merged commit 5535363 into isledecomp:master Jan 5, 2025
4 checks passed
@disinvite disinvite deleted the alignment-report branch January 5, 2025 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants