Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report function alignment accuracy #59

Merged
merged 3 commits into from
Jan 5, 2025
Merged

Report function alignment accuracy #59

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6c7ca8f
Report function alignment accuracy
disinvite Jan 4, 2025
73d7e55
Merge branch 'master' into alignment-report
disinvite Jan 5, 2025
2046591
Review comments
disinvite Jan 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions reccmp/tools/asmcmp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,10 @@ def main():

### Compare everything.

# Count how many functions have the same virtual address in orig and recomp.
functions_aligned_count = 0

# Number of functions compared (i.e. excluding stubs)
function_count = 0
total_accuracy = 0
total_effective_accuracy = 0
Expand All @@ -275,6 +279,12 @@ def main():
match, show_both_addrs=args.print_rec_addr, is_plain=args.no_color
)

if (
match.match_type == SymbolType.FUNCTION
and match.orig_addr == match.recomp_addr
):
functions_aligned_count += 1

if match.match_type == SymbolType.FUNCTION and not match.is_stub:
function_count += 1
total_accuracy += match.ratio
Expand Down Expand Up @@ -322,15 +332,20 @@ def main():

implemented_funcs = function_count

# Substitute an alternate value for the total number of functions in the file
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that you added multiple explanations! How about Substitute an alternate value for the total number of functions in the target (e.g. to account for functions that have not been stubbed yet but have been counted by some analysis tool like Ghidra)? Or have I misunderstood the purpose of this argument?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, that's correct. I'll expand the comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're here, should this really be function_count = max(function_count, int(args.total)) to prevent exceeding 100% accuracy?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes 👍

if args.total:
function_count = int(args.total)

if function_count > 0:
effective_accuracy = total_effective_accuracy / function_count * 100
actual_accuracy = total_accuracy / function_count * 100
alignment_accuracy = functions_aligned_count / function_count * 100
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe alignment_percentage? I think accuracy is fine for the other variables since the code can be matched to a fractional degree, but alignment is binary

print(
f"\nTotal effective accuracy {effective_accuracy:.2f}% across {function_count} functions ({actual_accuracy:.2f}% actual accuracy)"
)
print(
f"{functions_aligned_count} functions are aligned ({alignment_accuracy:.2f}%)"
)

if args.svg is not None:
gen_svg(
Expand Down
Loading