Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential issue with parsing fio output in parser_fio.sh #433

Open
vermagit opened this issue Dec 16, 2020 · 4 comments
Open

Potential issue with parsing fio output in parser_fio.sh #433

vermagit opened this issue Dec 16, 2020 · 4 comments
Labels
bug Something isn't working

Comments

@vermagit
Copy link

vermagit commented Dec 16, 2020

Not sure if the format of the output produced by fio has changed, but parser_fio.sh does not seem to be picking up the right fields for the jq parsing.

To Reproduce
Steps to reproduce the behavior:

  1. sudo yum install fio (fio-3.19, HC, CentOS-HPC 8.1)
  2. See sample output file produced by a variation of fio.pbs. Line of interest is as follows:
    READ: bw=611MiB/s (641MB/s), 611MiB/s-611MiB/s (641MB/s-641MB/s), io=512MiB (537MB), run=838-838msec
  3. Parser fails when run as-is. Remove the '|tonumber' from the parser to see that the fields being picked up appear incorrectly
@vermagit vermagit added the bug Something isn't working label Dec 16, 2020
@vermagit vermagit changed the title parser_fio.sh Potential issue with parsing fio output in parser_fio.sh Dec 16, 2020
@xpillons
Copy link
Collaborator

@vermagit is this specific to CentOS 8.1 ?

@vermagit
Copy link
Author

This should not be specific to 8.1, but I'd guess specific to the fio-3.19 version maybe. I can share sample output files being written since that is the source of the discrepancy as the parser seems to be expecting a different format.

@garvct
Copy link
Collaborator

garvct commented Dec 16, 2020

I think if may be easier to use the fio --output-format=json to specify you want fio output in json format directly and then parse that file.
e.g to extract fio File size, Block size, numjobs and read BW

jq '.jobs[]."job options".size,.jobs[]."job options".bs,.jobs[]."job options".numjobs,.jobs[].read.bw' fio_output.json
"2G"
"4M"
"4"
2616533

@xpillons
Copy link
Collaborator

good suggestion @garvct , @vermagit please feel free to contribute via a PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants