You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am running pymultinest on many cutouts of a .fits file, essentially iterating over about 1000 times and running pymultinest to completion before moving on to the next iteration. I am using TACC's Lonestar6 high performance computing cluster for this. The issue I am having is that, randomly, one of the cutouts will start and never progress past this point:
Starting MultiNest
generating live points
live points generated, starting sampling
The following files are the only ones that are created:
run_F160W_233_760_20.7th_mag_singles_IS.points
run_F160W_233_760_20.7th_mag_singles_params.json
run_F160W_233_760_20.7th_mag_singles_ev.dat
run_F160W_233_760_20.7th_mag_singles_IS.ptprob
run_F160W_233_760_20.7th_mag_singles_phys_live.points
run_F160W_233_760_20.7th_mag_singles_IS.iterinfo
run_F160W_233_760_20.7th_mag_singles_live.points
run_F160W_233_760_20.7th_mag_singles_resume.dat
This issue happens completely randomly, does not usually repeat on the same cutout when the program is rerun, and will cause an entire run to halt until the max time set in the SLURM file is hit.
Is this an issue you have ever encountered before and have any idea how to solve? Even if there was some way to abort the mutlinest run if it does not progress past a certain point, that would be useful to keep the scheduled job going. I can provide the details in any of the files above if it helps.
Thank you!
The text was updated successfully, but these errors were encountered:
I don't know. You probably need to put some prints into your function to see where it is stuck.
One feature of multinest is that prior samples below a threshold (by default log_zero = -1e100) are ignored and not used. Maybe your likelihood function only returns values below that?
I am running pymultinest on many cutouts of a .fits file, essentially iterating over about 1000 times and running pymultinest to completion before moving on to the next iteration. I am using TACC's Lonestar6 high performance computing cluster for this. The issue I am having is that, randomly, one of the cutouts will start and never progress past this point:
MultiNest v3.10
Copyright Farhan Feroz & Mike Hobson
Release Jul 2015
no. of live points = 400
dimensionality = 6
Starting MultiNest
generating live points
live points generated, starting sampling
The following files are the only ones that are created:
run_F160W_233_760_20.7th_mag_singles_IS.points
run_F160W_233_760_20.7th_mag_singles_params.json
run_F160W_233_760_20.7th_mag_singles_ev.dat
run_F160W_233_760_20.7th_mag_singles_IS.ptprob
run_F160W_233_760_20.7th_mag_singles_phys_live.points
run_F160W_233_760_20.7th_mag_singles_IS.iterinfo
run_F160W_233_760_20.7th_mag_singles_live.points
run_F160W_233_760_20.7th_mag_singles_resume.dat
This issue happens completely randomly, does not usually repeat on the same cutout when the program is rerun, and will cause an entire run to halt until the max time set in the SLURM file is hit.
Is this an issue you have ever encountered before and have any idea how to solve? Even if there was some way to abort the mutlinest run if it does not progress past a certain point, that would be useful to keep the scheduled job going. I can provide the details in any of the files above if it helps.
Thank you!
The text was updated successfully, but these errors were encountered: