Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor observe_process function #33

Closed
athowes opened this issue May 14, 2024 · 10 comments · Fixed by #375
Closed

Refactor observe_process function #33

athowes opened this issue May 14, 2024 · 10 comments · Fixed by #375
Assignees

Comments

@athowes
Copy link
Collaborator

athowes commented May 14, 2024

New plan

  • Create add_event_vars function to add upper and lower given either lower and window or lower and upper
  • Enforce our names structure which is ptime_lwr and ptime_upr (i.e. change the names)
  • Find a way to slot the back into existing functionality so as not to break anything for now

Old

I've just used observe_process(). Some notes on possible refactor:

  • The name "observe process" is vague and seems misleading for what it is doing. For my understanding, what it is doing is implementing interval censoring. If so, the name should reflect that.
  • If it is implementing interval censoring, then I think that there should be arguments available for specifying the censoring intervals. At the moment it looks like it's just using daily intervals, and some kind of fixed upper and lower.
@seabbs
Copy link
Contributor

seabbs commented May 14, 2024

I think its implementing the observation process which in this instance is censoring but agree it can be renamed.

FYI I think this should be organised under some kind of simulation label (as this is part of the simulation/validation code) - in general if we start organising issues like this it may help us gain some clarity around what is happening

@athowes
Copy link
Collaborator Author

athowes commented May 14, 2024

Even in this instance I'd think of the observation process as having multiple stages, including but not limited to censoring. I think having functions to modularly implement each part of the observation process would be nice.

@seabbs
Copy link
Contributor

seabbs commented May 15, 2024

Yes agree

@athowes athowes changed the title Refactoring observe_process() Refactor interval censoring function May 15, 2024
@athowes
Copy link
Collaborator Author

athowes commented May 15, 2024

As a part of this issue, I propose to:

  • rename observe_process() to something to represent interval censoring,
  • include arguements for the intervals to be used,
  • create unit tests for the function,
  • fill out the roxygen2 documentation for the function.

@athowes athowes self-assigned this May 30, 2024
@athowes athowes removed their assignment Aug 8, 2024
@athowes athowes changed the title Refactor interval censoring function Refactor observe_process() function Sep 23, 2024
@athowes athowes added high and removed medium labels Sep 23, 2024
@athowes
Copy link
Collaborator Author

athowes commented Sep 23, 2024

As a part of this issue, I propose to:
* [ ] rename observe_process() to something to represent interval censoring,
* [ ] include arguements for the intervals to be used,
* [ ] create unit tests for the function,
* [ ] fill out the roxygen2 documentation for the function.

I am now working in this direction.

@athowes
Copy link
Collaborator Author

athowes commented Sep 23, 2024

Note on implementation:

  • I think we should be using .floor_mult here to round down to the nearest multiple of x
  • I guess there is a question about whether people are censoring their data or they are taking already censored data and adding some columns. Maybe it just works the same either way...
  • I think we should be letting users input x
  • Probably we can let there be different censoring windows used for primary and secondary
    • Then I think we can re-use the names pwindow and swindow for the sizes of these
  • I don't know why we have _daily as new columns. If I just delete these does it do anything? Use code search to find all instances of _daily

@athowes
Copy link
Collaborator Author

athowes commented Sep 23, 2024

I've drafted a function that would look like this:

observe_process <- function(linelist, pwindow = 1, swindow = 1) {
  assert_numeric(pwindow, lower = 0)
  assert_numeric(swindow, lower = 0)
  
  linelist |>
    mutate(
      ptime_lwr = .floor_mult(.data$ptime, f = pwindow),
      ptime_upr = .data$ptime_lwr + pwindow,
      stime_lwr = .floor_mult(.data$stime, f = pwindow),
      stime_upr = .data$stime_lwr + swindow,
      delay_lwr = pmax(0, .data$stime_lwr - .data$ptime_lwr - 1),
      delay_upr = .data$delay_lwr + pwindow + swindow
    )
}

If users provide data where stime and ptime are already censored to the lower bound then this would do nothing and add the upper bounds.

I'm also interested as to why in the original code the upper bound for the delay was the lower bound plus one. Isn't this wrong? The upper bound should be plus two?

@athowes athowes self-assigned this Oct 1, 2024
@athowes
Copy link
Collaborator Author

athowes commented Oct 2, 2024

Expanding upon

if users provide data where stime and ptime are already censored to the lower bound then this would do nothing and add the upper bounds.

then I think we can do two things with this function:

  1. Add the upr columns needed for input into future model functionality to data that already is censored to the lower
  2. Add the lwr and upr columns to data that has no censoring

Things to be worried about here:

  • What if data are censored to the upper? This would seem weird to me

@athowes
Copy link
Collaborator Author

athowes commented Oct 2, 2024

To me, the function should also be renamed. I am thinking of it as a "add censoring" function, which might exist as some part of a more modular observation process collection of functions. As I write above, I think this function can also be used within preprocessing for data that are censored but don't have the windows in the data.frame.

add_censor_columns is a example bad name from the top of my head to show where I am thinking about going with this.

@athowes
Copy link
Collaborator Author

athowes commented Oct 2, 2024

Tag @seabbs for comment on above before I begin working on PR #352 again (don't want to go off in wrong direction).

@athowes athowes changed the title Refactor observe_process() function Refactor observe_process function Oct 10, 2024
@athowes athowes linked a pull request Oct 14, 2024 that will close this issue
9 tasks
seabbs pushed a commit that referenced this issue Oct 15, 2024
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!
seabbs pushed a commit that referenced this issue Jan 10, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 96671b56707c79c32c3b90aae51f07fb1c69343d [formerly c053cfd78c114c248605ed72f1cbabb4b32b3569]
Former-commit-id: cc406257db083a003725ac53a1ac52c64129f234
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 05053e1
Former-commit-id: 0fb7ec80e140a7887bb091dac128d53ce950c5ec
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 05053e1
Former-commit-id: 0fb7ec80e140a7887bb091dac128d53ce950c5ec
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 05053e1
Former-commit-id: 0fb7ec80e140a7887bb091dac128d53ce950c5ec
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 96671b56707c79c32c3b90aae51f07fb1c69343d [formerly c053cfd78c114c248605ed72f1cbabb4b32b3569]
Former-commit-id: cc406257db083a003725ac53a1ac52c64129f234
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 96671b56707c79c32c3b90aae51f07fb1c69343d [formerly c053cfd78c114c248605ed72f1cbabb4b32b3569]
Former-commit-id: cc406257db083a003725ac53a1ac52c64129f234
Former-commit-id: 99acbd4
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 05053e1
Former-commit-id: 0fb7ec80e140a7887bb091dac128d53ce950c5ec
Former-commit-id: 30c9d47dc3d7266df6c74eb77fa4a870232d5c36 [formerly 0f90b0e]
Former-commit-id: c4777e5410e516ba0ae27961dc3e9bfb94decb54
seabbs pushed a commit that referenced this issue Jan 21, 2025
* Add template for new add_event_vars function

* First draft complete of add_event_vars functionality

* Reduce complexity of add_event_vars using a helper function

* Template for add_obs_vars

* Revert commit to wrong branch!

Former-commit-id: 05053e1
Former-commit-id: 0fb7ec80e140a7887bb091dac128d53ce950c5ec
Former-commit-id: 30c9d47dc3d7266df6c74eb77fa4a870232d5c36 [formerly 0f90b0e]
Former-commit-id: c4777e5410e516ba0ae27961dc3e9bfb94decb54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants