Merge pull request #182 from DIDSR/LOA

Merge LOA branch into master.

README.md

@@ -1,4 +1,4 @@
-<h3>README for iMRMC and iRoeMetz</h3>
+# README for iMRMC and iRoeMetz
 
 Here is a high-level overview of the iMRMC software:
 
@@ -7,39 +7,31 @@ Here is a high-level overview of the iMRMC software:
 
 Examples of specific MRMC analyses and other FAQs are available in the [iMRMC repository's Wiki](https://github.com/DIDSR/iMRMC/wiki).
 
-Visit the <a href="https://github.com/DIDSR/iMRMC/releases" rel="nofollow">release Page</a> to download the software or access the repository [HERE](https://github.com/DIDSR/iMRMC).
+The R package is available from [CRAN](https://cran.r-project.org/web/packages/iMRMC/index.html). You can also visit the <a href="https://github.com/DIDSR/iMRMC/releases" rel="nofollow">release Page</a> to download the R package (source tar.gz file and binary zip file), the zip and tar.gz files of the entire repository, or the iMRMC java app (jar file).
 
-The documentation for both packages can be found <a href="http://didsr.github.io/iMRMC/" rel="nofollow">here.</a>
+Documentation of the java app can be found here: http://didsr.github.io/iMRMC/.
 
-Here is a summary of updates to [iMRMC](https://github.com/DIDSR/iMRMC/blob/master/UPDATES_iMRMC.md) and a summary of updates to [iRoeMetz](https://github.com/DIDSR/iMRMC/blob/master/UPDATES_iRoeMetz.md).
+Here are summaries of updates to the now frozen java apps:  [iMRMC](https://github.com/DIDSR/iMRMC/blob/master/UPDATES_iMRMC.md) and [iRoeMetz](https://github.com/DIDSR/iMRMC/blob/master/UPDATES_iRoeMetz.md).
 
 Please check out a statement of the licenses related to our software [here](LICENSE.md).
 
-<h5>iMRMC</h5>
+## iMRMC
 
 Desc: The primary objective of the iMRMC application is to assist investigators with analyzing and sizing multi-reader multi-case (MRMC) reader studies that compare the difference in the area under Receiver Operating Characteristic curves (AUCs) from two modalities. The core elements of this application include the ability to perform MRMC variance analysis and the ability to size an MRMC trial.
 
  * The core iMRMC application is a stand-alone, precompiled, license-free Java applications and the source code. It can be used in GUI mode or on the command line.
  * There is also an R package that utilizes the core Java application. Examples for using the programs can be found in the R help files.
 
 Here are some ways to cite our work:
+* FDA/CDRH, “iMRMC: Software for the Statistical Analysis of multi-reader multi-case studies,” *RST Catalog*, 2022, https://doi.org/10.5281/ZENODO.6628838.
 * Gallas, Brandon D., Andriy Bandos, Frank Samuelson, and Robert F. Wagner. “A Framework for Random-Effects ROC Analysis: Biases with the Bootstrap and Other Variance Estimators.” Commun Stat A-Theory 38, no. 15 (2009): 2586–2603. https://doi.org/10.1080/03610920802610084.
 * Gallas, Brandon D. “One-Shot Estimate of MRMC Variance: AUC.” Acad Radiol 13, no. 3 (2006): 353–62. https://doi.org/10.1016/j.acra.2005.11.030.
 
-To cite the java code, please use this (with the appropriate version number):
-* Gallas, Brandon D., "IMRMC-java vX.XX: Application for Analyzing and Sizing MRMC Reader Studies." Silver Spring, MD, 2017. https://github.com/DIDSR/iMRMC/releases.
 
-To cite the R package, please use this (version number depends on whether you downloaded the code from GitHub or CRAN):
-* Gallas, Brandon D., "IMRMC-R vX.XX: Application for Analyzing and Sizing MRMC Reader Studies." Silver Spring, MD, 2017. https://cran.r-project.org/web/packages/iMRMC/index.html.
-
-<h5>iRoeMetz</h5>
+## iRoeMetz
 
 Desc: The iRoeMetz application can be used to simulate the reader scores for MRMC experiments via Monte Carlo methods given variance components of the ROC scores. The application also estimates the variance components of AUC and can calculate the variance components directly with numerical integration. The simulated experiments can be saved and used in MRMC variance analysis programs, such as iMRMC. The iRoeMetz application is a stand-alone, precompiled, license-free Java applications and the source code.
 
-<h5>R package iMRMC</h5>
-
-Desc: The R package iMRMC executes the iMRMC program which writes the results to the local files system, it reads the analysis results from the local file system, packs the analysis results into a list object, deletes the data and analysis results from the local file system, and returns the list object.
-
 Here is one way to cite our work:
 * Gallas, Brandon D., and Stephen L. Hillis. “Generalized Roe and Metz ROC Model: Analytic Link between Simulated Decision Scores and Empirical AUC Variances and Covariances.” J Med Img 1, no. 3 (2014): 031006. https://doi.org/doi:10.1117/1.JMI.1.3.031006.
 

Rpackage/iMRMC/DESCRIPTION

@@ -1,15 +1,15 @@
-Package: iMRMC
-Type: Package
-Title: Multi-Reader, Multi-Case Analysis Methods (ROC, Agreement, and Other Metrics)
-Version: 2.0.0
-Author: Brandon Gallas
-Maintainer: Brandon Gallas <Brandon.Gallas@fda.hhs.gov>
-Description: This software does Multi-Reader, Multi-Case (MRMC) analyses of data from imaging studies where clinicians (readers) evaluate patient images (cases). What does this mean? ... Many imaging studies are designed so that every reader reads every case in all modalities, a fully-crossed study. In this case, the data is cross-correlated, and we consider the readers and cases to be cross-correlated random effects. An MRMC analysis accounts for the variability and correlations from the readers and cases when estimating variances, confidence intervals, and p-values. The functions in this package can treat arbitrary study designs and studies with missing data, not just fully-crossed study designs. An overview of this software, including references presenting details on the methods, can be found here: <https://www.fda.gov/medical-devices/science-and-research-medical-devices/imrmc-software-do-multi-reader-multi-case-statistical-analysis-reader-studies>.
-URL: https://github.com/DIDSR/iMRMC, https://CRAN.R-project.org/package=iMRMC
-License: CC0
-Encoding: UTF-8
-LazyData: true
-Depends: R (>= 3.5.0)
-RoxygenNote: 7.3.1
-Imports: stats
-Suggests: testthat
+Package: iMRMC
+Type: Package
+Title: Multi-Reader, Multi-Case Analysis Methods (ROC, Agreement, and Other Metrics)
+Version: 2.1.0
+Author: Brandon Gallas
+Maintainer: Brandon Gallas <Brandon.Gallas@fda.hhs.gov>
+Description: This software does Multi-Reader, Multi-Case (MRMC) analyses of data from imaging studies where clinicians (readers) evaluate patient images (cases). What does this mean? ... Many imaging studies are designed so that every reader reads every case in all modalities, a fully-crossed study. In this case, the data is cross-correlated, and we consider the readers and cases to be cross-correlated random effects. An MRMC analysis accounts for the variability and correlations from the readers and cases when estimating variances, confidence intervals, and p-values. The functions in this package can treat arbitrary study designs and studies with missing data, not just fully-crossed study designs. An overview of this software, including references presenting details on the methods, can be found here: <https://www.fda.gov/medical-devices/science-and-research-medical-devices/imrmc-software-do-multi-reader-multi-case-statistical-analysis-reader-studies>.
+URL: https://github.com/DIDSR/iMRMC, https://CRAN.R-project.org/package=iMRMC
+License: CC0
+Encoding: UTF-8
+LazyData: true
+Depends: R (>= 3.5.0)
+RoxygenNote: 7.3.2
+Imports: stats, Matrix
+Suggests: testthat

Rpackage/iMRMC/NAMESPACE

@@ -1,34 +1,46 @@
-# Generated by roxygen2: do not edit by hand
-
-export(convertDFtoDesignMatrix)
-export(convertDFtoScoreMatrix)
-export(createGroups)
-export(createIMRMCdf)
-export(deleteCol)
-export(doAUCmrmc)
-export(doIMRMC)
-export(doIMRMC_java)
-export(doROCavg)
-export(doROCxy)
-export(extractPairedComparisonsBRBM)
-export(extractPairedComparisonsWRBM)
-export(getBRBM)
-export(getMRMCdataset)
-export(getMRMCscore)
-export(getWRBM)
-export(init.lecuyerRNG)
-export(laBRBM)
-export(laWRBM)
-export(renameCol)
-export(roc2binary)
-export(sim.gRoeMetz)
-export(sim.gRoeMetz.config)
-export(simMRMC)
-export(simRoeMetz.example)
-export(successDFtoROCdf)
-export(uStat11.conditionalD)
-export(uStat11.jointD)
-export(undoIMRMCdf)
-import(parallel)
-import(utils)
-importFrom(stats,qnorm)
+# Generated by roxygen2: do not edit by hand
+
+export(convertDFtoDesignMatrix)
+export(convertDFtoScoreMatrix)
+export(createGroups)
+export(createIMRMCdf)
+export(deleteCol)
+export(doAUCmrmc)
+export(doIMRMC)
+export(doIMRMC_java)
+export(doROCavg)
+export(doROCxy)
+export(extractPairedComparisonsBRBM)
+export(extractPairedComparisonsWRBM)
+export(getBRBM)
+export(getMRMCdataset)
+export(getMRMCscore)
+export(getWRBM)
+export(init.lecuyerRNG)
+export(laBRBM)
+export(laBRWM)
+export(laWRBM)
+export(laWRWM)
+export(renameCol)
+export(roc2binary)
+export(sim.NormalIG.Hierarchical)
+export(sim.NormalIG.Hierarchical.config)
+export(sim.gRoeMetz)
+export(sim.gRoeMetz.config)
+export(simMRMC)
+export(simRoeMetz.example)
+export(successDFtoROCdf)
+export(uStat11.conditionalD)
+export(uStat11.jointD)
+export(undoIMRMCdf)
+import(parallel)
+import(utils)
+importFrom(Matrix,Matrix)
+importFrom(Matrix,tcrossprod)
+importFrom(stats,aov)
+importFrom(stats,model.matrix)
+importFrom(stats,qnorm)
+importFrom(stats,rbeta)
+importFrom(stats,rgamma)
+importFrom(stats,rnorm)
+importFrom(stats,var)

Rpackage/iMRMC/NEWS.md

@@ -11,6 +11,12 @@ A big thank you to all that have worked on this software throughout the years. W
 *	Yousun Ko          •	Rowhan Pathare  
 * Si Wen             •	Adam Wunderlich   
 
+# iMRMC 2.1.0
+*08/2024*
+- Add functions and tests to do MRMC analysis of limits of agreement `laWRWM`, `laWRBM`, `laBRWM`, `laBRBM`
+- Add function and test to simulate MRMC agreement data: `sim.NormalIG.Hierarchical`
+- Add scripts to demonstrate new simulation and analysis of MRMC agreement data in the `Rpackage/iMRMC/inst/extra/limitsOfAgreement` folder
+
 # iMRMC 2.0.0
 *04/2024*
 - Add R only functions to analyze MRMC studies, notably `doIMRMC()`