Make Tests Parallel Again (MTPA)

Author: LPTK

.github/workflows/nix.yml

@@ -18,7 +18,8 @@ jobs:
       - name: Install TypeScript
         run: npm ci
       - name: Run test
-        run: sbt -J-Xmx4096M -J-Xss8M test
+        # Not running all tests because those outside of hkmc2 are obsolete (will be removed)
+        run: sbt -J-Xmx4096M -J-Xss8M hkmc2AllTests/test
       - name: Check no changes
         run: |
           git update-index -q --refresh

README.md

@@ -1,37 +1,18 @@
 # MLscript
 
-What would TypeScript look like if it had been designed with type inference and soundness in mind?
+This is the second iteration of the MLscript compiler,
+nicknamed _hkmc2_ (Hong Kong MLscript Compiler v2).
 
-We provide one possible answer in MLscript, an object-oriented and functional programming language with records, generic classes, mix-in traits, first-class unions and intersections, instance matching, and ML-style principal type inference.
-These features can be used to implement expressive class hierarchies as well as extensible sums and products.
-
-MLscript supports union, intersection, and complement (or negation) connectives, making sure they form a Boolean algebra, and add enough structure to derive a sound and complete type inference algorithm.
 
 ## Getting Started
 
 ### Project Structure
 
 #### Sub-Projects
 
-- The ts2mls sub-project allows you to use TypeScript libraries in MLscript. It can generate libraries' declaration information in MLscript by parsing TypeScript AST, which can be used in MLscript type checking.
-
-#### Directories
-
-- The `shared/src/main/scala/mlscript` directory contains the sources of the MLscript compiler.
-
-- The `shared/src/test/scala/mlscript` directory contains the sources of the testing infrastructure.
-
-- The `shared/src/test/diff` directory contains the actual tests.
-
-- The `ts2mls/js/src/main/scala/ts2mls` directory contains the sources of the ts2mls module.
-
-- The `ts2mls/js/src/test/scala/ts2mls` directory contains the sources of the ts2mls declaration generation test code.
+Most SBT subprojects are obsolete and will be removed in the future.
 
-- The `ts2mls/jvm/src/test/scala/ts2mls` directory contains the sources of the ts2mls diff test code.
-
-- The `ts2mls/js/src/test/typescript` directory contains the TypeScript test code.
-
-- The `ts2mls/js/src/test/diff` directory contains the declarations generated by ts2mls.
+Most of the important code of the new compiler is in the `hkmc2` folder.
 
 ### Prerequisites
 
@@ -54,14 +35,30 @@ brew install mimalloc boost gmp
 
 ### Running the tests
 
-Running the main MLscript tests only requires the Scala Build Tool installed.
-In the terminal, run `sbt mlscriptJVM/test`.
+Running the tests requires the Scala Build Tool (SBT) installed.
+
+We recommend running all tests in the SBT shell,
+i.e., do not restart SBT every time,
+but launch it in shell mode (with command `sbt`)
+and then use one of the following commands.
+
+- `hkmc2AllTests/test` for running all hkmc2 tests.
+- `hkmc2JVM/test` for running only the compilation tests, in `hkmc2/shared/src/test/mlscript-compile`.
+- `hkmc2DiffTests/test` for running only the diff-tests, in `hkmc2/shared/src/test/mlscript`.
+- `~hkmc2DiffTests/Test/run` for running the test watcher,
+  which updates test files as you save them and recompiles the Scala sources automatically on change.
+- `test` for compiling all JVM and JS subprojects
+  and running every single test in the repository,
+  including obsolete ones.
 
-Running the ts2mls MLscript tests requires NodeJS, and TypeScript in addition.
-In the terminal, run `sbt ts2mlsTest/test`.
+Another useful SBT incantation is `; hkmc2AllTests/test; ~hkmc2DiffTests/Test/run`.
+This command runs all hkmc2 tests once and then starts the test watcher.
+This is a useful command to use periodically while making changes to the compiler,
+to check that you haven't broken anything.
 
-You can also run all tests simultaneously.
-In the terminal, run `sbt test`.
+Note that when saved, the special file `ChangedTests.cmd` will trigger the test watcher to run
+all tests that currently have unstaged changes in git.
+This is useful when you have a working subset of tests that you want to run periodically.
 
 ### Running tests individually
 
@@ -89,6 +86,8 @@ private val testsData = List(
   )
 ```
 
+
+<!--
 ### Running the web demo locally
 
 To run the demo on your computer, compile the project with `sbt fastOptJS`, then open the `local_testing.html` file in your browser.
@@ -98,3 +97,6 @@ in `shared/src/main/scala/mlscript`,
 have it compile to JavaScript on file change with command
 `sbt ~fastOptJS`,
 and immediately see the results in your browser by refreshing the page with `F5`.
+-->
+
+

build.sbt

@@ -19,7 +19,7 @@ ThisBuild / scalacOptions ++= Seq(
 )
 
 lazy val root = project.in(file("."))
-  .aggregate(mlscriptJS, mlscriptJVM, ts2mlsTest, compilerJVM, hkmc2JS, hkmc2JVM, coreJS, coreJVM)
+  .aggregate(mlscriptJS, mlscriptJVM, ts2mlsTest, compilerJVM, hkmc2AllTests, coreJS, coreJVM)
   .settings(
     publish := {},
     publishLocal := {},
@@ -49,8 +49,6 @@ lazy val hkmc2 = crossProject(JSPlatform, JVMPlatform).in(file("hkmc2"))
       baseDirectory.value.getParentFile()/"shared"/"src"/"test"/"mlscript", "*.mls", NothingFilter),
     watchSources += WatchSource(
       baseDirectory.value.getParentFile()/"shared"/"src"/"test"/"mlscript", "*.cmd", NothingFilter),
-    
-    Test/run/fork := true, // so that CTRL+C actually terminates the watcher
   )
   .jvmSettings(
   )
@@ -59,6 +57,22 @@ lazy val hkmc2 = crossProject(JSPlatform, JVMPlatform).in(file("hkmc2"))
 lazy val hkmc2JVM = hkmc2.jvm
 lazy val hkmc2JS = hkmc2.js
 
+lazy val hkmc2DiffTests = project.in(file("hkmc2DiffTests"))
+  .dependsOn(hkmc2JVM)
+  .settings(
+    scalaVersion := scala3Version,
+    
+    libraryDependencies += "org.scalactic" %%% "scalactic" % "3.2.18",
+    libraryDependencies += "org.scalatest" %%% "scalatest" % "3.2.18" % "test",
+    
+    Test/run/fork := true, // so that CTRL+C actually terminates the watcher
+  )
+
+lazy val hkmc2AllTests = project.in(file("hkmc2AllTests"))
+  .settings(
+    Test / test := ((hkmc2DiffTests / Test / test) dependsOn (hkmc2JVM / Test / test)).value
+  )
+
 lazy val core = crossProject(JSPlatform, JVMPlatform).in(file("core"))
   .settings(
     sourceDirectory := baseDirectory.value.getParentFile()/"shared",

hkmc2/jvm/src/test/scala/hkmc2/CompileTestRunner.scala

@@ -0,0 +1,51 @@
+package hkmc2
+
+import org.scalatest.{funsuite, ParallelTestExecution}
+import org.scalatest.time._
+import org.scalatest.concurrent.{TimeLimitedTests, Signaler}
+import os.up
+
+import mlscript.utils._, shorthands._
+
+
+class CompileTestRunner
+  extends funsuite.AnyFunSuite
+  with ParallelTestExecution
+  // with TimeLimitedTests // TODO
+:
+  
+  private val inParallel = isInstanceOf[ParallelTestExecution]
+  
+  // val timeLimit = TimeLimit
+  
+  val pwd = os.pwd
+  val workingDir = pwd
+  
+  val dir = workingDir/"hkmc2"/"shared"/"src"/"test"
+  
+  val validExt = Set("mls")
+  
+  val allFiles = os.walk(dir)
+    .filter(_.toIO.isFile)
+    .filter(_.ext in validExt)
+    
+  protected lazy val compileTestFiles = allFiles.filter: file =>
+      file.segments.contains("mlscript-compile")
+  
+  // TODO dedup path stuff with DiffTestRunner?
+  compileTestFiles.foreach: file =>
+    
+    val basePath = file.segments.drop(dir.segmentCount).toList.init
+    val relativeName = basePath.map(_ + "/").mkString + file.baseName
+    
+    test(relativeName):
+      
+      println(s"Compiling: $relativeName")
+      
+      val preludePath = dir/"mlscript"/"decls"/"Prelude.mls"
+      
+      MLsCompiler(preludePath).compileModule(file)
+  
+end CompileTestRunner
+
+

hkmc2/shared/src/main/scala/hkmc2/codegen/Printer.scala

@@ -78,7 +78,7 @@ object Printer:
       val docBody = if publicFields.isEmpty && privateFields.isEmpty then doc"" else doc" { #{ ${docPrivFlds}${docPubFlds} #}  # }"
       val docCtorParams = if clsParams.isEmpty then doc"" else doc"(${ctorParams.mkString(", ")})"
       doc"class ${sym.nme}${docCtorParams}${docBody}"
-
+  
   def mkDocument(arg: Arg)(using Raise, Scope): Document =
     val doc = mkDocument(arg.value)
     if arg.spread

hkmc2/jvm/src/test/scala/hkmc2/ReplHost.scala renamed to hkmc2/shared/src/main/scala/hkmc2/utils/ReplHost.scala

@@ -5,7 +5,7 @@
 :sjs
 let folderName1 = process.env.PWD.split("/").pop()
 in let folderName2 = process.cwd().split("/").pop()
-in folderName2 === folderName1 || folderName2 === "jvm"
+in folderName2 === folderName1 || folderName2 === "shared"
 //│ JS (unsanitized):
 //│ let tmp, tmp1, tmp2, tmp3, tmp4, tmp5, tmp6;
 //│ tmp = this.process.env.PWD.split("/") ?? null;
@@ -16,7 +16,7 @@ in folderName2 === folderName1 || folderName2 === "jvm"
 //│ tmp4 = tmp3.pop() ?? null;
 //│ this.folderName2 = tmp4;
 //│ tmp5 = this.folderName2 === this.folderName1;
-//│ tmp6 = this.folderName2 === "jvm";
+//│ tmp6 = this.folderName2 === "shared";
 //│ tmp5 || tmp6
 //│ = true
 

hkmc2/shared/src/test/scala/hkmc2/ReportFormatter.scala renamed to hkmc2/shared/src/main/scala/hkmc2/utils/ReportFormatter.scala

@@ -120,12 +120,13 @@ abstract class DiffMaker:
 
   val tests = Command("tests"):
     case "" =>
-      new DiffTestRunner(
-        // * I don't understand why when I use `new` here
-        // * the test framework seems to reinstantiate `State` for every test
-        // new DiffTestRunner.State
-        DiffTestRunner.State
-      ){}.execute()
+      // * Note that making `DiffTestRunnerBase` extend `ParallelTestExecution`,
+      // * as we used to do, is quite dangerous, because of the way ScalaTest works (which is pretty dumb):
+      // * it would try to re-instantiate the test classes haphazardly without passing it any arguments,
+      // * which either crashes (as it would here) or recomputes the state every time
+      // * (as would be the case if we created an anonymous subclass here),
+      // * even when the tests, when run with `execute()`, are not run in parallel (also for dumb reasons).
+      DiffTestRunnerBase(new DiffTestRunner.StateWithGit).execute()
 
 
   val fileName = file.last
@@ -198,7 +199,8 @@ abstract class DiffMaker:
           failures += globalStartLineNum
           unexpected("warning", blockLineNum, d.mkExtraInfo)
       case Diagnostic.Kind.Internal =>
-        failures += globalStartLineNum
+        if !tolerateErrors then
+          failures += globalStartLineNum
         // unexpected("internal error", blockLineNum)
         throw d
       report(blockLineNum, d :: Nil, showRelativeLineNums.isSet)