[LintDiff] AutoRest - JavaScript heap out of memory

[danieljurek]

Something is causing a crash in Node when running autorest directly. Similar crashes occur locally (even when running on a machine with 64GB memory) when using with the latest OAV (v2.2.4) or the OVA used in Legacy LintDiff (v2.2.3)

Logs show:

Autorest command: npm exec --no -- autorest --v3 --spectral --azure-validator --semantic-validator=false --model-validator=false --message-format=json --openapi-type=arm --openapi-subtype=arm --use=/home/runner/work/azure-rest-api-specs/azure-rest-api-specs/node_modules/@microsoft.azure/openapi-validator --tag=package-preview-2025-04  after/specification/machinelearningservices/resource-manager/readme.md
  (node:2438) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
  (Use `node --trace-deprecation ...` to show where the warning was created)
  
  <--- Last few GCs --->
  
  [2474:0x4429e000]    74947 ms: Mark-Compact 4041.6 (4130.3) -> 4023.9 (4131.5) MB, pooled: 3 MB, 1169.70 / 0.00 ms  (average mu = 0.193, current mu = 0.200) allocation failure; scavenge might not succeed
  [2474:0x4429e000]    76370 ms: Mark-Compact 4039.6 (4131.5) -> 4025.1 (4129.3) MB, pooled: 7 MB, 1389.50 / 0.00 ms  (average mu = 0.113, current mu = 0.024) allocation failure; scavenge might not succeed
  
  
  <--- JS stacktrace --->
  
  FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
  ----- Native stack trace -----
  
   1: 0xe09a56 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   2: 0x11ba250 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   3: 0x11ba527 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   4: 0x13e7f55  [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   5: 0x13e7f83  [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   6: 0x140105a  [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   7: 0x1404228  [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
   8: 0x1c69ef1  [/opt/hostedtoolcache/node/22.15.0/x64/bin/node]
  {"level":"information","message":"AutoRest core version selected from configuration: ^3.2.0."}
  {"level":"information","message":"> Loading local AutoRest extension '@microsoft.azure/openapi-validator' (/home/runner/work/azure-rest-api-specs/azure-rest-api-specs/node_modules/@microsoft.azure/openapi-validator)"}
  {"pluginName":"spectral","extensionName":"@microsoft.azure/openapi-validator","level":"information","message":"openapiValidatorPluginFunc: Enter"}
  {"pluginName":"spectral","extensionName":"@microsoft.azure/openapi-validator","level":"information","message":"spectralPluginFunc: Enter"}
  {"pluginName":"spectral","extensionName":"@microsoft.azure/openapi-validator","level":"information","message":"openapiValidatorPluginFunc: Validating files:"}
  {"pluginName":"spectral","extensionName":"@microsoft.azure/openapi-validator","level":"information","message":"openapiValidatorPluginFunc: File 1/5: 'file:///home/runner/work/azure-rest-api-specs/azure-rest-api-specs/after/specification/machinelearningservices/resource-manager/Microsoft.MachineLearningServices/preview/2025-04-01-preview/machineLearningServices.json'"}

Example LintDiff run with the issue: https://github.com/Azure/azure-rest-api-specs/actions/runs/14931189663/job/41947606290

Example Legacy LintDiff run without this problem: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=4848946&view=logs&j=0574a2a6-2d0a-5ec6-40e4-4c6e2f70bea2&t=80c3e782-49f0-5d1c-70dd-cbee57bdd0c7

Legacy LintDiff uses a different command when running autorest, and does not exhibit crashing behavior:

node /mnt/vss/_work/_tasks/AzureApiValidation_5654d05d-82c1-48da-ad8f-161b817f6d41/0.0.112/private/azure-swagger-validation/azureSwaggerValidation/node_modules/autorest/dist/app.js --v3 --spectral --azure-validator --semantic-validator=false --model-validator=false --message-format=json --openapi-type=arm --openapi-subtype=arm  --use=@microsoft.azure/openapi-validator@2.2.3 /mnt/vss/_work/1/lint-c93b354fd9c14905bb574a8834c4d69b/specification/machinelearningservices/resource-manager/readme.md

When comparing commands used, autorest parameters are the same except --use which uses a local path in new LintDiff and a package name and version in legacy LintDiff and the file paths which are specific to the machine context.

[danieljurek]

Legacy LintDiff uses an environment variable:

...
NODE_OPTIONS: '--max-old-space-size=8192'
...

That'll be straightforward.