Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: omnibus release cleanup #1251

Merged
merged 1 commit into from
Mar 2, 2025
Merged

feat: omnibus release cleanup #1251

merged 1 commit into from
Mar 2, 2025

Conversation

sgammon
Copy link
Member

@sgammon sgammon commented Mar 2, 2025

Ready for review Powered by Pull Request Badge

Summary

Implements the new TypeScript native parser, and switches to it from the language engine; fixes several binary bloat issues which were causing a compounding size problem in the final binary. Re-generated PGO profiles and switched back to PGO optimization for release. Fixed C compiler flag alignment, LLVM 19 support, and LTO. Fixed link-time section GC.

BoringSSL and vendored native TLS in Orogene have both been dropped in favor of linking to system OpenSSL (a size tradeoff).

Elide's binary, with this PR, reduces from 1.3G (!!) on x86-64 to about 400MB (estimated, with Python). It's still too large, but it is a fantastic improvement.

Several other bugs were fixed with regard to JNA and static modules. Elide's runtime "natives" root has been eliminated, which fixes a longstanding circular build flaw. Instead, the build-time "target root" is now considered a staging area for finalized native libraries, which will be copied (in a future PR) to be peers with the final binary, clearing the way for a feature which ultimately splits several slices of Elide into shared libraries.

Elide's Rust build now uses explicit target triples, and the Gradle build is aware of how to build the third-party natives and Rust libs. Running a Gradle build should be sufficient for these tools to run as well. Now that we use explicit target triples, we can begin to plan for cross-compiled native libs, which will solve downstream issues with libraries like elide-sqlite.

  • TypeScript
    • Switch to native parser
    • Support for context pre-initialization
    • Fix bug with import syntax
  • Native Image
    • Reflection fixes for Pkl/Clikt
    • Support for LLVM 19
    • Compiler flag alignment, LTO fixes
    • Link-time section GC fix
    • Reduction in size for Rust libs
    • Early build support for cross-lang LTO (-fuse-linker-plugin)
  • Chores & Fixes
    • Always build Linux binaries with debug info
    • General dependency upgrades/stability fixes
    • Upgrade Gradle to latest stable release
    • Upgrade Maven, NPM, and Cargo libs
  • Pre-merge Checklist
    • Re-enable Python support
    • Fix embedded kotlinc/javac
    • Testing and fixes on macOS
    • CI fixes and updates for LLVM 19
    • Find and eliminate remaining OpenSSL build

@sgammon sgammon added the bug Something isn't working label Mar 2, 2025
@sgammon sgammon added this to the Release R7: Beta 1 milestone Mar 2, 2025
@sgammon sgammon self-assigned this Mar 2, 2025
Copy link

socket-security bot commented Mar 2, 2025

Updated dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
npm/@types/node@22.13.522.13.8 None +1 2.4 MB types
npm/@typescript-eslint/eslint-plugin@8.24.18.25.0 Transitive: environment, filesystem +36 3.2 MB
npm/@typescript-eslint/parser@8.24.18.25.0 Transitive: environment, filesystem +28 1.45 MB
npm/preact@10.26.210.26.4 None 0 1.43 MB developit, drewigg, jdecroock, ...4 more
npm/typescript@5.7.35.8.2 None 0 22.9 MB typescript-bot

View full report↗︎

Comment on lines +36 to +42
kotlin {
jvm()
}

sourceSets.all {
kotlin.setSrcDirs(listOf("jmh/src"))
resources.setSrcDirs(listOf("jmh/resources"))
// kotlin.srcDirs(listOf("jmh/src"))
// resources.srcDirs(listOf("jmh/resources"))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleanup

@@ -1,3 +1,4 @@
[versions]
micronaut-core = "4.7.1"
micronaut-platform = "4.7.4"
micronaut-picocli = "5.7.0"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are probably other overrides/upgrades we can apply here

Comment on lines 141 to 143
private const val ENABLE_JVM = true
private const val ENABLE_RUBY = true
private const val ENABLE_PYTHON = true
private const val ENABLE_JVM = false
private const val ENABLE_RUBY = false
private const val ENABLE_PYTHON = false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nts: flip before merge

private const val ENABLE_COLOR_SUPPORT = true
private const val ENABLE_COLOR_SUPPORT = false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tmp: multi-width chars and other quirks are breaking formatting here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

facade is missing the Elide global (it is being converted into a native symbol); this is breaking serving from js, which mounts its symbols there

Comment on lines 23 to 24
// Register security providers at JVM startup time.
@JvmStatic @Synchronized private fun registerProviders() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete entirely

@sgammon sgammon force-pushed the chore/release-cleanup branch 4 times, most recently from 5d0e2ae to e3b2592 Compare March 2, 2025 04:34
@sgammon sgammon force-pushed the chore/release-cleanup branch 6 times, most recently from 9ba96f5 to c99bf57 Compare March 2, 2025 08:01
Copy link

codecov bot commented Mar 2, 2025

Codecov Report

Attention: Patch coverage is 36.73469% with 93 lines in your changes missing coverage. Please review.

Project coverage is 48.40%. Comparing base (cf62b54) to head (08da217).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...time/lang/typescript/AbstractTypeScriptLoader.java 3.22% 30 Missing ⚠️
...de/runtime/lang/typescript/TypeScriptLanguage.java 52.08% 19 Missing and 4 partials ⚠️
...e/runtime/lang/javascript/JavaScriptPrecompiler.kt 0.00% 17 Missing ⚠️
...e/lang/typescript/TypeScriptPrecompiledLoader.java 50.00% 11 Missing and 4 partials ⚠️
...de/runtime/core/internals/graalvm/GraalVMEngine.kt 50.00% 3 Missing and 2 partials ⚠️
...untime/lang/javascript/JavaScriptCompilerConfig.kt 0.00% 1 Missing ⚠️
...in/elide/runtime/lang/javascript/JavaScriptLang.kt 0.00% 1 Missing ⚠️
...e/runtime/core/internals/graalvm/GraalVMContext.kt 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1251      +/-   ##
==========================================
- Coverage   50.81%   48.40%   -2.41%     
==========================================
  Files         459      483      +24     
  Lines       18037    18449     +412     
  Branches     3195     3277      +82     
==========================================
- Hits         9165     8930     -235     
- Misses       7841     8486     +645     
- Partials     1031     1033       +2     
Flag Coverage Δ
jvm 48.40% <36.73%> (-2.41%) ⬇️
lib 48.40% <36.73%> (-2.41%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...kotlin/elide/tool/cli/cmd/repl/ToolShellCommand.kt 43.65% <ø> (+20.66%) ⬆️
.../elide/tool/cli/options/EngineJavaScriptOptions.kt 50.00% <ø> (ø)
.../elide/tool/control/AbstractScriptEngineFactory.kt 0.00% <ø> (ø)
...s/cli/src/main/kotlin/elide/tool/err/ErrPrinter.kt 0.00% <ø> (ø)
.../main/kotlin/elide/runtime/core/PolyglotContext.kt 0.00% <ø> (ø)
...c/main/kotlin/elide/runtime/diag/SourceLocation.kt 0.00% <ø> (ø)
.../src/main/kotlin/elide/runtime/gvm/GraalVMGuest.kt 0.00% <ø> (ø)
...src/main/kotlin/elide/runtime/gvm/GuestLanguage.kt 0.00% <ø> (ø)
...rc/main/kotlin/elide/runtime/gvm/InvocationMode.kt 0.00% <ø> (ø)
.../elide/runtime/typescript/TypeScriptPrecompiler.kt 100.00% <100.00%> (ø)
... and 17 more

... and 53 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf62b54...08da217. Read the comment docs.

feat: support for exact metadata in native image
feat: static-umbrella and prefer-shared linkage modes
feat: stronger hardening flags for both clang and gcc
feat: use oxc parser from truffle typescript
feat: integrate cargo and gradle builds
feat: switch to explicit target triples for cargo
feat: early musl support for rustlibs
feat: build conventions for target info + triples
feat: use native typescript parser by default
feat: ability to enforce custom cargo configs by name
fix: javascript format for esm should follow ts
fix: gc of sections by linker
fix: lto and cflags alignment
fix: cross-lang lto with linker plugins
fix: always produce debug symbols on linux
fix: remove vendored native tls in orogene
fix: remove vendored boringssl
fix: native access to oxc parser
fix: bloat from static rust libs
fix: make orogene and uv conditional
fix: target `x86-64-v3`
fix: better support for clang
fix: aligned cflags for hardening
fix: some issues building against musl
fix: drop sqlite math fns for libm dep
fix: build embedded + benchmarks by default
fix: consolidate `elide.target` and related props
fix: entirely eliminate the "elide natives" root
fix: drop bouncycastle (unused)
fix: drop mosaic and compose (tmp, unused)
fix: respect cflags from all native builds
fix: continued support for cfi/branch protection
fix: drop unused / irrelevant resources
fix: colors break native error printing
fix: copy method visibility for node child process records
fix: drop module preloading
chore: bump version for release
chore: add support for es2024 setting
chore: add run configurations for idea
chore: update llvm/clang → `19.x`
chore: update gradle → `8.13`
chore: update typescript → `5.8.2`
chore: update pnpm → `10.5.2`
chore: update protobuf → `4.28.2`
chore: generate initial new-format reachability metadata
chore: update crates
chore: update pnpm deps
chore: update maven deps
chore: trim unused crates and deps
chore: relocate vm feature classes to `engine`
chore: update api pins
chore: update detekt baselines
chore: update pgo profiles

Signed-off-by: Sam Gammon <sam@elide.dev>
@sgammon sgammon force-pushed the chore/release-cleanup branch from c99bf57 to 08da217 Compare March 2, 2025 09:10
Copy link

sonarqubecloud bot commented Mar 2, 2025

@sgammon sgammon merged commit ea4779c into main Mar 2, 2025
25 checks passed
@sgammon sgammon deleted the chore/release-cleanup branch March 2, 2025 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant