Fix large table issues

[ChaseLewis]

Description of Changes

Changes were made to handle large data table changes. Changes and why they were made are outlined in this discord chat.

https://discord.com/channels/1037340874172014652/1363015282381033613/1363972150922444935

Main changes are modifying the applyTableUpdates and applyOperations to handle larger row counts. Currently a million rows will completely fail in the sdk. This is due to a bug causing a stack overflow & the OperationMap having very poor performance characteristics.

I fixed this by not allocating all the pendingCallbacks on the stack and changing the applyOperations to use a normal Map rather than the OperationMap which uses an internal list for its data structure.

To make complex keys compatible with a normal map I introduced the toPrimaryKey on complex key types like connectionId and Identity. This converts them to a hashable format and allows them to pas all existing tests.

API

• This is an API breaking change to the SDK
  This changes no interface so should not be a breaking change. It has been tested with all key types as far as I can tell so do not believe it should break any underlying functionality.

Requires SpacetimeDB PRs

This is the only PR required.

Testing

I compiled this code and tested it in my app for testing and it worked great.
https://github.com/ChaseLewis/place-rs

I also ran all existing automated testing which it passed locally on my machine.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 13f0f54

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[bfops]

Thank you for opening this! We'll try to take a look soon.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@clockworklabs/spacetimedb-sdk@179

commit: 621fb2a