Skip to content

[Feature Request]: <title>speed up merge node with parallel #1384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 tasks done
TonicZhang opened this issue Apr 16, 2025 · 1 comment
Closed
2 tasks done

[Feature Request]: <title>speed up merge node with parallel #1384

TonicZhang opened this issue Apr 16, 2025 · 1 comment
Labels
enhancement New feature or request

Comments

@TonicZhang
Copy link

TonicZhang commented Apr 16, 2025

Do you need to file a feature request?

  • I have searched the existing feature request and this feature request is not already filed.
  • I believe this is a legitimate feature request, not just a question or bug.

Feature Request Description

Preliminary testing shows that the speed bottleneck is at the merge node stage. Is it possible to optimize this by changing to a concurrent merge?
Can I remove the “await” and use “create_task” and “gather” in the following code? Was the use of await here because some graph databases, like NetworkX, do not support concurrent node insertion, so everything was changed to a serial mode?

Image

@TonicZhang TonicZhang added the enhancement New feature or request label Apr 16, 2025
@danielaskdd
Copy link
Collaborator

Too many concurrent requests can generate excessive database connections, consuming a large amount of resources and leading to a performance decline. The correct approach is to use batch mode for data queries. The main branch has already been optimized following this principle. Currently, the performance of PostgreSQL and Neo4j has been significantly improved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants