Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local Storage Naming and Directory Scaling #188

Open
ifadams opened this issue Jul 24, 2024 · 0 comments
Open

Local Storage Naming and Directory Scaling #188

ifadams opened this issue Jul 24, 2024 · 0 comments
Labels
Discussion Needed Require additional context and discussion Enhancement New features or upgrades for functionality, performance, etc Priority: Backlog Tasks that are not pressing, but to be tracked and revisited

Comments

@ifadams
Copy link
Contributor

ifadams commented Jul 24, 2024

Is your feature request related to a problem? Please describe.
Currently there is a static directory hierarchy that's constructed for local storage of videos, images, etc, on server startup.

While a fan-out is necessary to prevent overloading a single storage location (i.e. directory), we should (re)investigate this and consider refactoring as its somewhat inefficient and rigid for larger scale use-cases.

We also have a somewhat naive method for generating local path and file names that may have collision risks at larger scale that are not well articulated.

Describe the solution you'd like

Ideally we should have a methodology that is akin to consistent hashing in distributed storage. This lets us dynamically scale and adjust our hierarchy.

As for internal naming, we could look at UUIDs to be automatically generated.

@ifadams ifadams added Enhancement New features or upgrades for functionality, performance, etc Discussion Needed Require additional context and discussion Priority: Backlog Tasks that are not pressing, but to be tracked and revisited labels Jul 24, 2024
@ifadams ifadams changed the title Local Storage Naming and Directory Hierarchy Problematic Local Storage Naming and Directory Scaling Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion Needed Require additional context and discussion Enhancement New features or upgrades for functionality, performance, etc Priority: Backlog Tasks that are not pressing, but to be tracked and revisited
Projects
None yet
Development

No branches or pull requests

1 participant