[Bug]: Using Cosmos DB: E11000 duplicate key error when new user attempts login or registration

[robsaccone]

What happened?

Using Azure Cosmos DB as backend, which (poorly) emulates Mongo DB, and as a result have encountered a number of issues. This one is due to unique indexes being created in the users collection for each of the "social login" options (facebookId, etc.).

Apparently Cosmos DB doesn't support sparse unique indexes, i.e. treats multiple NULL values as duplicates. I am using OpenID only, so manually deleting the other indexes in the collection solved the issue.

Not looking for a fix here per se, as I'm beginning to realize that Cosmos DB should be consider a semi-supported choice given it's API emulation. Adding detail for others who might try the same, and perhaps for those who might have ideas to better support both Cosmos and Mongo.

Steps to Reproduce

1. Deploy using Azure Cosmos DB as database vs. Mongo native.
2. Have more than one user attempt login or registration
3. Get duplicate key error.

What browsers are you seeing the problem on?

No response

Relevant log output

E11000 duplicate key error collection: test.users. Failed _id or unique index constraint.

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Yeah this is a known issue for me. I might be able to remove the sparse field as I’m not sure it’s needed. Still need to investigate the effects of removing it.

I’ve heard DocumentDB is a good enterprise level, cloud alternative (AWS)

[robsaccone]

Thanks Danny. Thinking of dropping Cosmos and either containerizing Mongo as prescribed or using Atlas. Most of my clients are big Azure shops, hence my first attempt at an Azure-first approach. Painful so far but it's working. About to deploy the assistants branch to Azure...

[illgitthat]

I can extend the Azure documentation of LibreChat

Please do! Also for what it's worth, this PR could potentially solve the root problem here: #1814

Merging it tomorrow.

Is there any update on this PR / fix to this issue?

[danny-avila]

PR does not actually fix the issue, except presents another compatibility issue with Azure CosmosDB, that's why it hasn't been merged

[philipbrembeck]

Any updates on this?

[gijs-voogd]

anyone who has a working solution for the azure cosmos db? running it in the container works but I prefer using the paas db

[rubentalstra]

@gijs-voogd this one works: https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/

don't use the RU

[gijs-voogd]

thanks @rubentalstra. switched to vcore but this gives me:

while there is nothing in the db:

are you using the vcore model as well?

[rubentalstra]

very strange because it works for me

[rubenaster]

@rubentalstra I'm not sure that really RU is the problem. What MongoDB version did you configure for your CosmosDB?

I have the same issue:

2025-03-24T21:08:58.984295886Z 2025-03-24 21:08:58 �[32minfo�[39m: �[32m[openidStrategy] user not found with openidId: dDUXXXXXXPGs�[39m
2025-03-24T21:08:59.011230544Z 2025-03-24 21:08:59 �[32minfo�[39m: �[32m[openidStrategy] user not found with email: user.name@my.company for openidId: dDUXXXXXXPGs�[39m
2025-03-24T21:08:59.039710762Z 2025-03-24 21:08:59 �[31merror�[39m: �[31m[openidStrategy] login failed E11000 duplicate key error collection: test.users. Failed _id or unique index constraint.�[39m
2025-03-24T21:08:59.048601278Z MongoServerError: E11000 duplicate key error collection: test.users. Failed _id or unique index constraint.
2025-03-24T21:08:59.048629258Z     at InsertOneOperation.execute (/app/node_modules/mongodb/lib/operations/insert.js:51:19)
2025-03-24T21:08:59.048632996Z     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2025-03-24T21:08:59.048635949Z     at async tryOperation (/app/node_modules/mongodb/lib/operations/execute_operation.js:203:20)
2025-03-24T21:08:59.048638727Z     at async executeOperation (/app/node_modules/mongodb/lib/operations/execute_operation.js:73:16)
2025-03-24T21:08:59.048657468Z     at async Collection.insertOne (/app/node_modules/mongodb/lib/collection.js:157:16)

[rubentalstra]

@rubentalstra I'm not sure that really RU is the problem. What MongoDB version did you configure for your CosmosDB?

I have the same issue:

2025-03-24T21:08:58.984295886Z 2025-03-24 21:08:58 �[32minfo�[39m: �[32m[openidStrategy] user not found with openidId: dDUXXXXXXPGs�[39m

2025-03-24T21:08:59.011230544Z 2025-03-24 21:08:59 �[32minfo�[39m: �[32m[openidStrategy] user not found with email: user.name@my.company for openidId: dDUXXXXXXPGs�[39m

2025-03-24T21:08:59.039710762Z 2025-03-24 21:08:59 �[31merror�[39m: �[31m[openidStrategy] login failed E11000 duplicate key error collection: test.users. Failed _id or unique index constraint.�[39m

2025-03-24T21:08:59.048601278Z MongoServerError: E11000 duplicate key error collection: test.users. Failed _id or unique index constraint.

2025-03-24T21:08:59.048629258Z     at InsertOneOperation.execute (/app/node_modules/mongodb/lib/operations/insert.js:51:19)

2025-03-24T21:08:59.048632996Z     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

2025-03-24T21:08:59.048635949Z     at async tryOperation (/app/node_modules/mongodb/lib/operations/execute_operation.js:203:20)

2025-03-24T21:08:59.048638727Z     at async executeOperation (/app/node_modules/mongodb/lib/operations/execute_operation.js:73:16)

2025-03-24T21:08:59.048657468Z     at async Collection.insertOne (/app/node_modules/mongodb/lib/collection.js:157:16)

@rubenaster i have it working on my azure account, with mongoDB version 5.0 or 6.0

[rubenaster]

@rubentalstra okay, that one's on you. It really works with vCore. Thanks mate!

[rubentalstra]

@gijs-voogd https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/quickstart-portal

[gijs-voogd]

yes the vcore model works for me too. the error i had yesterday with the vcore model had to do with the database. you also get that error when the connectionstring or somethings else is wrong, not sure what did the trick. deployed a new one today with mongoDB 6.0 instead of 7.0

[oxch]

Dropping indices on users collection resolves the issue - drop those BEFORE any data is placed into users collection - remove data if already present, otherwise you won't be able to drop indices (in my experience anyways). Only leave indices for _id, email fields only and RU-type Cosmos for Mongo DB deployment should work too without this error.