MongoDB Command insert failed: E11000 duplicate key error collection
Today in this article we shall see how to resolve the error E11000 duplicate key error collection in the MongoDB command execution.
Today in this article, we will cover below aspects,
- Issue Description
- Resolution
Issue Description
Mongo Insert/Update operation gives the below error,
MongoDB.Driver.MongoComamndException: Command insert failed: E11000 duplicate key error collection:[yourccollection] index :[key] dup key: { : }
Resolution
I had this error recently for Upsert operation,
var returnDoc = await collectionnew.FindOneAndUpdateAsync<Library>(
filter: Builders<Library>.Filter.Eq("_id", userId),
update: update.Combine(updates),
options: new FindOneAndUpdateOptions<Library, Library>
{
IsUpsert = true,
ReturnDocument = ReturnDocument.After,
});
if(returnDoc!=null)
{
//your logic here
}
- E11000 duplicate key error could occur due to many factors depending on the type of operation you are performing.
- This error however indicates that the issue has occurred because the key with which you performed the Mongo operation key already exists with the value specified.
- This also meant the indexed key in the error is a unique key and not the non-unique key or sparsed key. That means only one key with a unique value can exist in the given mongo collection.
Generally, you will have default _id as a unique key in the mongo collection.
Example:
But you can add more unique keys depending on your requirements provided you are 100% sure, its value will remain unique for the whole collection and won’t conflict with other ids.
To fix the issue, please try any of the below guidelines,
- Do you need the indexed field to be unique really? If not, then create an index with the non-unique index or sparsed index, etc.
Example:
Above, we have used a non-unique index since the data field can have duplicates and ideally can not remain unique logically.
- Delete unnecessary index – Don’t use indexed if not necessary – MongoDB Indexing Guidelines and Best Practices
- Apply unique index to only those fields which will have values. For example, if the value is not specified then it’s possible mongo assigns the null value, then a null value will be assigned to the very first record but the second record will throw the duplicate error issue.
Please make sure the indexed fields are properly indexed. If you are 100% sure of the index field will remain unique across the collection then only you can make it a unique indexed field else make it indexed but non-unique so that duplicated values can be allowed. Using this option also meant that you already have _id or any other unique id to be used for the query considering the performance.
References:
- MongoDB Indexing Guidelines and Best Practices
- MongoDB Collection Naming Convention
That’s all! Happy coding!
Does this help you fix your issue?
Do you have any better solutions or suggestions? Please sound off your comments below.
Please bookmark this page and share it with your friends. Please Subscribe to the blog to get a notification on freshly published best practices and guidelines for software design and development.
Are you sure you’re dropping the right database? Make sure you do use dbName; db.dropDatabase();.
Yes, I switched to dbName before dropping the database
The insert code is straightforward:
db.collection('users', function (err, collection) {
collection.insert(contacts, function(err, result) {
if (err) {
job.log('Failed to save contacts as user with error: ' + JSON.stringify(err));
done('Failed to save contacts as user with error: ' + JSON.stringify(err));
} else {
console.log(result.length);
done();
}
});
});
contacts is an array of json in accordance with the users' schema whose schema is defined such that no unique index exists.
I have about 760 records out of which about 93 are inserted without any issues, the processing stops at 94th entry. This is what the 94th entry looks like:
{
"phone": [
{
"number": "91xxxxxxxxxxx",
"numberType": "work",
"contactReferrer": "546a20a7a8b0aaf0175f3ae1"
},
{
"number": "91yyyyyyyyyyy",
"numberType": "home",
"contactReferrer": "546a20a7a8b0aaf0175f3ae1"
},
{
"number": "91zzzzzzzzzzz",
"numberType": "work",
"contactReferrer": "546a20a7a8b0aaf0175f3ae1"
}
],
"organization": [
{
"organization": "the org",
"title": null,
"employmentType": "employment"
}
],
"email": [
{
"emailId": "xxx@gmail.com",
"emailType": "other",
"contactReferrer": "546a20a7a8b0aaf0175f3ae1"
}
],
"address": [
{
"state": "Karnataka",
"postalCode": null,
"addressType": "other",
"poBox": null,
"street": null,
"city": "Bidar",
"country": "India"
},
{
"state": "Karnataka",
"postalCode": null,
"addressType": "other",
"poBox": null,
"street": null,
"city": "Bidar",
"country": "India"
}
],
"name": [
{
"firstName": "Rakesh",
"lastName": "Raman",
"contactReferrer": "546a20a7a8b0aaf0175f3ae1"
}
],
"_id": "546a2786044e2cd928698eb0"
}
The error:
{"code":11000,"index":109,"errmsg":"insertDocument :: caused by :: 11000 E11000 duplicate key error index: dbName.users.$_id_ dup key: { : ObjectId('546a2786044e2cd928698eb0') }
I have setup a replica set and want to add it in a mongos. But I got below error in mongos:
mongos> sh.addShard("rs3/172.19.0.12:27017,172.19.0.6:27017,172.19.0.5:27017")
{
"code" : 11000,
"ok" : 0,
"errmsg" : "E11000 duplicate key error collection: admin.system.version index: _id_ dup key: { : "shardIdentity" }"
}
from the above message, it says admin.system.version has duplicate key error. The collection admin.system.version should be set by mongodb. I don’t understand why it has a duplicate key.
All mongo instances are 3.4.4 version.
Below is the replica set status:
rs3:PRIMARY> rs.status()
{
"set" : "rs3",
"date" : ISODate("2017-07-09T02:13:37.146Z"),
"myState" : 1,
"term" : NumberLong(10),
"heartbeatIntervalMillis" : NumberLong(2000),
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"appliedOpTime" : {
"ts" : Timestamp(1499566407, 1),
"t" : NumberLong(10)
},
"durableOpTime" : {
"ts" : Timestamp(1499563905, 1),
"t" : NumberLong(9)
}
},
"members" : [
{
"_id" : 0,
"name" : "172.19.0.12:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2493,
"optime" : {
"ts" : Timestamp(1499566407, 1),
"t" : NumberLong(10)
},
"optimeDate" : ISODate("2017-07-09T02:13:27Z"),
"electionTime" : Timestamp(1499563936, 1),
"electionDate" : ISODate("2017-07-09T01:32:16Z"),
"configVersion" : 414750,
"self" : true
},
{
"_id" : 1,
"name" : "172.19.0.5:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 1403,
"optime" : {
"ts" : Timestamp(1499566407, 1),
"t" : NumberLong(10)
},
"optimeDurable" : {
"ts" : Timestamp(1499563905, 1),
"t" : NumberLong(9)
},
"optimeDate" : ISODate("2017-07-09T02:13:27Z"),
"optimeDurableDate" : ISODate("2017-07-09T01:31:45Z"),
"lastHeartbeat" : ISODate("2017-07-09T02:13:35.870Z"),
"lastHeartbeatRecv" : ISODate("2017-07-09T02:13:35.854Z"),
"pingMs" : NumberLong(0),
"syncingTo" : "172.19.0.12:27017",
"configVersion" : 414750
},
{
"_id" : 2,
"name" : "172.19.0.6:27017",
"health" : 1,
"state" : 3,
"stateStr" : "RECOVERING",
"uptime" : 2487,
"optime" : {
"ts" : Timestamp(1499070510, 1000),
"t" : NumberLong(3)
},
"optimeDurable" : {
"ts" : Timestamp(1499070510, 1000),
"t" : NumberLong(3)
},
"optimeDate" : ISODate("2017-07-03T08:28:30Z"),
"optimeDurableDate" : ISODate("2017-07-03T08:28:30Z"),
"lastHeartbeat" : ISODate("2017-07-09T02:13:35.865Z"),
"lastHeartbeatRecv" : ISODate("2017-07-09T02:13:36.965Z"),
"pingMs" : NumberLong(0),
"configVersion" : 414750
}
],
"ok" : 1
}