E11000 duplicate key error collection mongo

Following is my user schema in user.js model —

var userSchema = new mongoose.Schema({
    local: {
        name: { type: String },
        email : { type: String, require: true, unique: true },
        password: { type: String, require:true },
    },
    facebook: {
        id           : { type: String },
        token        : { type: String },
        email        : { type: String },
        name         : { type: String }
    }
});

var User = mongoose.model('User',userSchema);

module.exports = User;

This is how I am using it in my controller —

var user = require('./../models/user.js');

This is how I am saving it in the db —

user({'local.email' : req.body.email, 'local.password' : req.body.password}).save(function(err, result){
    if(err)
        res.send(err);
    else {
        console.log(result);
        req.session.user = result;
        res.send({"code":200,"message":"Record inserted successfully"});
    }
});

Error —

{"name":"MongoError","code":11000,"err":"insertDocument :: caused by :: 11000 E11000 duplicate key error index: mydb.users.$email_1  dup key: { : null }"} 

I checked the db collection and no such duplicate entry exists, let me know what I am doing wrong ?

FYI — req.body.email and req.body.password are fetching values.

I also checked this post but no help STACK LINK

If I removed completely then it inserts the document, otherwise it throws error «Duplicate» error even I have an entry in the local.email

Apr 29, 2019

MongoDB’s E11000 error is a common source of confusion. This error occurs when two documents have the
same value for a field that’s defined as unique in your Mongoose schema.

Mongoose models have an _id field that’s always unique. If you try to insert two documents with the same _id, you get the below error message.

MongoError: E11000 duplicate key error collection: test.customers index: _id_
dup key: { : ObjectId('5cc5ea092dca872442916cf5') }

The test.customers part represents the MongoDB collection that the error occurred in. The _id_ string is the name of the unique index, and the ObjectId() is the duplicate value.

The below code is one way you might get the above error message. MongoDB collections always have a unique index on _id, so trying to insert a document
with a duplicate id will cause a duplicate key error.

const CharacterModel = mongoose.model('Character',
  new Schema({ name: String }));

const doc = await CharacterModel.create({ name: 'Jon Snow' });

doc._id; // Something like "5cc5e9be172acd237a893610"

try {
  // Try to create a document with the same `_id`. This will always fail
  // because MongoDB collections always have a unique index on `_id`.
  await CharacterModel.create(Object.assign({}, doc.toObject()));
} catch (error) {
  // MongoError: E11000 duplicate key error collection: test.characters
  // index: _id_ dup key: { : ObjectId('5cc5ea092dca872442916cf5') }
  error.message;
}

This error is often caused by null or undefined field values. null and undefined count as distinct values, so if you declare a field email as unique, two documents cannot have email = undefined. The below example creates two documents without an email property, which causes a duplicate key error.

const UserModel = mongoose.model('User', new Schema({
  name: String,
  email: {
    type: String,
    unique: true
  }
}));

// Wait for the index to build. The index name will be `email_1`
await UserModel.init();

// Create a document with no `email` set
await UserModel.create({ name: 'user 1' });

try {
  await UserModel.create({ name: 'user 2' });
} catch (error) {
  // E11000 duplicate key error collection: test.users index: email_1
  // dup key: { : null }
  error.message;
}

To make MongoDB E11000 error messages user-friendly, you should use the mongoose-beautiful-unique-validation plugin.

const schema = new Schema({ name: String });
schema.plugin(require('mongoose-beautiful-unique-validation'));

const CharacterModel = mongoose.model('Character', schema);

const doc = await CharacterModel.create({ name: 'Jon Snow' });

try {
  // Try to create a document with the same `_id`. This will always fail
  // because MongoDB collections always have a unique index on `_id`.
  await CharacterModel.create(Object.assign({}, doc.toObject()));
} catch (error) {
  // Path `_id` (5cc60c5603a95a15cfb9204d) is not unique.
  error.errors['_id'].message;
}

More Mongoose Tutorials

• Implementing Soft Delete in Mongoose
• Using limit() with Mongoose Queries
• How to Fix «Buffering timed out after 10000ms» Error in Mongoose
• Using insertOne() in Mongoose
• Mongoose on(‘delete’)
• Enums in Mongoose
• Mongoose find() Certain Fields

Are you sure you’re dropping the right database? Make sure you do use dbName; db.dropDatabase();.

Yes, I switched to dbName before dropping the database

The insert code is straightforward:

db.collection('users', function (err, collection) {
  collection.insert(contacts, function(err, result) {
    if (err) {
      job.log('Failed to save contacts as user with error: ' + JSON.stringify(err));
      done('Failed to save contacts as user with error: ' + JSON.stringify(err));
    } else {
      console.log(result.length);
      done();
    }
  });
});

contacts is an array of json in accordance with the users' schema whose schema is defined such that no unique index exists.

I have about 760 records out of which about 93 are inserted without any issues, the processing stops at 94th entry. This is what the 94th entry looks like:

{
    "phone": [
        {
            "number": "91xxxxxxxxxxx",
            "numberType": "work",
            "contactReferrer": "546a20a7a8b0aaf0175f3ae1"
        },
        {
            "number": "91yyyyyyyyyyy",
            "numberType": "home",
            "contactReferrer": "546a20a7a8b0aaf0175f3ae1"
        },
        {
            "number": "91zzzzzzzzzzz",
            "numberType": "work",
            "contactReferrer": "546a20a7a8b0aaf0175f3ae1"
        }
    ],
    "organization": [
        {
            "organization": "the org",
            "title": null,
            "employmentType": "employment"
        }
    ],
    "email": [
        {
            "emailId": "xxx@gmail.com",
            "emailType": "other",
            "contactReferrer": "546a20a7a8b0aaf0175f3ae1"
        }
    ],
    "address": [
        {
            "state": "Karnataka",
            "postalCode": null,
            "addressType": "other",
            "poBox": null,
            "street": null,
            "city": "Bidar",
            "country": "India"
        },
        {
            "state": "Karnataka",
            "postalCode": null,
            "addressType": "other",
            "poBox": null,
            "street": null,
            "city": "Bidar",
            "country": "India"
        }
    ],
    "name": [
        {
            "firstName": "Rakesh",
            "lastName": "Raman",
            "contactReferrer": "546a20a7a8b0aaf0175f3ae1"
        }
    ],
    "_id": "546a2786044e2cd928698eb0"
}

The error:

{"code":11000,"index":109,"errmsg":"insertDocument :: caused by :: 11000 E11000 duplicate key error index: dbName.users.$_id_ dup key: { : ObjectId('546a2786044e2cd928698eb0') }

Доброй ночи, не могу отловить баг. Создаю юзера, но получаю ошибку :

{
    "name": "MongoError",
    "message": "E11000 duplicate key error collection: Udemy.users index: UserSchema.email_1 dup key: { : null }",
    "driver": true,
    "index": 0,
    "code": 11000,
    "errmsg": "E11000 duplicate key error collection: Udemy.users index: UserSchema.email_1 dup key: { : null }"
}

Код : User.js

const mongoose = require('mongoose');
const validator = require('validator');
const jwt = require('jsonwebtoken');
const _ = require('lodash');
 
 
let UserSchema = new mongoose.Schema({
 email: {
  type: String,
  required: true,
  trim: true,
  minlength: 1,
  unique: true,
  validate: {
   validator: validator.isEmail,
   message: '{VALUE} is not a valid email'
  }
 },
 password: {
  type: String,
  require: true,
  minlength: 6
 },
 tokens: [{
  access: {
   type: String,
   required: true
  },
  token: {
   type: String,
   required: true
  }
 }]
});
 
 
UserSchema.methods.toJSON = function () {
 let user = this;
 let userObject = user.toObject();
 
 return _.pick(userObject, ['_id', 'email']);
};
 
 
UserSchema.methods.generateAuthToken = function () {
 let user = this;
 let access = 'auth';
 let token = jwt.sign({_id: user._id.toHexString(), access}, 'abc123').toString();
 user.tokens = user.tokens.concat([{access, token}]);
 return user.save().then(() => {
  return token;
 });
};
 
let User = mongoose.model('User', UserSchema);
 
module.exports = {
 User
};

server.js :

app.post('/users', (req, res) => {
 let body = _.pick(req.body, ['email', 'password']);
 let user = new User(body);
 
 user.save().then(() => {
  return user.generateAuthToken();
 }).then((token) => {
  res.header('x-auth', token).send(user);
 }).catch((e) => {
  res.status(400).send(e);
 })
});

Полный код тут

Спасибо!

Summary

In this post, I discuss the symptoms and workaround for an atomicity bug in MongoDB.  This error manifests itself when performing collection updates from multiple threads, with upsert:true, on a unique key.  The bug has been open since 2014 and still unresolved.  https://jira.mongodb.org/browse/SERVER-14322

Bug Set Up

/*jshint esversion: 6 */

'use strict';
'use esversion 6';

const MongoClient = require('mongodb').MongoClient;
const CONNECTION_URL = 'mongodb://admin:mongo@localhost:27017';
const DB_NAME = 'testDB';
const COLLECTION = 'testCollection';

let connection;
let promises = [];

function update(i, db,  query, value, upsrt) {
 return db.collection(COLLECTION).updateOne(query, value, {'upsert' : upsrt})
 .then((result) => {
  return new Promise((resolve, reject) => { resolve('i: ' + i + ' time: ' + new Date().getTime());});
 })
 .catch(err => {
  throw err;
 });
}

MongoClient.connect(CONNECTION_URL, {useNewUrlParser : true})
.then((result) => {
 connection = result;
 const db = connection.db(DB_NAME);
 
 for (let i=0; i<5; i++) {
  let query = {'_id': 'johnDoe'};
  let value = { set : {'funds' : i*100}, $addToSet : {'indices' : i}};
  let p = updateWithRetry(i, db, query, value, true);
  promises.push(p);
 }
 return Promise.all(promises);
})
.then((results) => {
 results.forEach((result) => {
  console.log('result - ' + result);
 });
})
.catch(err => {
 console.log(err); 
})
.finally(() => {
 console.log('closing db connection');
 connection.close();
});

Explanation

The Node.js code above creates an array of promises, each of which attempts to perform a db update with upsert equal to ‘true’.  The promises sent thru Promise.all to execute them all as a batch.

Results

The duplicate key error is generated, as expected.

{ MongoError: E11000 duplicate key error collection: testDB.testCollection index: _id_ dup key: { : "johnDoe" }
    at Function.create (/nas/archive/dev/workspace/blackjack/node_modules/mongodb-core/lib/error.js:43:12)
    at toError (/nas/archive/dev/workspace/blackjack/node_modules/mongodb/lib/utils.js:149:22)
    at coll.s.topology.update (/nas/archive/dev/workspace/blackjack/node_modules/mongodb/lib/operations/collection_ops.js:1399:39)
    at /nas/archive/dev/workspace/blackjack/node_modules/mongodb-core/lib/connection/pool.js:532:18
    at process.internalTickCallback (internal/process/next_tick.js:70:11)
  driver: true,
  name: 'MongoError',
  index: 0,
  code: 11000,
  errmsg:
   'E11000 duplicate key error collection: testDB.testCollection index: _id_ dup key: { : "johnDoe" }',
  [Symbol(mongoErrorContextSymbol)]: {} }
closing db connection

Below is the output of a read of this collection.  One update (#4 index) completed before the error was thrown and halted the execution.

> db.testCollection.find({});
{ "_id" : "johnDoe", "funds" : 400, "indices" : [ 4 ] }

Workaround

function updateWithRetry(i, db,  query, value, upsrt) {
 return db.collection(COLLECTION).updateOne(query, value, {'upsert' : upsrt})
 .then((result) => {
  return new Promise((resolve, reject) => { resolve('i: ' + i + ' time: ' + new Date().getTime());});
 })
 .catch(err => {
  if (err.code === 11000){
   console.log('i: ' + i + ' 11000 error');
   return updateWithRetry(i, db, query, value, false);
  }
  else {
   throw err;
  }
 });
}

Explanation

In this revised update block, I added a check for the 11000 error code.  If that’s, in fact, the source of the error, I retry the update with a recursive call.

Results

i: 0 11000 error
i: 2 11000 error
i: 3 11000 error
i: 4 11000 error
result - i: 0 time: 1546649403149
result - i: 1 time: 1546649403146
result - i: 2 time: 1546649403149
result - i: 3 time: 1546649403149
result - i: 4 time: 1546649403149
closing db connection

> db.testCollection.find({});
{ "_id" : "johnDoe", "funds" : 400, "indices" : [ 1, 0, 2, 3, 4 ] }

Source