Develop Reference

exception: geoNear command failed: errmsg: more than one 2d index

I'm trying to retrieve documents from a collection by distance. I've tried to use the $geoNear aggregation, but I either run into errors (using node.js and postman) or 0 records get returned.
Sample Document:
{
"_id" : ObjectId("5cc37692fe9fd54b3cd136c9"),
"category" : "art",
"description" : "an eastbound description for The soda event.",
"geometry" : {
"type" : "Point",
"coordinates" : [
3.60178480057443,
6.46123057784917
]
}
"__v" : 0
}
.
.
Model:
const GeoSchema = new Schema({
type: {
type: String,
default: "Point"
},
coordinates: {
type: [Number],
index: "2dsphere"
}
}, { _id: false });
const EventSchema = new Schema({
category: String,
description: String,
geometry: GeoSchema,
});
Query
db.events.aggregate([
{
$geoNear: {
near: { type: "Point", coordinates: [3.60178480057443, 6.46123057784917] },
distanceField: "dist",
maxDistance: 90000,
spherical: true
}
}
]);
Running the query above returns zero results, an empty array (on postman) and sometimes displays this error:
{
"name": "MongoError",
"errmsg": "exception: geoNear command failed: { ok: 0.0, errmsg: \"more than one 2d index, not sure which to run geoNear on\" }",
"code": 16604,
"ok": 0
}
When this error is displayed, I run the db.events.dropIndex function. I copied and pasted the examples used in the docs, and it worked fine. Please How do I make this $geoNear function work. I have been struggling with it for a whole day.

A Single collection cannot have more then one 2dsphere index and You have created 2dsphere index on more than one field by running following command.
db.getCollection('test').createIndex({ "loc": "2dsphere" })
So remove the one of the index and it will get work. You can check the list of indices using this
db.getCollection('test').getIndexes()
Update:
You can use key parameter to specify on which field do you want to make search with if your collection have multiple $geoNear indices

Create ID with data of other fields

I'm new to mongodb, and I'm using mongoose to validate and order the data (I'm open to change it to MySQL if this doesn't work).
The app will be an e-shop, to buy merchandising related to movies, games, ext.
My schema is as follows:
var productSchema = {
id: {
type: String,
required: true
},
name: {
type: String,
required: true
},
img: {
type: String,
required: true
},
price: {
type: Number,
required: true
},
stock: {
type: Number,
required: true
},
category: {
object: {
type: String,
required: true
},
group: {
type: String,
required: true
},
name: {
type: String,
required: true
}
}
};
This is what I would like to do:
If I have the following data in category:
category.object = "ring"
category.group = "movies"
category.name= "lord of the rings"
I want the id to be made of the first letters of every field in category and a number (the number of the last item added plus 1). In this case, It would be RMLOTR1.
What I'm doing right now
I'm adding a lot of data at the same time, so every time I do it, I made a function that iterates through all the items added and does what I want but...
My question is
Is there a built-in way to do this with mongodb or mongoose, adding the data and creating the id at the same time? I know I can do a virtual, but I want the data to be stored.
Extras
If it's not posible to do this with mongodb, is there a way to do this with MySQL?
Is doing this kind of thing considered a correct/wrong approach?

You are basically looking for a "pre" middleware hook on the "save" event fired by creating new documents in the collection. This will inspect the current document content and extract the "strings" from values in order to create your "prefix" value for _id.
There is also another part, where the "prefix" needs the addition of the numeric counter when there is already a value present for that particular "prefix" to make it distinct. There is a common technique in MongoDB used to "Generate an auto-incrementing sequence field", which basically involves keeping a "counters" collection and incrementing the value each time you access it.
As a complete and self contained demonstration, you combine the techniques as follows:
var async = require('async'),
mongoose = require('mongoose'),
Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost/warehouse');
var counterSchema = new Schema({
"type": { "type": String, "required": true },
"prefix": { "type": String, "required": true },
"counter": Number
});
counterSchema.index({ "type": 1, "prefix": 1 },{ "unique": true });
counterSchema.virtual('nextId').get(function() {
return this.prefix + this.counter;
});
var productSchema = new Schema({
"_id": "String",
"category": {
"object": { "type": String, "required": true },
"group": { "type": String, "required": true },
"name": { "type": String, "required": true }
}
},{ "_id": false });
productSchema.pre('save', function(next) {
var self = this;
if ( !self.hasOwnProperty("_id") ) {
var prefix = self.category.object.substr(0,1).toUpperCase()
+ self.category.group.substr(0,1).toUpperCase()
+ self.category.name.split(" ").map(function(word) {
return word.substr(0,1).toUpperCase();
}).join("");
mongoose.model('Counter').findOneAndUpdate(
{ "type": "product", "prefix": prefix },
{ "$inc": { "counter": 1 } },
{ "new": true, "upsert": true },
function(err,counter) {
self._id = counter.nextId;
next(err);
}
);
} else {
next(); // Just skip when _id is already there
}
});
var Product = mongoose.model('Product',productSchema),
Counter = mongoose.model('Counter', counterSchema);
async.series(
[
// Clean data
function(callback) {
async.each([Product,Counter],function(model,callback) {
model.remove({},callback);
},callback);
},
function(callback) {
async.each(
[
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "four weddings and a funeral"
}
},
{
"category": {
"object": "ring",
"group": "movies",
"name": "lord of the rings"
}
}
],
function(data,callback) {
Product.create(data,callback)
},
callback
)
},
function(callback) {
Product.find().exec(function(err,products) {
console.log(products);
callback(err);
});
},
function(callback) {
Counter.find().exec(function(err,counters) {
console.log(counters);
callback(err);
});
}
],
function(err) {
if (err) throw err;
mongoose.disconnect();
}
)
This gives you output like:
[ { category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR1' },
{ category:
{ name: 'four weddings and a funeral',
group: 'movies',
object: 'ring' },
__v: 0,
_id: 'RMFWAAF1' },
{ category: { name: 'lord of the rings', group: 'movies', object: 'ring' },
__v: 0,
_id: 'RMLOTR2' } ]
[ { __v: 0,
counter: 2,
type: 'product',
prefix: 'RMLOTR',
_id: 57104cdaa774fcc73c1df0e8 },
{ __v: 0,
counter: 1,
type: 'product',
prefix: 'RMFWAAF',
_id: 57104cdaa774fcc73c1df0e9 } ]
To first understand the Counter schema and model, you are basically defining something where you are going to look up a "unique" key and also attach a numeric field to "increment" on match. For convenience this just has a two fields making up the unique combination and a compound index defined. This could just also be a compound _id if so wanted.
The other convenience is the virtual method of nextId, which just does a concatenation of the "prefix" and "counter" values. It's also best practice here to include something like "type" here since your Counter model can be used to service "counters" for use in more than one collection source. So here we are using "product" whenever accessing in the context of the Product model to differentiate it from other models where you might also keep a similar sequence counter. Just a design point that is worthwhile following.
For the actual Product model itself, we want to attach "pre save" middleware hook in order to fill the _id content. So after determining the character portion of the "prefix", the operation then goes off and looks for that "prefix" with the "product" type data in combination in the Counter model collection.
The function of .findOneAndUpdate() is to look for a document matching the criteria in the "counters" collection and then where a document is found already it will "increment" the current counter value by use of the $inc update operator. If the document was not found, then the "upsert" option means that a new document will be created, and at any rate the same "increment" will happen in the new document as well.
The "new" option here means that we want the "modified" document to be returned ( either new or changed ) rather than what the document looked like before the $inc was applied. The result is that "counter" value will always increase on every access.
Once that is complete and a document for Counter is either incremented or created for it's matching keys, then you now have something you can use to assign to the _id in the Product model. As mentioned earlier you can use the virtual here for convenience to get the prefix with the appended counter value.
So as long as your documents are always created by either the .create() method from the model or by using new Product() and then the .save() method, then the methods attached to your "model" in your code are always executed.
Note here that since you want this in _id, then as a primary key this is "immutable" and cannot change. So even if the content in the fields referenced was later altered, the value in _id cannot be changed, and therefore why the code here makes no attempt when an _id value is already set.

MongoDB with Mongoose - Find only certain child documents

MongoDB 3.0.7 and Mongoose 4.3.4.
Schema:
var schema = new mongoose.Schema({
confirmed: { type: Boolean, default: false },
moves: [new mongoose.Schema({
name: { type: String, default: '' },
live: { type: Boolean, default: true }
})]
});
mongoose.model('Batches', schema);
Query:
var Batch = mongoose.model('Batches');
var query = {
confirmed: true,
moves: {
$elemMatch: {
live: true
}
}
};
Batch.find(query).exec(function(err, batches){
console.log('batches: ', batches);
});
I need to return all batches that are confirmed, and all moves within the returned batches that are live.
At the moment, the above is returning only the confirmed batches (which is what I want), but all the moves in each returned batch (which is not what I want). So the limiting of moves by the live flag is not working.
How do I limit the sub-documents that are returned..?
Ideally, I would like to keep everything that controls the data returned within query passed to find, and not have to call more methods on Batch.

For Mongoose versions >=4.3.0 which support MongoDB Server 3.2.x, you can use the $filter operator with the aggregation framework to limit/select the subset of the moves array to return based on the specified condition. This returns an array with only those elements that match the condition, so you will use it in the $project stage to modify the moves array based on the filter above.
The following example shows how you can go about this:
var Batch = mongoose.model('Batches'),
pipeline = [
{
"$match": { "confirmed": true, "moves.live": true }
},
{
"$project": {
"confirmed": 1,
"moves": {
"$filter": {
"input": "$moves",
"as": "el",
"cond": { "$eq": [ "$$el.live", true ] }
}
}
}
}
];
Batch.aggregate(pipeline).exec(function(err, batches){
console.log('batches: ', batches);
});
or with the fluent aggregate() API pipeline builder:
Batch.aggregate()
.match({
"$match": { "confirmed": true, "moves.live": true }
})
.project({
"confirmed": 1,
"moves": {
"$filter": {
"input": "$moves",
"as": "el",
"cond": { "$eq": [ "$$el.live", true ] }
}
}
})
.exec(function(err, batches){
console.log('batches: ', batches);
});
For Mongoose versions ~3.8.8, ~3.8.22, 4.x which support MongoDB Server >=2.6.x, you could filter out the false values using a combination of the $map and $setDifference operators:
var Batch = mongoose.model('Batches'),
pipeline = [
{
"$match": { "confirmed": true, "moves.live": true }
},
{
"$project": {
"confirmed": 1,
"moves": {
"$setDifference": [
{
"$map": {
"input": "$moves",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.live", true ] },
"$$el",
false
]
}
}
},
[false]
]
}
}
}
];
Batch.aggregate(pipeline).exec(function(err, batches){
console.log('batches: ', batches);
});

The query does not limit moves by the live flag. The query reads: find all confirmed batches with at least one live move.
There are 2 options to retrieve live moves only: retrieve all moves, and filter the array clientside; or map-reduce it serverside - unwind all moves, filter live ones, and group by document id.
The former is simpler to implement, but will result with more data transfer, cpu and memory consumption on the client side. The later is more efficient, but a bit more complex to implement - if you expect more than 16Mb in the response, you will need to use a temporary collection.

You can use the aggregation framework's $unwind method to split them into separate documents, here are sample codes.
Batches.aggregate(
{ $match: {'confirmed': true, 'moves.live': true}},
{$unwind: '$moves'},
{$project: {
confirmed: 1
name: '$moves.name',
live:'$moves.live'
}
}, function(err, ret){
})

Adding GeometryCollections with Mongoose

I've build a form in Angular to add GeoJSON features to MongoDB using Mongoose. The features include several properties and a GeometryCollection with a point and a lineString.
Here comes the trouble: I was able to create features with just a single point in my geometry but I'm unable to create features with a geometry collection that uses a lineString. I get either:
16755 Can't extract geo keys from object, malformed geometry?
or:
{ [CastError: Cast to number failed for value "0,0,1,1" at path "coordinates"]
message: 'Cast to number failed for value "0,0,1,1" at path "coordinates"',
name: 'CastError',
type: 'number',
value: [[0,0],[1,1]],
path: 'coordinates' }'
I do realize it says type: 'number' while my schema is set to an array of arrays:
var featureSchema = new mongoose.Schema({
'type': {
type: String,
default: "Feature"
},
geometry: {
'type': {
type: String,
default: 'GeometryCollection',
}, geometries: [{
'type': {
type: String,
default: 'Point'
},
coordinates: [Number]
}, {
'type': {
type: String,
default: 'LineString'
},
coordinates: {
type: [Array],
default: [[0,0], [1,1]]
}
}]
},
properties: {
title: String
}
});
So my first question is: does anyone know how to properly add features using GeometryCollections with Mongoose?
My second question is how to add an array of arrays when using forms? When I use a text input now I get my array of arrays delivered as a string. I was able to convert the point coordinates using:
var array = req.body.feature.geometry.geometries.coordinates.split(',');
for(var i=0; i<array.length; i++) {
array[i] = +array[i];
}
Is there a way to convert a string (ie "[ [0,0], [1,1] ]") to an array of arrays to create the lineString coordinates?
Thanks in advance!

The proper way is to split this into multiple schemas, which is much easier to read, use and maintain. For example:
GeoJSON.FeatureCollection = {
"type" : {
"type": String,
"default": "FeatureCollection"
},
"features": [GeoJSON.Feature]
}
GeoJSON.Feature = {
"id": {
"type": "String"
},
"type": {
"type": String,
"default": "Feature"
},
"properties": {
"type": "Object"
},
"geometry": GeoJSON.Geometry
}
GeoJSON.GeometryCollection = {
"type": {
"type": String,
"default": "GeometryCollection"
},
"geometries": [GeoJSON.Geometry]
}
GeoJSON.Geometry = {
"type": {
"type": String,
"enum": [
"Point",
"MultiPoint",
"LineString",
"MultiLineString",
"Polygon",
"MultiPolygon"
]
},
"coordinates": []
}
Taken from: https://github.com/RideAmigosCorp/mongoose-geojson-schema

MongoDB : Map Reduce : Create one sub-document from another one

I have a mongodb collection which has documents like this :
{
"_id" : ObjectId("safdsd435tdg54trgds"),
"startDate" : ISODate("2013-07-02T17:35:01.000Z"),
"endDate" : ISODate("2013-08-02T17:35:01.000Z"),
"active" : true,
"channels" : [
1, 2, 3, 4
],
}
I want to convert this to something like this :
{
"_id" : ObjectId("safdsd435tdg54trgds"),
"startDate" : ISODate("2013-07-02T17:35:01.000Z"),
"endDate" : ISODate("2013-08-02T17:35:01.000Z"),
"active" : true,
"channels" : [
1, 2, 3, 4
],
"tags" :[
{
"name": one
"type": channel
},
{
"name": two
"type": channel
},
{
"name": three
"type": channel
},
{
"name": four
"type": channel
}
]
}
I already have a mapping of what 1,2,3,4 mean. Just for the sake of simplicity I put them as their alphabetical format. the values could be different, but they're static mappings.

You seem to be trying to do this update without a big iteration of your collection, So you "could" do this with mapReduce, albeit in a very "mapReduce way" as it has it's own way of doing things.
So first you want to define a mapper that encapsulates your current document :
var mapFunction = function (){
var key = this._id;
var value = {
startDate: this.startDate,
endDate: this.endDate,
active: this.active,
channels: this.channels
};
emit( key, value );
};
Now here the reducer is actually not going to be called as all the keys from the mapper will be unique, being of course the _id values from the original document. But to make the call happy:
var reduceFunction = function(){};
As this is a one to one thing this will go to finalize. It could be in the mapper, but for cleanliness sake
var finalizeFunction = function (key, reducedValue) {
var tags = [
{ name: "one", type: "channel" },
{ name: "two", type: "channel" },
{ name: "three", type: "channel" },
{ name: "four", type: "channel" }
];
reducedValue.tags = [];
reducedValue.channels.forEach(function(channel) {
reducedValue.tags.push( tags[ channel -1 ] );
});
return reducedValue;
};
Then call the mapReduce:
db.docs.mapReduce(
mapFunction,
reduceFunction,
{
out: { replace: "newdocs" },
finalize: finalizeFunction
}
)
So that will output to a new collection, but in the way that mapReduce does it so you have this:
{
"_id" : ObjectId("53112b2d0ceb66905ae41259"),
"value" : {
"startDate" : ISODate("2013-07-02T17:35:01Z"),
"endDate" : ISODate("2013-08-02T17:35:01Z"),
"active" : true,
"channels" : [ 1, 2, 3, 4 ],
"tags" : [
{
"name" : "one",
"type" : "channel"
},
{
"name" : "two",
"type" : "channel"
},
{
"name" : "three",
"type" : "channel"
},
{
"name" : "four",
"type" : "channel"
}
]
}
}
So all your document fields other than _id are stuck under that value field, so that's not the document that you want. But that is how mapReduce works.
If you really need to get out of jail from this and are willing to wait a bit, the upcoming 2.6 release has added an $out pipeline stage. So you "could" transform the documents in your new collection with $project like this:
db.newdocs.aggregate([
// Transform the document
{"$project": {
"startDate": "$value.startDate",
"endDate": "$value.endDate",
"active": "$value.active",
"channels": "$value.channels",
"tags": "$value.tags"
}},
// Output to new collection
{"$out": "fixeddocs" }
])
So that will be right. But of course this is not your original collection. So to back to that state you are going to have to .drop() collections and use .renameCollection() :
db.newdocs.drop();
db.docs.drop();
db.fixeddocs.renameCollection("docs");
Now please READ the documentation carefully on this, there are several limitations, and of course you would have to re-create indexes as well.
All of this, and in particular the last stage is going to result in a lot of disk thrashing and also keep in mind that you are dropping collections here. It almost certainly is a case for taking access to your database off-line while this is performed.
And even as such the dangers here are real enough that perhaps you can just live with running an iterative loop to update the documents, using arbitrary JavaScript. And if you really must have to do so, you could always do that using db.eval() to have that all execute on the server. But if you do, then please read the documentation for that very carefully as well.
But for completeness even if I'm not advocating this:
db.eval(function(){
db.docs.find().forEach(function(document) {
var tags = [
{ name: "one", type: "channel" },
{ name: "two", type: "channel" },
{ name: "three", type: "channel" },
{ name: "four", type: "channel" }
];
document.tags = [];
document.channels.forEach(function(channel) {
document.tags.push( tags[ channel -1 ] );
});
var id = document._id;
delete document._id;
db.docs.update({ "_id": id },document);
});
})