Develop Reference

Sails js and GeoJSON format

anybody knows how to store GEOJSON in sails js models?
i want to store features and points in sails js models.
var geojsonFeature = {
"type": "Feature",
"properties": {
"name": "Coors Field",
"amenity": "Baseball Stadium",
"popupContent": "This is where the Rockies play!"
},
"geometry": {
"type": "Point",
"coordinates": [-104.99404, 39.75621]
}
};

In Sails' Attributes documentation you can find that there are 5 types supported by Sail's Waterline (the underlying ORM within Sails)
string
number
boolean
json
ref
In your case, you have several objects embedded. Something like the following would probably work:
attributes: {
type: {
type: 'string',
enum: ['Feature','This','That','Other'], // not required...
},
properties: {
type: 'json'
},
geometry: {
type: 'json'
}
}

Trying to access dot notation variable with a string

I'm working on creating a web form that can dynamically read a swagger endpoint to create form fields. Specifically right now I am trying to read the schemas from the component section defined by openAPI 3.
Example json:
{
"openapi": "3.0.1",
"info": {
.......
},
"paths": {
........
},
"components": {
"schemas": {
"FakeAppConfiguration": {
"type": "object",
"properties": {
"setting1": {
"type": "string",
"nullable": true
},
"setting2": {
"type": "string",
"nullable": true
}
},
"additionalProperties": false
},
"OtherFakeAppConfiguration": {
........
},
"ThirdFakeAppConfiguration": {
........
}
}
}
}
}
Using this snippet of json as an example, I can easily get the names of the schemas that are defined by using (json has already been loaded into data using fetch)
for (let schema in data.components.schemas)
{
//this will print out FakeAppConfiguration, OtherFakeAppConfiguration, ThirdFakeAppConfiguration
console.log(schema);
}
My problem now comes in trying to access each of these schema trees without calling them directly. I could easily do data.components.schemas.FakeAppConfiguration, but that would defeat the purpose of making this dynamic. I've been trying to somehow use the strings obtained in the above loop to access what I want to no avail. Some examples of things I've tried are below. Anyone able to help me get further access without calling the variable directly with dot notation? I have also considered doing manual parsing of the JSON, but trying to avoid that. This is a react app, so if anyone can think of a library that could help, I'm all ears there as well.
//treating like a key
data.components.schemas['FakeAppConfiguration']
//trying to create a map
interface SchemaDef {
type: string,
properties: Properties,
//....etc,etc
}
let i = 0;
let schemas: Map<string, SchemaDef> = new Map<string, SchemaDef>();
for (let schema in data.components.schemas)
{
schemas.set(schema, data.components.schemas[i]);
i++;
}

You could iterate over the Object.entries() of your "schemas" object.
let schemas = {
"FakeAppConfiguration": {
"type": "object",
"properties": {
"setting1": {
"type": "string",
"nullable": true
},
"setting2": {
"type": "string",
"nullable": true
}
},
},
"FakeAppConfiguration2": {
"type": "object",
"properties": {
"setting1": {
"type": "string",
"nullable": true
},
"setting2": {
"type": "string",
"nullable": true
}
},
}
};
for (let [key, value] of Object.entries(schemas)) {
console.log(key, "\n\n", value);
}

exception: geoNear command failed: errmsg: more than one 2d index

I'm trying to retrieve documents from a collection by distance. I've tried to use the $geoNear aggregation, but I either run into errors (using node.js and postman) or 0 records get returned.
Sample Document:
{
"_id" : ObjectId("5cc37692fe9fd54b3cd136c9"),
"category" : "art",
"description" : "an eastbound description for The soda event.",
"geometry" : {
"type" : "Point",
"coordinates" : [
3.60178480057443,
6.46123057784917
]
}
"__v" : 0
}
.
.
Model:
const GeoSchema = new Schema({
type: {
type: String,
default: "Point"
},
coordinates: {
type: [Number],
index: "2dsphere"
}
}, { _id: false });
const EventSchema = new Schema({
category: String,
description: String,
geometry: GeoSchema,
});
Query
db.events.aggregate([
{
$geoNear: {
near: { type: "Point", coordinates: [3.60178480057443, 6.46123057784917] },
distanceField: "dist",
maxDistance: 90000,
spherical: true
}
}
]);
Running the query above returns zero results, an empty array (on postman) and sometimes displays this error:
{
"name": "MongoError",
"errmsg": "exception: geoNear command failed: { ok: 0.0, errmsg: \"more than one 2d index, not sure which to run geoNear on\" }",
"code": 16604,
"ok": 0
}
When this error is displayed, I run the db.events.dropIndex function. I copied and pasted the examples used in the docs, and it worked fine. Please How do I make this $geoNear function work. I have been struggling with it for a whole day.

A Single collection cannot have more then one 2dsphere index and You have created 2dsphere index on more than one field by running following command.
db.getCollection('test').createIndex({ "loc": "2dsphere" })
So remove the one of the index and it will get work. You can check the list of indices using this
db.getCollection('test').getIndexes()
Update:
You can use key parameter to specify on which field do you want to make search with if your collection have multiple $geoNear indices

pg-promise wrong insertion of multipolygons into postgis database

So I am using pg-promise to insert multiple geojson MultiPolygons into a postgis database. The insertion into the database work fine, but for some of the row in the database I get a strange behaviour, that is the cell is filled with two lines. The first line some load message and the second line is the actual geom object which more strangely is converted right from geojson to postgis geom.
function createBorder(pathToJSON, table) {
fs.readFile(pathToJSON,
{encoding: 'UTF-8'},
(err, data) => {
let geoJSON = JSON.parse(data);
geoJSON.features.forEach(f => {
f.geometry.crs = {
type: 'name',
properties: {
name: 'EPSG:4326'
}
}
db.none('INSERT INTO nyc_borders(geom)\
VALUES (ST_GeomFromGeoJSON(${geoJSON}))', {
geoJSON: f.geometry
})
.then((d) => {
console.log(f.geometry);
})
.catch(error => {
console.log("ERROR: ", error);
})
});
});
}
createBorder('./data/community_districts.geojson');
I shortend the geoJSON output, it is basically the community district borders from nyc downloaded from the opendata portal
Geojson:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"shape_leng": "51549.5578986",
"boro_cd": "311",
"shape_area": "103177785.347"
},
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
-73.97348373564797,
40.61137106069874
],
[
-73.97303089190211,
40.6090051063008
],
[
-73.97299433938896,
40.60881414180224
]
]
]
]
}
},
{
"type": "Feature",
"properties": {
"shape_leng": "65821.875617",
"boro_cd": "313",
"shape_area": "88195686.2688"
},
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
-73.96720294103956,
40.573326317397424
],
[
-73.96738975478877,
40.573258999904446
],
[
-73.9674356779313,
40.57320896967127
],
[
-73.96736390080571,
40.57304456895217
],
[
-73.98372152615246,
40.59582107821707
]
]
]
]
}
}
]
}
Some pictures from my database:
database table with rows that have two lines inside one cell
one cell expanded to see the actual tow lines better
So I am really stuck because I do not have an idea how to start debuging, singe the insertion does work some how and also the conversion of the geojson object looks fine. I actually can not figure out who is causing this wrong behaviour.

You can have full control over how pg-promise formats data, by using Custom Type Formatting.
For example, if you have an array[][2] (points as shown), you can convert them like this:
const toGeometry = g => ({ /* g = array[][2] (points) */
rawType: true,
toPostgres: a => {
const points = a.map(p => pgp.as.format('$1 $2', p));
return 'ST_GeomFromText(\'LINESTRING(' + points.join() + ')\')';
}
});
And then you can pass in toGeometry(f.geometry) to apply your custom formatting.
See also: ST_GeomFromText.

I found the solution for my problem the two lines displayed in the pictures that confused me where only information added by datagrip to tell me that the huge polygons where not loaded fully.
I had a look into the same rows with psql:
SELECT ST_ASGEOJSON(geom) FROM <tablename> WHERE id=<myid>
and there the second line would not show up.
Then I realised it is just additional information.

How do I populate sub-document after $geoNear

I am using NodeJs and Mongoose and building a feature to list near by deals.
Deal.db.db.command({
"geoNear": Deal.collection.name,
"near": [23,67],
"spherical": true,
"distanceField": "dis"
}, function (err, documents) {
if (err) {
return next(err);
}
console.log(documents);
});
Deal schema:
var dealSchema = new mongoose.Schema({
title: String,
merchant: {
type: mongoose.Schema.Types.ObjectId,
ref: Merchant,
index: true
}
});
Here, I get all deals and their distances from current location. Inside Deal schema I have a merchant as reference object.
How do I populate merchants with each returned Deal object?
Do I need to iterate through all returned Deal objects and populate manually?

This actually turns out to be kind of interesting. Certainly what you do not want to really do is "dive into" the native driver part, though you possibly could on solution 2. Which leads to that there are a few ways to go about this without actually rolling your own queries to match this manually.
First a little setup. Rather than reproduce your dataset I have just picked something I had lying around from previous testing. The principles are the same, so a basic setup:
var mongoose = require('mongoose'),
async = require('async'),
Schema = mongoose.Schema;
mongoose.connect('mongodb://localhost');
var infoSchema = new Schema({
"description": String
});
var shapeSchema = new Schema({
"_id": String,
"amenity": String,
"shape": {
"type": { "type": String },
"coordinates": []
},
"info": { "type": Schema.Types.ObjectId, "ref": "Info" }
});
var Shape = mongoose.model( "Shape", shapeSchema );
var Info = mongoose.model( "Info", infoSchema );
Essentially one model with the "geo" information and the other that is referenced with just some information we want to populate. Also my lazy alteration of data:
{
"_id" : "P1",
"amenity" : "restaurant",
"shape" : { "type" : "Point", "coordinates" : [ 2, 2 ] }
}
{
"_id" : "P3",
"amenity" : "police",
"shape" : { "type" : "Point", "coordinates" : [ 4, 2 ] }
}
{
"_id" : "P4",
"amenity" : "police",
"shape" : { "type" : "Point", "coordinates" : [ 4, 4 ] }
}
{
"_id" : "P2",
"amenity" : "restaurant",
"shape" : { "type" : "Point", "coordinates" : [ 2, 4 ] },
"info" : ObjectId("539b90543249ff8d18e863fb")
}
So there is one thing in there that we can expect to populate. As it turns out a $near query is pretty straightforward as does sort as expected:
var query = Shape.find(
{
"shape": {
"$near": {
"$geometry": {
"type": "Point",
"coordinates": [ 2, 4 ]
}
}
}
}
);
query.populate("info").exec(function(err,shapes) {
if (err) throw err;
console.log( shapes );
});
That is just a standard .find() with the $near operator invoked. This is MongoDB 2.6 syntax, so there is that. But the results sort correctly and populate as expected:
[
{ _id: 'P2',
amenity: 'restaurant',
info:
{ _id: 539b90543249ff8d18e863fb,
description: 'Jamies Restaurant',
__v: 0 },
shape: { type: 'Point', coordinates: [ 2, 4 ] } },
{ _id: 'P4',
amenity: 'police',
info: null,
shape: { type: 'Point', coordinates: [ 4, 4 ] } },
{ _id: 'P1',
amenity: 'restaurant',
info: null,
shape: { type: 'Point', coordinates: [ 2, 2 ] } },
{ _id: 'P3',
amenity: 'police',
info: null,
shape: { type: 'Point', coordinates: [ 4, 2 ] } }
]
That is pretty nice, and a simple way to invoke. The catch? Sadly changing the operator to $geoNear to take the spherical geometry into account starts throwing errors. So if you want that then you can't just do things as you are used to.
Another approach though is that mongoose has a .geoNear() function that is supported. But just like the db.command invocation, this does not return a mongoose document or other Model type object that will accept .populate(). Solution? Just play with the output a little:
var query = Shape.geoNear({
"type": "Point",
"coordinates": [ 2, 4 ]
},{spherical: true},function(err,shapes) {
if (err) throw err;
shapes = shapes.map(function(x) {
delete x.dis;
var a = new Shape( x.obj );
return a;
});
Shape.populate( shapes, { path: "info" }, function(err,docs) {
if (err) throw err;
console.log( docs );
});
So here the result returned is an array of raw objects. But with a little manipulation you can turn these into something that is going to work with the .populate() method which can also be invoked from the model class as shown.
The result of course is the same, though the field order may be a little different. And you didn't need to iterate an call the queries yourself. This is really all that .populate() is actually doing, but I think we can agree that using the .populate() method at least looks a lot cleaner and does not re-invent the wheel for this purpose.