Develop Reference

MongoDB Error: Cannot create property '_id' on string

I'm using Node.js and Express on Heroku, with the MongoDB addon.
My database connection works fine and I can successfully push some data in, but not other.
Here is the database connection:
mongodb.MongoClient.connect(mongoURI, function (err, database) {
if (err) {
console.log(err);
process.exit(1);
}
// Save database object from the callback for reuse.
db = database;
console.log("Database connection ready");
// Initialize the app.
var server = app.listen(process.env.PORT || dbport, function () {
var port = server.address().port;
console.log("App now running on port", port);
});
});
I can successfully push my Twitter API response into the database like this:
db.collection(TWEETS_COLLECTION).insert(data);
('data' is just a JSON variable)
But when I try to push another JSON variable into the database in the same method, I get an error. Code:
var jsonHash = '{"hashtag":"","popularity":1}';
var objHash = JSON.parse(jsonHash);
objHash.hashtag = req.body.hashtag;
JSON.stringify(objHash);
collection(HASHTAG_COLLECTION).insert(jsonHash);
And the error:
TypeError: Cannot create property '_id' on string '{"hashtag":"myhash","popularity":1}'
at Collection.insertMany...
...
Any ideas what I'm doing wrong?

I don't know where you are getting the jsonHash variable from but I think you are doing unecessary JSON-handling here. You are also inserting the wrong variable, you want to insert objHash which is a valid object to insert, now you are inserting jsonHash which is just a string. JSON.stringify(objHash); is not doing anything as you are not saving the JSON returned from the function. I think you want something like this?
var objHash = {
hashtag: "",
popularity:1
};
objHash.hashtag = req.body.hashtag;
collection(HASHTAG_COLLECTION).insert(objHash);

jsonHash is still a string. May be you want to save objHash instead without JSON.stringify ?

How do I use client side JavaScript to find and return data from MongoDB and a Node.js server with no framework?

I have read all the questions on SO that I could find. They all use Express, Mongoose or they leave something out. I understand that Node.js is the server. I understand the MongoDB require is the driver the Node.js server uses to open a connection to the MongoDB. Then, on the server, I can do (from the documentation):
var MongoClient = require('mongodb').MongoClient;
var assert = require('assert');
var ObjectId = require('mongodb').ObjectID;
var url = 'mongodb://localhost:27017/test';
var findRestaurants = function(db, callback) {
var cursor =db.collection('restaurants').find( );
cursor.each(function(err, doc) {
assert.equal(err, null);
if (doc != null) {
console.dir(doc);
} else {
callback();
}
});
};
// Connect to the db
MongoClient.connect(url, function(err, db) {
assert.equal(null, err);
findRestaurants(db, function() { //I don't want to do this as soon as the server starts
db.close();
});
});
//if I put findRestaurant here,
function findRestaurant(data){
}
How do I call it from the client?
I do not want to find data as soon as I start the server. I realize those are examples, but what I cannot find is a way where the client requests some data and where the Node.js server returns it.
I have seen close examples using jQuery, Angular on the client, and then Express, Mongoose, Meteor, , etc.
All I want to understand is how I make this request from the client's browser. I can do that with XMLhttpRequest(), so I can put that part together, I believe. But, any example is appreciated.
But what is waiting on the Node.js side of things (how do I set up my function to be called once the server is listening)?
How do I create a function on the server side, maybe "GetRestaurants" and have that return the data it gets using find()?
I cannot find this information, this simple, anywhere. Is it too complicated to do the example without a framework?
I do not wish to copy and paste from something using Express, etc. without understanding what's going on. Most explanations never say, this goes on the Node.js side. This is client. I know I am expected to do my own research, but I am not putting it together, too used to RDBMSes, IIS, Apache, PHP, and so on.
I believe I have a fundamental misunderstanding of what's going on in the paradigm.
Please. No REST API creation, no frameworks of any kind on Node.js other than using the MongoDB library (unless there is an absolute requirement), not even jQuery, Angular, Jade, or anything else for the client side, straight up JavaScript on all sides.
I have seen questions like this,
How to display data from MongoDB to the frontend via Node.js without using a framework
But they do not show what I am asking. They do it all at once, as soon as the database connects. What if I want to do a delete or insert or find? There are many SO questions like this, but I have not hit the one that shows what I am looking for.

This should give the guidance. Once you go to a browser and type http://localhost:5155 the callback function (request, response) { will be called and the request to db will be made. Make sure you get response and then start working on the client side code:
const http = require('http');
const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');
const url = 'mongodb://localhost:27017/test';
const server = http.createServer(function (request, response) {
getData(function (data) {
response.end(data);
});
});
function getData(callback) {
// Connect to the db
MongoClient.connect(url, function (err, db) {
assert.equal(null, err);
findRestaurants(db, function (data) {
db.close();
callback(data);
});
});
const findRestaurants = function (db, callback) {
const cursor = db.collection('restaurants').find();
const data = [];
cursor.each(function (err, doc) {
assert.equal(err, null);
data.push(doc);
if (doc === null) {
callback(data);
}
});
};
}
server.listen(5155);

Mongoose Schema for remote MongoDb

I have opened a connection to my remote mongodb ec2 instance but now am trying to retrieve data that is nested within a collection. The database has multiple collections (ie visitor, campaign, form, etc...) and has data already in it from another source. I am using node + express for the application.
1) Do I have to define a schema in my app to match the remote database or can I just query for the data and store it in an object?
mongoose schema creation
2) Actually retrieving the values within the visitor collection, can I just use dot notation to query within the visitor collection for visitor_id using:
db.find(visitor.visitor_id)
Here is the database connection code I am using if that helps
var uri = 'mongodb://xx.xxx.xx.x'
var mongoOptions = { db: { safe: true } };
db = mongoose.createConnection(uri, mongoOptions, function (err, res) {
if (err) {
console.log('ERROR connecting to: remote' + uri + '. ' + err);
} else {
console.log('Successfully connected to: remote' + uri);
}
});

If you're using mongoose, then yes, you need to define a schema in your app to match the database.
That notation won't work. If I understand the specific query you're trying to make (to fetch the document matching a visitor_id) then you'll need something roughly like this:
// Assuming you already have mongoose connected to the database elsewhere
var mongoose = require('mongoose');
var Schema = mongoose.Schema;
var visitorSchema = new Schema({
visitor_id: Number,
etc: etc // the rest of your schema
});
var Visitor = mongoose.model('Visitor', visitorSchema);
Visitor.findOne({ visitor_id: the_id_you_want_to_query }, function (err, doc) {
// doc contains the visitor document, if found
});
I suggest you familiarize yourself with queries with MongoDB and mongoose in particular—the docs aren't super easy to understand but cover most of the main cases.

How to get readable string from mysql with node.js?

When user register, i save its name in mysql (russian name in database seems like - ÐрминÑÐ). When I take data from database with PHP and print out, it works good (show russian letters), but when user connect to node.js server (using socket.io), make mysql query (using node-mysql module) and receive data from query, then his name seems like - ÐрминÑÐ.
How to get readable letters?
Here are some lines from my server file:
var mysql = require('mysql');
var db = mysql.createConnection({host:'x', user:'x', password:'x', database:'x', charset:'UTF8_GENERAL_CI'});
db.connect();
var http = require('http');
var server = http.createServer(function(req,res){
req.setEncoding("utf8");
}).listen(8080);
var io = require('socket.io').listen(server);

I think this should help:
function onRequest(request, response) {
...
request.setEncoding("utf8");
...
}
http.createServer(onRequest).listen(8888);

Here is what I'm using in the front-end:
function encode_utf8(s) {
return unescape(encodeURIComponent(s));
}
function decode_utf8(s) {
return decodeURIComponent(escape(s));
}
So, try passing your string to
decodeURIComponent(escape(name));

I had the same problem.
Try to debug it. If console.log data from fronted have incorrect charset, just update your socket.io script.

mongodb nodejs multiple insert issue

I'm trying to import a table from mysql to mongodb straight without any schema changes.
I wrote a small node script for that and my issue is with the way i implemented it.
Maybe I hit some limit of using mongo db insert limit while using it inside a loop.
I think this problem would not have had come if it was in reverse (maybe not! )
So here's the thing.
The row in the mysql table is more than 100,000 but when the loop hit's more than around 30000 the number of inserted items just get reduced.
so let's say if there was 100,000 items in the mysql table after complete import using the below mentioned script, i get only a maximum of 37000 or so.
My strong suscpicion is either in the node script/node mongodb connector, or some bug in the script or lastly a limit in mongodb concurrent db inserts.
I'm pasting the script below.
Hoping i get a way around it.
Thanks,
var http = require('http'),
mysql = require('mysql'),
mongo = require('mongodb').MongoClient,
format = require('util').format;
var connection = mysql.createConnection({
user: "xxx",
password: "xxx",
database: "mydb"
});
connection.connect();
var query = "select * from mytable";
var mysqlrows = '';
connection.query(query, function(err,rows,fields){
if(err) throw err;
console.log(rows.length+'rows found.');
mongo.connect('mongodb://root:root#127.0.0.1:27017/mydb', function(err, db){
if (err)
throw err;
var collection = db.collection('mytable');
for(var i=0; i<rows.length;i++)
{
//console.log(JSON.stringify(rows[i]));
(function(i){
collection.insert(rows[i],function(err,docs){});
console.log(i);
})(i);
}
db.close();
});
});
connection.end();

The problem is that you're not waiting for the insert operations to complete before closing your connection to MongoDb via the db.close(); call. You need to keep track of your outstanding asynchronous requests and then only call db.close(); when they've all completed.

To make sure that you are getting all the data from mySQL, try to access the last row. If you can get it, use the flag w and j of mongodb to make sure that each call inserts the data before moving to the next. with the w and j flag, you should consider multiple inserts by inserting multiple rows at each call using and array.