I have a collection in MongoDb where data in the collection has the following structure :
{userid = 1 (the id of the user), key1 = value1 , key2 = value2, .... }
I want to write mongodb mapreduce functions where i could put the userid in the map function and in the reduce function i need to write they ( key,value ) pairs in a csv (?) file such that they would be :
key1,key2, key3,...
value1,value2,value3,..
value1,value2,value3,..
value1,value2,value3,..
How can i do that with mongodb
Thanks
There is no "file output" option.
The MongoDB documentation has details on exporting data.
In particular, mongoexport allows for export by JSON or CSV which should be legible from other software.
If you want to significantly modify the data output, then you'll have to use a client library and cursor through the data while writing to a file.
You can not write data to a file directly. You have to implement such a functionality on the application level by reading the data from the collection and writing it to the filesystem in whatever format.
Related
Im triying to write a JSON object to a DB with FS but it´s not working as expected. What i want to do is write the JSON data inside this: [] Example:
[
{"name":"jhon"},
{"name":"jhonathan"}
]
Here is the code:
Thanks.
The comment you provided doesn't make much sense, because the JSON data you provided in the post and in the comments is completely different. But I get the gist of it. I guess you have a JSON file containing an array and you want to push new items to it. Let's do this.
The thing is, when you call fs.appendFile, you're only writing to the end of the file. You're not following JSON format by doing so.
You need to do this:
Read file content.
Parse JSON text into an object.
Update the object in memory.
Write object in JSON format back to the file system.
I'll call the synchronous methods for simplicity's sake, but you should be able to convert it to async quite easily.
const path = __dirname + 'db.json'
// Reading items from the file system
const jsonData = fs.readFileSync(path)
const items = JSON.parse(jsonData)
// Add new item to the item list
items.push(newItem)
// Writing back to the file system
const newJsonString = JSON.stringify(items)
fs.writeFileSync(path, newJsonString)
Whenever I read all my data from a database table and receive it as JSON from my API, I get my data like this (unique_name being the primary key in this example):
[{"unique_name":"alice", "age":18, "city":"kansas"},{"unique_name":"bob", "age":20, "city":"chicago"}]
In my javascript application, however, I need my data to be formatted like so:
const myData = {alice: {age:18, city:"kansas"}, bob: {age:20, city:"chicago"}}
I guess this could be done with object mapping of some sort, but I'm afraid this would be too slow with a lot of entries. Is there any clean way to do this?
There is no real problem to map a lot of entries.
If you want more performences you should do it server side (eg: on a nodejs server)
You can simply use a .forEach()
Exemple (not complete so you just get the idea)
const myData = {};
mySQL.forEach(entry => myData[entry.unique_key] = entry)
I'm trying to create a save/load function for my game in js, but I have basically no idea with how to go through with doing this. I can save variables to a JSON file or LocalStorage, but I don't know how to load them back into the program. I'm also pretty sure I'm exporting variables the wrong way as well. Any help?
Normally, I use JSON format to store and read data (of any type).
To save data (using key gamedata as example):
var myData = {
name: 'David',
score: 10
}
localStorage.setItem('gamedata', JSON.stringify(myData));
** without JSON.stringify, you data will be saved as string [Object object]
To retrieve the data back:
var savedData = localStorage.getItem('gamedata'); // savedData is string
var myData = JSON.parse(savedData); // parse JSON string to java object
setup a bin on www.myJSON.com. p5 has built in functionality for ajax requests such as loadJSON. that way it's not in local storage and you can access your data if you have it on github. I know your struggle, I used to deal with this sort of issue myself before I found myJSON
It is possible to get data from Firebase database from Javascript (WEB) directly in JSON format ?
Currently I'm using this function and I must to parse all data and convert it to JSON array
firebase.database().ref('posts/').once('value', function(snap){
snap.forEach(function(obj){
console.log(obj.val());
// Parsing obj and saving to JSON array ...
})
})
It would be nice if I can get data directly in JSON format (like in Firebase console).
What if you just JSON.stringify() it? Does that get you what you're looking for?
firebase.database().ref('posts/').once('value', function(snap){
console.log(JSON.stringify(snap.val()))
})
I'm trying to write a mongo script to import a jsonArray from a JSON file. My script is in .js format and I execute it with load() command in mongo shell. Is it possible to do it with a mongo script?
I know I can use mongoimport instead. But I want to know a way to do it with a script.
The contents of my current script in which the import part is missing is given below..
var db = connect("localhost:27017/fypgui");
//Import json to "crimes" collection here
var crimes = db.crimes.find();
while (crimes.hasNext()){
var item = crimes.next();
var year =(item.crime_date != null)?(new Date(item.crime_date)).getFullYear():null;
db.crimes.update( {_id: item._id}, {$set: {crime_year: year}});
}
There is another answer to this question. Even though it's a bit old I am going to respond.
It is possible to do this with the mongo shell.
You can convert your JSON to valid JavaScript by prefixing it with var myData= then use the load() command to load the JavaScript. After the load() you will be able to access your data from within your mongo script via the myData object.
data.js
var myData=
[
{
"letter" : "A"
},
{
"letter" : "B"
},
{
"letter" : "C"
}
]
read.js
#!/usr/bin/mongo --quiet
// read data
load('data.js');
// display letters
for (i in myData) {
var doc = myData[i];
print(doc.letter);
}
For writing JSON is easiest to just load your result into a single object. Initialize the object at the beginning with var result={} and then use printjson() at the end to output. Use standard redirection to output the data to a file.
write.js
#!/usr/bin/mongo --quiet
var result=[];
// read data from collection etc...
for (var i=65; i<91; i++) {
result.push({letter: String.fromCharCode(i)});
}
// output
print("var myData=");
printjson(result);
The shebang lines (#!) will work on a Unix type operating system (Linux or MacOs) they should also work on Windows with Cygwin.
It is possible to get file content as text using undocumented cat() function from the mongo shell:
var text = cat(filename);
If you are interested cat() and other undocumented utilities such as writeFile are defined in this file: shell_utils_extended.cpp
Having file's content you can modify it as you wish or directly pass it to JSON.parse to get JavaScript object:
jsObj = JSON.parse(text);
But be careful: unfortunately JSON.parse is not an equivalent of mongoimport tool in the sense of its JSON parsing abilities.
mongoimport is able to parse Mongo's extended JSON in canonical format. (Canonical format files are created by bsondump and mongodump for example. For more info on JSON formats see MongoDB extended JSON).
JSON.parse does not support canonical JSON format. It will read canonical format input and will return JavaScript object but extended data type info present in canonical format JSON will be ignored.
No, the mongo shell doesn't have the capability to read and write from files like a fully-fledged programming environment. Use mongoimport, or write the script in a language with an official driver. Node.js will have syntax very close to the mongo shell, although Node.js is an async/event-driven programming environment. Python/PyMongo will be similar and easy to learn if you don't want to deal with structuring the logic to use callbacks.
Hey I know this is not relevent but, every time I need to import some jsons to my mongo db I do some shity copy, paste and run, until I had enough!!!
If you suffer the same I v written a tiny batch script that does that for me. Intrested?
https://github.com/aminjellali/batch/blob/master/mongoImporter.bat
#echo off
title = Mongo Data Base importing tool
goto :main
:import_collection
echo importing %~2
set file_name=%~2
set removed_json=%file_name:.json=%
mongoimport --db %~1 --collection %removed_json% --file %~2
goto :eof
:loop_over_files_in_current_dir
for /f %%c in ('dir /b *.json') do call :import_collection %~1 %%c
goto :eof
:main
IF [%1]==[] (
ECHO FATAL ERROR: Please specify a data base name
goto :eof
) ELSE (
ECHO #author amin.jellali
ECHO #email a.j.amin.jellali#gmail.com
echo starting import...
call :loop_over_files_in_current_dir %~1
echo import done...
echo hope you enjoyed me
)
goto :eof