Converting BSON data saved in file to Javascript object - javascript

I'm trying to convert a BSON file generated by a python script into a Javascript object. I am using https://www.npmjs.com/package/bson for the BSON package and using XMLHttpRequest to load the file.
In case it matters, this is my package-lock.json entry for bson.
"bson": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/bson/-/bson-4.0.4.tgz",
"integrity": "sha512-Ioi3TD0/1V3aI8+hPfC56TetYmzfq2H07jJa9A1lKTxWsFtHtYdLMGMXjtGEg9v0f72NSM07diRQEUNYhLupIA==",
"requires": {
"buffer": "^5.1.0",
"long": "^4.0.0"
},
"dependencies": {
"buffer": {
"version": "5.6.0",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.6.0.tgz",
"integrity": "sha512-/gDYp/UtU0eA1ys8bOs9J6a+E/KWIY+DZ+Q2WESNUA0jFRsJOc0SNUO6xJ5SGA1xueg3NL65W6s+NY5l9cunuw==",
"requires": {
"base64-js": "^1.0.2",
"ieee754": "^1.1.4"
}
}
}
}
The generated file is based on a very simple python program
data = { 'key0': 'a', 'key1': [ 1, 2, 3 ], 'key2': 'b', 'key3': [ { 'k0': 'random', 'k1': 'string', 'k2': 'to use', 'k3': 3.145 }, { 'k0': 'other', 'k1': 'values', 'k2': 'here', 'k3': 0.0001}] }
with open('test.bson', 'wb') as fp:
encoded = bson.encode(data)
fp.write(encoded)
The package being used for the python is pymongo (Version 3.10.1).
NOTE: I've updated the data dict. The first version worked fine when I used Dekel's solution. However, my actual data doesn't work. I modified it, and now it doesn't work with this error:
Uncaught Error: buffer length 199 must === bson size 253
deserialize$1
I can load the file, however I cannot figure out the correct BSON calls to use in JS to get this to a Javascript object. I'm met with wrong type errors (needs a Buffer), transpilation errors, or exceptions.
My code looks like the following (it uses Dekel's deserialize in his answer).
import { deserialize } from 'bson'
let xmlHttp = new XMLHttpRequest();
xmlHttp.onreadystatechange = () => {
if (xmlHttp.status == 200 && xmlHttp.readyState == 4) {
const buf = Buffer.from(xmlHttp.responseText, 'binary');
const dat = deserialize(buf, {});
console.log(dat);
}
};
xmlHttp.open("GET", 'assets/test.bson');
xmlHttp.send();
If I didn't use the {} as the second argument to deserialize, it generates it results in
TS2554: Expected 2 arguments, but got 1
I am using Webpack and Typescript for development.
It's not clear if the data is perhaps incorrect from the point of view of the JS BSON implementation or if I am calling the JS BSON incorrectly.
I can decode the file in python and bsondump also properly decodes the file.
I've created a GitHub repo which has more details as well as the test data. https://github.com/mobileben/test-bson-js
Some other details discovered.
When converting to a Buffer, must include binary as the encoding, else it will not work right.
This json (note it is represented as a python dict) {"key3": [{"k0": "random", "k1": "string", "k2": "to use", "k3": 3.145}, {"k0": "other", "k1": "values", "k2": "here", "k3": 0.0001}]} will cause the Uncaught error where the sizing doesn't match
Float values, when using a dict that will not result in the Uncaught error, have the wrong value. If I use integer values, they are fine.
For the last item
{"key3": {"k9": "here", "k0": 1, "k1": 2, "k2": 3}}
Will work. It results in (on the JS-side)
{"key3":{"k9":"here","k0":1,"k1":2,"k2":3}}
However
{"key3": {"k9": "here", "k0": 0.1, "k1": 0.2, "k2": 0.3}}
Results in (on the JS side)
{"key3":{"k9":"here","k0":1.8745098039215684,"k1":1.8745098039215684,"k2":1.825}}
Running bsondump on the same file yields:
{"key3":{"k9":"here","k0":{"$numberDouble":"0.1"},"k1":{"$numberDouble":"0.2"},"k2":{"$numberDouble":"0.3"}}}

Assuming you have the content of the file inside the data variable, you can use the BSON lib the following:
import { deserialize } from 'bson';
import data from '!!raw-loader!assets/test.bson'
console.log(deserialize(Buffer.from(data)))
If you are using the xmlhttprequest:
import { deserialize } from 'bson';
let xmlHttp = new XMLHttpRequest();
xmlHttp.onreadystatechange = () => {
if (xmlHttp.status == 200 && xmlHttp.readyState == 4) {
console.log(deserialize(Buffer.from(xmlHttp.responseText)));
}
};
xmlHttp.open("GET", 'assets/test.bson');
xmlHttp.send();

Related

JavaScript parsing nested JSON data

I'm creating a discord bot (using discord.js)
I'm writing a help command using pages in embeds, I have the pages working fine - however, the problem seems to come when trying to get the data from the JSON.
I have the JSON file setup in a larger config file, so it's nested inside of it.
Each command has it's name attached to it
{
"other-json-data": "other data",
"commands": {
"rule": {
"info": "This command gives the rules",
"expectedArgs": "<number>"
},
"invite": {
"info": "Invite new users",
"expectedArgs": "<no-args>"
},
"flip": {
"info": "Flip a coin",
"expectedArgs": "<no-args>"
}
},
"other-json-data": "other data"
}
I need to get the data from the commands area for each page.
I only have a integer input (from the page number), but I haven't got a clue how I would get the data from whatever command needs to be shown.
For something else in my project, I am using this to get the expectedArgs from the JSON object config.commands[arguments].expectedArgs, where the config is just a reference to the JSON file, this works perfectly fine. The arguments is a string input (i.e. rule), which returns whatever the info from that command.
However, would there be a way to get say the second one down (invite). I've tried config.commands[pageNumber].expectedArgs}, however, this doesn't seem to work. pageNumber would be an integer, so would it would get whatever value and then I could grab the expectedArgs.
You can get all keys from an object and select one using their index.
const json = {
"other-json-data": "other data",
"commands": {
"rule": {
"info": "This command gives the rules",
"expectedArgs": "<number>"
},
"invite": {
"info": "Invite new users",
"expectedArgs": "<no-args>"
},
"flip": {
"info": "Flip a coin",
"expectedArgs": "<no-args>"
}
},
"other-json-data": "other data"
}
const pageNumber = 1
// key will be a command name, e.g. 'invite'
const key = Object.keys(json.commands)[pageNumber]
const { expectedArgs } = json.commands[key]
console.log(`${key} expects ${expectedArgs}`)
Remember that indexes range starts at zero.

How do I import JSON file data into JS file and output the JSON data in JS console?

I have a list of data in my JSON file. I am trying to output certain strings and arrays from my JSON file via my JS. How do I go on about this? All these files are saved on my desktop.
I've tried Xhttp code. But I think I need some server going on for that, and I don't have that. Also, I'm pretty sure this should be possible without having to use a server?
PS: the json file is named: movie.json
JSON CODE
{
"movie": {
"name": "drive",
"year": "2011",
"people": {
"actors": [
{
"name": "ryan gosling"
},
{
"name": "cary mulligan"
},
{
"name": "bryan cranston"
}
]
}
}
}
JS CODE
function preload() {
var movie = load.JSON("movie.json");
}
function(movie) {
var movie = JSON.parse(movie);
console.log(movie[0].name);
console.log(movie[0].year);
console.log(movie[0].actors);
}();
drive, 2011, ryan gosling, cary mulligan, bryan cranston
var movie;
var http = new XMLHttpRequest();
xhhtp.open( "GET", "movie.json", true);
xhttp.onreadystatechange = function () {
if(http.readyState == 4 && http.status == 200) {
movie = JSON.parse(http.responseText);
}
}
http.send();
console.log(movie[0].name);
console.log(movie[0].year);
console.log(movie[0].actors);
I do not know if the code above will help you. Using XMLHttpRequest will help you fetch the json file then you can parse and sort into array. Note: you do not need a server to use XMLHttpRequest, if you have text editor like VSCode you can us it to run live HTML codes then you can get the full link to the JSON file you want to parse e.g localhost:9000/movie.json

How to convert JSON to YAML in javascript

I want to convert a json string to yaml format in javascript.I am trying my hand on google from last two days didnt found any solution or libraries.
There are answers available for java but not for javascript.
Suppose i have json string like this:
{
"json": [
"fat and rigid"
],
"yaml": [
"skinny and flexible"
],
"object": {
"array": [
{
"null_value": null
},
{
"boolean": true
},
{
"integer": 1
}
]
}
}
conver to yaml:
json:
- fat and rigid
yaml:
- skinny and flexible
object:
array:
- null_value:
- boolean: true
- integer: 1
There is a online converter http://www.json2yaml.com/ , but how to convert to it in javascript.
You can use yaml NPM package.
const YAML = require('yaml');
const jsonObject = {
version: "1.0.0",
dependencies: {
yaml: "^1.10.0"
},
package: {
exclude: [ ".idea/**", ".gitignore" ]
}
}
const doc = new YAML.Document();
doc.contents = jsonObject;
console.log(doc.toString());
Output
version: 1.0.0
dependencies:
yaml: ^1.10.0
package:
exclude:
- .idea/**
- .gitignore
Use the 'js-yaml' npm package! That's the one that is officially recognized by yaml.org. (If you want to be extra sure && this post is outdated, go check yaml.org yourself to see which package it recommends.) I initially used 'json2yaml' instead and got strange parsing behavior when converting json (as string) to yaml.
If someone still wants to convert JSON to YAML, you may use this JavaScript library:
https://www.npmjs.com/package/json2yaml
I tried to package the answer to a bash script.
#!/bin/bash
#convert-json-to-yaml.sh
if [[ "$1" == "" ]]; then
echo "You must provide a json file in argument i.e. ./convert-json-to-yaml.sh your_file.json"
exit 1
fi
jsonFile=$1
yamlFile="$1.yaml"
if [[ "$2" != "" ]]; then
yamlFile=$2
fi
python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' < ${jsonFile} > ${yamlFile}
Here is how to do it :)
import * as YAML from 'yaml';
const YAMLfile = YAML.stringify(JSONFILE);
And if you want to create a YAML file, you do it with de FS.
import * as fs from 'fs';
fs.writeFileSync('./fileName.yaml', YAMLfile);

Integrating PHP into JavaScript to fetch JSON

I installed PHP5 and Apache2 on an Linux vServer (8 Gb Ram ; 8 vCores) and I was scripting but I can't get rid of an error in my code. I'm trying to fetch JSON with integrated PHP but it doesn't work. Here is part of the source code:
function GameDetails(servername, serverurl, mapname, maxplayers, steamid, gamemode) {
var data = '<?php header("access-control-allow-origin: http://api.steampowered.com");$jsonData = file_get_contents('http://api.steampowered.com/ISteamUser/GetPlayerSummaries/v0002/?key=*REMOVEDREMOVEDREMOVED&steamids=76561197960435530');echo $jsonData;?>';
document.getElementById('server_name').innerHTML = jsonObj.response.players[0].personaname;
});
}
And here is the version after PHP Processing:
function GameDetails(servername, serverurl, mapname, maxplayers, steamid, gamemode) {
var data = '{
"response": {
"players": [
{
"steamid": "76561197960435530",
"communityvisibilitystate": 3,
"profilestate": 1,
"personaname": "Robin",
"lastlogoff": 1405647102,
"profileurl": "http://steamcommunity.com/id/robinwalker/",
"avatar": "http://media.steampowered.com/steamcommunity/public/images/avatars/f1/f1dd60a188883caf82d0cbfccfe6aba0af1732d4.jpg",
"avatarmedium": "http://media.steampowered.com/steamcommunity/public/images/avatars/f1/f1dd60a188883caf82d0cbfccfe6aba0af1732d4_medium.jpg",
"avatarfull": "http://media.steampowered.com/steamcommunity/public/images/avatars/f1/f1dd60a188883caf82d0cbfccfe6aba0af1732d4_full.jpg",
"personastate": 0,
"realname": "Robin Walker",
"primaryclanid": "103582791429521412",
"timecreated": 1063407589,
"personastateflags": 0,
"loccountrycode": "US",
"locstatecode": "WA",
"loccityid": 3961
}]
}
}';
alert data;
document.getElementById('server_name').innerHTML = data.response.players[0].personaname;
});
}
I know its a loading screen for a Garrys Mod Server. The Debug Console from Firefox says:
SyntaxError: unterminated string literal.
But it's just normal JSON. JSON examples from web works without errors.
var data = '{
There's your unterminated string literal.
You can't have a literal new line in the middle of a string literal in JS.
Later, you are trying to access data as an object, not a string, anyway. Get rid of the quotes around it.

how to write json files in javascript

Ok, so I am programming a web operating system using js. I am using JSON for the file system. I have looking online for tutorials on JSON stuff for about a week now, but I cannot find anything on writing JSON files from a web page. I need to create new objects in the file, not change existing ones. Here is my code so far:
{"/": {
"Users/": {
"Guest/": {
"bla.txt": {
"content":
"This is a test text file"
}
},
"Admin/": {
"html.html": {
"content":
"yo"
}
}
},
"bin/": {
"ls": {
"man": "Lists the contents of a directory a files<br/>Usage: ls"
},
"cd": {
"man": "Changes your directory<br/>Usage: cd <directory>"
},
"fun": {
"man": "outputs a word an amount of times<br/>Usage: fun <word> <times>"
},
"help": {
"man": "shows a list of commands<br/>Usage: help"
},
"clear": {
"man": "Clears the terminal<br/>Usage: clear"
},
"cat": {
"man": "prints content of a file<br/>Usage: cat <filename>"
}
},
"usr/": {
"bin/": {
},
"dev/": {
}
}
}}
I think the better solution is to stringify your JSON, encode with base64 encoding and then send it to a server-side script (a PHP page, for instance) which could save this file. See:
var json = JSON.stringify(myJson);
var encoded = btoa(json);
You can use ajax for sending:
var xhr = new XMLHttpRequest();
xhr.open('POST','myServerPage.php',true);
xhr.setRequestHeader('Content-type','application/x-www-form-urlencoded');
xhr.send('json=' + encoded);
And in the server-side:
$decoded = base64_decode($_POST['json'])
$jsonFile = fopen('myJson.json','w+');
fwrite($jsonFile,$decoded);
fclose($jsonFile);
I'd take off the "/"'s from the keys then could split on "/" and walk the tree by shifting values off the result. For example, the following code will create the full path if it doesn't already exist, but preserving the folder & contents if it does.
var fs = {
"bin": {
"mkdir": function(inPath) {
// Gets rid of the initial empty string due to starting /
var path = inPath.split("/").slice(1);
var curFolder = fs;
while(path.length) {
curFolder[path[0]] = curFolder[path[0]] || {};
curFolder = curFolder[path.shift()];
}
}
}
}
fs.bin.mkdir("/foo/bar");
console.log(JSON.stringify(fs, function(key,val) {
if(key === 'mkdir') return undefined;
return val;
}, 2));
Output:
{
"bin": {},
"foo": {
"bar": {}
}
}
As others have mentioned, rather than building the JSON object by hand with strings, to avoid syntax errors (and frustration), building it through code then using JSON.stringify to get the final result would likely be simpler.

Categories

Resources