Develop Reference

I am having trouble appending dynamic data to my JSON file

I have been having a bit of trouble appending new dynamic data to a JSON file. To sum up my project, I take in the projectName from an input form at the /new page.
My API is then using the node.js's fs module to create a new JSON file with which I can then append the new dynamic data upon subsequential requests to my form. The variables are 1) projectName (is taken in from my form), 2) activeUser (which is programmed in through an environmental variable), 3) is the date of the request which I am acquiring through a timestamp variable with this function:
const timestamp = (JSON.parse(JSON.stringify(new Date())));
All three of these variables seem to print correctly for 2 subsequent requests and then on the third form submission there seems to be no new data appending to the JSON file. However i am relatively new to node.js and I can't seem to figure out where I am messing this up.
This is my API
pages/api/demos/index.js
import dbConnect from '../../../lib/dbConnect';
import Demo from '../../../models/Demo';
import fs from 'fs';
export default async function handler(req, res) {
const {
query: { id },
method,
} = req
await dbConnect()
switch (method) {
case 'POST':
try {
//check if file exist
if (!fs.existsSync('projects.json')) {
//create new file if not exist
fs.closeSync(fs.openSync('projects.json', 'w'));
}
// read file
const timestamp = (JSON.parse(JSON.stringify(new Date())));
const newFileName = req.body.projectName;
const activeUser = process.env.ACTIVE_USERNAME;
const file = fs.readFileSync('projects.json')
const data = {
"projects": [
{
"username": activeUser,
"pages": [
{
"display": "routes",
"subpages": [
{
"date": timestamp,
"route": newFileName,
"display": newFileName
}
]
}
]
}
]
}
//check if file is empty
if (file.length == 0) {
//add data to json file
fs.writeFileSync("projects.json", JSON.stringify([data]))
} else {
//append data to jso file
const json = JSON.parse(file.toString())
//add json element to json object
json.push(data);
fs.appendFileSync("projects.json", JSON.stringify(data))
}
const demo = await Demo.create(
req.body
)
res.status(201).json({ success: true, data: demo })
} catch (error) {
res.status(400).json({ success: false })
}
break
default:
res.status(400).json({ success: false })
break
}
}
After the first form submission my JSON file projects.json looks like
[
{
"projects": [
{
"username": "projectmikey",
"pages": [
{
"display": "routes",
"subpages": [
{
"date": "2022-09-12T19:03:09.547Z",
"route": "1",
"display": "1"
}
]
}
]
}
]
}
]
and then after the 2nd form submission
[
{
"projects": [
{
"username": "projectmikey",
"pages": [
{
"display": "routes",
"subpages": [
{
"date": "2022-09-12T19:03:09.547Z",
"route": "1",
"display": "1"
}
]
}
]
}
]
}
]{
"projects": [
{
"username": "projectmikey",
"pages": [
{
"display": "routes",
"subpages": [
{
"date": "2022-09-12T19:03:24.466Z",
"route": "2",
"display": "2"
}
]
}
]
}
]
}
Oddly it seems to work for two form submissions and then the data stops appending to my file. This is after the third attempt, (no change to the file)
[
{
"projects": [
{
"username": "projectmikey",
"pages": [
{
"display": "routes",
"subpages": [
{
"date": "2022-09-12T19:03:09.547Z",
"route": "1",
"display": "1"
}
]
}
]
}
]
}
]{
"projects": [
{
"username": "projectmikey",
"pages": [
{
"display": "routes",
"subpages": [
{
"date": "2022-09-12T19:03:24.466Z",
"route": "2",
"display": "2"
}
]
}
]
}
]
}
It seems to stop working at all when I remove the pair of brackets around the initial JSON object. The line I am refering to is fs.writeFileSync("projects.json", JSON.stringify([data]))
I could really use another pair of eyes on this so I can see where I am messing this up! lol Thanks in advance for your time...

Although it feels like you are "appending" to the file, you are actually doing something more complicated.
e.g. before state:
[ "one", "two" ]
desired after-state:
[ "one", "two", "three" ]
Notice that you can't just append text to the before-state JSON because there's already that pesky ] terminating the whole object.
Some failed attempts might look like:
failed attempt to append another entire array
[ "one", "two" ][ "three" ]
This is invalid because there are two root objects.
failed attempt to append just the rest of the array
[ "one", "two" ], "three" ]
That's no good either. The ] at the end of the original file needs to be overwritten or removed, so there's no way to just append. I suppose technically you could seek to the position of the final ] and then continue writing an incomplete object from there. But this is very awkward to remove the final ] from the source and to remove the initial [ from the chunk you're trying to append. It's just a difficult approach.
What you actually want to do is:
read the entire JSON file
parse the JSON into a JavaScript object (or create an empty object if the file didn't exist)
Modify the JavaScript object as necessary (e.g. push into the array to add another element)
stringify the JavaScript object into new JSON
overwrite the entire file with the new JSON.
/* In Node.js:
const fs = require('fs');
try {
initialJSON = fs.readFileSync('example.json');
} catch (ignore) {
initialJSON = '[]';
}
*/
/* Mocked for this example: */
initialJSON = '["one","two"]';
// Common
obj = JSON.parse(initialJSON);
obj.push("three");
finalJSON = JSON.stringify(obj);
/* In Node.js:
fs.writeFileSync('example.json', finalJSON);
*/
/* Mocked for this example: */
console.log(finalJSON);

Delete / remove am empty array from object from javascript using mongoose

I have database collection, that looks like this , how to remove this empty array .
Initially I have object in this (HUE) array
Which looks like this
"HUE": [{
"chartId": "timeseries",
"name": "TS",
}]
, but after deleting the objects, it does not delete the empty array
{
"userId": "adam",
"charts": {
"HUE": [],
"Color": [{
"chartId": "one",
"name": "TS",
}]
}
}
P.S I only want to delete the HUE array when its empty
delChartObj.updateOne(
{ 'userId': userId },
{ $pull: query } // this line actually find the chartId and delete it
// after the above line, I actually want to check, if after del the object , array became empty, then delete the array too
, function (err, obj) {
if (err) { res.send.err }
res.status(200).send("Deleted Successfully");
});

db.collection.update({userId, 'charts.HUE': {$exists:true, $size:0}}, { $unset: { 'charts.HUE': 1 } })
It works -> https://mongoplayground.net/p/uqU7d0Kp2eL
Update: The question changed and this solution is not completely correct. Users have to ask with more details.

If you want to filter the array and then if empty to delete the array, you can do the bellow
db.collection.update(
{
"$expr" : {
"$eq" : [ "$userId", "adam" ]
}
},
[ {
"$addFields" : {
"charts.HUE" : {
"$filter" : {
"input" : "$charts.HUE",
"as" : "hue",
"cond" : {
"$ne" : [ "$$hue.chartId", "timeseries" ]
}
}
}
}
}, {
"$addFields" : {
"charts.HUE" : {
"$cond" : [ {
"$eq" : [ {
"$size" : "$charts.HUE"
}, 0 ]
}, "$$REMOVE", "$charts.HUE" ]
}
}
} ]
)
Test code here
Its pipeline update requires MongoDB >= 4.2
Pipeline updates are very powerful, but for simple updates they can be more complicated.
$$REMOVE is a system variable, if a field gets this value, its removed.
Here the field gets this value only if empty array.

Javascript/json keep in database and special chars

I have layout builder in React to keep data structure and text I use ImmutableJS objects.
Such structure with attributes as text or css styles is saved into database as JSON.
To make it JSON I using json-immutable library: serialize, deserialize functions.
After save in database I provide configuration for react components as javascript variables. For example my backend generate js file with variables or small part is printing directly in html code using script tag.
Data are JSON or decoded javascript.
The biggest problem I have with save special chars.
For example if someone set ' single quote in some attribute it is saved directly.
But when I print it in html code as
var myConfig = '{anyjson}';
when inside JSON is single quote parser throw error. The same with double quotes, & (ampersant) or any chars used in html code like <,/>
Single quote I replace to \' when I print it in html code.
But I think does exists any way to save keep all data in JSON and still they will easy to decode by deserialize function to parse JSON to ImmutableJS objects.
Code example
https://jsfiddle.net/jaroapp/2yzud6ua/2/
var structure = {
"__iterable":"Map",
"data":[
[
"entityMap",
{
"__iterable":"Map",
"data":[
[
"html_el_qb7tyhi",
{
"__iterable":"Map",
"data":[
[
"imported",
false
],
[
"path",
"html_el_qb7tyhi"
],
[
"componentData",
{
"__iterable":"Map",
"data":[
]
}
],
[
"draftjsObject",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"draftjs",
true
],
[
"data",
{
"__iterable":"OrderedMap",
"data":[
[
"text",
"B&B is the best company. It's my hope for new markets."
]
]
}
],
[
"chunk",
null
],
[
"style",
{
"__iterable":"OrderedMap",
"data":[
[
"background-image",
"url(\"/path/to/image.jpg\")"
]
]
}
],
[
"attr",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"runEditor",
false
],
[
"entityMap",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"type",
"div"
],
[
"key",
"html_el_qb7tyhi"
]
]
}
],
[
"html_el_2dgupn7",
{
"__iterable":"Map",
"data":[
[
"imported",
false
],
[
"path",
"html_el_2dgupn7"
],
[
"componentData",
{
"__iterable":"Map",
"data":[
]
}
],
[
"draftjsObject",
{
"entityMap":{
},
"blocks":[
{
"key":"3ia22",
"text":"Text saved with html inside",
"type":"unstyled",
"depth":0,
"inlineStyleRanges":[
],
"entityRanges":[
],
"data":{
}
}
]
}
],
[
"draftjs",
true
],
[
"data",
{
"__iterable":"OrderedMap",
"data":[
[
"text",
null
],
[
"html",
"<p class=\"md-block-unstyled\">Text saved with html inside</p>"
]
]
}
],
[
"chunk",
null
],
[
"style",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"attr",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"runEditor",
false
],
[
"entityMap",
{
"__iterable":"OrderedMap",
"data":[
]
}
],
[
"type",
"div"
],
[
"key",
"html_el_2dgupn7"
]
]
}
]
]
}
],
[
"containersMap",
{
"__iterable":"Map",
"data":[
]
}
],
[
"componentsMap",
{
"__iterable":"Map",
"data":[
[
"entityMap",
{
"__iterable":"OrderedMap",
"data":[
]
}
]
]
}
]
]
};
Such structure I set as parameter into ReactJS component.
If I set it as JSON and wrap in quotes then the browser throws an error. If I set it as JavaScript object into the React component I can't make ImmutableJS from this one, because this structure is read by this
https://www.npmjs.com/package/json-immutable library (I use the same to make JSON from Immutable JS to save it in database);
Thanks in advance for any hints.

I solved this problem. Short description, maybe will helpful for someone.
I get file with deserialize function form json-immutable package. And I modified function deserialize to
export function deserialize(json){
var options = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : {};
if(typeof json==='string'){
return JSON.parse(json, function(key, value, options){
return revive(key, value, options);
});
}else{
return JSON.parse(JSON.stringify(json), function(key, value, options){
return revive(key, value, options);
});
}
}
Function in package use JSON.parse for param. Maybe it's not the most elegant solution but I don't have time to find way to don't "re-json" object. Standard Object.keys and map return origin object.

How to modify json data in fiddler.

I am gettring one issue so I want to change the day in fiddler,
which after ajax request I am getting this json data.
{
"ROOT": {
"DATA": {
"I": [
[
"ABC",
"123"
],
[
"DEF",
"124"
],
[
"GHI",
"125"
]
]
}
}
}
Now in fiddler url I want to change this like below.
{
"ROOT": {
"DATA": {
"I": [
[
"DEF",
"124"
],
[
"GHI",
"125"
]
]
}
}
}
Is there any way I can change data in fiddler by clicking on my url

Yes, there is a way. Go to Fiddler, right click on the url and then go to Replay -> Reissue and Edit
Now on the right side go to Inspectors -> TextView. You will see the json data that you want to change. Do the changes and then click on Run To Completion to hit the same url with the CHANGED data.

How to store City, State and Country data related json in single area

I like to know how to construct or store json data for City, State and Country in a single variable ?
I have seen one but it only deal with single kind of data means CAR
{
"cars": {
"Nissan": [
{"model":"Sentra", "doors":4},
{"model":"Maxima", "doors":4}
],
"Ford": [
{"model":"Taurus", "doors":4},
{"model":"Escort", "doors":4}
]
}
}
My intention is to store different kind of json data in single area or in single variable.

What about this ?
{
"countries": [
{...}
],
"states" [
{...}
],
"cities": [
{...}
]
}

maybe this is a direct answer.
{
"Countries":[{
"CountryName":"Indonesia",
"States":[{
"StateName":"Bali",
"Cities":["Denpasar",
"Kuta",
"Tuban"
]},
{
"StateName":"Jakarta",
"Cities":[
"Bandung",
"Tanggerang"
]
}
]
}]}
check the file here
jsonforcountriesstatecities