I am using express to get data from a public API and use the data in my front end.
This is my characters route which works fine on certain Public API URLs, but most that I try ends up in an unexpected end of input error.
I am also getting an
Unexpected token , in JSON at position 48. How can this happen when it appears to be valid JSON?
const express = require('express'); // Web Framework
const https = require('https');
const router = express.Router();
const api = 'https://www.cheapshark.com/api/1.0/games?title=batman&steamAppID=35140&limit=60&exact=0';
router.get("/", function(req, res) {
https.get(api, (response) => {
console.log(response.statusCode);
response.on('data', (d) => {
try{
const data = JSON.parse(d);
console.log(data);
res.send(data);
} catch (err) {
console.log(err);
}
})
// res.send("Running")
})
})
module.exports = router;
This is my index.js which uses the character route
const express = require('express'); // Web Framework
const app = express();
const PORT = 3000;
const charactersRoute = require('./routes/characters');
//Characters Route
app.use('/characters', charactersRoute)
app.listen(PORT, function(err) {
if(err) console.log(err);
console.log(`Server is listening on port ${PORT}`)
})
This is the JSON I am trying to parse. I validated this is valid JSON on those JSON validation sites.
[
{
"gameID": "146",
"steamAppID": "35140",
"cheapest": "14.95",
"cheapestDealID": "LNCZ5EicmEMiwyfYVw%2FNdGPos9V7MzoPId2UuwaBqvA%3D",
"external": "Batman: Arkham Asylum Game of the Year Edition",
"internalName": "BATMANARKHAMASYLUMGAMEOFTHEYEAREDITION",
"thumb": "https://cdn.cloudflare.steamstatic.com/steam/apps/35140/capsule_sm_120.jpg?t=1634156906"
}
]
https response is a stream that returns chunks of data in the data event, so you need to concatenate data there (store buffer raw data into an array), and parse it in the .end event, when the response is finished, and then you can use res.json to send it to the consumer:
try this:
const express = require('express'); // Web Framework
const https = require('https');
const router = express.Router();
const api = 'https://www.cheapshark.com/api/1.0/games?title=batman&steamAppID=35140&limit=60&exact=0';
router.get("/", function(req, res) {
https.get(api, (response) => {
console.log(response.statusCode);
const resData = [];
response.on('data', (chunk) => {
resData.push(chunk);
});
response.on('end', function() {
try {
const data = JSON.parse(Buffer.concat(resData).toString());
console.log(data);
res.json(data);
} catch (err) {
console.log(err);
}
});
})
});
module.exports = router;
Related
I've been trying to practice with some HTTP requests, and specifically I want to send a POST request, with data taken from an field, and sent via fetch() to a url on my localhost thats set up with express. From there i want to somehow get the response back and have that be displayed on my HTML doc.
However, I've ran into a real head scratcher when it comes to getting response.json() to be anything other than undefined.
here's my frontend script:
const url = "/result";
const inputField = document.querySelector("#write");
const submitButton = document.querySelector("#submit");
const responseField = document.querySelector("#text-goes-here");
const postText = async () => {
const text = inputField.value;
const data = JSON.stringify({ destination: text });
try {
const response = await fetch(url, {
method: "POST",
body: data,
headers: {
"Content-type": "application/json",
},
});
if (response.ok === true) {
const jsonResponse = await response.json();
responseField.innerHTML = jsonResponse;
}
} catch (error) {
console.log(error);
}
};
const displayText = (event) => {
event.preventDefault();
while (responseField.firstChild) {
responseField.removeChild(responseField.firstChild);
}
postText();
};
submitButton.addEventListener("click", displayText);
and my server script:
const express = require("express");
const bodyParser = require("body-parser");
const read = require('fs');
const router = express.Router();
const app = express();
const port = 3000;
app.use(express.static(__dirname + "/public"));
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.get("/", (req, res) => {
res.sendFile("public/index.html");
})
router.post("/result", (req, res) => {
console.log(req.body);
res.send();
});
app.use("/", router);
app.listen(port, () => {
console.log(`Server running at port: ${port}`)
});
I did some digging in the dev console and found that (response.ok) is in fact "true", yet it errors out into the catch statement saying "SyntaxError: Unexpected end of JSON input
at postText (script.js:23)"
which is this line exactly:
const jsonResponse = await response.json();
can anyone shed any light on what i'm doing wrong here? I'm at a loss at this point
This error means that you're trying to parse something that is not a valid JSON object.
"SyntaxError: Unexpected end of JSON input at postText (script.js:23)"
Which is true, because the response you're sending back to the frontend is not a JSON.
router.post("/result", (req, res) => {
console.log(req.body);
// The response is not a valid JSON object
res.send();
});
You can change res.send() to res.json() and give it a valid object.
res.json({ name:"John", age:30, car:null })
Sorry if I don't post the correct details, this is my first hands-on project after going through online tutorials.
I'm using React, node with axios to build a web app that captures status(available, in a meeting, lunch etc) and the time spent on each status.
The app works fine, it captures and writes the data onto the backend(JSON) however, I keep getting this error on the console.
POST https://localhost:5000/write net::ERR_CONNECTION_RESET
Uncaught (in promise) ERROR: Network Error
I've tried to look for a solution but can't find one that is similar to the tech-stack I used. Also, my lack of sufficient knowledge don't help either.
Any lead or read or solution will help.
pasting my code below:
My frontend code to push data into JSON file
const saveJson = (posts) => {
//api URL //end point from node server / express server
const url = "http://localhost:5000/write";
axios.post(url, posts).then((response) => {
//console.log(response);
}); };
The server.js code
const express = require("express");
const bodyParser = require("body-parser");
//calling packages
const fs = require("fs");
const morgan = require("morgan");
const cors = require("cors");
//Declare app
const app = express();
const port = 5000;
//middlewares
app.use(bodyParser.json());
app.use(morgan("dev"));
app.use(cors());
//default route for server
app.get("/", (req, res) =>
res.status(200).send({
message: "Server is running...",
})
);
const WriteTextToFileAsync = async (contentToWrite) => {
fs.writeFile("./src/data.json", contentToWrite, (err) => {
console.log(contenToWrite);
if (err) {
console.log(err);
} else {
console.log("Done writing to file...");
// res.json({ msg: "success" });
}
});
};
//Declare timerow/write route to accept incoming require with data
app.post("/write", async (req, res, next) => {
//take the body from incoming requestby using req.body and conver it into string
const requestContent = JSON.stringify(req.body);
await WriteTextToFileAsync(requestContent);
});
//404 route for server
app.use((req, res, next) =>
res.status(404).send({
message: "Could not find specified route requested...!",
})
);
//run server
app.listen(port, () => {
console.log(
`!!! server is running
!!! Listening for incoming requests on port ${port}
!!! http://localhost:5000
`
);
});
[`const express = require('express');
const app = express();
const https = require('https');
const url = "https://api.thevirustracker.com/free-api?countryTimeline=US";
app.get("/", (req ,res) => {
res.send("Server is Running")
https.get(url, (response) => {
response.on("data", (data) => {
const TimelineData = JSON.parse(data);
console.log(TimelineData);
})
})
})
app.listen(3000, ()=>console.log("Server is Running 0n 5000"));`]1
const express = require('express');
const app = express();
const https = require('https');
const url = "https://api.thevirustracker.com/free-api?countryTimeline=US";
app.get("/", (req ,res) => {
res.send("Server is Running")
https.get(url, (response) => {
response.on("data", (data) => {
const TimelineData = JSON.parse(data);
console.log(TimelineData);
})
})
})
app.listen(3000, ()=>console.log("Server is Running 0n 5000"));
To deliver large data in an effective manner API send data in chunk/stream format. and to receive each chunk it triggers the 'data' event and in your case, it might be possible that API sends data in chunk format. and it will not send you complete data in a single event.
Let's assume the complete response of your API is :
{ name: 'bella', age: 34, count: 40138 }
And API send it in 2 chunks :
Chunk1: { name: 'bella', age: 34, count: 4013
Chunk2: 8 }
In that case Json.Parse() on Chunk1 or Chunk2 will not work and threw an exception.
To deal with this problem you need to listen to the 'end' event and capture data from the'data' and parse it in the 'end' event.
Use the below code:
const express = require('express');
const app = express();
const https = require('https');
const url = "https://archive.org/advancedsearch.php?q=subject:google+sheets&output=json";
app.get("/", (req, res) => {
res.send("Server is Running")
https.get(url, (response) => {
var responseData = '';
response.on("data", (dataChunk) => {
responseData += dataChunk;
})
response.on('end', () => {
const TimelineData = JSON.parse(responseData);
console.log(TimelineData);
});
}).on('error', (e) => {
console.error(e);
});
})
app.listen(5000, () => console.log("Server is Running 0n 5000"));
The "data" event can be fired multiple times: https://nodejs.org/api/http.html#http_class_http_clientrequest
You have to listen for the "end" event and concat all chunks from the "data" event togehter for the full body response.
const express = require('express');
const app = express();
const https = require('https');
const url = "https://api.thevirustracker.com/free-api?countryTimeline=US";
app.get("/", (req, res) => {
res.send("Server is Running")
https.get(url, (response) => {
const chunks = [];
response.on("data", (data) => {
chunks.push(data);
})
response.on("end", () => {
let size = chunks.reduce((prev, cur) => {
return prev + cur.length;
}, 0);
let data = Buffer.concat(chunks, size).toString();
console.log(JSON.parse(data))
});
})
})
app.listen(3000, () => console.log("Server is Running 0n 5000"));
why are you using https?
replace https with http and run it again.
const express = require('express');
const app = express();
const http = require('http');
const url = "https://api.thevirustracker.com/free-api?countryTimeline=US";
app.get("/", (req ,res) => {
res.send("Server is Running")
http.get(url, (response) => {
response.on("data", (data) => {
const TimelineData = JSON.parse(data);
console.log(TimelineData);
})
})
})
const express = require('express')
const app = express()
const port = 3000
app.post('/', (req, res) => {
res.send('Hello World!")
})
app.listen(port, () => {
console.log('server running')
})
When you run the program in nodejs, open the brower and type http://localhost:3000. The output will be....
Listen for 'end ' the problem will be resolved
Try importing all the dependencies. Importing is better than requiring because you can selectively load only the pieces you need. Also in package.json file add "type":"module" before scripts. The days of const something= require('something') are a thing of the past now because of new ESM modules.
import express from 'express';
import https from 'https';
const app=express();
const port=3000;
In package.json file
"name": "restApiWithNode",
"version": "1.0.0",
"description": "",
"main": "index.js",
"type": "module",
Read this article for clarity https://formidable.com/blog/2021/node-esm-and-exports/
This is my index.js file and i think i have placed the routes after installing bodyParser but still getting the syntax error.
const express = require('express'); //Framework to build server side application
const morgan = require('morgan'); //Logging the nodejs requests
const bodyParser = require('body-parser'); //To get the JSON data
const urls = require('./db/urls');
const app = express();
app.use(morgan('tiny'));
app.use(bodyParser.json());
app.use(express.static('./public')); //If a request comes with '/' check if file is in there if it is then serve it up.
// app.get('/', (req, res) => {
// res.send('Hello, World !!');
// });
app.post('/api/shorty', async (req, res) => {
console.log(req.body);
try {
const url = await urls.create(req.body); //Passing the body data which is JSON to create function
res.json(url);
} catch (error) {
res.status(500);
res.json(error)
}
});
const port = process.env.PORT || 5000;
app.listen(port, () => {
console.log(`listening on port ${port}`);
});
This is the urls.js file,I am not getting where have i messed up to make Syntax.JSON error in this file.
const db = require('./connection');
const Joi = require('joi');//Schema validation
const urls = db.get('urls');
const schema = Joi.object().keys({
name : Joi.string().token().min(1).max(100).required(),
url : Joi.string().uri({
scheme: [
/https?/ //get http 's' is optional
]
}).required()
}).with('name','url');
//almostShorty = {
// name = ,
// url =
// }
function create(almostShorty){
const result = Joi.validate(almostShorty, schema);
if(result.error === null){
return urls.insert(almostShorty);//Inserting the object in the Data Base.
}else{
return Promise.reject(result.error);
}
};
module.exports = {create};//Exporting the create function.
I'm trying to convert the .xls table to .csv row.
I found the library that helps with such operation, its name XLSX
As the first step, I encode .xls table to base64 format.
Now I'm trying to send the post request with Postman (with base64 code as the body of the request)
But when I'm trying to console.log my req.body, I'm receiving undefined. Can you tell me where I could make a mistake in my app?
my app.js:
const express = require('express');
const dataConverter = require('./inputDataConverter');
const { errorHandler } = require('./../error/error-handler');
const app = express();
const port = process.env.PORT || 5000;
const domain = process.env.DOMAIN || '0.0.0.0';
app.post('/convert', dataConverter);
app.all('*', (req, res, next) => next('Invalid request'));
app.use(errorHandler);
app.listen(port, () => {
console.log(`Microservice runs on http://${domain}:${port}`);
});
inputDataConverter.js:
const XLSX = require('xlsx');
module.exports = (req, res, next) => {
console.log('First console.log ' + req.body);
const getWorkbook = (data) => {
let wb = null;
if (!data.type || data.type === 'base64') {
wb = XLSX.read(data.body, { type: 'base64' });
}
console.log('everything is working');
return wb;
};
const requestData = req.body;
console.log(requestData);
getWorkbook(requestData);
};
As Molda said
You may need body parser
https://www.npmjs.com/package/body-parser
On new version actually body-parser is embedded to express so you can use that like:
app.use(express.json())
I really suggest you use express-generator package to generate base Express application
https://www.npmjs.com/package/express-generator
You may find something you may not need
just strip that out