Convert buffered file data to Json object: express-fileupload - javascript

I am sending a JSON file over http. The file is available on req.files using the express-fileupload middleware. I am getting the file as a buffered data. I want to convert the file to a JSON object.
app.post('/start', function(req, res){
if(!req.files.seed)
res.status(400).send("Please upload the seed file");
var file = req.files.seed;
var obj = //Convert the file to JSON object and send it to create instance;
instance.createInstance(obj);
res.status(200).send("Started....");
});
When printed, the file looks like something this
{ name: 'seed.json',
data: <Buffer 7b 0d 0a 09 22 61 72 72 61 79 22 3a 20 5b 20 31 2c 20 31 20 5d 2 c 0d 0a 09 22 72 65 63 75 72 72 65 6e 63 65 22 3a 20 7b 0d 0a 09 09 22 73 65 63 6f 6e ... >,
encoding: '7bit',
mimetype: 'application/json',
mv: [Function: mv] }
I tried using JSON.parse(file) but SyntaxError: Unexpected token o in JSON at position 1 pops up.
I also tried using converting it to a string using
var text = file.toString(file,'utf8')
var obj = JSON.parse(text)
but this also doesn't seem to work. The properties of this objects, when accessed are undefined.
The JSON file structure.
{
"array": [ 1, 1 ],
"recurrence": {
"second": 50,
"minute": null,
"hour": null,
"dayOfweek": null
},
"campaign": {
"sender": "StartUp India Yatra",
"email": "fashion#getposhaq.com",
"subject": "{Invitation} StartUp India Yatra Chapter",
"title": "StartUp India Yatra Campaign"
},
"condition": {
"open": {
"greaterThanEqual": 1,
"lessThan": 2
},
"campaignSummary": null
},
"textPath": "../template.txt",
"htmlPath": "../template.html",
"path": "../emailer/index.js"
"retailerId": "4"
}

Given what you presented in the debug, your encoding is not utf8 but 7bit. So for a proper decoding you will need to change a bit your code.
var file = req.files.seed;
var obj = JSON.parse(file.data.toString('ascii'));
// ... do your creation logic
Any way you can play with the utf8, ascii econdings to see if you do not have JSON.parse problems.

Related

How to convert buffer to JSON?

I am getting this raw output from Twitter API stream
<Buffer 7b 22 64 61 74 61 22 3a 7b 22 65 64 69 74 5f 68 69 73 74 6f 72 79 5f 74 77 65 65 74 5f 69 64 73 22 3a 5b 22 31 35 38 38 32 31 37 34 39 32 37 34 30 30 ... 349 more bytes>
I am trying to convert it to JSON like this
const obj = JSON.parse(data);
console.log(obj)
But I am getting this error
SyntaxError: Unexpected end of JSON input
at JSON.parse (<anonymous>)
at Request.<anonymous> (/home/scooprank/Github/scooprank/tweet_listener/server.js:63:26)
at Request.emit (node:events:513:28)
at IncomingMessage.<anonymous> (/home/scooprank/Github/scooprank/node_modules/request/request.js:1073:12)
at IncomingMessage.emit (node:events:513:28)
at addChunk (node:internal/streams/readable:315:12)
at readableAddChunk (node:internal/streams/readable:289:9)
at IncomingMessage.Readable.push (node:internal/streams/readable:228:10)
at HTTPParser.parserOnBody (node:_http_common:140:24)
at TLSSocket.socketOnData (node:_http_client:521:22)
How do I resolve this?
assign the buffer to a variable, them use the method of the buffer to transfer to a JSON, something like this
let url = 'something'
let obj: Buffer = Buffer.from(url)
let obj2 = obj.toJSON()

iam getting buffer and hex codes [duplicate]

This question already has answers here:
Why does Node.js' fs.readFile() return a buffer instead of string?
(8 answers)
Closed 12 months ago.
var fs = require('fs');
fs.readFile('TestFile.txt', function (err, data) {
if (err) throw err;
console.log(data);
});
//TestFile.txt This is test file to test fs module of Node.js
Iam getting buffer and hex codes in place of console data
<Buffer 54 68 69 73 20 69 73 20 74 65 73 74 20 66 69 6c 65 20 74 6f 20 74 65 73 74 20 66 73 20 6d 6f 64 75 6c 65 20 6f 66 20 4e 6f 64 65 2e 6a 73>
According to the documentation :
If no encoding is specified (using options.encoding), the data is returned as a <Buffer> object. Otherwise, the data will be a string.
Solution : add an encoding (and read the doc! :)
fs.readFile('TestFile.txt', { encoding : 'utf8' } , function()...
By converting data to String(data) you will get text

Render PDF from buffer using Node.js and JavaScript

I have my files object on back-end:
[{ Id: 3,
UserId: 7,
FileType: 'application/pdf',
FileContent:
<Buffer 25 50 44 46 2d 31 2e 33 0d 0a 25 e2 e3 cf d3 0d 0a 0d 0a 31 20 30 20 6f 62 6a 0d 0a 3c 3c 0d 0a 2f 54 79 70 65 20 2f 43 61 74 61 6c 6f 67 0d 0a 2f 4f ... >,
FileName: '1',
UserUploadId: 7 },
...]
I am sending this object to the view:
res.render('dashboard/files/index',{'title': 'My files', 'my_files' : files})
On HTML I am rendering with handlebarsjs a table containing a row per file and a button that executes a function receiving as unique parameter the file Id
<table class="table">
<thead class="thead-dark">
<tr>
<th scope="col">UserId</th>
<th scope="col">View</th>
</tr>
</thead>
<tbody>
{{#each my_files}}
<tr>
<td>{{this.UserId}}</td>
<td>
<button onclick="viewPdf({{this.Id}})"></button>
</td>
</tr>
{{/each}}
</tbody>
</table>
In the same document, with JavaScript, I am trying to get the FileContent property of the object, but for this I need to find the file in the array my_files searching by Id parameter.
<script>
function viewPdf(Id){
var found_file = my_files.find(function(element) {
return element.Id == Id;
});
console.log(found_file)
}
</script>
But I am getting this error output:
Uncaught ReferenceError: my_files is not defined
at viewPdf (......:465)
at HTMLButtonElement.onclick
So,
JavaScript running server and browser side is different. You can't find a declared server side variable in the browser, vice versa.
A way to shared variable between server and client :
function viewPdf(Id){
$.ajax('/getFiles', {
success:function(my_files){
// put logic here
}
})
}

AJAX send data to Node.js server

I am using AJAX POST data to server which use Node.js code. Very briefly, two file in this test project. Here is main.html:
<!DOCTYPE HTML>
<html>
<head>
</head>
<body>
<script>
var xhr = new XMLHttpRequest();
xhr.open('POST', '/', true);
xhr.send('hello');
</script>
</body>
</html>
Here the server code:
const http = require('http');
http.createServer(function(req,res) {
console.log(req.body);
}).listen(3000);
Perhaps,you have known that server will console.log() 'undefined'.
So the question is why it's 'undefined'? How to get AJAX data on server?
I know other solution according to this question.
Could you help me any other way to get data conveniently?
You've created server allright, but there's two problems:
Connection endpoint. Your AJAX is POST, so you need to parse POST requests correctly (not exactly solves your issue, but you need to distinguish request methods).
Node.js is not parsing your POST body by default, you need to tell it how it's done, by using querystring module, for example.
Combining this together:
var qs = require('querystring');
http.createServer(function(req, res) {
if (request.method == 'POST') {
var body = '';
request.on('data', function(data) {
body += data;
});
request.on('end', function() {
var post = qs.parse(body);
// Use your POST here
console.log(post);
});
}
}).listen(3000);
JSON data - cleaner solution:
You need to encode your AJAX data with JSON, and than parse it on the server side like this:
http.createServer(function(req,res) {
if (req.method == 'POST') {
var jsonPost = '';
req.on('data', function(data) {
jsonPost += data;
});
req.on('end', function() {
var post = JSON.parse(jsonPost);
// Use your POST here
console.log(post);
});
}
}).listen(3000);
You'd be even better off using Express.js framework for that, with it's bodyParser module.
UPDATE - How chunks are buffered:
Consider simple example - send in xhr.send() a LARGE amount of text, that exceeds Content-Length. Than do the following on data event:
req.on('data', function(data) {
console.log(data);
body += data;
});
You will see something like:
<Buffer 0a 0a 4c 6f 72 65 6d 20 69 70 73 75 6d 20 64 6f 6c 6f 72 20 73 69 74 20 61 6d 65 74 2c 20 63 6f 6e 73 65 63 74 65 74 75 72 20 61 64 69 70 69 73 63 69 ... >
<Buffer 65 74 20 71 75 61 6d 20 63 6f 6e 67 75 65 20 6c 6f 62 6f 72 74 69 73 2e 20 50 65 6c 6c 65 6e 74 65 73 71 75 65 20 74 65 6d 70 75 73 20 75 6c 6c 61 6d ... >
<Buffer 61 2e 20 56 69 76 61 6d 75 73 20 76 69 74 61 65 20 61 6e 74 65 20 6d 65 74 75 73 2e 20 4d 61 75 72 69 73 20 71 75 69 73 20 61 6c 69 71 75 65 74 20 65 ... >
That shows that the data received in chunks on data event. Only on end event you will get the whole data sent (if you've aggregated it before that). Node.js doesn't handle this, that's why you need third-party modules.
That is - you cannot just get req.body of the request, as it wasn't set at all.

Parsing string to JSON using node gives unexpected token, validator says ok

I have the following string that I would like to parse to JSON :
{
"STATUS": [
{
"STATUS": "S",
"When": 1394044643,
"Code": 17,
"Msg": "GPU0",
"Description": "cgminer 3.7.3"
}
],
"GPU": [
{
"GPU": 0,
"Enabled": "Y",
"Status": "Alive",
"Temperature": 70,
"Fan Speed": 3089,
"Fan Percent": 70,
"GPU Clock": 1180,
"Memory Clock": 1500,
"GPU Voltage": 1.206,
"GPU Activity": 99,
"Powertune": 20,
"MHS av": 0.4999,
"MHS 5s": 0.5009,
"Accepted": 4335,
"Rejected": 7,
"Hardware Errors": 0,
"Utility": 27.8007,
"Intensity": "0",
"Last Share Pool": 0,
"Last Share Time": 1394044643,
"Total MH": 4676.7258,
"Diff1 Work": 69436,
"Difficulty Accepted": 69360,
"Difficulty Rejected": 112,
"Last Share Difficulty": 16,
"Last Valid Work": 1394044643,
"Device Hardware%": 0,
"Device Rejected%": 0.1613,
"Device Elapsed": 9356
}
],
"id": 1
}
When I use e.g. http://jsonlint.com/ it says that the JSON is correct but when I use in node.js:
console.log(JSON.parse(data.toString()));
I get the following :
undefined:1
e Hardware%":0.0000,"Device Rejected%":0.1570,"Device Elapsed":9554}],"id":1}
^
SyntaxError: Unexpected token
Any clue what am I doing wrong here ?
EDIT
The data is coming as ByteStream :
.on('data',function(data){
console.log(data.toString());
console.log();
console.log(data);
console.log();
console.log("data "+ data.GPU);
//...
//...
{"STATUS":[{"STATUS":"S","When":1394045650,"Code":17,"Msg":"GPU0","Description":"cgminer 3.7.3"}],"GPU":[{"GPU":0,"Enabled":"Y","Status":"Alive","Temperature":70.00,"Fan Speed":3090,"Fan Percent":70,"GPU Clock":1180,"Memory Clock":1500,"GPU Voltage":1.206,"GPU Activity":99,"Powertune":20,"MHS av":0.4999,"MHS 5s":0.5007,"Accepted":4841,"Rejected":8,"Hardware Errors":0,"Utility":28.0261,"Intensity":"0","Last Share Pool":0,"Last Share Time":1394045638,"Total MH":5181.3734,"Diff1 Work":77548,"Difficulty Accepted":77456.00000000,"Difficulty Rejected":128.00000000,"Last Share Difficulty":16.00000000,"Last Valid Work":1394045638,"Device Hardware%":0.0000,"Device Rejected%":0.1651,"Device Elapsed":10364}],"id":1}
<Buffer 7b 22 53 54 41 54 55 53 22 3a 5b 7b 22 53 54 41 54 55 53 22 3a 22 53 22 2c 22 57 68 65 6e 22 3a 31 33 39 34 30 34 35 32 34 38 2c 22 43 6f 64 65 22 3a 31 ...>
data undefined
EDIT
When I do this :
console.log(data.toString());
console.log(JSON.stringify(data.toString()));
I get the following result :
"{"STATUS":[{"STATUS":"S","When":1394046864,"Code":17,"Msg":"GPU0","Description":"cgminer 3.7.3"}],"GPU":[{"GPU":0,"Enabled":"Y","Status":"Alive","Temperature":70.00,"Fan Speed":3087,"Fan Percent":70,"GPU Clock":1180,"Memory Clock":1500,"GPU Voltage":1.206,"GPU Activity":99,"Powertune":20,"MHS av":0.5000,"MHS 5s":0.5016,"Accepted":5396,"Rejected":8,"Hardware Errors":0,"Utility":27.9597,"Intensity":"0","Last Share Pool":0,"Last Share Time":1394046864,"Total MH":5789.2352,"Diff1 Work":86428,"Difficulty Accepted":86336.00000000,"Difficulty Rejected":128.00000000,"Last Share Difficulty":16.00000000,"Last Valid Work
":1394046864,"Device Hardware%":0.0000,"Device Rejected%":0.1481,"Device Elapsed":11580}],"id":1}"
"{\"STATUS\":[{\"STATUS\":\"S\",\"When\":1394046864,\"Code\":17,\"Msg\":\"GPU0\",\"Description\":\"cgminer 3.7.3\"}],\"GPU\":[{\"GPU\":0,\"Enabled\":\"Y\",\"Status\":\"Alive\",\"Temperature\":70.00,\"Fan Sp
eed\":3087,\"Fan Percent\":70,\"GPU Clock\":1180,\"Memory Clock\":1500,\"GPU Voltage\":1.206,\"GPU Activity\":99,\"Powertune\":20,\"MHS av\":0.5000,\"MHS 5s\":0.5016,\"Accepted\":5396,\"Rejected\":8,\"Hardw
are Errors\":0,\"Utility\":27.9597,\"Intensity\":\"0\",\"Last Share Pool\":0,\"Last Share Time\":1394046864,\"Total MH\":5789.2352,\"Diff1 Work\":86428,\"Difficulty Accepted\":86336.00000000,\"Difficulty Re
jected\":128.00000000,\"Last Share Difficulty\":16.00000000,\"Last Valid Work\":1394046864,\"Device Hardware%\":0.0000,\"Device Rejected%\":0.1481,\"Device Elapsed\":11580}],\"id\":1}\u0000"
Notice the last unicode character of the second message \u0000 what can I do with it?
The problem was with the terminating null character. After removing it I can now parse the string no problemo ( post more efficient way if you have it)
var re = /\0/g;
str = data.toString().replace(re, "");
var o = JSON.parse(str);
console.log(o);
or
var str = data.toString().slice(0, - 1);
or work directly on bytes in Buffer
var buf = data.slice(0,data.length-1);
console.log(JSON.parse(buf.toString()));
I copied your console log output for data.toString() and and as #patryk pointed out, there is a unicode character \u0000 at the end of the string. That might be causing the problem. I removed the trailing character, assigned the string to a variable and created a buffer out of it and everything is hunky dory.
var a = '{"STATUS":[{"STATUS":"S","When":1394045650,"Code":17,"Msg":"GPU0","Description":"cgminer 3.7.3"}],"GPU":[{"GPU":0,"Enabled":"Y","Status":"Alive","Temperature":70.00,"Fan Speed":3090,"Fan Percent":70,"GPU Clock":1180,"Memory Clock":1500,"GPU Voltage":1.206,"GPU Activity":99,"Powertune":20,"MHS av":0.4999,"MHS 5s":0.5007,"Accepted":4841,"Rejected":8,"Hardware Errors":0,"Utility":28.0261,"Intensity":"0","Last Share Pool":0,"Last Share Time":1394045638,"Total MH":5181.3734,"Diff1 Work":77548,"Difficulty Accepted":77456.00000000,"Difficulty Rejected":128.00000000,"Last Share Difficulty":16.00000000,"Last Valid Work":1394045638,"Device Hardware%":0.0000,"Device Rejected%":0.1651,"Device Elapsed":10364}],"id":1}';
var b = new Buffer(a);
console.log(JSON.parse(b.toString());
output -
{ STATUS:
[ { STATUS: 'S',
When: 1394045650,
Code: 17,
Msg: 'GPU0',
Description: 'cgminer 3.7.3' } ],
GPU:
[ { GPU: 0,
Enabled: 'Y',
Status: 'Alive',
Temperature: 70,
'Fan Speed': 3090,
'Fan Percent': 70,
'GPU Clock': 1180,
'Memory Clock': 1500,
'GPU Voltage': 1.206,
'GPU Activity': 99,
Powertune: 20,
'MHS av': 0.4999,
'MHS 5s': 0.5007,
Accepted: 4841,
Rejected: 8,
'Hardware Errors': 0,
Utility: 28.0261,
Intensity: '0',
'Last Share Pool': 0,
'Last Share Time': 1394045638,
'Total MH': 5181.3734,
'Diff1 Work': 77548,
'Difficulty Accepted': 77456,
'Difficulty Rejected': 128,
'Last Share Difficulty': 16,
'Last Valid Work': 1394045638,
'Device Hardware%': 0,
'Device Rejected%': 0.1651,
'Device Elapsed': 10364 } ],
id: 1 }

Categories

Resources