This question already has answers here:
Why does Node.js' fs.readFile() return a buffer instead of string?
(8 answers)
Closed 12 months ago.
var fs = require('fs');
fs.readFile('TestFile.txt', function (err, data) {
if (err) throw err;
console.log(data);
});
//TestFile.txt This is test file to test fs module of Node.js
Iam getting buffer and hex codes in place of console data
<Buffer 54 68 69 73 20 69 73 20 74 65 73 74 20 66 69 6c 65 20 74 6f 20 74 65 73 74 20 66 73 20 6d 6f 64 75 6c 65 20 6f 66 20 4e 6f 64 65 2e 6a 73>
According to the documentation :
If no encoding is specified (using options.encoding), the data is returned as a <Buffer> object. Otherwise, the data will be a string.
Solution : add an encoding (and read the doc! :)
fs.readFile('TestFile.txt', { encoding : 'utf8' } , function()...
By converting data to String(data) you will get text
Related
Before marking this question as a duplicate , I must state that I am aware of the other posts that transform a buffer in an array of buffers (I have been trying virtually every thread's comments on this , but is not what Im looking for).
I am using Node's child process to run a python script .The script returns a list in the form of :
[["a","21","67"],["b","22","69"]]
As per the Node child process docs , the result is UTF8 encoded. The problem is that whenever I try with textDecoder :
const response=new TextDecoder(data).decode(data);
or with regex:
var response = data.toString()
var ressult= a.match(/\d+/g);
None of them (and many variations of them that I have tried) work as I expect. Ultimately , I just want to get a javascript array ( [["a","21","67"],["b","22","69"]] ) so that I can perform functions on it(or , in other words, the list returned from the python script itself).Thank you !
Edit:
This is my child process:
const { spawn } = require('child_process');
const pyProg= spawn(`${srcDirectory}/venv/Scripts/python.exe`,[`${srcDirectory}/optimization/optimizationFunction.py`]);
pyProg.stdout.on('data', function(data) {
console.log(data.toString())
});
And this is what is logged by data.toString():
[['r1', '1e45200a-f990-4fda-a76b-b00fa05d9f53', 200.0], ['r2', '1e45200a-f990-4fda-a76b-b00fa05d9f53', 199.9999999999999], ['r1', '1ea3890a-f990-4fda-a76b-b00fa05d9f53', 0.0], ['r2', '1ea3890a-f990-4fda-a76b-b00fa05d9f53', 0.0]]
This is what console.log(data) logs:
<Buffer 53 65 74 20 70 61 72 61 6d 65 74 65 72 20 55 73 65 72 6e 61 6d 65 0d 0a>
<Buffer 41 63 61 64 65 6d 69 63 20 6c 69 63 65 6e 73 65 20 2d 20 66 6f 72 20 6e 6f 6e 2d 63 6f 6d 6d 65 72 63 69 61 6c 20 75 73 65 20 6f 6e 6c 79 20 2d 20 65 ... 19 more bytes>
<Buffer 47 75 72 6f 62 69 20 4f 70 74 69 6d 69 7a 65 72 20 76 65 72 73 69 6f 6e 20 39 2e 35 2e 30 20 62 75 69 6c 64 20 76 39 2e 35 2e 30 72 63 35 20 28 77 69 ... 6 more bytes>
<Buffer 54 68 72 65 61 64 20 63 6f 75 6e 74 3a 20 34 20 70 68 79 73 69 63 61 6c 20 63 6f 72 65 73 2c 20 38 20 6c 6f 67 69 63 61 6c 20 70 72 6f 63 65 73 73 6f ... 334 more bytes>
<Buffer 50 72 65 73 6f 6c 76 65 20 72 65 6d 6f 76 65 64 20 31 33 20 72 6f 77 73 20 61 6e 64 20 34 20 63 6f 6c 75 6d 6e 73 0d 0a>
<Buffer 50 72 65 73 6f 6c 76 65 20 74 69 6d 65 3a 20 30 2e 30 30 73 0d 0a 50 72 65 73 6f 6c 76 65 64 3a 20 36 20 72 6f 77 73 2c 20 37 20 63 6f 6c 75 6d 6e 73 ... 67 more bytes>
<Buffer 46 6f 75 6e 64 20 68 65 75 72 69 73 74 69 63 20 73 6f 6c 75 74 69 6f 6e 3a 20 6f 62 6a 65 63 74 69 76 65 20 32 32 32 30 30 2e 30 30 30 30 30 30 0d 0a>
<Buffer 46 6f 75 6e 64 20 68 65 75 72 69 73 74 69 63 20 73 6f 6c 75 74 69 6f 6e 3a 20 6f 62 6a 65 63 74 69 76 65 20 32 31 32 30 30 2e 30 30 30 30 30 30 0d 0a ... 227 more bytes>
<Buffer 0d 0a 20 20 20 20 20 30 20 20 20 20 20 30 20 20 20 20 20 63 75 74 6f 66 66 20 20 20 20 30 20 20 20 20 20 20 32 31 32 30 30 2e 30 30 30 30 20 32 31 32 ... 761 more bytes>
<Buffer 47 75 72 6f 62 69 20 4f 70 74 69 6d 69 7a 65 72 20 76 65 72 73 69 6f 6e 20 39 2e 35 2e 30 20 62 75 69 6c 64 20 76 39 2e 35 2e 30 72 63 35 20 28 77 69 ... 250 more bytes>
<Buffer 20 20 4d 61 74 72 69 78 20 72 61 6e 67 65 20 20 20 20 20 5b 31 65 2b 30 30 2c 20 39 65 2b 30 33 5d 0d 0a 20 20 4f 62 6a 65 63 74 69 76 65 20 72 61 6e ... 90 more bytes>
<Buffer 50 72 65 73 6f 6c 76 65 20 72 65 6d 6f 76 65 64 20 31 33 20 72 6f 77 73 20 61 6e 64 20 34 20 63 6f 6c 75 6d 6e 73 0d 0a 50 72 65 73 6f 6c 76 65 20 74 ... 55 more bytes>
<Buffer 56 61 72 69 61 62 6c 65 20 74 79 70 65 73 3a 20 35 20 63 6f 6e 74 69 6e 75 6f 75 73 2c 20 32 20 69 6e 74 65 67 65 72 20 28 32 20 62 69 6e 61 72 79 29 ... 2 more bytes>
Additionally, this is what typeof data.toString() returns :
string
string
string
string
string
string
string
string
string
string
string
I'm not 100% sure if I got you right here, but it sounds like the output from Python is already present when you access it. However, when you got your data it's a string first that needs to be parsed to a JavaScript Array.
Try
let myArray = JSON.parse(result);
to see, if the array is afterwards what you expected.
I am using node-Red the function.
I have got a string like this one:
{"txpk":{"imme":false,"tmst":145559484,"freq":869.525,"rfch":0,"powe":27,"modu":"LORA","datr":"SF9BW125","codr":"4/5","ipol":true,"size":17,"data":"YA0iASalAAADMf8AAbKwI3I="}}
This is the message. The payload is coming in the function block.
All I want is to add some Bytes in front of the string:
buf = new Buffer('buff');
buf1 = new Buffer ('0000');
buf[0]=0x02;
buf[1]=0xbd;
buf[2]=0x45;
buf[3]=0x03;
buf1= msg.payload;
msg.payload = buf+buf1;
return [null,msg];
The result is als follows writing in hex:
02 EF BF BD 45 03 7B 22 74 78 70 6B 22 3A 7B 22 69 6D 6D 65 22 3A 66 61 6C 73 65 2C 22 74 6D 73 74 22 3A 31 34 35 35 35 39 34 38 34 2C 22 66 72 65 71 22 3A 38 36 39 2E 35 32 35 2C 22 72 66 63 68 22 3A 30 2C 22 70 6F 77 65 22 3A 32 37 2C 22 6D 6F 64 75 22 3A 22 4C 4F 52 41 22 2C 22 64 61 74 72 22 3A 22 53 46 39 42 57 31 32 35 22 2C 22 63 6F 64 72 22 3A 22 34 2F 35 22 2C 22 69 70 6F 6C 22 3A 74 72 75 65 2C 22 73 69 7A 65 22 3A 31 37 2C 22 64 61 74 61 22 3A 22 59 41 30 69 41 53 61 6C 41 41 41 44 4D 66 38 41 41 62 4B 77 49 33 49 3D 22 7D 7D 22 3B 0A
Instead of :
02 BD 45 03
I see at the first bytes:
02 EF BF BD 45 03
Could anyone explain what I did wrong?
You are trying to join a Buffer object with a String - which is the cause of your problem. It is better to convert the String to a Buffer then use Buffer.concat() to join them:
buf = new Buffer('buff');
buf1 = new Buffer (msg.payload);
buf[0]=0x02;
buf[1]=0xbd;
buf[2]=0x45;
buf[3]=0x03;
msg.payload = Buffer.concat([buf, buf1]);
return [null,msg];
This question already has answers here:
JavaScript closure inside loops – simple practical example
(44 answers)
Closed 5 years ago.
I am running this code, which should compare saved webpages with the current live versions, but it seems to only do so for the last entry:
ipcMain.on('compare', (event, arg) => {
index.find({
"type": "website"
}, function(err, docs) {
var doc;
var data;
for (i = 0; i < docs.length; i++) {
console.log(doc = docs[i]);
console.log(i);
request('GET', doc.URL).done(function(res) {
data = fs.readFileSync(DataPath + "/savedpages/" + doc.Name + ".html");
console.log(data);
console.log(res.getBody())
if (data == res.getBody()) {
index.update({
_id: doc.ID
}, {
$set: {
Changes: false
}
}, function(err, updateval) {
if (err) throw err;
console.log(doc.Name);
event.sender.send('update-false', doc.Name + "-changescell")
})
} else {
index.update({
_id: doc.ID
}, {
$set: {
Changes: true
}
}, function(err, updateval) {
if (err) throw err;
console.log(doc.Name);
event.sender.send('update-true', doc.Name + "-changescell")
})
}
})
}
})
})
The output looks like this:
{ Changes: false,
type: 'website',
Date: 2017-07-30T14:04:35.592Z,
Name: 'petra',
URL: 'http://petra.oldisoft.de',
_id: 'SXtHgqAWniDBAibJ' }
0
{ Changes: false,
type: 'website',
Date: 2017-07-30T14:03:26.658Z,
Name: 'heise',
URL: 'http://heise.de',
_id: 'zOc1Wnm801huVPjs' }
1
<Buffer 3c 21 44 4f 43 54 59 50 45 20 68 74 6d 6c 3e 0a 3c 68 74 6d 6c 20 6c
61 6e 67 3d 22 64 65 22 3e 0a 0a 3c 68 65 61 64 3e 0a 20 20 20 20 3c 74 69
74 6c ... >
<Buffer 3c 2f 68 65 61 64 3e 0a 3c 62 6f 64 79 20 6f 6e 6c 6f 61 64 3d 27 6a
61 76 61 73 63 72 69 70 74 3a 64 6f 63 75 6d 65 6e 74 2e 61 6e 6d 65 6c 64
75 6e ... >
heise
<Buffer 3c 21 44 4f 43 54 59 50 45 20 68 74 6d 6c 3e 0a 3c 68 74 6d 6c 20 6c
61 6e 67 3d 22 64 65 22 3e 0a 0a 3c 68 65 61 64 3e 0a 20 20 20 20 3c 74 69
74 6c ... >
<Buffer 3c 21 44 4f 43 54 59 50 45 20 68 74 6d 6c 3e 0a 3c 68 74 6d 6c 20 6c
61 6e 67 3d 22 64 65 22 3e 0a 0a 3c 68 65 61 64 3e 0a 20 20 20 20 3c 74 69
74 6c ... >
heise
It seems to me that the code skips a part of the code while the for loop is running. I don't understand why it is behaving as such.
this is because you are running async code in a loop.
To overcome this you need to create a closure by wrapping the async code in a function:
for(i = 0; i < docs.length; i++){
console.log(doc = docs[i]);
console.log(i);
runAsyncCode(doc)
}
function runAsyncCode(doc){
request('GET', doc.URL).done(function(res) {
data = fs.readFileSync(DataPath + "/savedpages/" + doc.Name + ".html");
console.log(data);
console.log(res.getBody())
if(data == res.getBody()){
index.update({_id: doc.ID}, {$set: {Changes:false}}, function(err, updateval){
if(err) throw err;
console.log(doc.Name);
event.sender.send('update-false', doc.Name + "-changescell")
})
} else {
index.update({_id: doc.ID}, {$set: {Changes:true}}, function(err, updateval){
if(err) throw err;
console.log(doc.Name);
event.sender.send('update-true', doc.Name + "-changescell")
})
}
})
}
I am using AJAX POST data to server which use Node.js code. Very briefly, two file in this test project. Here is main.html:
<!DOCTYPE HTML>
<html>
<head>
</head>
<body>
<script>
var xhr = new XMLHttpRequest();
xhr.open('POST', '/', true);
xhr.send('hello');
</script>
</body>
</html>
Here the server code:
const http = require('http');
http.createServer(function(req,res) {
console.log(req.body);
}).listen(3000);
Perhaps,you have known that server will console.log() 'undefined'.
So the question is why it's 'undefined'? How to get AJAX data on server?
I know other solution according to this question.
Could you help me any other way to get data conveniently?
You've created server allright, but there's two problems:
Connection endpoint. Your AJAX is POST, so you need to parse POST requests correctly (not exactly solves your issue, but you need to distinguish request methods).
Node.js is not parsing your POST body by default, you need to tell it how it's done, by using querystring module, for example.
Combining this together:
var qs = require('querystring');
http.createServer(function(req, res) {
if (request.method == 'POST') {
var body = '';
request.on('data', function(data) {
body += data;
});
request.on('end', function() {
var post = qs.parse(body);
// Use your POST here
console.log(post);
});
}
}).listen(3000);
JSON data - cleaner solution:
You need to encode your AJAX data with JSON, and than parse it on the server side like this:
http.createServer(function(req,res) {
if (req.method == 'POST') {
var jsonPost = '';
req.on('data', function(data) {
jsonPost += data;
});
req.on('end', function() {
var post = JSON.parse(jsonPost);
// Use your POST here
console.log(post);
});
}
}).listen(3000);
You'd be even better off using Express.js framework for that, with it's bodyParser module.
UPDATE - How chunks are buffered:
Consider simple example - send in xhr.send() a LARGE amount of text, that exceeds Content-Length. Than do the following on data event:
req.on('data', function(data) {
console.log(data);
body += data;
});
You will see something like:
<Buffer 0a 0a 4c 6f 72 65 6d 20 69 70 73 75 6d 20 64 6f 6c 6f 72 20 73 69 74 20 61 6d 65 74 2c 20 63 6f 6e 73 65 63 74 65 74 75 72 20 61 64 69 70 69 73 63 69 ... >
<Buffer 65 74 20 71 75 61 6d 20 63 6f 6e 67 75 65 20 6c 6f 62 6f 72 74 69 73 2e 20 50 65 6c 6c 65 6e 74 65 73 71 75 65 20 74 65 6d 70 75 73 20 75 6c 6c 61 6d ... >
<Buffer 61 2e 20 56 69 76 61 6d 75 73 20 76 69 74 61 65 20 61 6e 74 65 20 6d 65 74 75 73 2e 20 4d 61 75 72 69 73 20 71 75 69 73 20 61 6c 69 71 75 65 74 20 65 ... >
That shows that the data received in chunks on data event. Only on end event you will get the whole data sent (if you've aggregated it before that). Node.js doesn't handle this, that's why you need third-party modules.
That is - you cannot just get req.body of the request, as it wasn't set at all.
I am implementing websocket v07 server protocol in C/C++. It's pretty easy and I have almost coded it. Though I can't understand what mean numbers in first initial packet (not handshake)? Client uses "socket.io" framework. So client sends Date and server answer with another number. I think its something like checksum. Can anyone explain me please how to generate answer and what does it mean?
Examples:
client GET./socket.io/1/?t=1322647367092&jsonp=0
server 52 io.j[0]("1880965230667822746 :15:25:websocket,htmlfile,xhr-polling,jsonp-polling");
client GET./socket.io/1/?t=1322647141334&jsonp=0
server 52 io.j[0]("7826289221657265491 :15:25:websocket,htmlfile,xhr-polling,jsonp-polling");
1322647367092 - Date, server answer - 1880965230667822746
1322647141334 - Date, server answer - 7826289221657265491
My Packet ->
000000 48 54 54 50 2F 31 2E 31 | 20 32 30 30 20 4F 4B 0D HTTP/1.1.200.OK.
000010 0A 43 6F 6E 74 65 6E 74 | 2D 54 79 70 65 3A 20 61 .Content-Type:.a
000020 70 70 6C 69 63 61 74 69 | 6F 6E 2F 6A 61 76 61 73 pplication/javas
000030 63 72 69 70 74 0D 0A 43 | 6F 6E 6E 65 63 74 69 6F cript..Connectio
000040 6E 3A 20 6B 65 65 70 2D | 61 6C 69 76 65 0D 0A 54 n:.keep-alive..T
000050 72 61 6E 73 66 65 72 2D | 45 6E 63 6F 64 69 6E 67 ransfer-Encoding
000060 3A 20 63 68 75 6E 6B 65 | 64 0D 0A 0D 0A 35 30 0D :.chunked....50.
000070 0A 69 6F 2E 6A 5B 30 5D | 28 22 32 37 38 38 39 39 .io.j[0]("278899
000080 38 30 38 38 32 36 32 34 | 38 38 30 3A 31 35 3A 32 80882624880:15:2
000090 35 3A 77 65 62 73 6F 63 | 6B 65 74 2C 68 74 6D 6C 5:websocket,html
0000A0 66 69 6C 65 2C 78 68 72 | 2D 70 6F 6C 6C 69 6E 67 file,xhr-polling
0000B0 2C 6A 73 6F 6E 70 2D 70 | 6F 6C 6C 69 6E 67 22 29 ,jsonp-polling")
0000C0 3B 0D 0A 30 0D 0A 0D 0A ;..0....
Original Packet ->
000000 48 54 54 50 2F 31 2E 31 | 20 32 30 30 20 4F 4B 0D HTTP/1.1.200.OK.
000010 0A 43 6F 6E 74 65 6E 74 | 2D 54 79 70 65 3A 20 61 .Content-Type:.a
000020 70 70 6C 69 63 61 74 69 | 6F 6E 2F 6A 61 76 61 73 pplication/javas
000030 63 72 69 70 74 0D 0A 43 | 6F 6E 6E 65 63 74 69 6F cript..Connectio
000040 6E 3A 20 6B 65 65 70 2D | 61 6C 69 76 65 0D 0A 54 n:.keep-alive..T
000050 72 61 6E 73 66 65 72 2D | 45 6E 63 6F 64 69 6E 67 ransfer-Encoding
000060 3A 20 63 68 75 6E 6B 65 | 64 0D 0A 0D 0A 35 30 0D :.chunked....50.
000070 0A 69 6F 2E 6A 5B 30 5D | 28 22 33 38 34 35 36 31 .io.j[0]("384561
000080 33 32 31 33 36 32 32 39 | 34 30 37 3A 31 35 3A 32 32136229407:15:2
000090 35 3A 77 65 62 73 6F 63 | 6B 65 74 2C 68 74 6D 6C 5:websocket,html
0000A0 66 69 6C 65 2C 78 68 72 | 2D 70 6F 6C 6C 69 6E 67 file,xhr-polling
0000B0 2C 6A 73 6F 6E 70 2D 70 | 6F 6C 6C 69 6E 67 22 29 ,jsonp-polling")
0000C0 3B 0D 0A 30 0D 0A 0D 0A ;..0....
It is a "session ID" according to the socket.io protocol specification:
The body of the response should contain the session id (sid) given to the client, followed by the heartbeat timeout, the connection closing timeout, and the list of supported transports separated by :