There is the error i am facing when i am sending the input to the child-process written in python.
When i send the data for first time it gives output but with the second input i send me error. Hint that the pipe to covers is ended just after the first output i receive.
Can you help me.
here is the node code.
var bodyParse = require('body-parser');
var urlencodedParser = bodyParse.urlencoded({extended: false});
var spawn = require('child_process').spawn
var py = spawn('python', ['dialogue_management_model.py'])
module.exports = function(app) {
app.get('/', function(req, res) {
res.render('index');
});
app.post('/', urlencodedParser, function(req, res) {
var typed = (JSON.stringify(req.body).substring(2, JSON.stringify(req.body).indexOf(":") - 1));
console.log(typed)
module.exports.typed = typed
var data = typed;
dataString = '';
// Handling the Input data from the Front End With the Post request.
// taking computed/operated data from the python file
py.stdout.on('data', function(data){
dataString += data.toString();
});
// Simply logging it to the Console
py.stdout.on('end', function(){
console.log(dataString);
res.send(dataString);
});
// python doesn't understand the data without string format
py.stdin.write(data);
py.stdin.end();
})
}
Just Server is started in other file and is passing full control to here and from here i am calling python code to take input do computation and pass me the result.
You are ending the input stream entirely after the first call. Move var py = spawn('python', ['dialogue_management_model.py']) into the post request handler, so each request will spawn a child process, write the data, end the input stream, wait for a response, and return a result when the output stream ends.
This gives you the added benefit of making it more thread safe. That is to say, if you have two requests come in at the same time, both will end up adding listeners for py.stdout.on('data', ..., resulting in both getting a mixture of output. Also I'm fairly certain py.stdout.on('end', would only trigger for once, so any request that comes in after the stdout.end callback runs from the first request(s) would hang until they timeout.
Also, unrelated to your question, but when you do:
var typed = (JSON.stringify(req.body).substring(2, JSON.stringify(req.body).indexOf(":") - 1))
you should assign the JSON.stringify() to a variable so you don't have to run it twice.
ie. var payload = JSON.stringify(req.body); var typed = (payload.substring(2, payload.indexOf(":") - 1))
But even then, if you just need the first key you can do Object.keys(req.body)[0] rather than converting the object to a string and parsing the string.
Related
I have a C# dotnetcore console app running on an AWS instance, and I would like to add in communication between this and my playfab cloudscript.
I can communicate from the C# console to playfab, that was simple just using the playfab nuget package. However I'm having trouble going the other way.
I only ever want to send a few different simple messages, so im not looking for anything too complex. What I have done so far, is I added the following to the start of my console application:
var listener = new HttpListener();
listener.Prefixes.Add("http://+:80/");
listener.Start();
Writer.WriteBuffer("Listening...");
HttpListenerContext context = listener.GetContext();
HttpListenerRequest request = context.Request;
// Obtain a response object.
HttpListenerResponse response = context.Response;
// Construct a response.
Writer.WriteBuffer("Context: " + context.ToString());
Writer.WriteBuffer("request: " + request.ToString());
Writer.WriteBuffer("response: " + response.ToString());
string responseString = "<HTML><BODY> Hello world!</BODY></HTML>";
byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);
// Get a response stream and write the response to it.
response.ContentLength64 = buffer.Length;
System.IO.Stream output = response.OutputStream;
output.Write(buffer, 0, buffer.Length);
// You must close the output stream.
output.Close();
listener.Stop();
Writer.WriteBuffer is a wrapper for writing Console.Write essentially, just formats stuff in a much better way for me. I see the "listening..." come up, so great its listening.
Now, I copied an example from playfab and just adapted it very slightly. cloudscript is in js, so here is what I am running from playfab:
var headers = {
"X-MyCustomHeader": "Some Value"
};
var body = {
input: args,
userId: currentPlayerId,
mode: "foobar"
};
var url = "http://11.111.111.1/";
var content = JSON.stringify(body);
var httpMethod = "post";
var contentType = "application/json";
// The pre-defined http object makes synchronous HTTP requests
var response = http.request(url, httpMethod, content, contentType, headers);
return { responseContent: response };
11.111.111.1 is where I put the IP address of the AWS instance (I've changed it for obvious reasons).
When running this, I get "httpRequestError": "Timeout".
When I check on AWS, I have nothing else printed out other than "Listening...", so it hasn't handled anything yet.
I'm not too sure where the problem lies to be honest.
I have a fully functioning PHP application that I am trying to make a Node.js version of. It deals with serving image tiles. When it's ready to display the image it does:
// Stream out the image
echo self::$tile;
How would I do something similar in Node.js? I understand this is a broad question, but I think my biggest issue is that I don't understand how PHP "echoes" an image.
Details:
I'm using AWS to get the image. The AWS call returns a Buffer. At this point of time, in the Javascript I have left the image as a Buffer.
The site populates a map with tiled images, so there are multiple calls with the image placed at a particular location on the page. I am using express to handle the requests.
app.get(/^\/omb\/1.0.0\/(.+)\/(.+)\/(.+)\/(.+)\.[a-zA-Z]*$/, function(req, res){
var MosaicStreamer = require('./models/MosaicStreamer.js');
var ms = new MosaicStreamer;
var configs = {library: req.params[0], zoom: req.params[1], column: req.params[2], row: req.params[3]};
ms.handleTile(configs);
});
handleTile grabs the image and ultimately brings me to where I am now. The image is grabbed using the following:
var aws = new AWS.S3();
var params = {
Bucket: this.bucket,
Key: this.tileDirectory + this.filepath,
Range: 'bytes=' + (this.toffset + 4) + "-" + (this.tsize + this.toffset + 4)
};
var ts = this;
aws.getObject(params, function(err, data){
if(ts.tile == null){
ts.tile = data.Body; //S3 get object
}
}
I think what you want to do is take a given URL which represents closely the naming convention of folders/files in your S3 Bucket. So assuming that you've established a client connection to your S3, you can use the readFile method. The 2nd argument is an imageStream which you can pass in the response. Once the stream has ended from S3, it will automatically end the res from the client, outputting the image directly to the client (as you intend).
Some psuedo code:
app.get(/^\/omb\/1.0.0\/(.+)\/(.+)\/(.+)\/(.+)\.[a-zA-Z]*$/, function(req, res){
var MosaicStreamer = require('./models/MosaicStreamer.js');
var ms = new MosaicStreamer;
var configs = {library: req.params[0], zoom: req.params[1], column: req.params[2], row: req.params[3]};
return ms.handleTile(configs, res);
//return the handleTile function, add 2nd argument and pass res through
});
Inside of handleTile function you can make the call for the S3
function handleTile(configs, res){
client.readFile('filename', function(error, imageStream){
imageStream.pipe(res);
});
}
Now requests to images like this:
<img src="/path/to/my/file/that/matches/regexp/expression"/>
It will request that image from the S3 Bucket and stream the resource back to the client directly.
To successfully render an image, you have to implement three steps:
Retrieve the image data (for instance as a Buffer read via fs.readFile) or a stream (for instance via fs.createReadStream
Set the appropriate headers in the web request handler with the arguments (req, res); something like
res.writeHead(200, {'Content-Type': 'image/png'});
Write the file. If you have the file in a Buffer, with
res.end(buf, 'binary');
If you have a stream via
read_stream.pipe(res)
The whole code may look like (assuming you want to serve the file image.jpg from the current directory):
'use strict';
var fs = require('fs');
var http = require('http');
http.createServer(function(req, res) {
fs.readFile('image.jpg', function(err, buf) {
if (err) {
res.writeHead(500);
res.end('Cannot access file.');
return;
}
res.writeHead(200, {'Content-Type': 'image/jpeg'});
res.end(buf, 'binary');
});
}).listen(8002, '');
Using a stream, a very simple version (beware: no error handling, with error handling it can get a little bit more complex, depending how you want to handle errors occurring while the file is being read)
'use strict';
var fs = require('fs');
var http = require('http');
http.createServer(function(req, res) {
var stream = fs.createReadStream('image.jpg');
// Error handling omitted here
res.writeHead(200, {'Content-Type': 'image/jpeg'});
stream.pipe(res);
}).listen(8003, '');
Code that uses a Buffer is easier to write, but means that your server must hold the whole file in memory - for instance, you will be unable to serve a 320 Gigapixel image file. You also only start sending data once you have the whole file.
Using a stream allows sending the file as soon as you get it, so it will be a little faster. If you're reading from file or a local fast server the speed difference is likely negligible. In addition, you'll only need a little bit of memory. On the other hand, error handling is more complex.
I'm a node.js newbie and I'm creating my first big app with it (I'm using express). I need to have my webpage perform some javascript canvas-drawing when the user loads an id with a get request, e.g.
www.mywebsite.com/page?id=22
I know I can handle this with a simple
var express = require("express");
var app = express();
app.get('handle',function(request,response){
//request.id
});
but I don't know how to start my webpage with the asked drawing for that id. Every tutorial on the internet on express and get explains how to handle get requests... well this question is about "what happens next?"
Rephrased: I'm not sure how should I tell the html page "you need to draw what is associated with this id" from express and then send that page back to the user.
You can take the id from params and after this to return a response based on that id.
var express = require('express');
var app = express();
app.get("/page/:id",function(request, response){
var id = request.params.id;
// do something with id
// send a response to user based on id
var obj = { id : id, Content : "content " +id };
response.writeHead(200, {"Content-Type": "application/json"});
response.write(JSON.stringify(obj));
});
Notes:
You use /page/:id to make urls like www.mywebsite.com/page/22 or www.mywebsite.com/page?id=22 and you can have acces to id on server with request.params.id (output: 22).
With response you can write a response to the server. In this example i returned a json object.
In writeHead 200 come from status which means OK , and content-type means that I return a json object
You can return what you want, a page or something else, this is just an example (PoC).
If you want to pass multiple variables in the request, you can pass it in the following way:
var emailVar = "someEmail#gmail.com";
var nameVar = "someName";
var url = `/home?email=${emailVar}&name=${nameVar}`;
//Now make the request.
and in the backend logic, you can retrieve these values as:
app.get('/home', function(request, response)
{
console.log(request.query.email+" "+request.query.name);
var email = request.query.email;
var name = request.query.name;
response.setHeader('Content-Type', 'application/json');
if(request.query.email)
{
response.send(JSON.stringify({
message: 'Got the email'
}));
}
else
{
response.send(JSON.stringify({
message: 'No email sent'
}));
}
});
This approach is useful for performing query operations in the backend.
EDITED
I have a nodeJS http server that is meant for receiving uploads from multiple clients and processing them separately.
My problem is that I've verified that the first request blocks the reception of any other request until the previous request is served.
This is the code I've tested:
var http = require('http');
http.globalAgent.maxSockets = 200;
var url = require('url');
var instance = require('./build/Release/ret');
http.createServer( function(req, res){
var path = url.parse(req.url).pathname;
console.log("<req>"+path+"</req>");
switch (path){
case ('/test'):
var body = [];
req.on('data', function (chunk) {
body.push(chunk);
});
req.on('end', function () {
body = Buffer.concat(body);
console.log("---req received---");
console.log(Date.now());
console.log("------------------");
instance.get(function(result){
postHTTP(result, res);
});
});
break;
}
}).listen(9999);
This is the native side (omitting obvious stuff) where getInfo is the exported method:
std::string ret2 (){
sleep(1);
return string("{\"image\":\"1.JPG\"}");
}
Handle<Value> getInfo(const Arguments &args) {
HandleScope scope;
if(args.Length() == 0 || !args[0]->IsFunction())
return ThrowException(Exception::Error(String::New("Error")));
Persistent<Function> fn = Persistent<Function>::New(Handle<Function>::Cast(args[0]));
Local<Value> objRet[1] = {
String::New(ret2().c_str())
};
Handle<Value> ret = fn->Call(Context::GetCurrent()->Global(), 1, objRet);
return scope.Close(Undefined());
}
I'm resting this with 3 curl parallel requests
for i in {1..3}; do time curl --request POST --data-binary "#/home/user/Pictures/129762.jpg" http://192.160.0.1:9999/test & done
This is the output from the server:
<req>/test</req>
---req received---
1397569891165
------------------
<req>/test</req>
---req received---
1397569892175
------------------
<req>/test</req>
---req received---
1397569893181
------------------
These the response and the timing from the client:
"1.JPG"
real 0m1.024s
user 0m0.004s
sys 0m0.009s
"1.JPG"
real 0m2.033s
user 0m0.000s
sys 0m0.012s
"1.JPG"
real 0m3.036s
user 0m0.013s
sys 0m0.001s
Apparently requests are received after the previous has been served. The sleep(1) simulates a synchronous operation that requires about 1s to complete and can't be changed.
The client receives the responses with an incremental delay of ~1s.
I would like to achieve a kind of parallelism, although I'm aware I'm in a single threaded environment such as nodeJS. What I would like to achieve is receiving all 3 answers is ~1s.
Thanks in advance for your help.
This:
for(var i=0;i<1000000000;i++) var a=a+i;
Is a pretty severe blocking operation. As soon as the first block ends. Your whole server hangs until this for loop is done. I'm interested in why you are trying to do this.
Perhaps you are trying to simulate a delayed response ?
setTimeout(function)({
send404(res);
}, 3000);
Right now you are turning a non-flowing stream into flowing mode by attaching a data event handler, and subsequently loading the whole stream into memory. You probably don't want to do this.
You can use the stream in now-flowing mode as illustrated below, this is useful if you want to send the data to some place that is only accessible after some other event.
However, using the stream in flowing mode is the fastest. If you want to write your own body parser I suppose you might want to use flowing mode, it depends on your use case.
req.on('readable', function () {
var chunk;
while (null !== (chunk = readable.read())) {
body.push(chunk);
}
});
Flowing and non-flowing mode is also know as respectively v1 and v2 streams, as the older streams used in node only supported flowing mode.
I am developing a web application where the user can type in a "device id" on the web page (we have 100's of devices out on the field in production use each with a unique ID), that result entered by the user will be sent to the Node.js server that in return will store it into a variable and use it in a SQL Query to retrieve results about that particular device from the database server and then display the results back to the client web page.
Currently the form input feature has not been implemented yet even though I've already coded the form code in html.
The program works fine as it is if I were to manually change the DEVICE_ID to the device I wish to retrieve data from in the code but of course I want to be able to enter this on the client page instead of me having to change it in the server-side source code manually.
"use strict";
var pg = require('pg').native;
var http = require('http');
var $ = require('cheerio');
var fs = require('fs');
var url = require('url');
var htmlString = fs.readFileSync('index.html').toString();
var parsedHTML = $.load(htmlString);
var dbUrl = "tcp://URL HERE/";
// The Sign-ID
var DEVICE_ID = '2001202';
// Create the http server
http.createServer(function (request, response) {
var request = url.parse(request.url, true);
var action = request.pathname;
// Connect and query the database
pg.connect(dbUrl, function(err, client) {
// The complete sql query for everything that's needed for the app!
client.query("SQL QUERY IS HERE" + DEVICE_ID + "etc..",
function (err, result) {
// Remaining program code is here that performs DOM based
// manipulation to display results returned from the server
// to the client side page.
// Time to Render the document and output to the console
console.log(parsedHTML.html());
// Render the document and project onto browser
response.end(parsedHTML.html());
}
); // End client.query
}); // End pg.connect
}).listen(8080); // End http.CreateServer
pg.end();
I've considered the following:
1) Use An OnClick() function from within the HTML code, like this:
onclick="lookupSignID()
Then include an external JS file from within the HTML that includes the lookupSignID() function however I soon found out this is only performing client-side function and is not what I want.
2) AJAX is only good for if the server is generating new information by itself, therefore I can't use AJAX since the user is entering the device ID to get information from it.
3) Possibly using POST/ GET
Can anyone please advise on what course of action I should take? If solution (3) is the best way how would I go about doing this? Can it be integrated into my existed code (shown above) without many changes?
Thanks!
If you used jQuery on the client side with an AJAX POST function, and then on the server side, you have express.js, you could do this:
app.post('/deviceid', function(req, res) {
var deviceid = req.param('deviceid')
console.log(deviceid)
...
})