I'm a total noob in node.js (and internet tech. in general). On my assignment from the uni. in which we were asked to develop an http server, I have been requested to do the following:
Implementation details: upon receiving a message from the ‘client’ (socket, data and end events), assume it’s an HTTP message and parse it, if it’s not(an HTTP message) you should return an HTTP response(status 400).
My question is: How to parse the the message given, and how should I expect the message to look like? Bottom line, how does an http message looks like?
Thank you!
Node itself uses a http_parser written in C.
It's based on NGINX's HTTP parser with some extensions by the node core team.
Node's http module then [uses it](var HTTPParser = process.binding('http_parser').HTTPParser;)
var HTTPParser = process.binding('http_parser').HTTPParser;
For example the ClientRequest::onSocket uses a parser.
ClientRequest.prototype.onSocket = function(socket) {
var req = this;
process.nextTick(function() {
var parser = parsers.alloc();
// [snip]
});
};
If you actually want to write your own parser then have fun parsing the HTTP protocol.
If you don't know how to write a parser, then read up on Parsing
Related
Is there any way to get responses to a specific client when another client has a different request at the same time to the same server?
This is code snippet for an exchange server. The given function is present in a library named "ccxt", this function "exchange.fetchMarkets()" has an API which requests to a third party server which is an exchange server like 'bitfinex', 'crex24', 'binance', etc. The issue I am facing is when one client is requesting for an exchange like 'crex24' at the same time when another client is requesting for different exchange like 'binance', they are getting the same response as the function calls for the last recent exchange.
I want it to give responses according to the client's requests independent of each other.
this one is controller function:
const ccxt = require("ccxt");
exports.fetchMarkets = function(req, res){
let API = req.params.exchangeId;
let exchange = new ccxt[API]();
if (exchange.has["fetchMarkets"]) {
try{
var markets = await exchange.fetchMarkets();
res.send(markets)
}catch (err) {
let error = String(err);
res.send({ failed: error });
}
}else{
res.send({loadMarkets : "not available"})
}
}
This is end point for the server request:
app.route('/markets/:exchangeId')
.get(exchange.fetchMarkets)
Here you can find the ccxt library: https://github.com/ccxt/ccxt/wiki/Manual and can be included in the project by "npm install ccxt"
I don't see why the code you mentioned wouldn't work the way you are expecting it to work. I created a small app and it is working as expected. You can check here
https://repl.it/repls/IllfatedStrangeRepo
I am hitting four different request with different ids and I am getting different response.
Hope it clear the doubts.
We're automating our test with karate framework. In one of our features we need to decode a token and get a scope in the response. Everything works well, except this code in js.
function(token) {
return JSON.parse(new Buffer(token.split('.')[1],'base64').toString('ascii')).scope;
}
Error:
Caused by: <eval>:2 ReferenceError: "Buffer" is not defined
Caused by: jdk.nashorn.internal.runtime.ECMAException
In official tutorials it is said that javascript is 'native' to karate, so we don't understand why Buffer is not recognized? What we should do? Thanks for any help
I was able to successfully base64Decode a JWT token payload to JSON using the following code without the need for a Java method:
Background:
* def parseJwt =
"""
function(token) {
var base64Url = token.split('.')[1];
var base64Str = base64Url.replace(/-/g, '+').replace(/_/g, '/');
var Base64 = Java.type('java.util.Base64');
var decoded = Base64.getDecoder().decode(base64Str);
var String = Java.type('java.lang.String')
return new String(decoded)
};
"""
Scenario: JWT Token
Given path 'jwt/authenticate'
And header x-goog-authenticated-user-email = 'email'
And request {}
When method get
Then status 200
* json result = parseJwt(responseHeaders['Set-Cookie'][0])
* match result == {permissions: [1,2,3], iss: "us", exp: "#number", email: "email"}
Note: It does seem to be required to use json rather than def as Karate does better if it parses the string to json itself. Also, you may obtain the token from a header rather than a cookie as in this example if so, just change the responseHeader that you are looking for.
I'm pretty sure that Buffer is advanced / non-standard or NodeJS so it probably is not supported by the JVM JS engine (Nashorn).
Here's my recommendation. For this case, do the work using Java utilities.
For example, look at the Karate basic-auth example in the doc which uses Base64 encoding.
If it is really complex, simply create a Java static function, it will be much easier to test as a side-benefit. Hope this helps !
With respect to the post Karate: Problem with requests using bearer token | Karate is showing "org/apache/commons/codec/binary/Base64" in response instead of HTTP 401
My query was marked duplicate hence posting an answer/solution which we have found in our project.
We did some debugging in karate core about the error. It seemed a dependency was missing.
We added the dependency in the POM.xml
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.15</version>
</dependency>
Afterwards the problem was resolved and we started getting HTTP 401.
I am not sure it can be added to the karate core.
I am trying to proxy an api call from client side through my server for some third party service, reasons for this being CORS issues and adding in a secret key on the server side. I usually get it done in the following way:
app.use('/someService', (req, res) => {
let url = `https://example.com/${config.SECRET_KEY}/endpoint${req.url}`
req.pipe(request(url).pipe(res))
})
this way I can use any ajax library on the client side and perform get request for example: get: '/someService/params' and it would normally go through and perform that request and then pipe it back. However now I started getting:
Error: write after end
in express and I am not entirely sure about what could be causing it.
Your piping is wrong. As it is now, you're piping to res twice (.pipe() returns the argument passed to it for chainability).
Instead try this:
req.pipe(request(url)).pipe(res)
I should point out however that properly proxying the HTTP response is not quite that simple since currently this line will always respond with HTTP status code 200, no matter what the remote server for the middle request responds with. Also, any headers from that response will not be sent to res. With that in mind, you could naively try something like:
var proxiedRes = req.pipe(request(url));
proxiedRes.on('response', function(pres) {
res.writeHead(pres.statusCode, pres.headers);
// You will want to add a `pres` 'error' event handler too in case
// something goes wrong while reading the proxied response ...
pres.pipe(res);
});
I've distilled my issue to some really basic functionality here. Basically, we're sending a request to a server (you can go ahead and c/p the URL and see the json document we get in response).
We get the response, we pipe it into a write stream and save it as a .json file - but the problem is that the file keeps being cut off. Is the .json file too large? Or am I missing something? Node.js newbie - massively appreciate any help I can get.
var fs = require('fs');
var url = 'https://crest-tq.eveonline.com/market/10000002/history/?type=https://crest-tq.eveonline.com/inventory/types/34/'
var request = require('request');
request(url).pipe(fs.createWriteStream('34_sell.json'));
var fs = require('fs');
var request = require('request');
var url = 'https://crest-tq.eveonline.com/market/10000002/history/?type=https://crest-tq.eveonline.com/inventory/types/34/';
request(url).pipe(fs.createWriteStream('34_sell.json'));
Not an answer but this is the code I used. I'm using request version 2.74.0. And node version v5.4.1.
Try writing a get request to the url and send the json as response and write an error handling statement like if err then throw err..console log and see the result..hope it works
So I'm trying to make a very basic node.js server that with take in a request for a string, randomly select one from an array and return the selected string. Unfortunately I'm running into a few problems.
Here's the front end:
function newGame()
{
guessCnt=0;
guess="";
server();
displayHash();
displayGuessStr();
displayGuessCnt();
}
function server()
{
xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET","server.js", true);
xmlhttp.send();
string=xmlhttp.responseText;
}
This should send the request to server.js:
var http = require('http');
var choices=["hello world", "goodbye world"];
console.log("server initialized");
http.createServer(function(request, response)
{
console.log("request recieved");
var string = choices[Math.floor(Math.random()*choices.length)];
console.log("string '" + string + "' chosen");
response.on(string);
console.log("string sent");
}).listen(8001);
So clearly there are several things going wrong here:
I get the feeling the way I am "connecting" these two files isn't correct both in the xmlhttp.open method and in using response.on to send the string back to the front end.
I'm a little confused with how I call this page on localhost. The front end is named index.html and the sever posts to 8001. What address should I be go to on localhost in order to access the initial html page after I have initialized server.js? Should I change it to .listen(index.html) or something like that?
are there other obvious problems with how I am implementing this (using .responsetext etc.)
(sorry for the long multi-question post but the various tutorials and the node.js source all assume that the user already has an understanding of these things.)
Your request should be to the server, NOT the server.js file which instantiates it. So, the request should look something like this: xmlhttp.open("GET","http://localhost:8001/", true); Also, you are trying to serve the front-end (index.html) AND serve AJAX requests at the same URI. To accomplish this, you are going to have to introduce logic to your server.js that will differentiate between your AJAX requests and a normal http access request. To do this, you'll want to either introduce GET/POST data (i.e. call http://localhost:8001/?getstring=true) or use a different path for your AJAX requests (i.e. call http://localhost:8001/getstring). On the server end then, you'll need to examine the request object to determine what to write on the response. For the latter option, you need to use the 'url' module to parse the request.
You are correctly calling listen() but incorrectly writing the response. First of all, if you wish to serve index.html when navigating to http://localhost:8001/, you need to write the contents of the file to the response using response.write() or response.end(). First, you need to include fs=require('fs') to get access to the filesystem. Then, you need to actually serve the file.
XMLHttpRequest needs a callback function specified if you use it asynchronously (third parameter = true, as you have done) AND want to do something with the response. The way you have it now, string will be undefined (or perhaps null), because that line will execute before the AJAX request is complete (i.e. the responseText is still empty). If you use it synchronously (third parameter = false), you can write inline code as you have done. This is not recommended as it locks the browser during the request. Asynchronous operation is usually used with the onreadystatechange function, which can handle the response once it is complete. You need to learn the basics of XMLHttpRequest. Start here.
Here is a simple implementation that incorporates all of the above:
server.js:
var http = require('http'),
fs = require('fs'),
url = require('url'),
choices = ["hello world", "goodbye world"];
http.createServer(function(request, response){
var path = url.parse(request.url).pathname;
if(path=="/getstring"){
console.log("request recieved");
var string = choices[Math.floor(Math.random()*choices.length)];
console.log("string '" + string + "' chosen");
response.writeHead(200, {"Content-Type": "text/plain"});
response.end(string);
console.log("string sent");
}else{
fs.readFile('./index.html', function(err, file) {
if(err) {
// write an error response or nothing here
return;
}
response.writeHead(200, { 'Content-Type': 'text/html' });
response.end(file, "utf-8");
});
}
}).listen(8001);
console.log("server initialized");
frontend (part of index.html):
function newGame()
{
guessCnt=0;
guess="";
server();
displayHash();
displayGuessStr();
displayGuessCnt();
}
function server()
{
xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET","http://localhost:8001/getstring", true);
xmlhttp.onreadystatechange=function(){
if (xmlhttp.readyState==4 && xmlhttp.status==200){
string=xmlhttp.responseText;
}
}
xmlhttp.send();
}
You will need to be comfortable with AJAX. Use the mozilla learning center to learn about XMLHttpRequest. After you can use the basic XHR object, you will most likely want to use a good AJAX library instead of manually writing cross-browser AJAX requests (for example, in IE you'll need to use an ActiveXObject instead of XHR). The AJAX in jQuery is excellent, but if you don't need everything else jQuery offers, find a good AJAX library here: http://microjs.com/. You will also need to get comfy with the node.js docs, found here. Search http://google.com for some good node.js server and static file server tutorials. http://nodetuts.com is a good place to start.
UPDATE: I have changed response.sendHeader() to the new response.writeHead() in the code above !!!
Express makes this kind of stuff really intuitive. The syntax looks like below :
var app = require('express').createServer();
app.get("/string", function(req, res) {
var strings = ["rad", "bla", "ska"]
var n = Math.floor(Math.random() * strings.length)
res.send(strings[n])
})
app.listen(8001)
https://expressjs.com
If you're using jQuery on the client side you can do something like this:
$.get("/string", function(string) {
alert(string)
})
I was facing following error with code (nodejs 0.10.13), provided by ampersand:
origin is not allowed by access-control-allow-origin
Issue was resolved changing
response.writeHead(200, {"Content-Type": "text/plain"});
to
response.writeHead(200, {
'Content-Type': 'text/html',
'Access-Control-Allow-Origin' : '*'});
Here is a fully functional example of what you are trying to accomplish. I created the example inside of hyperdev rather than jsFiddle so that you could see the server-side and client-side code.
View Code:
https://hyperdev.com/#!/project/destiny-authorization
View Working Application: https://destiny-authorization.hyperdev.space/
This code creates a handler for a get request that returns a random string:
app.get("/string", function(req, res) {
var strings = ["string1", "string2", "string3"]
var n = Math.floor(Math.random() * strings.length)
res.send(strings[n])
});
This jQuery code then makes the ajax request and receives the random string from the server.
$.get("/string", function(string) {
$('#txtString').val(string);
});
Note that this example is based on code from Jamund Ferguson's answer so if you find this useful be sure to upvote him as well. I just thought this example would help you to see how everything fits together.