I want my http respond script to respond with data from my SQL server. So I can use AJAX to update HTML with data from my SQL server. And I cant find a way to do this. I'm just learning about async and I have a feeling that if I can save the output of my async function to a global var then it will work. Any help would save my headache.
My simple Listen script is:
var test = "hello!"
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(test);
res.end();
}).listen(8080);
and my sql code is:
const util = require('util');
var mysql = require('mysql');
var con = mysql.createConnection({
host: "XXXXX",
user: "XXXXX",
password: "XXXXX",
database: "XXX"
});
var DBresult=null;
function getdb(){
const query = util.promisify(con.query).bind(con);
(async () => {
try {
const rows = await query('SELECT * FROM mylist');
DBresult=rows;
} finally {
con.end();
}
})()
}
Do NOT use any globals or shared, higher scoped variables for your asynchronous result in a server. Never do that. That is an anti-pattern for good reason because that can create intermittent concurrency problems because more than one request can be in process at the same time on your server so these would cause conflicting access to those variables, creating intermittent, very-hard-to-debug problems. I repeat again, NEVER.
You didn't describe an exact situation you are trying to write code for, but here's an example.
Instead, you use the result inside the context that it arrives from your asynchronous call. If you can use await, that generally makes the coding cleaner.
Here's a simple example:
const con = mysql.createConnection({
host: "XXXXX",
user: "XXXXX",
password: "XXXXX",
database: "XXX"
});
const query = util.promisify(con.query).bind(con);
http.createServer(async function(req, res) {
if (req.path === "/query" && req.method === "GET") {
try {
const rows = await query('SELECT * FROM mylist');
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(rows));
} catch(e) {
console.log(e);
res.statusCode = 500;
res.end();
}
} else {
// some other respone
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.write("hello");
res.end();
}
}).listen(8080);
Things to note here:
Checking both path and method before handling the request.
Making callback function async so it can use await.
Making sure any promise rejection from await is caught by try/catch and an error response is sent if there's an error.
Sending result as JSON and setting appropriate content-type.
You may be using the plain http module as a learning experience, but you will very quickly find that using the simple Express framework will save you lots of programming time and make things lots easier.
If you want to use async functions, it would be something like this, take care of "async" in function for the createServer
var http = require('http');
var server = http.createServer(async (req, res) => {
async function mgetdata() {
// Async code goes here
return 'Hello, world!';
}
// Wait for the async function to complete and get its return value
const response = await mgetdata();
// Send the response
res.end(response);
});
Related
I'm trying to get a client token from the BrainTree API. I can establish a connection normally but when the code is executed the client token is always null. I've tried multiple solutions including this one here, but no luck.
The code is deployed in an Azure Function.
My Server Side Code :
module.exports = async function (context, req) {
const express = require("express");
const router = express.Router();
var braintree = require("braintree");
var clientToken;
var gateway = new braintree.BraintreeGateway({
environment: braintree.Environment.Sandbox,
merchantId: 'xxxx',
publicKey: 'xxxx',
privateKey: 'xxx'
});
context.log('about to generate a client key');
var Ctk = await gateway.clientToken.generate({}, (err, response) => {
clientToken = response.clientToken
});
var responseC = "The Client token is +" +clientToken;
context.res = {
body: responseC
};
context.log('client key generated');
}
The await keyword only works with functions that return Promise objects. While the Braintree documentation isn't 100% crystal-clear on this, the implication that's raised by switching the sample code on the linked page from "callbacks" to "Promises" is that when you specify a callback function via the second positional parameter a Promise isn't returned, thus nuking the value of using await in the first place. In this scenario, it doesn't work as you expect it and will continue to process asynchronously via the callback, which is why your clientToken isn't getting set the way you think it should.
If you're invested in using Promises all the way through, consider chaining the callback within a .then() call as the documentation prescribes. You also don't need to capture the return value of the call to generate(), as you're storing it in clientToken and not using Ctk at all.
await gateway.clientToken.generate({}).then((err, response) => {
clientToken = response.clientToken
});
I have resolved this using the following code in the end.
It looks like braintree's functions are async and therefore we need to wait for them to complete. Promisify() did the job.
module.exports = async function (context, req) {
const express = require("express");
const router = express.Router();
var app = express();
var braintree = require("braintree");
var clientToken;
var gateway = new braintree.BraintreeGateway({
environment: braintree.Environment.Sandbox,
merchantId: 'xxx',
publicKey: 'xxx',
privateKey: 'xxx'
});
const util = require('util');
const sale = util.promisify(gateway.clientToken.generate);
result = await gateway.clientToken.generate({
}).then(response => {
// pass clientToken to your front-end
clientToken = response.clientToken
});
var cToken= "client token is : " +clientToken;
context.res = {
body: cToken
};
}
I'm trying to write a header of an MD5 hash token using crypto then return it back as a response. For some reason, it isn't actually running synchronously. I know JS is an asynchronous language, and that's really the only part I'm struggling with right now. Any help would be appreciated.
This is what I have so far:
const crypto = require('crypto');
const bodyParser = require('body-parser');
const formidable = require('formidable');
const async = require('async')
app.post('/pushurl/auth', (req, res) =>
var data = req.body.form1data1 + '§' + req.body.form1data2
async.waterfall([
function(callback) {
var token = crypto.createHash('md5').update(data).digest("hex");
callback(null, token);
},
function(token, callback) {
res.writeHead(301,
{Location: '/dashboard?token=' + token}
);
callback(null)
},
function(callback) {
res.end();
callback(null)
}
]);
}
});
Output:
Uncaught Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
<node_internals>/internal/errors.js:256
No debugger available, can not send 'variables'
Process exited with code 1
JavaScript is an asynchronous language, yes, but it can also do synchronous tasks very well. In your case, you don't need to do any async expect if you're dealing with promises.
If you write your code like in the example below it will just execute from top to bottom.
But the error (probably) occurred because you forgot to add an opening curly brace to your app.post callback, which results in the data var being immediately returned because of an implied return statement () => (implied), () => {} (explicit).
const crypto = require('crypto');
const bodyParser = require('body-parser');
const formidable = require('formidable');
app.post('/pushurl/auth', (req, res) => {
const data = req.body.form1data1 + '§' + req.body.form1data2;
const token = crypto.createHash('md5').update(data).digest("hex");
res.writeHead(301, {
Location: '/dashboard?token=' + token
});
res.end();
});
I have an http-proxy to proxy any website and inject some custom JS file before to serve the HTML back to the client. Whenever I try to access the proxied website, it will hang up or the browser seems to load indeterminately. But when I check the HTML source, I successfully managed to inject my custom JavaScript file. Here is the code:
const cheerio = require('cheerio');
const http = require('http');
const httpProxy = require('http-proxy');
const { ungzip } = require('node-gzip');
_initProxy(host: string) {
let proxy = httpProxy.createProxyServer({});
let option = {
target: host,
selfHandleResponse: true
};
proxy.on('proxyRes', function (proxyRes, req, res) {
let body = [];
proxyRes.on('data', function (chunk) {
body.push(chunk);
});
proxyRes.on('end', async function () {
let buffer = Buffer.concat(body);
if (proxyRes.headers['content-encoding'] === 'gzip') {
try {
let $ = null;
const decompressed = await ungzip(buffer);
const scriptTag = '<script src="my-customjs.js"></script>';
$ = await cheerio.load(decompressed.toString());
await $('body').append(scriptTag);
res.end($.html());
} catch (e) {
console.log(e);
}
}
});
});
let server = http.createServer(function (req, res) {
proxy.web(req, res, option, function (e) {
console.log(e);
});
});
console.log("listening on port 5051");
server.listen(5051);
}
Can someone please tell me if I am doing anything wrong, it looks like node-http-proxy is dying a lot and can't rely much on it since the proxy can work sometimes and die at the next run, depending on how many times I ran the server.
Your code looked fine so I was curious and tried it.
Although you do log a few errors, you don't handle several cases:
The server returns a body with no response (cheerio will generate an empty HTML body when this happens)
The server returns a response that is not gzipped (your code will silently discard the response)
I made a few modifications to your code.
Change initial options
let proxy = httpProxy.createProxyServer({
secure: false,
changeOrigin: true
});
Don't verify TLS certificates secure: false
Send the correct Host header changeOrigin: true
Remove the if statement and replace it with a ternary
const isCompressed = proxyRes.headers['content-encoding'] === 'gzip';
const decompressed = isCompressed ? await ungzip(buffer) : buffer;
You can also remove the 2 await on cheerio, Cheerio is not async and doesn't return an awaitable.
Final code
Here's the final code, which works. You mentioned that "it looks like node-http-proxy is dying a lot [...] depending on how many times I ran the server." I experienced no such stability issues, so your problems may lie elsewhere if that is happening (bad ram?)
const cheerio = require('cheerio');
const http = require('http');
const httpProxy = require('http-proxy');
const { ungzip } = require('node-gzip');
const host = 'https://github.com';
let proxy = httpProxy.createProxyServer({
secure: false,
changeOrigin: true
});
let option = {
target: host,
selfHandleResponse: true
};
proxy.on('proxyRes', function (proxyRes, req, res) {
console.log(`Proxy response with status code: ${proxyRes.statusCode} to url ${req.url}`);
if (proxyRes.statusCode == 301) {
throw new Error('You should probably do something here, I think there may be an httpProxy option to handle redirects');
}
let body = [];
proxyRes.on('data', function (chunk) {
body.push(chunk);
});
proxyRes.on('end', async function () {
let buffer = Buffer.concat(body);
try {
let $ = null;
const isCompressed = proxyRes.headers['content-encoding'] === 'gzip';
const decompressed = isCompressed ? await ungzip(buffer) : buffer;
const scriptTag = '<script src="my-customjs.js"></script>';
$ = cheerio.load(decompressed.toString());
$('body').append(scriptTag);
res.end($.html());
} catch (e) {
console.log(e);
}
});
});
let server = http.createServer(function (req, res) {
proxy.web(req, res, option, function (e) {
console.log(e);
});
});
console.log("listening on port 5051");
server.listen(5051);
I ended up writing a small Python Server using CherryPy and proxied the web app with mitmproxy. Everything is now working smoothly. Maybe I was doing it wrong with node-http-proxy but I also became sceptic about using it in a production environment.
I'm running a server with nodejs+mongodb:
let MongoClient = require('mongodb').MongoClient.connect("mongodb://localhost:27017/mydb", { useNewUrlParser: true } ),
(async () =>{
let client;
try {
client = await MongoClient;
...
I'm creating some data-visualizations and I need a simple way to access my backend data from javascript, is this possible? Ideally I would like full access.
You have to build a bridge, e.g. using a REST API:
// server.js
// npm install express, body-parser, mongodb
const app = require("express")();
const bodyParser = require("body-parser");
const db = require("mongodb").MongoClient.connect(/*...*/);
app.use(bodyParser.json());
app.post("/findOne", async (req, res) => {
try {
const connection = await db;
const result = await connection.findOne(req.body);
if(!result) throw new Error("Not found!");
res.status(200).json(result);
} catch(error) {
res.status(500).json(error);
}
});
// ... all those other methods ...
app.listen(80);
That way you can easily connect to it on the client:
// client.js
function findOne(query) {
const result = await fetch("/findOne/", {
method: "POST",
body: JSON.stringify(query),
headers:{
'Content-Type': 'application/json'
}
});
if(!result.ok) throw await result.json();
return await result.json();
}
Note: I hope you are aware that you also allow some strangers to play with your database if you do not validate the requests properly / add authentication.
For security purposes you should never do this, but hypothetically you could make an AJAX endpoint or WebSockets server on the node application that passes the input straight to mongoDB and takes the output straight back to the client.
It would be a much better practice to write a simple API using AJAX requests or WS to prevent the user from compromising your database.
This is my first ever question on here so please excuse any abnormalities in etiquette.
I am new to Node.js and backend programming in general.
Right now I am using Node and Tedious to connect to a local SQL server. I'd like to keep my main.js file clean and so am trying to put everything related to my SQL connection in a separate js file. Below would be the simplest possible form I have for my main.js.
var http = require('http');
var sqlmodule = require('./SQLconnection');
http.createServer(function (req, res) {
sqlmodule.makeConnection();
}).listen(8080);
I then have my SQLconnection.js file.
var Connection = require('tedious').Connection;
exports.makeConnection = function () {
var config = {
userName: 'XXXXXX',
password: 'XXXXXX',
server: 'XXXXXX'
};
var connection = new Connection(config);
};
//The below code is my event listener but I don't know how
//to incorporate it as part of the module.
connection.on('connect', function(err) {
if (err) {
console.error('Connection error', err);
} else {
console.log('Connected');
}
});
I have no problems when the listener isn't present in the file, but I can't find a way to have it part of the SQLconnection.js module. I've tried adding exports and module.exports before it in a few ways but to no success. It listening for an event and not being a normal function is stumping me.
How would I go about getting the event listeners in the separate file?
I'm also trying to go about this as vanilla as possible, so I'm just using Node.js and Tedious at this point.
change
exports.makeConnection = function () {
to
function makeConnection() {
...
module.exports = {makeConnection}
As an additional change, you need to put your connection listener in the sames scope as the connection variable. Personally, I would also have makeConnection return a Promise with the connection so you are not operating on a connection that has failed/not yet connected. Something like
var Connection = require('tedious').Connection;
function makeConnection() {
var config = {
userName: 'XXXXXX',
password: 'XXXXXX',
server: 'XXXXXX'
};
return new Promise((resolve, reject) => {
var connection = new Connection(config);
connection.on('connect', function(err) {
if (err) return reject(err);
resolve(connection);
});
}
};
module.exports = {makeConnection}