I'm running a very basic NodeJS application just to mess around with and learn ZSH but the queryStringObject I've defined from my parsedUrl in bash returns {fizz:'buzz'} and zsh returns [Object: null prototype] {}. What can I do in ZSH or in my app to make this console.log the JSON formatting instead of what ZSH is currently giving me?
var http = require('http');
var url = require('url');
var server = http.createServer(function(req,res){
var parsedUrl = url.parse(req.url, true); //true indicates to include query object
var queryStringObject = parsedUrl.query;
res.end('Hello World\n'); //curls to dom
console.log('Request received with query string:',queryStringObject);
}
I'll happily go back to using Bash to get what I'm used to seeing or simply use postman for this kind of testing, but I'm trying to learn little bits and pieces of ZSH as I go along and this behavior is likely to hit me often so I'd like to know how to best handle it.
It isn't a bash, zsh or any specific shell problem. Try JSON.parse() to print your object in JSON format.
console.log('Request received with query string:',JSON.parse(queryStringObject));
Related
Preamble
To start off, I'm not a developer; I'm just an analyst / product owner with time on their hands. While my team's actual developers have been busy finishing off projects before year-end I've been attempting to put together a very basic API server in Node.js for something we will look at next year.
I used Swagger to build an API spec and then used the Swagger code generator to get a basic Node.js server. The full code is near the bottom of this question.
The Problem
I'm coming across an issue when writing out to a log file using the fs module. I know that the ENOENT error is usually down to just specifying a path incorrectly, but the behaviour doesn't occur when I comment out the Swagger portion of the automatically generated code. (I took the logging code directly out of another tool I built in Node.js, so I'm fairly confident in that portion at least...)
When executing npm start, a few debugging items write to the console:
"Node Server Starting......
Current Directory:/mnt/c/Users/USER/Repositories/PROJECT/api
Trying to log data now!
Mock mode: disabled
PostgreSQL Pool created successfully
Your server is listening on port 3100 (http://localhost:3100)
Swagger-ui is available on http://localhost:3100/docs"
but then fs throws an ENOENT error:
events.js:174
throw er; // Unhandled 'error' event
^
Error: ENOENT: no such file or directory, open '../logs/logEvents2021-12-24.log'
Emitted 'error' event at:
at lazyFs.open (internal/fs/streams.js:277:12)
at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Investigating
Now normally, from what I understand, this would just mean I've got the paths wrong. However, the file has actually been created and the first line of the log file has been written just fine
My next thought was that I must've set the fs flags incorrectly, but it was set to 'a' for append:
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
Removing Swagger Code
Now here's the weird bit: if I remove the Swagger code, the log files write out just fine and I don't get the fs exception!
This is the specific Swagger code:
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
When I comment out this code, the log file writes out just fine.
The only thing I can think that might be happening is that somehow Swagger is modifying (?) the app's working directory so that fs no longer finds the same file?
Full Code
'use strict';
var path = require('path');
var fs = require('fs');
var http = require('http');
var oas3Tools = require('oas3-tools');
var serverPort = 3100;
// I am specifically tried using path.join that I found when investigating this issue, and referencing the app path, but to no avail
const __logdir = path.join(__dirname,'./logs');
//These are date and time functions I use to add timestamps to the logs
function dateNow(){
var dateNow = new Date().toISOString().slice(0,10).toString();
return dateNow
}
function rightNow(){
var timeNow = new Date().toTimeString().slice(0,8).toString();
return "["+timeNow+"] "
};
console.info("Node Server Starting......");
console.info("Current Directory: " + __dirname)
// Here I create the WriteStreams
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
var errorsFile = fs.createWriteStream(__logdir+"/errorEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Error Log File to location: %s \nWith error description: %s',__logdir, err);
});
// And create an additional console to write data out:
const Console = require('console').Console;
var logOut = new Console(logsFile,errorsFile);
console.info("Trying to log data now!") // Debugging logging
logOut.log("========== Server Startup Initiated ==========");
logOut.log(rightNow() + "Server Directory: "+ __dirname);
logOut.log(rightNow() + "Logs directory: "+__logdir);
// Here is the Swagger portion that seems to create the behaviour.
// It is unedited from the Swagger Code-Gen tool
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
In case it helps, this is the project's file structure . I am running this project within a WSL instance in VSCode on Windows, same as I have with other projects using fs.
Is anyone able to help me understand why fs can write the first log line but then break once the Swagger code gets going? Have I done something incredibly stupid?
Appreciate the help, thanks!
Edit: Tried to fix broken images.
Found the problem with some help from a friend. The issue boiled down to a lack of understanding of how the Swagger module works in the background, so this will likely be eye-rollingly obvious to most, but keeping this post around in case anyone else comes across this down the line.
So it seems that as part of the Swagger initialisation, any scripts within the utils folder will also be executed. I would not have picked up on this if it wasn't pointed out to me that in the middle of the console output there was a reference to some PostgreSQL code, even though I had taken all reference to it out of the main index.js file.
That's when I realised that the error wasn't actually being generated from the code posted above: it was being thrown from to that folder.
So I guess the answer is don't add stuff to the utils folder, but if you do, always add a bunch of console logging...
When I run my coffeescript test application I get this error"
2018-12-06 02:19:24,681 <NodeTest> [ERROR] [MainThread] node_test.run - NodeJS test for Node v7.9.0 did not pass. Exit status: 1
Std Out:
Std Error: /opt/node_js/conf.js:25
osVersion: 'MyOS 1.10.1.21
^^^^^^^^^^^^^^^^^
SyntaxError: Invalid or unexpected token
This is the contents of conf.js:
const require('https');
module.exports = {
// Endpoint
endpoint: 'https://123.456.789.876',
// creds
access: 'accessblablabla',
secret: 'secret blablabla',
// Other options
s3BucketEndpoint: false,
s3ForcePathStyle: true,
httpOptions: {
agent: new https.Agent({ca: '-----BEGIN CERTIFICATE-----'})
},
// OS version
myOsVersion: 'MyOS 1.10.1.21'
}
I can't understand why myOsVersion: '%s is any different compared with anything else in the file. Can anybody spot what I'm doing wrong?
I do not use MacOS at all but from my view, I think you should declare:
const https = require('https');
At the top of your code, because I see you use the instance of this (the new keyword). Hope this can help you a bit!
It turns out the problem was the space in the string.
As with #phix's comment, if I manually create the file there is no issue. I have a Python application which generates it. Perhaps it's including some hidden character or something.
Anyway, I only need the version number from the string so edited my Python code to look like this:
version = std_out.split()[1]
I have a NodeJS server running on Heroku (free version). The server accepts a HTTP POST from a client (passing a parameter) and does a web request (using the parameter) in a headless browser. The headless browser is called HorsemanJS. "Horseman is a Node.js module that makes using PhantomJS a pleasure. It has a straight-forward chainable API, understandable control-flow, support for multiple tabs, and built-in jQuery."
When I send a request (actually for loop of 20 requests) from a client (my computer) to the server, the server code works correctly (does 20 HorsemanJS web requests), and returns the expected values. Then, it waits for the next connection. This is all good.
The problem is, when I try to connect to the server with two different clients (my computer and phone) at the same time, it crashes. I can reboot the server and return to using one client successfully. How can I make it handle multiple clients?
Error when crashed:
child_process.js:788
child.stdout.addListener('data', function(chunk) {
^
TypeError: Cannot read property 'addListener' of undefined
at Object.exports.execFile (child_process.js:788:15)
at exports.exec (child_process.js:649:18)
at Socket.<anonymous> (/app/node_modules/node-horseman/node_modules/node-phantom-simple/node-phantom-simple.js:237:7)
at Socket.g (events.js:199:16)
at Socket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at Pipe.onread (net.js:538:20)
Extract from my server code:
var express = require('express');
var app = express();
var Horseman = require('node-horseman');
app.set('port', (process.env.PORT || 5000));
var printMessage = function() { console.log("Node app running on " + app.get('port')); };
var getAbc = function(response, input)
{
var horseman = new Horseman();
horseman
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0") // building web browser
.open('http://www.stackoverflow.com')
.html()
.then(function (result) {
var toReturn = ijk(result));
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end(toReturn);
}).close();
}
var handleXyz = function(request, response)
{
getAbc(response, request.query.input);
}
app.listen(app.get('port'), printMessage);
app.post('/xyz', handleXyz);
I tried moving .close inside .then, and also before .then. The code still worked for a single client, but not multiple.
I suspect the problem is that one client is closes a PhantomJS instance while/before the client tries to use it.
Putting horseman.close() inside of the finally may not give it enough time to actually close our the phantomJS process completely. Try changing the finally into a .then(), and return horseman.close().
I have fswebcam running on a Raspberry Pi. Using the command line, this saves JPG images.
I now want to receive these images in a Node.js application, and send them on to be used in a browser via dataURI.
On Node.js, I do:
var exec = require('child_process').exec;
exec("fswebcam -d /dev/video0 -r 160x120 --no-banner --save '-'", function(err, stdout, stderr) {
var imageBase64 = new Buffer(stdout).toString('base64');
I then send imageBase64 to the browser.
In the browser, setting the received data as the data URI fails:
image.src = "data:image/jpg;base64," + imageBase64;
Doing the above with a data URI created from a stored JPG created by fswebcam (via an online generator) works fine.
What am I not seeing here regarding formats and encodings?
The Content-Type should probably be image/jpeg and not image/jpg.
Also, the new Buffer(stdout) is redundant since stdout is already a Buffer, so you can just do stdout.toString('base64').
Lastly, if it's the data itself that is bad, you can double-check your base64-encoded output with this webpage or by writing stdout to disk and using the file command on it to ensure it's intact.
A little late but as I had same problem while playing with fswebcam from Node, the correct way would be to either use spawn and listen to "data" events on the spawned child's stdout stream. Or if you use exec then pass the encoding to be "buffer" or null as then the stdout argument to the callback will be again a Buffer instance, because otherwise by default it's utf-8 encoded string as stated in the exec docs
child_process.exec("fswebcam -", { encoding: "buffer" }, (error, stdout, stderr) => {
// stdout is Buffer now;
});
I need to use LOAD DATA LOCAL INFILE using mysql-libmysqlclient in node. However I get an error
[Error: Query error #1148: The used command is not allowed with this MySQL version]
This does not happen running the same command from sequel pro so the issue is most likely with the connection used by mysql-libmysqlclient.
Anoyone know what parameters to send to mysql-libmysqlclient in order to fix this?
initializing connection:
secrets = stuff
mysql_db = mysql.createConnectionQueuedSync()
mysql_db.initSync()
mysql_db.realConnectSync secrets.host, secrets.user, secrets.password, secrets.database
mysql_db.connectError
module.exports = exports = mysql_db
Sending query:
mysql_db = require('./config/mysql_db')
sql_insert_into = "LOAD DATA local INFILE 'file_to_send.txt'
into table #{table.name}
FIELDS TERMINATED BY ';'
LINES TERMINATED by '\\n'
"
mysql_db.query sql_insert_into, (err, answer)->
console.log "sent " + sql_insert_into
The error I get:
[Error: Query error #1148: The used command is not allowed with this MySQL version]
Minor note: I program in coffee script but this shouldn't change anything
If LOAD DATA LOCAL is disabled, either in the server or the client, a client that attempts to issue such a statement receives the following error message:
ERROR 1148: The used command is not allowed with this MySQL version
You can refer MySql website link below for how to enable:
https://dev.mysql.com/doc/refman/5.5/en/load-data-local.html