Simple issue with the Winston logger for NodeJS, taking a ridiculous amount of hours to solve:
const winston = require('winston');
exports.logger = winston.createLogger({
transports: [
new winston.transports.File({
format: winston.format.json(),
level: 'info',
timestamp: true,
maxsize: 512000, // ~ 5 MB
maxFiles: 10,
filename: '../logs/info.log'
}),
new winston.transports.Console({
format: winston.format.simple(),
level: 'info'
})
]
});
Just as a funny fact, it can create the file inside the folder if it exists, but cannot create the folder if it does not exists (WTF?).
The problem is that never saves any information to the file. I tried modifying the access privileges. Nothing. No feedback received anywhere about the issue.
I may be too dumb for the documentation because nothing is covered about this but many people seem to be facing the same issue.
Ok, I just discovered the problem: depending on where the file is called from, the path will be interpreted differently.
Diagnose: The chunk I am showing was not in the project's root directory, but the main file I was executing is (and is requiring that chunk of code), and because of that the path was being interpreted from the root file's perspective.
Result: I was having my logs written right outside of my project's directory.
Related
Preamble
To start off, I'm not a developer; I'm just an analyst / product owner with time on their hands. While my team's actual developers have been busy finishing off projects before year-end I've been attempting to put together a very basic API server in Node.js for something we will look at next year.
I used Swagger to build an API spec and then used the Swagger code generator to get a basic Node.js server. The full code is near the bottom of this question.
The Problem
I'm coming across an issue when writing out to a log file using the fs module. I know that the ENOENT error is usually down to just specifying a path incorrectly, but the behaviour doesn't occur when I comment out the Swagger portion of the automatically generated code. (I took the logging code directly out of another tool I built in Node.js, so I'm fairly confident in that portion at least...)
When executing npm start, a few debugging items write to the console:
"Node Server Starting......
Current Directory:/mnt/c/Users/USER/Repositories/PROJECT/api
Trying to log data now!
Mock mode: disabled
PostgreSQL Pool created successfully
Your server is listening on port 3100 (http://localhost:3100)
Swagger-ui is available on http://localhost:3100/docs"
but then fs throws an ENOENT error:
events.js:174
throw er; // Unhandled 'error' event
^
Error: ENOENT: no such file or directory, open '../logs/logEvents2021-12-24.log'
Emitted 'error' event at:
at lazyFs.open (internal/fs/streams.js:277:12)
at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Investigating
Now normally, from what I understand, this would just mean I've got the paths wrong. However, the file has actually been created and the first line of the log file has been written just fine
My next thought was that I must've set the fs flags incorrectly, but it was set to 'a' for append:
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
Removing Swagger Code
Now here's the weird bit: if I remove the Swagger code, the log files write out just fine and I don't get the fs exception!
This is the specific Swagger code:
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
When I comment out this code, the log file writes out just fine.
The only thing I can think that might be happening is that somehow Swagger is modifying (?) the app's working directory so that fs no longer finds the same file?
Full Code
'use strict';
var path = require('path');
var fs = require('fs');
var http = require('http');
var oas3Tools = require('oas3-tools');
var serverPort = 3100;
// I am specifically tried using path.join that I found when investigating this issue, and referencing the app path, but to no avail
const __logdir = path.join(__dirname,'./logs');
//These are date and time functions I use to add timestamps to the logs
function dateNow(){
var dateNow = new Date().toISOString().slice(0,10).toString();
return dateNow
}
function rightNow(){
var timeNow = new Date().toTimeString().slice(0,8).toString();
return "["+timeNow+"] "
};
console.info("Node Server Starting......");
console.info("Current Directory: " + __dirname)
// Here I create the WriteStreams
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
var errorsFile = fs.createWriteStream(__logdir+"/errorEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Error Log File to location: %s \nWith error description: %s',__logdir, err);
});
// And create an additional console to write data out:
const Console = require('console').Console;
var logOut = new Console(logsFile,errorsFile);
console.info("Trying to log data now!") // Debugging logging
logOut.log("========== Server Startup Initiated ==========");
logOut.log(rightNow() + "Server Directory: "+ __dirname);
logOut.log(rightNow() + "Logs directory: "+__logdir);
// Here is the Swagger portion that seems to create the behaviour.
// It is unedited from the Swagger Code-Gen tool
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
In case it helps, this is the project's file structure . I am running this project within a WSL instance in VSCode on Windows, same as I have with other projects using fs.
Is anyone able to help me understand why fs can write the first log line but then break once the Swagger code gets going? Have I done something incredibly stupid?
Appreciate the help, thanks!
Edit: Tried to fix broken images.
Found the problem with some help from a friend. The issue boiled down to a lack of understanding of how the Swagger module works in the background, so this will likely be eye-rollingly obvious to most, but keeping this post around in case anyone else comes across this down the line.
So it seems that as part of the Swagger initialisation, any scripts within the utils folder will also be executed. I would not have picked up on this if it wasn't pointed out to me that in the middle of the console output there was a reference to some PostgreSQL code, even though I had taken all reference to it out of the main index.js file.
That's when I realised that the error wasn't actually being generated from the code posted above: it was being thrown from to that folder.
So I guess the answer is don't add stuff to the utils folder, but if you do, always add a bunch of console logging...
I run a gulp task using NodeJS module browser-sync as below.
=== File gulpfile.js ===
let browserSync = require('browser-sync').create();
gulp.task('browser-sync', function(){
browserSync.init( {
open: true,
injectChanges: true,
proxy: 'https://generalgulp.devsunset',
host: '192.168.1.76',
serveStatic: ['.'],
https: {
key: 'C:\\WebProjects\\GeneralGulp\\resources\\certificates\\server-generalgulp.key',
cert: 'C:\\WebProjects\\GeneralGulp\\resources\\certificates\\server-generalgulp.crt'
}
});
});
=== ===
My local project information is as below (I use latest up to current post date):
Node version: 17.1.0
NPM versions: 8.1.3
gulp: 4.0.2
NPM module browser-sync: 2.27.7
I run the browser-sync task. The output looks good.
==>
Using gulpfile C:\WebProjects\GeneralGulp\gulpfile.js
[Browsersync] Starting 'browser-sync'...
[Browsersync] Proxying: https://generalgulp.devsunset
Access URLs:
Local: https://localhost:3000
External: https://192.168.1.76:3000
UI: http://localhost:3001
UI External: http://localhost:3001
==>
I already add the SSL certificate for this domain to trusted root. I also have DNS records pointing from this domain ( https://generalgulp.devsunset ) - IP addresses ( 127.0.0.1 & 192.168.1.76)
I can access the site from both local & external address.
However, when I try to access the local resources using proxied domain ( https://generalgulp.devsunset
) , it gets an HTTP 403 :
Access to <my_custom_domain> was denied. You are not authorize to
view this page
I suppose when running my gulp "browser-sync" task, it will translate the custom domain to the https://localhost:3000 or https://192.168.1.76:3000
I have followed exactly the documents of https://browsersync.io/docs . I have also made an attempt with all solutions I could find. Those solutions led me to the gulp task that I wrote at the beginning.
I would appreciate if you can suggest me which things I should do further to troubleshoot why does my browser-sync cannot “proxy” my domain? Is there any parameter missing in my Gulp task?
Thanks !
I have modified the "proxy" parameter as below and it works when i access the proxied domain with given port:
(for my case is http(s)://generalgulp.devsunset:3000 )
`gulp.task('browser-sync', function(){
browserSync.init( {
open: true,
injectChanges: true,
proxy: 'generalgulp.devsunset',
host: '192.168.1.76',
serveStatic: ['.'],
https: {
key: 'C:\\WebProjects\\GeneralGulp\\resources\\certificates\\server-generalgulp.key',
cert: 'C:\\WebProjects\\GeneralGulp\\resources\\certificates\\server-generalgulp.crt'
}
});
});
`
This is a temporary acceptable solution regarding to the current question scope.
However, What i expect is the browser-sync will auto-forward traffic from custom domain ( http(s)://generalgulp.devsunset ) to : ( http://192.168.1.76:3000 ).
Does browser-sync allow users to do it ?
When I run my coffeescript test application I get this error"
2018-12-06 02:19:24,681 <NodeTest> [ERROR] [MainThread] node_test.run - NodeJS test for Node v7.9.0 did not pass. Exit status: 1
Std Out:
Std Error: /opt/node_js/conf.js:25
osVersion: 'MyOS 1.10.1.21
^^^^^^^^^^^^^^^^^
SyntaxError: Invalid or unexpected token
This is the contents of conf.js:
const require('https');
module.exports = {
// Endpoint
endpoint: 'https://123.456.789.876',
// creds
access: 'accessblablabla',
secret: 'secret blablabla',
// Other options
s3BucketEndpoint: false,
s3ForcePathStyle: true,
httpOptions: {
agent: new https.Agent({ca: '-----BEGIN CERTIFICATE-----'})
},
// OS version
myOsVersion: 'MyOS 1.10.1.21'
}
I can't understand why myOsVersion: '%s is any different compared with anything else in the file. Can anybody spot what I'm doing wrong?
I do not use MacOS at all but from my view, I think you should declare:
const https = require('https');
At the top of your code, because I see you use the instance of this (the new keyword). Hope this can help you a bit!
It turns out the problem was the space in the string.
As with #phix's comment, if I manually create the file there is no issue. I have a Python application which generates it. Perhaps it's including some hidden character or something.
Anyway, I only need the version number from the string so edited my Python code to look like this:
version = std_out.split()[1]
Dear Strongloop community
I am following the example provided by loopback for evaluating local-storage data store.
It works fine and I can use the rest API to create and retrieve files from the local file system (used as a data source).
I had to used "$ npm install loopback-component-storage --save".
the datasource defined in datasource.json is:
{
"db": {
"name": "db",
"connector": "memory"
},
"localFileSystemStorage": {
"name": "localFileSystemStorage",
"connector": "loopback-component-storage",
"provider": "filesystem",
"root": "./property_data"
}
}
the ./property_data directory is where all the data is stored. (relative to the project directory) and I can access it through rest with no issues.
Next I was trying to generate Angular service component so I can connect to the loopback server using Angular application. So inside the client directory I executed:
../example-2.0/client$ lb-ng ../server/server.js js/lb-services.js
and it failed:
/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/providers/filesystem/index.js:25
throw new Error('FileSystemProvider: Path does not exist: ' + this.root);
^
Error: FileSystemProvider: Path does not exist: ./server/storage
at new FileSystemProvider (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/providers/filesystem/index.js:25:11)
at Object.module.exports.createClient (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/providers/filesystem/index.js:17:10)
at Object.createClient (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/factory.js:86:18)
at new StorageService (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/storage-service.js:29:25)
at Object.exports.initialize (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-component-storage/lib/storage-connector.js:12:19)
at DataSource.setup (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-datasource-juggler/lib/datasource.js:332:17)
at new DataSource (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback-datasource-juggler/lib/datasource.js:109:8)
at Registry.createDataSource (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback/lib/registry.js:349:12)
at dataSourcesFromConfig (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback/lib/application.js:415:19)
at EventEmitter.app.dataSource (/home/eval/projects/loopback-component-storage/example-2.0/node_modules/loopback/lib/application.js:222:12)
Am I missing something?
I could really use some help here.
Thanks
In datasource.json file you have provided the relative path ./property_data with respect to /server folder.
Thus you need to call any method from that folder only.
Now when you are running it through the server then you are running it from /server folder thats why it runs without an issue but for angular Sdk generation you are doing it from client/ folder. Thus its genetating Path does not exist error.
Thus the correct syntax would be:
../example-2.0/server$ lb-ng server.js ../client/js/lb-services.js
How can I update log file when using Winston to handle logging for node.js. Below is the code which I tried. Only first time the data is getting saved in log file.
var logger = new (winston.Logger)({
transports: [
new (winston.transports.Console)(),
new (winston.transports.File)({ filename: '2012-07-09.log' })
]
});
logger.log('info', 'Test Log Message', { anything: 'This is metadata' });
Im using Node version 0.10.10 and winston 0.7.1
Any help on this will be really helpful. Thanks
Update to winston 7.2
https://github.com/mozilla/persona/commit/ecd695c52d9edbfa46b710ec8832c9fdf890ae16
I use 0.10.24 .It works for me now