I am having trouble pinpointing where my problem is coming from. Currently,
there is a constant request? for /socket.io/ and I am not sure why.
I checked my developer tools in chrome in the network panel and I don't actually see requests being made. However, the console.logs I put in my code are continuously being hit showing that there is a continuous attempt for /socket.io/.
Here is my project:
https://github.com/liondancer/ReviewWeb
I performed a grep on my node_modules and this was the only result:
$grep -rn "socket.io" node_modules/
node_modules//hapi/API.md:324:var SocketIO = require('socket.io');
I am using webpack so grep on the file would give me huge, unreadable results.
In my project's ReviewWeb/server/index.js I put some console.logs
server.ext('onPreResponse', function(request, reply) {
if (typeof request.response.statusCode !== 'undefined') {
console.log('Found server path');
return reply.continue();
}
console.log(request.response.statusCode);
console.log('rendering server side react', request.path)
Router.run(routes, request.path, function(Handler, router) {
var html = React.renderToStaticMarkup(<Handler />);
// http://cheeriojs.github.io/cheerio/
// Server knows HTML with Cheerio
var $ = Cheerio.load(HTMLBASE);
// http://cheeriojs.github.io/cheerio/#manipulation
// prepend inserts content as the first child of each of the selected elements.
$('#app').prepend(html);
// convert to html
var output = $.html();
reply(output);
});
});
This is the result from the console.logs
undefined
rendering server side react /socket.io/
Warning: No route matches path "/socket.io/". Make sure you have <Route path="/socket.io/"> somewhere in your routes
undefined
rendering server side react /socket.io/
Warning: No route matches path "/socket.io/". Make sure you have <Route path="/socket.io/"> somewhere in your routes
...
...
...
continuously...
Not sure how why this is happening and I am having a hard time finding the source of the problem.
* UPDATE *
I was using port 8080 and it was causing a problem. I changed to port 8000 and the problem didn't occur. I have no idea why but perhaps this could be a reason:
http://webpack.github.io/docs/webpack-dev-server.html#hot-mode
But I only have devtools set up to be source-map in my webpack.config.js
Related
Preamble
To start off, I'm not a developer; I'm just an analyst / product owner with time on their hands. While my team's actual developers have been busy finishing off projects before year-end I've been attempting to put together a very basic API server in Node.js for something we will look at next year.
I used Swagger to build an API spec and then used the Swagger code generator to get a basic Node.js server. The full code is near the bottom of this question.
The Problem
I'm coming across an issue when writing out to a log file using the fs module. I know that the ENOENT error is usually down to just specifying a path incorrectly, but the behaviour doesn't occur when I comment out the Swagger portion of the automatically generated code. (I took the logging code directly out of another tool I built in Node.js, so I'm fairly confident in that portion at least...)
When executing npm start, a few debugging items write to the console:
"Node Server Starting......
Current Directory:/mnt/c/Users/USER/Repositories/PROJECT/api
Trying to log data now!
Mock mode: disabled
PostgreSQL Pool created successfully
Your server is listening on port 3100 (http://localhost:3100)
Swagger-ui is available on http://localhost:3100/docs"
but then fs throws an ENOENT error:
events.js:174
throw er; // Unhandled 'error' event
^
Error: ENOENT: no such file or directory, open '../logs/logEvents2021-12-24.log'
Emitted 'error' event at:
at lazyFs.open (internal/fs/streams.js:277:12)
at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Investigating
Now normally, from what I understand, this would just mean I've got the paths wrong. However, the file has actually been created and the first line of the log file has been written just fine
My next thought was that I must've set the fs flags incorrectly, but it was set to 'a' for append:
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
Removing Swagger Code
Now here's the weird bit: if I remove the Swagger code, the log files write out just fine and I don't get the fs exception!
This is the specific Swagger code:
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
When I comment out this code, the log file writes out just fine.
The only thing I can think that might be happening is that somehow Swagger is modifying (?) the app's working directory so that fs no longer finds the same file?
Full Code
'use strict';
var path = require('path');
var fs = require('fs');
var http = require('http');
var oas3Tools = require('oas3-tools');
var serverPort = 3100;
// I am specifically tried using path.join that I found when investigating this issue, and referencing the app path, but to no avail
const __logdir = path.join(__dirname,'./logs');
//These are date and time functions I use to add timestamps to the logs
function dateNow(){
var dateNow = new Date().toISOString().slice(0,10).toString();
return dateNow
}
function rightNow(){
var timeNow = new Date().toTimeString().slice(0,8).toString();
return "["+timeNow+"] "
};
console.info("Node Server Starting......");
console.info("Current Directory: " + __dirname)
// Here I create the WriteStreams
var logsFile = fs.createWriteStream(__logdir+"/logEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Log File to location: %s \nWith error description: %s',__logdir, err);
});
var errorsFile = fs.createWriteStream(__logdir+"/errorEvents"+dateNow()+'.log',{flags: 'a'},(err) =>{
console.error('Could not write new Error Log File to location: %s \nWith error description: %s',__logdir, err);
});
// And create an additional console to write data out:
const Console = require('console').Console;
var logOut = new Console(logsFile,errorsFile);
console.info("Trying to log data now!") // Debugging logging
logOut.log("========== Server Startup Initiated ==========");
logOut.log(rightNow() + "Server Directory: "+ __dirname);
logOut.log(rightNow() + "Logs directory: "+__logdir);
// Here is the Swagger portion that seems to create the behaviour.
// It is unedited from the Swagger Code-Gen tool
// swaggerRouter configuration
var options = {
routing: {
controllers: path.join(__dirname, './controllers')
},
};
var expressAppConfig = oas3Tools.expressAppConfig(path.join(__dirname, '/api/openapi.yaml'), options);
var app = expressAppConfig.getApp();
// Initialize the Swagger middleware
http.createServer(app).listen(serverPort, function () {
console.info('Your server is listening on port %d (http://localhost:%d)', serverPort, serverPort);
console.info('Swagger-ui is available on http://localhost:%d/docs', serverPort);
}).on('error',console.error);
In case it helps, this is the project's file structure . I am running this project within a WSL instance in VSCode on Windows, same as I have with other projects using fs.
Is anyone able to help me understand why fs can write the first log line but then break once the Swagger code gets going? Have I done something incredibly stupid?
Appreciate the help, thanks!
Edit: Tried to fix broken images.
Found the problem with some help from a friend. The issue boiled down to a lack of understanding of how the Swagger module works in the background, so this will likely be eye-rollingly obvious to most, but keeping this post around in case anyone else comes across this down the line.
So it seems that as part of the Swagger initialisation, any scripts within the utils folder will also be executed. I would not have picked up on this if it wasn't pointed out to me that in the middle of the console output there was a reference to some PostgreSQL code, even though I had taken all reference to it out of the main index.js file.
That's when I realised that the error wasn't actually being generated from the code posted above: it was being thrown from to that folder.
So I guess the answer is don't add stuff to the utils folder, but if you do, always add a bunch of console logging...
INFO: I'm referring to this question I asked on Super User, but couldn't get an answer and I think this is a good place to ask, since the problem is probably code related.
I'm currently running a simple Node.JS server with express.js on my RaspberryPi with Debian installed on it. Everything works fine, but every morning I wake up to see my server isn't running anymore (the server process I started with the command node main.js).
My first guess was, that the Pi has some kind of sleep mode, which it enters after a couple of hours without traffic/etc, and which shuts down the server, but I also run a dydns-client, which is still up every morning (I also was informed, that the RaspberryPi doesn't come with a sleep mode).
I wrote a simple script to check whether the process is running and writes it into a log file, but today morning I had to notice, that this script was wasn't running as well (only for around two hours, it logs the server state every 15 seconds and the last state was running).
Here is the script:
#!/bin/sh
MATCH="SOME_PROCESS_NAME"
while [ true ]
do
if [ "$(ps -ef | grep ${MATCH} | grep -v grep)" ]; then
echo "$(date): Process is running ..."
else
echo "$(date): Process has ended ..."
fi
sleep 15
done
Is there a way to track a process after I started it to check tomorrow morning, what killed my process or why it ended (the script obviously didn't work)?
The server itself looks pretty simple and I don't think there is some kind of auto-shutdown I missed. Here is the code I used.
var express = require('express');
var path = require('path');
var server = express();
server.use(express.static(path.join(__dirname, 'public')));
server.listen(1337);
console.log("Server listening (PORT: " + 1337 + ") ...");
Any idea what to do, to keep the server running/find out what is the stopping reason?
UPDATE: I received a working answer over at RaspberryPi-stackexchange.
My guess is the Raspberry Pi restarts at midnight or something similar. to fix this maybe add an entry for your server process rc.local file. you can add commands to the rc.local file by editing /etc/rc.local
Would this helps https://raspberrypi.stackexchange.com/questions/8741/when-does-the-os-kill-an-application ?
I would like to suggest a different approach to monitor your process until you can get more information, to edit, then check, then start (wrote on the fly)
var fs = require('fs')
var spawn = require('child_process');
var child = spawn(process.argv[0], 'your/bin.js', {stdio:['ignore', 'pipe', 'pipe']})
child.stdout.pipe(fs.createReadStream('stdout.log'))
child.stderr.pipe(fs.createReadStream('stderr.log'))
child.on('error', function (err) {
fs.writeFile('error.log', JSON.stringify(err), function () { /* void */ })
})
child.on('close', function (code, signal) {
fs.writeFile('exit.log', "code="+code+" signal="+signal, function () { /* void */ })
})
Ok, I'm writing a pyqt software to generate a webpage. Due to some security issues with Chrome and other things, I need a webserver to test the webpage.
So I thought to create a button called run, that you can click or press f5 to start a server, and open the browser to the page. The snippet of code that this button calls, simplified (there is some code to do things, including changing current directory and such), looks like this:
import sys
import webbrowser
from SimpleHTTPServer import SimpleHTTPRequestHandler as HandlerClass
from BaseHTTPServer import HTTPServer as ServerClass
Protocol = 'HTTP/1.0'
port = 8080
ip = '127.0.0.1'
new = 2 #goes to new tab
url = "http://"+ip+":{0}".format(port)
serverAddress = (ip,port)
HandlerClass.protocol = Protocol
httpd = ServerClass(serverAddress, HandlerClass)
sa = httpd.socket.getsockname()
webbrowser.open(url,new=new)
httpd.serve_forever()
Ok, the problem is as serve_forever is called, it can be expected to serve forever. Is there a way to kill the server after browser is closed?
Edit: I understand many people recommend using threads but I can't find a way to detect that the browser has closed or killing the thread in system monitor (I'm on Ubuntu) while testing.
Edit2: ok, I've read webbrowser.py, it doesn't seem to return any sort of process identifier...
Edit3: I'm reflecting on this, maybe the correct approach would be checking if someone is accessing the server, and if not, then kill it... This way I can detect if the tab was closed... Problem is the only way I can think uses a dummy page with this power that loads whatever page to test inside, which seems too hackish...
It seems if I can find a way of doing this, maybe through error responses...I can do a webserver in a subprocess that has a while and exits by itself like the one here: https://docs.python.org/2/library/basehttpserver.html#more-examples
import sys
#from threading import Thread
import webbrowser
import BaseHTTPServer
import SimpleHTTPServer
serverClass=BaseHTTPServer.HTTPServer
handlerClass=SimpleHTTPServer.SimpleHTTPRequestHandler
Protocol = "HTTP/1.0"
port = 8080
ip = '127.0.0.1'
new = 2 #2 goes to new tab, 0 same and 1 window.
url = "http://"+ip+":{0}".format(port)
handlerClass.protocol = Protocol
httpd = serverClass((ip,port), handlerClass)
sa = httpd.socket.getsockname()
print("\n---\nServing HTTP on {0}, port {1}\n---\n".format(sa[0],sa[1]) )
browserOk = webbrowser.open(url,new=new)
def runWhileTrue():
while True:
#print(vars(httpd))
httpd.handle_request()
runWhileTrue()
Right now I'm thinking about using a timer like a watchdog, if the server is not used more then a period, it get's killed... But I think this is an awful solution... I wanted the browser to ping for it for some time while the tab is opened...maybe, don't know if optimal, looking this code right now : SimpleHTTPServer and SocketServer .
Thinking maybe if the server could understand a message from the website it could break loop. The tab closure could be detected in javascript like here : Browser/tab close detection using javascript (or any other language). Don't know how to communicate this to the server.
EditFinal:
In the javascript code of the webpage, I've inserted:
window.addEventListener('unload', function (e) { e.preventDefault(); jsonLevelGet("http://127.0.0.1:8081/exit.json"); }, false);
Then, the python code is this server.py:
import sys
from threading import Thread
import webbrowser
import BaseHTTPServer
import SimpleHTTPServer
serverClass=BaseHTTPServer.HTTPServer
handlerClass=SimpleHTTPServer.SimpleHTTPRequestHandler
Protocol = "HTTP/1.0"
port = 8080
ip = '127.0.0.1'
admIp = ip
admPort = 8081
new = 2 #2 goes to new tab, 0 same and 1 window.
url = "http://"+ip+":{0}".format(port)
handlerClass.protocol = Protocol
httpdGame = serverClass((ip,port), handlerClass)
httpdAdm = serverClass((admIp,admPort), handlerClass)
sa = httpdGame.socket.getsockname()
sb = httpdAdm.socket.getsockname()
print("\n---\nServing HTTP on {0}, port {1}\n---\n".format(sa[0],sa[1]) )
print("\n---\nAdm HTTP listening on {0}, port {1}\n---\n".format(sb[0],sb[1]) )
browserOk = webbrowser.open(url,new=new)
def runGameServer():
httpdGame.serve_forever()
print("\nrunGameServer stopped\n")
httpdAdm.shutdown()
return
def runAdmServer():
httpdAdm.handle_request()
httpdGame.shutdown()
print("\nrunAdmServer stopped\n")
return
gameServerThread = Thread(target=runGameServer)
gameServerThread.daemon = True
admServerThread = Thread(target=runAdmServer)
gameServerThread.start()
admServerThread.start()
admServerThread.join()
It works! When the tab is closed, the server.py code exits! Thanks #st.never!
As you said, you could detect (in Javascript, in the browser) that the window is being closed, and send one last request to the server to shut it down.
If you don't want to inspect all the requests searching for the "poweroff request", you can instead have your server listen on two different ports (probably on different threads). For example, the "main" server listens on port 8080 with the current behaviour, and a separate instance listens on port 8081. Then you can simply shut down the server whenever any request reaches port 8081.
I am having a problem similar to socket.io issue using sails.js. Every once in a while (once per day, or even few hours, it varies), a visitor to the web site/app will crash Node, seemingly due to the way his websocket client tries to connect. Anyway, here's the crash log:
debug: Lowering sails...
/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/express/node_modules/connect/lib/utils.js:216
return 0 == str.indexOf('s:')
^
TypeError: Cannot call method 'indexOf' of undefined
at exports.parseSignedCookie (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/express/node_modules/connect/lib/utils.js:216:19)
at Manager.socketAttemptingToConnect (/Volumes/Two/Sites/lsdfinder/node_modules/sails/lib/hooks/sockets/authorization.js:35:26)
at Manager.authorize (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:910:31)
at Manager.handleHandshake (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:786:8)
at Manager.handleRequest (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:593:12)
at Server.<anonymous> (/Volumes/Two/Sites/lsdfinder/node_modules/sails/node_modules/socket.io/lib/manager.js:119:10)
at Server.EventEmitter.emit (events.js:98:17)
at HTTPParser.parser.onIncoming (http.js:2076:12)
at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:120:23)
at Socket.socket.ondata (http.js:1966:22)
9 Oct 10:42:24 - [nodemon] app crashed - waiting for file changes before starting...
In config/sockets.js, authorization is set to true. Not sure what else to do, where to fix this. Any suggestions? I can read the Sails docs too, but this appears to be a problem in Express/Connect, no? Thanks.
...René
The problem is that once every so often, a client will connect that has no cookies. Sails.js is using util.parseSignedCookie() from Connect without checking for errors, and therefore an error is thrown. This is what it looks like in Sails:
if (handshake.headers.cookie) {
handshake.cookie = cookie.parse(handshake.headers.cookie);
handshake.sessionID = parseSignedCookie(handshake.cookie[sails.config.session.key], sails.config.session.secret);
}
If you take a look into the cookieParser() middleware of Connect, you can see error checking is required:
if (cookies) {
try {
req.cookies = cookie.parse(cookies);
if (secret) {
req.signedCookies = utils.parseSignedCookies(req.cookies, secret);
req.signedCookies = utils.parseJSONCookies(req.signedCookies);
}
req.cookies = utils.parseJSONCookies(req.cookies);
} catch (err) {
err.status = 400;
return next(err);
}
}
I've created a Gist here that fixes the problem, and will submit a pull request to Sails.js when I have the time. The Gist uses Connect's cookieParser() middleware to automatically handle errors. If you want to use this, modify this file in your modules folder:
node_modules/sails/lib/hooks/sockets/authorization.js
If you are doing a crossdomain request, you could turn off authorization.
In *site_dir/config/sockets.js* set authorization to false. One way of doing it. You can also call your api with something like this
bash
**http://localhost:1337?cookie=smokeybear**
Its is in the comments on the sockets.js file.
I've encountered a very weird problem in RingoJS. Consider the following code saved in main.js:
var {Application} = require("stick");
var {main} = require("ringo/httpserver");
var app = exports.app = new Application;
app.configure("error", "notfound");
if (module === require.main)
main(module.id);
Running ringo main.js starts the http server on http://localhost:8080/ as expected, displaying the default notfound page.
Now consider the following code, which is the same as above, except that I hard coded the if condition as true:
var {Application} = require("stick");
var {main} = require("ringo/httpserver");
var app = exports.app = new Application;
app.configure("error", "notfound");
if (true) main(module.id);
Running ringo main.js starts the http server, but on opening the site it gives me an error page instead of the notfound page. It says Wrapped java.net.BindException: Address already in use and gives me the following stack trace:
at ringo/httpserver.js:327 (Server)
at ringo/httpserver.js:428 (init)
at ringo/httpserver.js:506 (main)
at /home/aaditmshah/main.js:5
at ringo/jsgi/connector.js:28 (handleRequest)
What's happening? How does the if condition affect the server? After all both the conditions express the value true (which is the reason why the http server works). This has me abosultely confounded. Is it a bug?
the request coming in will re-execute your main module in the second version of your script. there's a reason why we write "if (module === require.main)", see the bottom of this page: http://ringojs.org/tutorial/httpserver.md