Need help for my thesis project: electron, express & sqlite3 - javascript

i'm currently working on my thesis project which consists of an educational desktop videogame for public and private schools in my city.
I also really like electron so i thought it would be a nice idea to make my app using it, however it's here when the problem starts. My university demands that all apps must use a relational database (SQL) and not non-relational db like MongoDB. Since the database i'll have it's relative small i chose SQLite.
For some reason i began to get a certain error:
C:\Users\Alejandro\Documents\Proyectos\express\node_modules\sqlite3\lib\binding\electron-v1.4-win32-ia32\node_sqlite3.node
After some research i found out that the reason was because electron executes on the client side and databases can only work on server side, to solve this issue i installed express on my app to execute a server in the background while electron executes the client (local client-server desktop app).
After some coding this was the result:
Main.js (Electron code changes)
const express = require('./resources/express')
const electron = require('electron')
// Module to control application life.
const app = electron.app
// Module to create native browser window.
const BrowserWindow = electron.BrowserWindow
const path = require('path')
const url = require('url')
// Keep a global reference of the window object, if you don't, the window will be closed automatically when the JavaScript object is garbage collected.
let mainWindow
function createWindow () {
// Create the browser window.
mainWindow = new BrowserWindow({width: 800, height: 600})
// and load the index.html of the app.
mainWindow.loadURL('http://localhost:8080/')
// Emitted when the window is closed.
mainWindow.on('closed', function () {
console.log('app server is closing')
express.closeServer()
mainWindow = null
})
}
Express.js (Express code):
const http = require('http')
const express = require('express')
const dbmanager = require('./dbmanager.js')
app = express()
app.set('view engine', 'pug')
app.get('/',function(request, response){
console.log('server started :D')
response.end()
})
app.get('/checktable',function(request, response){
dbmanager.createTable()
response.redirect('/receive')
})
app.get('/receive',function(request, response){
dbmanager.selectAll(function(err,data){
data.forEach(function(row){
console.log(row.id + " " + row.name + "Edad: " + row.age)
})
response.render('index', { title: 'Usuario', message: data[0].name + " " + data[0].last })
response.end()
})
})
var server = http.createServer(app).listen(8080)
var serverManager = {}
serverManager.closeServer = function(){// This executes
console.log('This is executing')
console.log('Server should close now')
server.close()
}
module.exports = serverManager
dbmanager.js (SQLite queries)
var sqlite3 = require('sqlite3').verbose()
var db = new sqlite3.Database('memory')
var queryManager = {}
queryManager.createTable = function(){
db.run("CREATE TABLE IF NOT EXISTS students (id int, name TEXT,last TEXT, age int)")
}
queryManager.deleteTable = function(){
db.run('DROP TABLE students')
}
queryManager.insertStudent = function(student){
var stmt = db.prepare("INSERT INTO students VALUES (?,?,?,?)")
stmt.run(student.id,student.name,student.last,student.age)
stmt.finalize()
console.log('Insert Successful')
}
queryManager.selectAll = function(callback){
db.all('SELECT * FROM students', function(err,rows){
if(err)
{
throw err;
}
else
{
callback(null, rows);
}
})
}
module.exports= queryManager
Index.pug (view)
html
head
title= title
body
table#table
h1= message
Before trying to execute the entire app i tried only executing the server and it worked.
I modified the npm start line from "electron ." to "electron . && node ./resources/express.js" so i could execute both the client and the server.
At that moment i commented all the lines related to the dbmanager.js to test if the client and server were working.
Needing to close the server at the moment the app windows was close i created a function to close the server when window close funcion is called, but it doesn't.
That's the first problem. The second problem shows up when i undo the comment on the dbmanager lines, i get the same error as before:
'C:\Users\Alejandro\Documents\Proyectos\express\node_modules\sqlite3\lib\binding\electron-v1.4-win32-ia32\node_sqlite3.node'
Am i doing something wrong? Please i really need help.

You're spinning up an Express server on the same machine as your Electron app, if your server can access a local database then what makes you think your app can't? You don't need an Express server. To use the sqlite3 module in an Electron app you just have to rebuild it to target the specific Electron version using one of the documented approaches.

Related

NodeJS HTTPS server serving over HTTP instead of HTTPS when running inside Docker container

I'm creating a website using NextJS and Docker so that I can easily deploy it. I used npx-create-next-app to initialize it and used this Dockerfile (slightly modified) to containerize it. Since I wanted to use SSL with my server without going through the hassle of setting up a proxy, I followed this article, and setup the custom server.
This worked fine when I ran it outside of a docker container, and performed as expected, serving over HTTPS. However when I containerized it, and tried to open the webpage over HTTPS, I came up with SSL_ERROR_RX_RECORD_TOO_LONG, but I could open the page using just HTTP (which I could not do when running outside of a container). Some googling led me to this question, from which I concluded that when running outside of a docker container, the custom server runs the server over HTTPS, as expected, however when I containerize it, it starts running HTTP, even though no code has been changed.
I'd expect the behavior to be the same when running locally or containerized.
At first I assumed this was due to invalid key and cert values in httpsOptions however I wasn't able to find anything that would make them invalid, and I don't see how that would cause this strange behavior. I tried changing the Docker run environment from node:alpine-16 to just node:latest to see if it had something to do with the parent image, but that was fruitless.
One other minor issue I had is that console.log does not seem to output to the container's log for some reason, I tried googling this but didn't find much of anything pertaining to it. This has made debugging much harder as I can't really output any debug data. The only log I get when running inside of a container is Listening on port 3000 url: http://localhost:3000, which I assume is output by some library/package as it isn't anywhere in my code.
Here is my custom server code in case it would be helpful:
const https = require('https');
const fs = require('fs');
const { parse } = require('url');
const next = require('next');
const dev = process.env.NODE_ENV !== 'production';
const hostname = "127.0.0.1";
const port = process.env.PORT || 3000
const app = next({ dev, hostname, port })
const handle = app.getRequestHandler()
const httpsOptions = {
key: fs.readFileSync('./cert/privkey.pem'),
cert: fs.readFileSync('./cert/fullchain.pem')
};
app.prepare().then(() => {
https.createServer(httpsOptions, async (req, res) => { // When running on docker this creates an HTTP server instead of HTTPS
const parsedUrl = parse(req.url, true)
const { pathname, query } = parsedUrl
await handle(req, res, parsedUrl)
}).listen(port, (err) => {
if(err) throw err
console.log(`Ready on https://localhost:${port}`)
})
})
Link to a reproducible example here.
The thing is, based on your sample repo, that your server.js file that is in the root of your repo gets overwritten in the image because of this line in the Dockerfile:
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
So the actual server.js that is running in the container is the server.js that is created by the yarn build command and it looks like this (you can exec into the container and see it for yourself):
const NextServer = require('next/dist/server/next-server').default
const http = require('http')
const path = require('path')
process.env.NODE_ENV = 'production'
process.chdir(__dirname)
// Make sure commands gracefully respect termination signals (e.g. from Docker)
// Allow the graceful termination to be manually configurable
if (!process.env.NEXT_MANUAL_SIG_HANDLE) {
process.on('SIGTERM', () => process.exit(0))
process.on('SIGINT', () => process.exit(0))
}
let handler
const server = http.createServer(async (req, res) => {
try {
await handler(req, res)
} catch (err) {
console.error(err);
res.statusCode = 500
res.end('internal server error')
}
})
const currentPort = parseInt(process.env.PORT, 10) || 3000
server.listen(currentPort, (err) => {
if (err) {
console.error("Failed to start server", err)
process.exit(1)
}
const nextServer = new NextServer({
hostname: 'localhost',
port: currentPort,
dir: path.join(__dirname),
dev: false,
customServer: false,
conf: {"env":{},"webpack":null,"webpackDevMiddleware":null,"eslint":{"ignoreDuringBuilds":false},"typescript":{"ignoreBuildErrors":false,"tsconfigPath":"tsconfig.json"},"distDir":"./.next","cleanDistDir":true,"assetPrefix":"","configOrigin":"next.config.js","useFileSystemPublicRoutes":true,"generateEtags":true,"pageExtensions":["tsx","ts","jsx","js"],"target":"server","poweredByHeader":true,"compress":true,"analyticsId":"","images":{"deviceSizes":[640,750,828,1080,1200,1920,2048,3840],"imageSizes":[16,32,48,64,96,128,256,384],"path":"/_next/image","loader":"default","loaderFile":"","domains":[],"disableStaticImages":false,"minimumCacheTTL":60,"formats":["image/webp"],"dangerouslyAllowSVG":false,"contentSecurityPolicy":"script-src 'none'; frame-src 'none'; sandbox;","remotePatterns":[],"unoptimized":false},"devIndicators":{"buildActivity":true,"buildActivityPosition":"bottom-right"},"onDemandEntries":{"maxInactiveAge":15000,"pagesBufferLength":2},"amp":{"canonicalBase":""},"basePath":"","sassOptions":{},"trailingSlash":false,"i18n":{"locales":["en"],"defaultLocale":"en"},"productionBrowserSourceMaps":false,"optimizeFonts":true,"excludeDefaultMomentLocales":true,"serverRuntimeConfig":{},"publicRuntimeConfig":{},"reactStrictMode":true,"httpAgentOptions":{"keepAlive":true},"outputFileTracing":true,"staticPageGenerationTimeout":60,"swcMinify":true,"output":"standalone","experimental":{"middlewarePrefetch":"flexible","optimisticClientCache":true,"manualClientBasePath":false,"legacyBrowsers":false,"newNextLinkBehavior":true,"cpus":7,"sharedPool":true,"profiling":false,"isrFlushToDisk":true,"workerThreads":false,"pageEnv":false,"optimizeCss":false,"nextScriptWorkers":false,"scrollRestoration":false,"externalDir":false,"disableOptimizedLoading":false,"gzipSize":true,"swcFileReading":true,"craCompat":false,"esmExternals":true,"appDir":false,"isrMemoryCacheSize":52428800,"fullySpecified":false,"outputFileTracingRoot":"","swcTraceProfiling":false,"forceSwcTransforms":false,"largePageDataBytes":128000,"enableUndici":false,"adjustFontFallbacks":false,"adjustFontFallbacksWithSizeAdjust":false,"trustHostHeader":false},"configFileName":"next.config.js"},
})
handler = nextServer.getRequestHandler()
console.log(
'Listening on port',
currentPort,
'url: http://localhost:' + currentPort
)
})
And as you see it starts a http server not a https. Also this is why the console.log("lksdfjls"); in your own server.js will not get executed.
What I would suggest is to leave node as it is, running on http://localhost:3000 and set up a reverse proxy that would forward incoming requests to this node backend that is accessible only from the reverse proxy. And of course reverse proxy would handle TLS termination. A docker compose setup would be more convenient for this so you could put the reverse proxy container (nginx for example) in the compose project too and map a directory from the docker host where your cert files are stored into the reverse proxy container at runtime - DO NOT BAKE CERTS OR ANY OTHER SECRETS INTO ANY IMAGE, not even if it is an internally used image only because it could leak out accidentally any time.
Also you could just manually run the two container with docker run but compose would make life easier it has a lot of capabilities for example you could scale compose services up and down so your backend service would run not in one but many containers. But if this would be a high load and/or business critical production stuff then you are better off with a better (real) container orchestrator like kubernetes, docker swarm, nomad etc but today as I see it the de facto container orchestrator is kubernetes.

Does log4js require any extra code to work on an Apache server?

I'm trying to add Log4js-Node to a Node.js server running on Apache. Here's my code:
const path = require("path");
const express = require("express");
const log4js = require('log4js');
const app = express();
const logger = log4js.getLogger();
logger.level = "debug";
const port = 443;
log4js.configure({
appenders: { everything: { type: 'file', filename: 'logs.log', flags: 'w' } },
categories: { default: { appenders: ['everything'], level: 'ALL' } }
});
const server = app.listen(port, () => {
logger.debug("listening to requests on port " + port);
});
app.get("/log", (req, res) => {
res.sendFile(path.join(__dirname + "/logs.log"));
});
When I run the script on Node.js on my computer and navigate to localhost:443/log I see what I expect, which is this:
[2020-03-17T22:50:43.145] [DEBUG] default - listening to requests on port 443
But when I run the code on a remote server it crashes and I get this in the error page (with part of the path replaced by me with "[removed]"):
App 25925 output: at Server. ([removed]/index.js:27:9)
App 25925 output: at Logger. [as debug] ([removed]/12/lib/node_modules/log4js/lib/logger.js:124:10)
App 25925 output: at Logger.log ([removed]/12/lib/node_modules/log4js/lib/logger.js:73:12)
App 25925 output: at Logger._log ([removed]/12/lib/node_modules/log4js/lib/logger.js:90:16)
App 25925 output: at Object.send ([removed]/12/lib/node_modules/log4js/lib/clustering.js:97:15)
App 25925 output: [removed]/12/lib/node_modules/log4js/lib/clustering.js:97
App 25925 output: at Object. ([removed]/12/lib/node_modules/log4js/lib/clustering.js:8:13)
I'm using A2 Hosting which uses Apache 2.4.41. I opted for Node.js 12.9.0, and Log4js 6.1.2. The package.json should be the same on both my computer and the server, and I've run npm install on both.
Is this just an issue with Log4js and the server, or have I missed something somewhere?
This was actually a relatively simple fix. The path referenced by the last error in the stack trace is a Log4js module that implements clustering support through Node's "cluster" module. The line "8" referenced is cluster = require("cluster"). It's wrapped in a try/catch block like this:
try {
cluster = require("cluster"); //eslint-disable-line
} catch (e) {
debug("cluster module not present");
disabled = true;
}
The installation of Node.js on my computer came with the "cluster" module, however as far as I can tell, the server I'm using doesn't support it. Also, the version of Node I'm using on my computer is newer than what I'm using on the server (so I've now installed 12.9 on my machine). I believe the older version of Node doesn't bother trying to catch the exception and tries to load the cluster module, fails, and then throws the error.
So the simple fix was to comment out most of the "try/catch" block, leaving just the contents of "catch" like this:
// try {
// cluster = require("cluster"); //eslint-disable-line
// } catch (e) {
debug("cluster module not present");
disabled = true;
// }
If someone has a better fix, I'm open to suggestions.
The same response of #skittleswrapper,thx, it work for me.
I use Node.js 14.18.1 with log4js 6.3.0.
But i wondering what'is the necessary of this module 'cluster' and if we can
add it to our app in other way.

What protocol should I use and what are the docs for it?

So I'm trying to build a Discord bot. These types of threads tend to get downvoted a lot on stackoverflow, so I'm hoping this doesn't happen to me.
This particular feature is acting as a temporary solution to my dashboard problem. Due to the nature of glitch.com's hosting, it's supposed to fall asleep after 5 minutes of http inactivity. I solved that already by adding a script that pings the URL every 4 minutes, but that caused another issue. I think what's happening is that because that script and the bot script are constantly running, and never technically 'finish', it never lets any incoming connection actually load the webpage. So my solution to that problem was to create another glitch project that would act as the dashboard website, and transfer information from the bot project. Of course then I'd need to create more scripts that communicate with each other via some internet protocol. The info recorded by the bot is all recorded in a private JSON database using the node-json-db npm library.
My problem is: I don't know what protocol would be best for this kind of thing. Even if I did know, then I'd have to go digging through the docs for the info I'm looking for.
My question is: What protocol should I use, and what docs do I need to read for this?
I've included some snippets of the code here:
The bot's server code (where I would add the script for communicating with the dashboard):
// server.js
// where your node app starts
// init project
const express = require('express');
const app = express();
const JsonDB = require('node-json-db');
const db = new JsonDB("myDataBase", true, true);
// we've started you off with Express,
// but feel free to use whatever libs or frameworks you'd like through `package.json`.
// http://expressjs.com/en/starter/static-files.html
app.use(express.static('public'));
// http://expressjs.com/en/starter/basic-routing.html
app.get('/', function(request, response) {
response.sendFile(__dirname + '/views/index.html');
});
app.post('/login/c3RvcCBoYWNrZXIh', function(request, response) {
var servername = request.param('servername');
var password = request.param('password');
if (db.getData("/" + servername + "/password") === password) {
response.json(db.getData("/" + servername));
} else {
response.json(null);
}
});
// listen for requests :)
const listener = app.listen(process.env.PORT, function() {
console.log('Your app is listening on port ' + listener.address().port);
});
// to keep the bot alive, since glitch puts projects to sleep after 5 mins of inactivity.
const http = require('http');
setInterval(() => {
http.get(`http://${process.env.PROJECT_DOMAIN}.glitch.me/`);
}, 270000);
The server.js on the dashboard website:
// server.js
// where your node app starts
// init project
const express = require('express');
const app = express();
const request = require('request');
// we've started you off with Express,
// but feel free to use whatever libs or frameworks you'd like through `package.json`.
// http://expressjs.com/en/starter/static-files.html
app.use(express.static('public'));
// http://expressjs.com/en/starter/basic-routing.html
app.get('/', function(request, response) {
response.sendFile(__dirname + '/views/index.html');
});
app.post('/login', function(request, response) {
var servername = request.param('servername');
var password = request.param('password');
if ("thereisnopassword" === password) {
response.sendFile(__dirname + '/dashboard/index.html');
} else {
response.sendFile(__dirname + '/views/wronginfo.html');
}
});
// listen for requests :)
const listener = app.listen(process.env.PORT, function() {
console.log('Your app is listening on port ' + listener.address().port);
});
I had this too, but solved it by simply putting the code to start the express server before the http loop.
// Require packages
const http = require('http');
const express = require('express');
const app = express();
// Express
app.get("/", (request, response) => {
response.sendStatus(200);
});
app.listen(process.env.PORT);
// Interval
setInterval(() => {
http.get(`http://${process.env.PROJECT_DOMAIN}.glitch.me/`);
}, 240000);
// Bot code
const Discord = require('discord.js');
const client = new Discord.Client();

Nodejs App doesnt spawn python child process when Nodejs app started using Systemd

I'd like to start my node js application on boot. Therefore I start a service from Systemd:
[Unit]
Description=Node.js server
After=network.target
[Service]
ExecStart=/usr/bin/node /var/www/Raspberry-Pi-Status/js/server.js
Restart = always
RestartSec=10
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=nodejs-server
Environment=NODE_ENV=production PORT=8000
Environment=PYTHONPATH=/usr/bin/python
[INSTALL]
WantedBy=multi-user.target
The server.js looks like this:
var util = require('util'),
spawn = require('child_process').spawn,
py = spawn('python',['temperature.py'],{detached: true});
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'monitor',
password : 'password',
database : 'temps'});
var app = require('http').createServer(handler),
io = require('socket.io').listen(app),
fs = require('fs'),
sys = require('util'),
exec = require('child_process').exec,
child;
// Listen on port 8000
app.listen(8000);
// If all goes well when you open the browser, load the index.html file
function handler(req, res) {
fs.readFile(__dirname+'/../index.html', function(err, data) {
if (err) {
// If no error, send an error message 500
console.log(err);
res.writeHead(500);
return res.end('Error loading index.html');
}
res.writeHead(200);
res.end(data);
});
}
py.stdout.on('data', function(data){
console.log('testing');
date = new Date().getTime();
temp = parseFloat(data);
io.sockets.emit('temperatureUpdate',date,temp);
});
// When we open the browser establish a connection to socket.io.
// Every 5 seconds to send the graph a new value.
io.sockets.on('connection', function(socket) {
console.log('user connected');
});
The node.js application should start a python script which reads out a temperature sensor. When I start node.js via the console everything works fine. However, when I start from Systemd, the python script is not spawned.
What's the problem there? Am I missing something?
Thanks in advance
Alexander
An issue could be the difference in the current working directory when run manually vs systemd. The spawn call used is documented has a default to inherit the current working directory.
When run via the shell, that would be whatever directory you are currently in. In man systemd.exec, you find the "WorkingDirectory=` directive, which documents systemd's default current working directory: "defaults to the root directory when systemd is running as a system instance".
So if your temperature.py is located in /var/www/Raspberry-Pi-Status, then set:
workingDirectory=/var/www/Raspberry-Pi-Status in your `[Service]` section.

Difficulty setting up a 'post' route in an express server issue

I'm attempting to validate that my client can post info to it's server. I've set up this 'route' in my Express server.
// server.js this is the server for the PvdEnroll application.
//
var express = require("express"),
http = require("http"),
mongoose = require( "mongoose" ),
app = express();
// static file directory for default routing
app.use(express.static(__dirname + "/client"));
app.use(express.urlencoded());
// create Express-powered HTTP server
http.createServer(app).listen(3000);
console.log("Server listening at http://127.0.0.1:3000/");
// set up a post route in the server
app.post("/selections", function (req, res) {
console.log("data has been posted to the server!");
});
app.post("/selections", function (req, res) {
console.log("Some data has been posted to the server from app.js");
})
The client uses this file:
var main = function () {
"use strict";
$.getJSON("../data/checkBoxesA.json", function(checkBoxTxt) {
checkBoxTxt.forEach(function (data) {
var $checkbox = "<input type ='checkbox' name = "
+ data.label + "id = 0 UNCHECKED/>";
$(".enroll_actions").append($checkbox);
$(".enroll_actions").append(' ' + data.label + "<br/>");
$(".enroll_actions").append(' ' + data.note + "<br/>");
$(".enroll_actions").append(' '+ "<br/>");
});
});
$(".comment-input").on("click", function (event) {
console.log("Hello World!");
// here we'll do a post to our selections route
$.post("selections", {}, function (response) {
console.log("Client says - We posted and the server responded!");
console.log("Response from server :", response);
console.log("STUBB1");
});
});
console.log("STUBB2");
};
$(document).ready(main);
In the Chrome console I'm getting:
POST file:///Users/*******/Projects/r_PE/app/PvdEnroll/client/html/selections net::ERR_FILE_NOT_FOUND
A path is being sought but a tutorial's example (which works!) and is structurally identical to mine(?) uses a name i.e. "selections" to establish a route between client and server.
I'm running the server on my Mac using Virtual Box and Vagrant.
Thanks for any clarification.
On restarting the server the log message is now "POST 127.0.0.1:3000/html/selections 404 (Not Found).
Okay. This is some helpful information!
Basically, your file structure is more or less this (some file names will be different, just look at the general structure):
Node.js code (including main .js file and the module.json)
client: a folder for your static content
html: a folder
index.html: the file you are currently using
Anyway, jQuery.post() is fed a relative path (as apposed to an absolute path). That means that, based on the location of where the code was, it will "guess" the location of the file.
That means that it was trying to find a route at [locahost:port]/html/selections when your server is listening at [localhost:port]/selections! You'll need to have jQuery post to /selections instead of just selections.
For more information, this blog post is helpful (relative paths work the same in Javascript/CSS). The helpful snippet:
Here is all you need to know about relative file paths:
Starting with "/" returns to the root directory and starts there
Starting with "../" moves one directory backwards and starts there
Starting with "../../" moves two directories backwards and starts there (and so on...)
To move forward, just start with the first subdirectory and keep moving forward

Categories

Resources