Develop Reference

API request through Docker & Nodejs fails but works perfectly when accessed directly using the Nodejs

I am developing an application using Nodejs and Docker. Within the code, I need to make the request to GitLab API to get the data. When I run the application through the Docker command docker-compose exec web sh -c "project=GitLab type=New npm start" then I get the error as it is unable to get the response from the API call but the same code and API request works perfectly when running with the direct command node index.js.
Following is the code I have:
./web/index.js:
const express = require('express');
const http = require("http");
const bodyParser = require('body-parser');
const app = express();
const port = process.env.PORT || 9000;
const gitlabDump = require("./controller/GitLabDump");
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
//Make NodeJS to Listen to a particular Port in Localhost
app.listen(port, function(){
var project = process.env.project;
var type = process.env.type;
if(project.trim() === "GitLab" && (type.trim() === "New" || type.trim() === "Update")){
//If porject is GitLab then fetch the data from Gitlab
console.log("Fetching GitLab Data.......");
gitlabDump.gitlabDump(type, function(data){
console.log("Completed Execution for GitLab")
process.exit();
})
}
}
Following is my controller code where I am making the API request:
./web/controller/GitLabDump.js
const request = require('request');
exports.gitlabDump = function(callback){
var gitlabAPI = "https://gitlab.com/api/v4/projects/<project_id>/repository/tree?ref=<target_branch>&path=path/to/subdirectory";
console.log("BEFORE \n")
request(gitlabAPI, function(error, response, body) {
console.log(JSON.parse(body))
callback("Completed");
})
}
Following is my DockerFile:
./docker.compose.yml
version: '3'
services:
db:
container_name: db
image: mysql:5.7
volumes:
- "./data/mysql:/var/lib/mysql:rw"
environment:
MYSQL_DATABASE: myDatabase
MYSQL_ROOT_PASSWORD: myPassword
MYSQL_PASSWORD: myPassword
DATABASE_HOST: localhost
restart: always
web:
container_name: web
image: node:8
volumes:
- ./web:/usr/src/app
working_dir: /usr/src/app
depends_on:
- db
restart: on-failure
command: "tail -f /dev/null"
environment: ["project=${project}", "type=${type}"]
Following is the command I am using to run the application:
docker-compose exec web sh -c "project=GitLab type=New npm start"
Following is the error that I get:
Fetching GitLab Data.......
BEFORE
undefined:1
undefined
^
The error is coming due to the line console.log(JSON.parse(body)). Because the body is undefined as the API call returns no data.
Please Note:
API URL is correct and I have the proper access because the same URL gives me data when accessing through Chrome, Postman, and even when running the code using the node index.js command.
I am using the same application to make other API calls apart from GitLab and they are working fine.
Can someone please help me what's the issue here and why is failing for GitLab API?

Posting the answer here as it can be useful to someone else in the future:
Finally, I was able to find the resolution. The issue is not happening because of the docker but rather because of the Nodejs itself. If we console.log the error then we get the certificate has expired message.
The fix is to request something like this:
request({
url: gitlabAPI,
agentOptions: {
rejectUnauthorized: false
}
}, function (error, response, body) {
console.log(JSON.parse(response.body))
});
Refer to the questions: Node.js request CERT_HAS_EXPIRED

Does log4js require any extra code to work on an Apache server?

I'm trying to add Log4js-Node to a Node.js server running on Apache. Here's my code:
const path = require("path");
const express = require("express");
const log4js = require('log4js');
const app = express();
const logger = log4js.getLogger();
logger.level = "debug";
const port = 443;
log4js.configure({
appenders: { everything: { type: 'file', filename: 'logs.log', flags: 'w' } },
categories: { default: { appenders: ['everything'], level: 'ALL' } }
});
const server = app.listen(port, () => {
logger.debug("listening to requests on port " + port);
});
app.get("/log", (req, res) => {
res.sendFile(path.join(__dirname + "/logs.log"));
});
When I run the script on Node.js on my computer and navigate to localhost:443/log I see what I expect, which is this:
[2020-03-17T22:50:43.145] [DEBUG] default - listening to requests on port 443
But when I run the code on a remote server it crashes and I get this in the error page (with part of the path replaced by me with "[removed]"):
App 25925 output: at Server. ([removed]/index.js:27:9)
App 25925 output: at Logger. [as debug] ([removed]/12/lib/node_modules/log4js/lib/logger.js:124:10)
App 25925 output: at Logger.log ([removed]/12/lib/node_modules/log4js/lib/logger.js:73:12)
App 25925 output: at Logger._log ([removed]/12/lib/node_modules/log4js/lib/logger.js:90:16)
App 25925 output: at Object.send ([removed]/12/lib/node_modules/log4js/lib/clustering.js:97:15)
App 25925 output: [removed]/12/lib/node_modules/log4js/lib/clustering.js:97
App 25925 output: at Object. ([removed]/12/lib/node_modules/log4js/lib/clustering.js:8:13)
I'm using A2 Hosting which uses Apache 2.4.41. I opted for Node.js 12.9.0, and Log4js 6.1.2. The package.json should be the same on both my computer and the server, and I've run npm install on both.
Is this just an issue with Log4js and the server, or have I missed something somewhere?

This was actually a relatively simple fix. The path referenced by the last error in the stack trace is a Log4js module that implements clustering support through Node's "cluster" module. The line "8" referenced is cluster = require("cluster"). It's wrapped in a try/catch block like this:
try {
cluster = require("cluster"); //eslint-disable-line
} catch (e) {
debug("cluster module not present");
disabled = true;
}
The installation of Node.js on my computer came with the "cluster" module, however as far as I can tell, the server I'm using doesn't support it. Also, the version of Node I'm using on my computer is newer than what I'm using on the server (so I've now installed 12.9 on my machine). I believe the older version of Node doesn't bother trying to catch the exception and tries to load the cluster module, fails, and then throws the error.
So the simple fix was to comment out most of the "try/catch" block, leaving just the contents of "catch" like this:
// try {
// cluster = require("cluster"); //eslint-disable-line
// } catch (e) {
debug("cluster module not present");
disabled = true;
// }
If someone has a better fix, I'm open to suggestions.

The same response of #skittleswrapper,thx, it work for me.
I use Node.js 14.18.1 with log4js 6.3.0.
But i wondering what'is the necessary of this module 'cluster' and if we can
add it to our app in other way.

Deploy OpenShift Node.js application from github

I created a node.js (with express) application and have been unsucessfully trying to deploy to OpenShift from github. I am attempting to deploy from the web interface (providing the URL to the github repository root and "master" in the branch/tag field) and am getting an error I'm having trouble understanding:
The initial build for the application failed:
Shell command '/sbin/runuser -s /bin/sh 5724c3b42d5271363b000191 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c4,c687' /bin/sh -c \"gear postreceive --init >> /tmp/initial-build.log 2>&1\""' returned an error. rc=255 .
Last 10 kB of build output: Stopping NodeJS cartridge
Repairing links for 1 deployments
Building git ref 'master', commit a5ca0f7
Building NodeJS cartridge Preparing build for deployment
Deployment id is c2527992
Activating deployment
Starting NodeJS cartridge Sat Apr 30 2016 10:41:09 GMT-0400 (EDT):
Starting application 'profile' ...
Script = server.js
Script Args = Node Options = !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! It is highly recommended that you add a package.json file to your application. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Waiting for application port (8080) become available ...
Application 'profile' failed to start (port 8080 not available) -------------------------
Git Post-Receive Result: failure
Activation status: failure
Activation failed for the following gears: 5724c3b42d5271363b000191
(Error activating gear: CLIENT_ERROR: Failed to execute: 'control start' for /var/lib/openshift/5724c3b42d5271363b000191/nodejs #<IO:0x000000019b0298> #<IO:0x000000019b0220> )
Deployment completed with status: failure postreceive failed
I read a couple of posts about some errors above like port 8080 not available and failed to execute control start but the directives I was able to follow did not solve my issue. I am finding the line that says "using a package.json file is highly recommended" strange as I do have one. My package.json file is:
{
"name": "Portfolio_Memoria",
"version": "0.0.0",
"private": true,
"scripts": {
"start": "node server.js"
},
"main": "server.js",
"description": "Portfolio_Memoria",
"author": {
"name": "gorra",
"email": ""
},
"dependencies": {
"express": "~4.9.0",
"body-parser": "~1.8.1",
"cookie-parser": "~1.3.3",
"morgan": "~1.3.0",
"serve-favicon": "~2.1.3",
"debug": "~2.0.0",
"jade": "~1.6.0",
"stylus": "0.42.3"
}
}
And server.js file is:
#!/usr/bin/env node
var debug = require('debug')('Portfolio_Memoria');
var app = require('./app');
if(typeof process.env.OPENSHIFT_NODEJS_PORT === 'undefined'){
app.set('port', process.env.PORT || 3000);
var server = app.listen(app.get('port'), function() {
debug('Express server listening on port ' + server.address().port);
});
} else {
app.set('port', process.env.OPENSHIFT_NODEJS_PORT || 3000);
app.set('ip', process.env.OPENSHIFT_NODEJS_IP || '127.0.0.1');
var server = app.listen(app.get('port'), app.get('ip'), function() {
debug('Express server listening on port ' + server.address().port);
});
}
Of course, the application runs without issue locally. I don't know what I am missing here.
EDIT:
I got it to work by creating a blank application in OpenShift, cloning the repository OpenShift creates via command line, copying my whole project to it and pushing it back. This is a workaround and not a solution to the original problem, though.

Cannot GET /cool on Heroku nodejs tutorial locally

Here are the dependencies of my package.json file where i've added "cool-ascii-faces. I then need to update my index.js file to GET the /cool page so that on each reload I would see an ascii face. I'm getting a 404 error and it says 'Cannot GET /cool'
"dependencies": {
"ejs": "2.3.3",
"express": "4.13.3",
"cool-ascii-faces": "~1.3"
}
Below is my index.js file that calls declares cool
var cool = require('cool-ascii-faces');
var express = require('express');
var app = express();
app.set('port', (process.env.PORT || 5000));
app.use(express.static(__dirname + '/public'));
// views is directory for all template files
app.set('views', __dirname + '/views');
app.set('view engine', 'ejs');
app.get('/cool', function(request, response) {
response.render('pages/index')
});
app.get('/cool', function(request, response) {
response.send(cool());
});
app.listen(app.get('port'), function() {
console.log('Node app is running on port', app.get('port'));
I then run npm install to update the dependencies and then heroku local, but get the 404 error.
Any help in the right direction would be great!

You're probably getting an exception when you start the node web server, due to a module dependency error.
Check your command/terminal window. If you see a red warning message pointing to your module.js file, you have an exception:
$ heroku local
forego | starting web.1 on port 5000
web.1 | module.js:341
In this case, you need to install the cool-ascii-faces module. In your 'node-js-getting-started' directory, use the following npm command to install:
$ npm i -S cool-ascii-faces
Also... you'll want to convert your index page route back to '/'. Your routes logic should look like this:
app.get('/', function(request, response) {
response.render('pages/index')
});
app.get('/cool', function(request, response) {
response.send(cool());
});
Otherwise, you'll always get the default 'pages/index' page when you hit the '/cool' route instead of a smiley face.

You don't have to include
app.set('port', (process.env.PORT || 5000));
app.listen(app.get('port'), function() {
console.log('Node app is running on port', app.get('port'));
}
Heroku will run "npm start" to start your server and dynamically choose the port. You don't have to specify port explicitly.

this problem happened for me. After I typed "git push heroku master", I typed "heroku open" and it worked.

Guide to upgrading loopback API core files and configuration to latest?

I just ran a new scaffolding using slc loopback and then compared server/server.js and the server/ contents, and they are slightly different.
My main goal is to re-enable the loopback explorer and get up to date with the current latest core code and configuration. Is there a guide yet written for this? Originally we just removed explorer.js from the server/boot folder to secure production, now wondering how to do the same, but also enable it on other non-public hosts.
server/ now lists a component-config.js file, as well as a middleware.production.json that we do not have. I tried to add these files (and updated server.js to the below) and npm install loopback-component-explorer, but I still can't get an explorer to mount.
server/boot now has a root.js and authentication.js, but we also still have rest-api.js. Does root.js take over from the more verbose rest-api.js?
The core code is now basically the same (also updated to latest with salita, so all package.json versions match—the only difference is the empty project app.get('loopback-component-explorer') returns something but in my modified project it's undefined.
What am I missing?
Our existing server.js:
var loopback = require('loopback');
var boot = require('loopback-boot');
var app = module.exports = loopback();
// Bootstrap the application, configure models, datasources and middleware.
// Sub-apps like REST API are mounted via boot scripts.
boot(app, __dirname);
app.start = function() {
// start the web server
return app.listen(function() {
app.emit('started');
console.log('Web server listening at: %s', app.get('url'));
});
};
// start the server if `$ node server.js`
if (require.main === module) {
app.start();
}
server.js from a slc loopback scaffolding:
var loopback = require('loopback');
var boot = require('loopback-boot');
var app = module.exports = loopback();
app.start = function() {
// start the web server
return app.listen(function() {
app.emit('started');
var baseUrl = app.get('url').replace(/\/$/, '');
console.log('Web server listening at: %s', baseUrl);
if (app.get('loopback-component-explorer')) {
var explorerPath = app.get('loopback-component-explorer').mountPath;
console.log('Browse your REST API at %s%s', baseUrl, explorerPath);
}
});
};
// Bootstrap the application, configure models, datasources and middleware.
// Sub-apps like REST API are mounted via boot scripts.
boot(app, __dirname, function(err) {
if (err) throw err;
// start the server if `$ node server.js`
if (require.main === module)
app.start();
});