Develop Reference

I'm attempting to put my logic in a separate file using module.exports and require, however, I can't seem to figure it out

Recently I picked up Node.js + Express.js in order to pair them with Socket.io and make a real time chat application. The problem is that since I'm relatively inexperienced with Node.js and Express.js, I'm having some trouble figuring out where to put my logic and how to separate it in different files. Right now, the logic that creates my Socket.io namespaces is in the www file and I'm trying to figure out where to place it. Right now it looks like this:
www File:
var app = require('../app');
var http = require('http');
var server = http.createServer(app);
let io = require('socket.io')(server);
servers.forEach((server) => {
console.log(server.name)
io.of(server.endpoint).on('connection',(socket) => {
socket.on('messageToServer', (message) => {
let roomName = Object.keys(socket.rooms)[1]
let room = server.room.find((room) => {
return room.name == roomName
})
room.history.push(message)
io.of(server.endpoint).to(roomName).emit('messageToClient', message)
})
socket.on('joinRoom', (roomToJoin) => {
let roomToLeave = Object.keys(socket.rooms)[1]
socket.leave(roomToLeave)
socket.join(roomToJoin)
let room = server.room.find((room) => {
return room.name == roomToJoin
})
socket.emit('chatHistory', room.history)
})
})
});
What I tried is the following - I created a socket.js file and put it in a folder called utility, moved the code from www to the socket.js file and tried exporting the server from my www file to my socket.js file so that Socket.io can access it:
socket.js File:
let server = require('../bin/www').server
let io = require('socket.io')(server);
function createSocketServers() {
servers.forEach((server) => {
console.log(server.name)
io.of(server.endpoint).on('connection',(socket) => {
socket.on('messageToServer', (message) => {
let roomName = Object.keys(socket.rooms)[1]
let room = server.room.find((room) => {
return room.name == roomName
})
room.history.push(message)
io.of(server.endpoint).to(roomName).emit('messageToClient', message)
})
socket.on('joinRoom', (roomToJoin) => {
let roomToLeave = Object.keys(socket.rooms)[1]
socket.leave(roomToLeave)
socket.join(roomToJoin)
let room = server.room.find((room) => {
return room.name == roomToJoin
})
socket.emit('chatHistory', room.history)
})
})
});
}
module.exports = createSocketServers
www File:
var app = require('../app');
var http = require('http');
var server = http.createServer(app);
module.exports = server
require('../utility/socket').createSocketServers
As soon as I did that, my code stopped working, so I can only assume I haven't done it correctly, hence why I'm here asking for help. Thanks!

Everything looks fine except this line
require('../utility/socket').createSocketServers
When you do, module.exports = createSocketServers in socket.js file, you actually exporting the constructor.
If you run what you require, it should work;
require('../utility/socket')()
If you want to keep it like you do now, you have to export an object from socket.js file;
module.exports = { createSocketServers }
After exporting your module like above, you have to run it like below;
require('../utility/socket').createSocketServers()

Access socket emitter with socket.io in other modules nodejs express

I'll preface this by saying I am new to nodejs in general. Coming from the world of C#, it's a completely different way of thinking for me.
I've gone through a few courses and I'm setting up a little website as sort of a test for myself. And I'm failing!
I'm using socket.io with node, and I'm trying to broadcast a message with the emitter once in a while. I don't care about specific socket points right now (although I will in the future), so the emitter for this should go out to everyone.
I am having trouble accessing the io object from other modules.
I'll post my server.js file, as well as app/index.js, socket/index.js, helpers/index.js and api/index.js. I hope that posting these will show how it's supposed to work.
Ideally, I'd like to keep all socket-related items in the socket module, for consistency. Right now, I'm trying to get a method to run in the helpers module, but ideally the socket module would be better.
Anyway, server.js:
'use strict';
const express = require('express');
const app = express();
const cryptometers = require('./app');
const api = require('./app/api');
const fs = require('fs');
const sources = require('./app/api/sources.json');
app.set('port', process.env.PORT || 3000);
app.set('view engine', 'ejs');
app.use(express.static('public'));
app.use(cryptometers.session);
app.use('/', cryptometers.router);
cryptometers.ioServer(app).listen(app.get('port'), () =>{
console.log('app listening on port ' + app.get('port'));
api.getData(sources[0].source, sources[0].url, app);
setInterval(function(){api.getData(sources[0].source, sources[0].url, app)}, 60000);
});
Standard fare here. I just added a data retriever that calls to an api once every minute, which updates the database.
app/index.js:
'use strict';
const config = require('./config');
// create an IO server instance
let ioServer = app => {
app.locals.globalMarketCap = [];
const server = require('http').Server(app);
const io = require('socket.io')(server);
io.set('transports', ['websocket']);
io.use((socket, next) => {
require('./session')(socket.request, {}, next);
});
require('./socket')(io,app);
return server;
}
// turns router into a module
module.exports = {
router: require('./routes')(),
session: require('./session'),
ioServer,
}
Here I'm initializing socket.io, binding it to the app. It's also where I initialize a local storage array of data. (Is this a good spot to do this??)
socket/index.js:
'use strict';
const h = require('../helpers');
module.exports = (io, app) => {
io.of('/').on('connection', socket =>{
console.log('socket.io connected to client');
if(app.locals.globalMarketCap){
socket.emit('globalMarketCap', JSON.stringify(app.locals.globalMarketCap));
}
})
}
Here I'm responding to connection events, and pushing out the array of data that I defined in the last file above. Again, ideally I'd like all socket type stuff to stay in here.
helpers/index.js:
'use strict';
const router = require('express').Router();
const db = require('../db');
// iterate through the routes object and mount the routes
let _registerRoutes = (routes, method) => {
for(let key in routes){
if(typeof routes[key] === 'object' && routes[key] !== null && !(routes[key] instanceof Array)){
_registerRoutes(routes[key], key);
} else {
// Register the routes
if(method === 'get'){
router.get(key, routes[key]);
} else if(method === 'post'){
router.post(key, routes[key]);
} else {
router.use(routes[key]);
}
}
}
}
let route = routes => {
_registerRoutes(routes);
return router;
}
let updateGlobalMarketCap = (app) =>{
//app.io.socket.emit('globalMarketCap', JSON.stringify(app.locals.globalMarketCap))
}
module.exports = {
route,
updateGlobalMarketCap
}
The commented out line for updateGlobalMarketCap is where my pain is. Trying to get access to the io object there.
api/index.js
'use strict';
const axios = require("axios");
const db = require("../db");
const h = require("../helpers");
let getData = (source, url, app, cryptoMeters) => {
axios
.get(url)
.then(response => {
//console.log(response.data);
response.data["source"] = source;
var data = new db.globalMarketCapModel(response.data);
app.locals.globalMarketCap = response.data;
var query = { source: source};
db.globalMarketCapModel.findOne({
"source":source
}, 'source old_total_market_cap_usd total_market_cap_usd', function(err, market) {
if (market) {
if(market.old_total_market_cap_usd != response.data.total_market_cap_usd
&& market.total_market_cap_usd != response.data.total_market_cap_usd){
response.data["old_total_market_cap_usd"] = market.total_market_cap_usd;
h.updateGlobalMarketCap(app);
}
db.globalMarketCapModel.update(query, response.data, function (err) {
if (err) {
console.log("uhoh")
} else {
return true;
}
});
} else {
data.save(function (err) {
if (err) {
console.log("uhoh")
} else {
return true;
}
})
}
})
return true;
})
.catch(error => {
console.log(error);
return false;
});
}
module.exports = {
getData
}
The getData function here is where a call to the update emitter would take place.
I've considered using standard node event emitters as a solution to my problem, but that might be gumming up the works and there's a simpler answer.
Anyway, thanks for reading, and I'm interested in any commentary on what i've written so far. pitfalls, mistakes, etc. Learning here! :)

There are many different ways to organize your code to accomplish sharing of the io object. Here's one such scheme. You break out your socket.io initialization code into its own module. You give that module two main features:
A constructor function (that you pass the server to) that allows socket.io to initialize itself on your server.
A method to get the io instance after it's been initialized.
This will allow any other code in your project that wants to get access to the io object to do something like this:
const io = require('./io.js').getIO();
Here's how that io module could be structured:
// io.js
// singleton instance of socket.io that is stored here after the
// constructor function is called
let ioInstance;
modules.exports = function(server) {
const io = require('socket.io')(server);
io.set('transports', ['websocket']);
io.use((socket, next) => {
require('./session')(socket.request, {}, next);
});
// save in higher scope so it can be obtained later
ioInstance = io;
return io;
}
// this getIO method is designed for subsequent
// sharing of the io instance with other modules once the module has been initialized
// other modules can do: let io = require("./io.js").getIO();
module.exports.getIO = function() {
if (!ioInstance) {
throw new Error("Must call module constructor function before you can get the IO instance");
}
return ioInstance;
}
And, this module would be initialized like this:
const io = require('./io.js')(server);
Where you pass it your web server so it can hook to that. It has to be initialized like this before anyone can use .getIO() on it. The storage in the module of the ioInstance makes use of the module caching. The module initialization code is only run once. After that, the same exports are just returned each time which have access to the saved ioInstance inside the module.

In React - How to get value from .sh(Shell Script) file?

I have .sh file like :
echo "Hello"
Which outputs :
Hello
Here is the query :
What I want to achieve is, to get output from the .sh file and use it in
my react application.
I have searched for npm packages but can't find any useful

Assuming you are not on Windows machine, you can write simple Node Express server, which can receive GET request, then execute your shell script with Node's built-in child_process.exec() and then send response containing stdout, which, in this case, will be your shell script's output - "Hello"
Code for the server. 'static' folder contains index.html and index.js, code for last is below this:
const express = require('express')
const { exec } = require('child_process')
const { join } = require('path')
const port = process.env.PORT || 24587
const app = express()
app.use(express.static(join(__dirname, 'static')))
app.get('/hello', (req, res) => {
exec(join(__dirname, 'hello.sh'), (err, stdout, stderr) => {
if (err) {
return res.status(400).json({ output: null, error: err.message })
}
res.status(200).json({ output: stdout, error: null })
})
})
app.listen(port, () => {
console.log('server listening on port', port)
})
Example code for the client (you can also wrap React around this code because the only thing you need is XHR made here with the help of fetch):
const btn = document.querySelector('#btn') // <button id="btn">get</button>
const result = document.querySelector('#result') // <div id="result"></div>
btn.addEventListener('click', () => {
if (self.fetch) {
fetch('/hello')
.then(res => res.json())
.then(data => {
result.innerText = data.output
})
.catch(data => {
result.innerText = data.error
})
}
})

I think you should put output to any file (echo $var > $destdir or using sed), read this file using nodejs and for example pass value through ajax request to react.

Ensuring server app runs before mocha tests start

This is similar to Ensuring Express App is running before each Mocha Test , but the specified solution still isnt working + i'm using a websocket server
in short , i'm using a websocket framework called socketcluster and this is my server file
import {SocketCluster} from 'socketcluster';
const socketCluster = new SocketCluster({
workers:1,
brokers:1,
port: 3000,
appName:null,
initController: __dirname + '/init.js',
workerController: __dirname + '/worker.js',
brokerController: __dirname + '/broker.js',
socketChannelLimit: 1000,
crashWorkerOnError: true
})
export default socketCluster
running node server.js starts the worker process specified in worker.js
export const run = (worker) => {
console.log(' >> worker PID: ',process.pid);
const app = express();
const httpServer = worker.httpServer;
const scServer = worker.scServer;
app.use(cookieParser())
httpServer.on('request', app);
app.get('/',(req,res) => {
console.log('recieved')
res.send('Hello world')
})
}
I want to test the server , but the tests are finishing (and failing) way before the server actually starts. is there a way i can force the server to fully load before going ahead with tests? this is what i have so far
describe('Express server',() =>{
beforeEach((done) => {
require('../../server/server')
done()
})
it('should return "Hello World"',(done) => {
http.get('http://127.0.0.1:3000',(res) => {
expect(res).to.contain('wtf world')
done()
})
})
})
the above doesnt seem to work. the server doesnt fully load in the before block despite providing the done() call as well.
edit - i've tried splitting the server.js file to invoke a different server based on how its imported.
const main = () => {
console.log('main server')
new SocketCluster({
workers:1,
brokers:1,
port: 3000,
appName:null,
initController: __dirname + '/init.js',
workerController: __dirname + '/worker.js',
brokerController: __dirname + '/broker.js',
socketChannelLimit: 1000,
crashWorkerOnError: true
})
}
export const test = (port,done) => {
console.log('testing server')
new SocketCluster({
workers:1,
brokers:1,
port: port,
appName:null,
initController: __dirname + '/init.js',
workerController: __dirname + '/worker.js',
brokerController: __dirname + '/broker.js',
socketChannelLimit: 1000,
crashWorkerOnError: true
})
done()
}
if (require.main === module){
main()
}
and in test.js , i do this - still doesnt seem to work though
import {expect} from 'chai';
import {test} from '../../server/server'
describe('Express server',() =>{
before(function(done){
test(3000,done)
})
it('should return "Hello World"',(done) => {
http.get('http://127.0.0.1:3000',(res) => {
expect(res).to.contain('world')
done()
})
})
})
edit:2 - trie another way by returning a promise from the server.js file. still doesnt work
export const test = (port) => {
console.log('testing server')
return Promise.resolve(new SocketCluster({
workers:1,
brokers:1,
port: port,
appName:null,
initController: __dirname + '/init.js',
workerController: __dirname + '/worker.js',
brokerController: __dirname + '/broker.js',
socketChannelLimit: 1000,
crashWorkerOnError: true
}))
}
and in the before hook
before(function(done,port){
test(3000).then(function(){
console.log('arguments: ',arguments)
done()
})
})

Your server module doesn't have a callback, so it could not be ready when you call done() in your beforeEach method.
First, export your app in your server module.
Then, do something like:
const app = require('../../server/server').app;
let server;
before(done => {
server = app.listen(3000, done);
});
/**
...
your tests here
...
**/
/** and, if you need to close the server after the test **/
after(done => {
server.close(done);
});
This way, done() will be called in the listen callback, so in your tests the server will be listening correctly. Then, remember to close it after tests end (useful if server is required in one or more test suites).

The solution explained here worked for me, in particular:
At the end of server.js ( or app.js ):
app.listen( port, ip, function()
{
console.log( 'Server running on http://%s:%s', ip, port )
app.emit( "app_started" )
})
module.exports = app
and in test.js:
var server = require( '../server' )
before( done =>
{
server.on( "app_started", function()
{
done()
})
})
In this case, app sends an "app_started" event when it is listening, and the test code waits for it. The provided URL contains more details.
Hope it helps !

You need to wait until the server actually listens on the given port.
This could be accomplished by exporting some kind of init function in your server.js, which takes the done callback from mocha.
In your server.js
let initCallback;
[..]
app.listen(port, function() {
if (initCallback) {
initCallback();
}
});
exports = {
init: function(cb) {
initCallback = cb;
}
}
In your test
beforeEach((done) => {
require('../../server/server').init(done)
})
Also see: How to know when node.js express server is up and ready to use

I combined the first two posts and it worked for mine.
First, make sure you have init code in your app.js or server.js
// put this in the beginning of your app.js/server.js
let initCallback;
//put this in the end of your app.js/server.js
if (initCallback) {
// if this file was called with init function then initCallback will be used as call back for listen
app.listen(app.get('port'),"0.0.0.0",(initCallback)=>{
console.log("Server started on port "+app.get('port'));
});
}
else{
// if this file was not called with init function then we dont need call back for listen
app.listen(app.get('port'),"0.0.0.0",()=>{
console.log("Server started on port "+app.get('port'));
});
}
//now export the init function so initCallback can be changed to something else when called "init"
module.exports = {
init: function(cb) {
initCallback = cb;
}
}
Next in your test.js you will need this
//beginning of your code
const app = require("../../server/server").app;
before(done => {
require("../app").init();
done();
});
//end of your code
after(done => {
done();
});
I am no expert in javascript but this works for me. Hope it helps!

Mocha API Testing: getting 'TypeError: app.address is not a function'

My Issue
I've coded a very simple CRUD API and I've started recently coding also some tests using chai and chai-http but I'm having an issue when running my tests with $ mocha.
When I run the tests I get the following error on the shell:
TypeError: app.address is not a function
My Code
Here is a sample of one of my tests (/tests/server-test.js):
var chai = require('chai');
var mongoose = require('mongoose');
var chaiHttp = require('chai-http');
var server = require('../server/app'); // my express app
var should = chai.should();
var testUtils = require('./test-utils');
chai.use(chaiHttp);
describe('API Tests', function() {
before(function() {
mongoose.createConnection('mongodb://localhost/bot-test', myOptionsObj);
});
beforeEach(function(done) {
// I do stuff like populating db
});
afterEach(function(done) {
// I do stuff like deleting populated db
});
after(function() {
mongoose.connection.close();
});
describe('Boxes', function() {
it.only('should list ALL boxes on /boxes GET', function(done) {
chai.request(server)
.get('/api/boxes')
.end(function(err, res){
res.should.have.status(200);
done();
});
});
// the rest of the tests would continue here...
});
});
And my express app files (/server/app.js):
var mongoose = require('mongoose');
var express = require('express');
var api = require('./routes/api.js');
var app = express();
mongoose.connect('mongodb://localhost/db-dev', myOptionsObj);
// application configuration
require('./config/express')(app);
// routing set up
app.use('/api', api);
var server = app.listen(3000, function () {
var host = server.address().address;
var port = server.address().port;
console.log('App listening at http://%s:%s', host, port);
});
and (/server/routes/api.js):
var express = require('express');
var boxController = require('../modules/box/controller');
var thingController = require('../modules/thing/controller');
var router = express.Router();
// API routing
router.get('/boxes', boxController.getAll);
// etc.
module.exports = router;
Extra notes
I've tried logging out the server variable in the /tests/server-test.js file before running the tests:
...
var server = require('../server/app'); // my express app
...
console.log('server: ', server);
...
and I the result of that is an empty object: server: {}.

You don't export anything in your app module. Try adding this to your app.js file:
module.exports = server

It's important to export the http.Server object returned by app.listen(3000) instead of just the function app, otherwise you will get TypeError: app.address is not a function.
Example:
index.js
const koa = require('koa');
const app = new koa();
module.exports = app.listen(3000);
index.spec.js
const request = require('supertest');
const app = require('./index.js');
describe('User Registration', () => {
const agent = request.agent(app);
it('should ...', () => {

This may also help, and satisfies #dman point of changing application code to fit a test.
make your request to the localhost and port as needed
chai.request('http://localhost:5000')
instead of
chai.request(server)
this fixed the same error message I had using Koa JS (v2) and ava js.

The answers above correctly address the issue: supertest wants an http.Server to work on. However, calling app.listen() to get a server will also start a listening server, this is bad practice and unnecessary.
You can get around by this by using http.createServer():
import * as http from 'http';
import * as supertest from 'supertest';
import * as test from 'tape';
import * as Koa from 'koa';
const app = new Koa();
# add some routes here
const apptest = supertest(http.createServer(app.callback()));
test('GET /healthcheck', (t) => {
apptest.get('/healthcheck')
.expect(200)
.expect(res => {
t.equal(res.text, 'Ok');
})
.end(t.end.bind(t));
});

Just in case, if someone uses Hapijs the issue still occurs, because it does not use Express.js, thus address() function does not exist.
TypeError: app.address is not a function
at serverAddress (node_modules/chai-http/lib/request.js:282:18)
The workaround to make it work
// this makes the server to start up
let server = require('../../server')
// pass this instead of server to avoid error
const API = 'http://localhost:3000'
describe('/GET token ', () => {
it('JWT token', (done) => {
chai.request(API)
.get('/api/token?....')
.end((err, res) => {
res.should.have.status(200)
res.body.should.be.a('object')
res.body.should.have.property('token')
done()
})
})
})

Export app at the end of the main API file like index.js.
module.exports = app;

We had the same issue when we run mocha using ts-node in our node + typescript serverless project.
Our tsconfig.json had "sourceMap": true . So generated, .js and .js.map files cause some funny transpiling issues (similar to this). When we run mocha runner using ts-node. So, I will set to sourceMap flag to false and deleted all .js and .js.map file in our src directory. Then the issue is gone.
If you have already generated files in your src folder, commands below would be really helpful.
find src -name ".js.map" -exec rm {} \;
find src -name ".js" -exec rm {} \;

I am using Jest and Supertest, but was receiving the same error. It was because my server takes time to setup (it is async to setup db, read config, etc). I needed to use Jest's beforeAll helper to allow the async setup to run. I also needed to refactor my server to separate listening, and instead use #Whyhankee's suggestion to create the test's server.
index.js
export async function createServer() {
//setup db, server,config, middleware
return express();
}
async function startServer(){
let app = await createServer();
await app.listen({ port: 4000 });
console.log("Server has started!");
}
if(process.env.NODE_ENV ==="dev") startServer();
test.ts
import {createServer as createMyAppServer} from '#index';
import { test, expect, beforeAll } from '#jest/globals'
const supertest = require("supertest");
import * as http from 'http';
let request :any;
beforeAll(async ()=>{
request = supertest(http.createServer(await createMyAppServer()));
})
test("fetch users", async (done: any) => {
request
.post("/graphql")
.send({
query: "{ getQueryFromGqlServer (id:1) { id} }",
})
.set("Accept", "application/json")
.expect("Content-Type", /json/)
.expect(200)
.end(function (err: any, res: any) {
if (err) return done(err);
expect(res.body).toBeInstanceOf(Object);
let serverErrors = JSON.parse(res.text)['errors'];
expect(serverErrors.length).toEqual(0);
expect(res.body.data.id).toEqual(1);
done();
});
});
Edit:
I also had errors when using data.foreach(async()=>..., should have use for(let x of... in my tests