I am trying to build a set of utils for my NodeJS project. These helpers will include: text utils (like substringing, console logging etc.), and more specific helpers like parsing the text of a tweet.
So I am trying to divide the module in different files and with a very clear idea of what each thing is meant to do.
For example I would like to achieve this:
var helpers = require("helpers");
var Utils = new helpers.Utils();
// working with text
Utils.text.cleanText("blahblalh");
// working with a tweet
Utils.twitter.parseTweet(tweet);
As you can see I am using Utils for different things, by calling very specific methods and sub methods.
I tried to understand how inheritance works here but I got a little bit lost.
This is what I am doing (some rough sample code):
//node_modules/helpers/index.js
var Text = require('./text');
var Twitter = require('./twitter');
function Utils() {
}
Utils.prototype.text = {
cleanText: function(text) {
Text.cleanText(text);
}
};
Utils.prototype.twitter = {
parseTweet(tweet) {
Twitter.parseTweet(tweet);
}
};
//node_modules/helpers/text.js
function Text() {
}
Text.prototype.cleanText = function(text) {
if (typeof text !== 'undefined') {
return text.replace(/(\r\n|\n|\r)/gm,"");
}
return null;
};
module.exports = Text;
//node_modules/helpers/twitter.js
function Twitter() {
};
Twitter.prototype.parseTweet = function(data) {
return data;
};
module.exports = Twitter
Is this a correct way? Am I doing something wrong or that could slow down the performances, etc?
To clarify how I'm understanding your post, I see two questions:
How do I structure code/methods within files, files that represent a category of utility functions
How do I organize the those categorical files into one larger library
Structuring methods within a category
Rather than making all of the category specific functions methods of objects (e.g. Twitter or Text), you could just export the functions in files named after them. Since it seems you are passing in the data you want to use, there is no need to make the functions instance methods of some empty class.
If your usage patterns of Twitter or Text usually have class variables you want to keep state on, and you want to instantiate Text or Twitter objects to use your examples, then I suppose that would be appropriate. When I setup util libs in my projects it usually is a bunch of exported functions that make up a module, rather than an exported javascript class.
To provide an example of what a text.js file made up of text-based utility functions might look like:
module.exports = {
cleanText:function(text) {
// clean it and return
},
isWithinRange(text, min, max) {
// check if text is between min and max length
}
}
Alternatively, you could do it this way:
exports.cleanText = function(text) {
// clean it and return
}
exports.isWithinRange = function (text, min, max) {
// check if text is between min and max length
}
Structuring utility category files to make a larger utility library
As far as organizing the utility methods, Luca's example is nice. I've organized some similarly like this:
utils-module/
lib/
text.js <-- this is the example file shown above
twitter.js
test/
index.js
Where index.js does something like
var textUtils = require('./lib/text');
exports.Text = textUtils;
Then when I want to use the util lib in say some User model in my node API, it's simply:
/*
* Dependencies
*/
var textUtils = require('path/to/lib').Text;
/*
* Model
*/
function User() {}
/*
* Instance Methods
*/
User.prototype.setBio = function(data) {
this.bio = textUtils.cleanText(data);
}
module.exports = User;
When I was first learning it was very helpful to look at popular, well-respected libraries to see how more experienced node/javascript devs were doing things. There are so many good (and bad) ones out there!
You can see a utils library example with lodash.
Lodash is an utility lib like underscorejs.
This library have file sustem structure like your.
It divides the functions in categories. Each category is a folder with an index.js file that includes into a namespace (literal object) each functions for that category!
Lodash/
Objects/
Function1.js
Functions2.js
....
Index.js
Array/
Function1.js
...
Index.js
Then in your code you can do this:
var objectsUtils = require("lodash/objects");
var foreach = require("lodash/array/each");
You can create a similar file system structure in order to have more flexibility.
You can require the entire lib, only one namespace or a single function.
This is better for performance, because you use only what you need and have a memory usage gain.
Related
I'm working with JS modules bundled with Webpack.
Suppose I have a common module, used everywhere. This module require another module, let's say it is an output for errors
var err = require("./errors.js");
var common = {
doSomething: function() {
var a = 1;
if (a == 1) {
err.output('An error occured');
}
}
}
module.exports = common;
Now this error management module could be somthing like
var $ = require("jquery");
var errors = {
output: function(msg) {
$('<div class="someClass">'+msg+'</div>).appendTo('body');
}
}
module.exports = errors;
And finally I have my entry point:
var common = require("./common.js");
common.doSomething();
My question is: since I have got two applications, let's call them backend and frontend, I would like different implementation of errors.output, because it's view-bounded. Maybe in backend I want to use a class different from "someClass", or maybe I don't want to use a DIV, but a P, or else.
I have multi entry points, actually. So I wouldn't like to have each of them passing around lots of callbacks or configuration. I would like to have only one implementation of errors module for all the entry point of the backend and only one for all the entry point of the frontend, without any code repetition.
This is something that I would implement with interface, in PHP.
Which is the best approach for Javascript?
Thank you
perhaps I have not worded the title correctly. Below is the explanation of what I'm trying to do.
I'm creating a helper.js file for my project. Inside it contains many functions, one I've pasted below. I export this function using module.exports.
function generateInternalError(message, stack_trace) {
if (process.env.NODE_ENV == 'dev') {
console.log({ message: message, stack_trace: stack_trace });
} else {
console.log({message: message});
}
}
module.exports = {
generateInternalError: generateInternalError
};
Where I want to utilize this function I would call:
helper.generateInternalError('Not Found',new Error().stack);
And it works as expected.
But, what I have been tasked with, is creating categories of functions. Essentially I need the following:
helper.errors.generateInternalError('Not Found',new Error().stack);
I can not seem to figure out the right way to export a class of functions or an object of functions in NodeJS such that I don't get an error like:
TypeError: helper.errors.generateClientError is not a function
Any assistance is appreciated.
Thank you
The module.exports property of a file is simply an object that maps names to functions. You can define it arbitrarily, for example:
module.exports = {
errors: {
generateInternalError,
...
},
...
};
Then, require('./helper').errors.generateInternalError will be defined.
helpers is just noise if everything is a helper, so drop that. Just use regular modules and unless you are sure you only have one function in that category export multiple functions. If you only export one function with module.exports then you don't need to do that as a property of an object which also means you can just say const genError=require('./errors')
Don't make something like helpers.errors.someErrorFunc because helpers is noise and you make categories with just separate module files. Don't try to make Node.js look like Java or something equally horrible.
It might be better to structure your helper sub classes in separate files.
Example
src/helpers.js
src/helpers/
src/helpers/errors.js
File helpers.js
module.exports = {
errors: require('./helpers/errors')
}
File helpers/errors.js
module.exports = {
generateInternalError: function(){
//write some internal error code here
}
};
Structuring your code like this will keep your root helpers file very organized and create a pattern that is easy to replicate for new subclasses.
If you prefer a less modular approach you could simply just return one big JSON object as other's have demonstrated...
module.exports = {
errors: {
generateInternalError: function(){
//internal error code
},
generateDatabaseError: function(){
//db error code
}
}
}
My question: How would one best go about breaking a large, monolithic javascript object literal into multiple, discrete, files?
I have a single javascript file that consists of an object literal with many methods attached to it. It's getting quite long and I want to break it into smaller parts that can more easily be managed.
I've heard I can use AMD or CommonJS to organize things, I've heard I should use RequireJS, that I should use Webpack or Browserify, that I should use any number of other tools/techniques. After looking at these things I am confused as to what the best approach is.
How would you do it? How would you take a single object literal consisting of a few thousands lines of javascript (made up of functions like "search" and "login" and "user") and reorganize it into multiple files that are more easily dealt with by a group of developers? The single, giant file thing is just getting to unwieldy and the options seems to varied and unclear. This is a fairly simple app that uses vanilla JS, a little jQuery and sits on top of a Grails backend.
I think the question is pretty clear but if you really need code to look at here is an example of the sort of object literal I am talking about:
var myObj = {
foo: "one",
bar: "two",
baz: false,
deez: -1,
login: function() {
// lots and lots of code
},
user: function() {
// lots and lots of code
},
beers: function() {
// lots and lots of code
},
varieties: function() {
// lots and lots of code
}
init: function() {
myObj.login.init();
myObj.user.init();
// lots of jQuery document.ready stuff
}
}
myObj.init();
You will a lot of suggestions and approaches to solve your problems, and I can't say any of them are wrong, they are just different.
My approach would be to use ES6 and its native module support.
To accomplish this I always use my own boilerplate named fabric which uses Webpack to compile the modules, Browsersync to help you on your development, Tape for unit testing, SASS for your CSS preprocessing, and Babel to compile a compatible ES5 bundle that you can easily use in your application.
Now, the way to use the ES6 modules is something like this with named exports:
//------ lib.js ------
export const sqrt = Math.sqrt;
export function square(x) {
return x * x;
}
export function diag(x, y) {
return sqrt(square(x) + square(y));
}
//------ main.js ------
import { square, diag } from 'lib';
console.log(square(11)); // 121
console.log(diag(4, 3)); // 5
Or using default exports:
//------ myFunc.js ------
export default function () { ... };
//------ main1.js ------
import myFunc from 'myFunc';
myFunc();
You can learn more about ES6 modules at 2ality
Here's the pattern I use:
When possible, break concepts into their own sub-object
Regardless of sub-objects or not, declare any non-broken-up properties first, then add to it as needed
If the files are across multiple files and you do not wish to use sub-objects per-file, use a temporary object to hold additional properties, and then extend the original.
Sample:
var myObj = {
foo: "one",
bar: "two",
baz: false,
deez: -1
}
myObj.login = function() {
// lots and lots of code
};
myObj.user = function() {
// lots and lots of code
};
myObj.drinks = {
beer: function() {},
wine: function() {},
sunnyDelight: {
drinkIt: function() {},
burp: function() {}
}
};
myObj.init = function() {
myObj.login.init();
myObj.user.init();
// lots of jQuery document.ready stuff
}
myObj.init();
Note that "drinks" is a concept unto itself, containing multiple properties and methods. Your concepts might be something like "ui", "utils", "data" or whatever the role of the contained properties happens to be.
For the extend point I made, there's not much code needed there either
// "utilities.js"
var myObj = {
// a bunch of properties and/or methods
};
myObj.moreStuff = "more stuff!";
and then in another file you have two choices. Either add to the object without overwriting it (you will need the dot-notation to do this):
// "ui.js"
var myObj = myObj || {};
// adds the render object to the existing myObj
myObj.render = {
header: function() {},
dialogBox: function() {}
}
The above works particularly well if you sub-divide your concepts... because you can still have fairly monolithic objects that will not trample over the rest of myObj. But maybe you want to add directly to myObj without trampling and without subdividing concerns:
// "ui.js"
var myObj = myObj || {};
// ultimately, the CONTENTS of this object get merged into the existing myObj
var myObjSupplement = {
header: function() {},
dialogBox: function() {},
heroBiscuit: "A yummy biscuit made from heroes!"
}
// using jQuery here, but it's not the only way to extend an object
$.extend(myObj, myObjSupplement)
I don't see TOO many opportunities to use the above, since myObjSupplement is now in the global namespace and defeats the purpose of limiting additions to the global namespace, but it's there if you need it.
[edited to add: ]
It might not go "without saying" as I thought-- but dividing into many different files probably works best if you have a build process in place that can concatenate them into one file suitable for minifying. You don't want to have 100 or even 6 separate files each requiring a synchronous HTTP call to fetch.
There are more modern and possibly 'better' approaches with technologies like AMD/RequireJS... but if the question is, "how do I divide up an object literal into several files", the above answer I've given is one I can stand behind.
While there are automated ways of doing this I'm sure, and I am also interested in seeing the answers this question gets, I would recommend simply going in and moving the method definitions into different files and calling the functions normally method(param); and linking the files to your html page.
This would serve multiple purposes, including the one you are looking to acheive of breaking your code down into more manageable modules. Among those purposes also include the fact that instead of having those definitions written to memory for every instance of the object, you would only define it once and make references to it whenever you need it.
Sorry I can't be of more help without actually seeing the JavaScript File.
You can reference this stack overflow example if you need more guidance in achieving this.
You don't have to have all of the methods defined in your objects or classes, it's better to modularize these methods into different files and use the <script src="path/to/your/script.js"> </script> tags to include them all with your html/php page
I use UglifyJS to minify a concatenated set of files, which works fine but not good enough. The built lib uses namespaces, so classes, functions and constants are stored in a root namespace variable:
(function() {
var root = { api:{}, core:{}, names:{} };
/* util.js file */
root.names.SOME_LONG_NAMED_CONST='Angel';
/* Person.js file */
root.core.Person = function(name) { this.name = name };
/* API.js with the functions we want to expose */
root.api.perform = function(param_for_api) { /* do something */ }
window.lib_name.perform = root.api.perform;
})();
which is minified to the not-so-minimal version
(function(){var a={api:{},core:{},names:{}};a.names.SOME_LONG_NAMED_CONST="Angel",a.core.Person=function(a){this.name=a},a.api.perform=function(){},window.lib_name.perform=a.api.perform})();
I understand uglify probably thinks that root var is a data structure that must be kept as-is and can't be changed. Is there a way to let UglifyJS mangle the nested names in the root namespace?
When you minimize Javascript you can only change names of variables, the api, core and names are not variables but properties of an object. If these were changed by the minimizer, you would potentially get unexpected results. What if in your code you would call
root["api"].perform = function()...
or even something like
function doIt(section, method, argument) {
root[section][method](argument);
}
doIt('api','perform', 101);
All perfectly legal JS, but a minimizer could never figure out what's going on.
I have been trying to use --mangle-props of UglifyJS2 and can tell you: 'it makes a mess'.
As someone pointed out: 'Developer should decide what properties to mangle, not uglifyjs'
I am approaching the problem using this options:
--mangle-props
--mangle-regexp="/_$/"
The regex matches any property with a underscore at the end.
You asked to mangle nested names in the root namespace. So, your code:
(function() {
var root = { api:{}, core:{}, names:{} };
root.names.SOME_LONG_NAMED_CONST_='Angel';
root.core.Person_ = function(name) { this.name = name };
root.api.perform_ = function(param_for_api) { }
window.lib_name.perform = root.api.perform;
})();
Would result in this:
(function() {
var n = {
api: {},
core: {},
names: {}
};
n.names.a = "Angel";
n.core.b = function(n) {
this.name = n;
};
n.api.c = function(n) {};
window.lib_name.perform = n.api.c;
})();
Command:
uglifyjs --beautify --mangle --mangle-props --mangle-regex="/_$/" -- file.js
If you want to mangle first level of root namespace (api, core, names) just put a underscore on them (api_, core_, names_), you are in control ;)
Just a side note: when you are mangling properties usable by other js files, you should mangle all files together with the same command, so the same identifier will be used over all files.
Aside from #JanMisker 's point (which is completely valid), rewriting properties is unsafe because they can be exposed to code outside the scope of the minification.
Although the self executing function has a scope, and if the code is only
(function() {
var root = { api:{}, core:{}, names:{} };
root.names.SOME_LONG_NAMED_CONST='Angel';
alert(root.names.SOME_LONG_NAMED_CONST); // some code that does something
})();
It is true that outside of the function, there is no way to access the root object, so rewriting the property names is safe, and the following code would result in the same:
(function() {
var a = { b:{}, c:{}, d:{} };
a.d.e='Angel';
alert(a.d.e);
})();
But even if you are inside your private scope you can access, and more importantly assign to variables from an outer scope! Imagine this:
(function() {
var root = { api:{}, core:{}, names:{} };
root.api.perform = function(param_for_api) { /* do something */ }
window.lib_name = root.api;
})();
You are not only exposing a function but an object with a function on it. And the function will be visible from any place where window is visible.
So, for example writing the following in the javascript console would yield different results with and without minification:
window.lib_name.perform(asdf);
With minification you would have to write:
window.lib_name.f(asdf);
Or something similar.
Remember that there can always be code outside your minification.
It is not that crucial to have the absolute minimal JS, but if IT IS that crucial for some reason (for example: aliens abducted your stepdaughter, and the only way to have her back is to minify this below 100 characters or so), you can manually replace an undesirably long property name to a shorter one, just be sure that it will not be exposed anywhere, and isn't be accessed through associative array notation (root['api']).
as #Jan-Misker explained in his answer, property name mangling is NOT an good idea because it could potentially break your code.
However, you can workaround it by define the property names as local variables, and modify all .properties to [keys], to make smaller file size:
(function() {
var API = 'api';
var CORE = 'core';
var NAMES = 'names';
var SLNC = 'SOME_LONG_NAMED_CONST';
var root = {};
root[API]={};
root[CORE]={};
root[NAMES]={};
/* util.js file */
root[NAMES][SLNC] ='Angel';
/* Person.js file */
root[CORE].Person = function(name) { this.name = name };
/* API.js with the functions we want to expose */
root[API].perform = function(param_for_api) { /* do something */ }
window.lib_name.perform = root[API].perform;
})();
Because now all the properties became a local variable, uglify js will mangle/shorten the variable names and as consequence you overall file size reduced:
!function(){var a="api",b="core",c="names",d="SOME_LONG_NAMED_CONST",e={};e[a]={},e[b]={},e[c]={},e[c][d]="Angel",e[b].Person=function(a){this.name=a},e[a].perform=function(){},window.lib_name.perform=e[a].perform}();
However, reduced file size doesn't mean you will get shorter downloading time on real server, because usually our http transport is gzipped, most of the repetitions will be compressed by your http server and it does a better job than human.
The latest release of uglify (today) has object property mangling, see v2.4.18. It also supports reserved files for excluding both object properties and variables that you don't want mangled. Check it out.
Use the --mangle-props option and --reserved-file filename1.json filename2.json etc....
I have a question related to the way of using the RequireJs.
Our app should grow a lot in the near feature, so the major problem is to prepare a skeleton that would be followed by the developers involved in the project.
we tried this kind of wrapper on RequireJS(trying to enforce the OOP approach):
//each file will contains such a definition
//this file should be located under: appNamespace/Client/Widget.js
//attaching the class definition to the namespace
with ({public : appNamespace.Client})
{
//using a Class defined in the file: appNamespace/DomainModel/DataTable.js
var dt = using('appNamespace.DomainModel.DataTable');
//Class definition
public.Widget = function(iName)
{
//private variable
var dSort = new dt.SortableTable();
this.Draw = function(iRegion)
{
//public method implementation
}
}
}
Or, we could use the natural RequireJS model like structure:
<pre><code>
//this module definition should be located in the file: appNamespace/Client/Widget.js
define('appNamespace/DomainModel/DataTable', function(dt)
{
function Widget(iName)
{
//private variable
var dSort = new dt.SortableTable();
this.Draw = function(iRegion)
{
//public method implementation
}
}
return Widget;
})
I would prefer the first example because:
1. it will enforce developers to write scripts in a more OOP style
2. it looks like the C# or Java notation - so it will allow a faster switching between the back-end code and the client code
3. I really don't like the model structure because it allows to write any style of code. More of that, it's not enough to define your class, you should define the API that this file is exposing - so you can actually define an API that has no relation to what that file contains.
So - why would I use the second example - the natural RequireJS model style?
Is there anything that I miss?
Any suggestion is welcome