Node.js module directory structure philosophy - javascript

There seems to be silent agreement on directory structure of node.js. At least I couldn't find any information on official site.
From what I've understand browsing opensource projects.
Usually project has /bin and /lib directory.
/bin contains modules entry point[s].
/lib contains 'helper' code.
I guess this is based on Unix directory structure.
And it makes sense for good-old compiled programs with executables and dlls.
AFAIK /lib was used to share libraries between programs.
In node, however, true dependencies are in node_modules.
/lib is used for application code.
Why not use /src?
This model is designed from execution point of view. But all projects, I've seen, use it to structure the code under develpment too. Also I've seen github-projects where whole module is just one file in bin directory.
I'm new to node.js and would like to know how people in community came to those decisions.

I guess what you mentioned about lib and src happened because of node philosophy to design small modules, not only in terms of code size, but most importantly in terms of scope. This principle has its roots in unix philosophy, particularly -
Small is beautiful.
Make each program do one thing well.
The architecture that I follow for developing web applications is broken into models, services, routes and controllers. I use lib for utilities and helper functions. Also a lot of node applications are built using express which itself has minimal api. In short, it is basically a matter of choice to choose convention or configuration.

Related

Node express project architecture when use versioning

I'm working on a node+express+mongo project, and I am wondering how the project architecture would be when working with versioning, should I put all the folders in /v1 and then make the same thing for /v2, or is it just the routes, I'm searching for the best practice of this case
It depends on what versioning tool you use. With git I would do different branches for major versions at least and add the version number to the config by all of them, so the routing and URI creation would know what the version number we are talking about. I would not add the version number to the directory structure.
I am not sure about other versioning systems, maybe in some of them you would need to have separate directories for each major version, but if so, then code reusage will be an issue and you have to separate the common parts to a common directory or to a different project.

Why use bin/www and not index.js?

The express-generator tool creates a file called bin/www and uses it as the application's main entry-point. I believe I've seen a couple of other modules do this as well, but the vast majority simply use index.js.
What is the rationale behind this? Of course I understand why you would split the server and the code for setting up the program into a separate modules, but why bin/www and not index.js? Why nest the main-entry point to a program two levels deeper than the stuff it calls? And remove the file-extension, making it even less descriptive?
Is there a clever, non-obvious reason behind this? Should I use this for my node-modules as well?
Thank you!
[edit]:
All good answers, thank you folks! I've accepted the one pointing out that this is standard behaviour for packages that include executables. Here's some more reading I've come across on this:
https://docs.npmjs.com/files/package.json#bin
https://blog.npmjs.org/post/118810260230/building-a-simple-command-line-tool-with-npm
You're used to running npm run, but not sysadmin. He will look for executables (attribute x) in thebin directory.
The entry point index.js is for node module. All packages that provide commands to run on the console contain the bin directory.
The extension is removed because it is not a script, but as a program. And these do not have extensions.
express-generator create a basic structure for an express application. By convention, the entry point of the app is index.js or app.js. In fact, express-generator create an app.js at the root of the application with the initial setup of express.
Also by convention, the bin/ directory is used for binary files, and by extension for the scripts you can directly launch (note the shebang at the first line of www file). This is common on Linux that binary file has no extension and it could explain the choice to keep this habit for this file.
www, by convention again, is used for naming web application (like /var/www/html in Apache server)
Anyway, as the documentation says,
The app structure created by the generator is just one of many ways to
structure Express apps. Feel free to use this structure or modify it
to best suit your needs.
See also this answer who talk about the core structure of express between version 3 & 4, with the removing of external module.

Node.js/Express project structure

so I've had strictly a MVC programming background (Laravel, CodeIgniter, Django, etc.) and more and more I'm build bigger projects in Node.js but I'm having a difficult time figuring out a solid way of structuring my project that makes sense...
I've done some research and I love the way J. Cole Morrison does it here but it makes for extremely fat route files.
The structure is something like
app
-router
-index.js (your main route def. that imports individual route files)
--routes
This is nice and readable but all of your logic is stuffed in the routes folder which just doesn't make sense (or does it?). I guess I'd expect something more along the lines of seperate controllers that will handle your thinking. Does anyone else have Ideas or structures they use or is this a good way to go?
After using node.js / express.js in some projects, I have found Kraken's (1.+) structure very useful. It uses folders as part of the routes. Also, it comes with a scaffolding tool, so the mvc structure is easy to keep. This is not the only solution, but it is a good one indeed.
Disclosure: This is copy from kraken.js
/config Application configuration including environment-specific
configs
/controllers Routes and logic
/locales Language specific content bundles
/lib Common libraries to be used across your app
/models Models
/public Web resources that are publicly available
/public/templates Server and browser-side templates
/tasks Grunt tasks to be automatically registered by
[grunt-config-dir]
/tests Unit and functional test cases
index.js Application entry point

JavaScript styleguide on organizing files

I have worked in a web project with a heavy part on JavaScript, and I have noticed that there was no style how to use JavaScript. What unsettled me most is that everyone added files here and there, which resulted in a mess to organize and deliver them. Because this will be happen in every new project, I would like to have something like a styleguide for JavaScript. This styleguide should address the following questions:
How should JavaScript files be organized in the file system during development?
How should the JavaScript parts be separated from the HTML and other parts of the application?
How should JavaScript files be delivered in the real application, so that less code has to be loaded on each request and not too much requests have to be sent?
Is there something public available as a starting point for developing our own styleguide? What are your experiences in using your styleguide? Are developers following it easily, what are the simple and what are the difficult parts in it?
(I know, more question than one, but I'm interested in the whole story here. As a background, we have used JQuery and JSF, but I don't think that will have an impact on the answer.)
If you're doing heavy client side, you are probably going the MVC way.
So I'll answer your questions with the approach taken by the brunch. Brunch projects use MVC library Backbone.js, and have strict directory structure.
How should JavaScript files be organized in the file system during development?
src/
app/
collections/
controllers/
models/
styles/
templates/
views/
vendor/
build/
web/
config.yaml
Use Stitch to organize your files as CommonJS modules. Then you will be able to use require() to define dependency between them, as well as to combine them into one file later.
How should the JavaScript parts be separated from the HTML and other parts of the application?
build directory is used to store html; build/web is used to store javascript, images, and css.
How should JavaScript files be delivered in the real application, so that less code has to be loaded on each request and not too much requests have to be sent?
At the build stage, all JavaScript is minified and combined into one file (build/web/js/app.js), so that client will have to make only one HTTP request when he / she visits your site for the first time.
It's probably a good idea to make building process as easy as possible. Brunch does that by offering brunch watch command, which monitors filesystem for changes and builds code instantly with the help of Stitch and some other tools.
(It should be noted that during development brunch projects also use CoffeeScript as the primary language; it is transparently compiled by brunch before stitching the resulting JavaScript. However, this doesn't matter as long as file organization is concerned, and is out of scope of your question.)
For all JavaScript files definitely use a separate directory. Have as many files as possible semantically. One large constructor should correspond to a separate file. Never use filename prefixes where you can create a directory.
Unix-style directory structure is often found on GitHub:
src -- for the source JavaScript.
lib -- for libraries.
tests -- for unit tests.
bin -- for executables.
dist -- for compiled files.
For compiling we use a Makefile with targets for production and development. The production version is all of files JSHint`ed, minified and concatenated into one. The development target is generating a server-side script that includes all JavaScript files dynamically (for easy inclusion into HTML).
But generally it depends. We used a widget directory for one project. This widget directory had a set of separate widget subdirectories (e.g. slider, tabs, modal-window), each of which had the following layout (inspired by DOMLoader):
html -- for HTML templates.
css -- for CSS files necessary for the widget.
js -- for the widget JavaScript constructor.
Crockford has a few stylistic guidelines and the Yahoo exceptional performance site has details which might be useful to you.
I can recommend a book: JavaScript Patterns by Stoyan Stefanov.
I think one of the best book about javascript

How can I convert a multi-file node.js app to a single file?

If I have a node.js application that is filled with many require statements, how can I compile this into a single .js file? I'd have to manually resolve the require statements and ensure that the classes are loaded in the correct order. Is there some tool that does this?
Let me clarify.
The code that is being run on node.js is not node specific. The only thing I'm doing that doesn't have a direct browser equivalent is using require, which is why I'm asking. It is not using any of the node libraries.
You can use webpack with target: 'node', it will inline all required modules and export everything as a single, standalone, one file, nodejs module
https://webpack.js.org/configuration/target/#root
2021 edit: There are now other solutions you could investigate, examples.
Namely:
https://esbuild.github.io
https://github.com/huozhi/bunchee
Try below:
npm i -g #vercel/ncc
ncc build app.ts -o dist
see detail here https://stackoverflow.com/a/65317389/1979406
If you want to send common code to the browser I would personally recommend something like brequire or requireJS which can "compile" your nodeJS source into asynchronously loading code whilst maintaining the order.
For an actual compiler into a single file you might get away with one for requireJS but I would not trust it with large projects with high complexity and edge-cases.
It shouldn't be too hard to write a file like package.json that npm uses to state in which order the files should occur in your packaging. This way it's your responsibility to make sure everything is compacted in the correct order, you can then write a simplistic node application to reads your package.json file and uses file IO to create your compiled script.
Automatically generating the order in which files should be packaged requires building up a dependency tree and doing lots of file parsing. It should be possible but it will probably crash on circular dependencies. I don't know of any libraries out there to do this for you.
Do NOT use requireJS if you value your sanity. I've seen it used in a largish project and it was an absolute disaster ... maybe the worst technical choice made at that company. RequireJS is designed to run in-browser and to asynchronously and recursively load JS dependencies. That is a TERRIBLE idea. Browsers suck at loading lots and lots of little files over the network; every single doc on web performance will tell you this. So you'll very very quickly end up needing a solution to smash your JS files together ... at which point, what's the point of having an in-browser dependency resolution mechanism? And even though your production site will be smashed into a single JS file, with requireJS, your code must constantly assume that any dependency might or might not be loaded yet; in a complex project, this leads to thousands of async load barriers wrapping every interaction point between modules. At my last company, we had some places where the closure stack was 12+ levels deep. All that "if loaded yet" logic makes your code more complex and harder to work with. It also bloats the code increasing the number of bytes sent to the client. Plus, the client has to load the requireJS library itself, which burns another 14.4k. The size alone should tell you something about the level of feature creep in the requireJS project. For comparison, the entire underscore.js toolkit is only 4k.
What you want is a compile-time step for smashing JS together, not a heavyweight framework that will run in the browser....
You should check out https://github.com/substack/node-browserify
Browserify does exactly what you are asking for .... combines multiple NPM modules into a single JS file for distribution to the browser. The consolidated code is functionally identical to the original code, and the overhead is low (approx 4k + 140 bytes per additional file, including the "require('file')" line). If you are picky, you can cut out most of that 4k, which provides wrappers to emulate common node.js globals in the browser (eg "process.nextTick()").

Categories

Resources