This is vague - I apologize in advance, I am trying to be as succinct as I can with my limited understanding, while exposing my potentially incorrect assumptions.
I have a website that's literally one huge HTML file. It runs scripts defined in-line in a <scripts> tag.
My goal is to move all the scripts into individual .js files and pull them into index.html (or into one another where required). I am familiar with the usage of Node's require and I am familiar with import, as used in Angular. I stress usage because I don't really know how they work.
Assumption 1: I cannot use require - it is purely for Node.js. The reason I bring it up is that I am pretty sure I have seen require in AngularJS 1.5 code, so assuming makes me uncomfortable. I am guessing this was stitched together by WebPack, Gulp, or similar.
Assumption 2: I should use import, but import only works with a URL address, if I have my .js hosted on a server or CDN, it will be be pulled. BUT, I cannot give local pathing (on the server) here - index.html will NOT automatically pull in the dependencies while being served. I need npm/Webpack/other to pre-compile my index.html if I want the deps pulled in on the server.
Assumption 3: Pre-compiling into a single, ready-to-go, html file is the desired way to build things, because the file can be served quickly (assuming it's ready to go). I make the assumption with the recent trend of serving Markdown from CDNs, the appearance of the JAMstack, and the number of sites using Jekyll and such (though admittedly for traditional Jekyll sites).
Assumption 4: I also want to go to Typescript eventually, but I assume that changes nothing, since I will just pull in TS to compile it down to .js and then use whatever solution I used above
Question: If it's clear what I am trying to do and what confuses me, is a decent solution to look into npm/Webpack to stich together my .js files? What would prepare them for being stiched together, using import/export? If so, is there an example of how this is usually done?
As you mentioned, require cannot be used for your purposes, since it is a part of CommonJS and NodeJS's module system. More info on require can be found here: What is this Javascript "require"?
Import is a ES2015 standard JS syntax. But ES2015 standard and everything above it has very limited browser support. You can read more about it here: Import Reference
However, you can still code in the latest standard (thereby enabling the use of import/export etc.,) and then transpile the code to be able to run on the browser. Inorder to do this, you require a transpiler. You can refer Babel which is one of the most popular transpilers : https://babeljs.io/
For your exact purpose, you need to use a module bundler. Webpack/Rollup etc., are some popular module bundlers. They can automatically identify all the JS files referenced through import, combine them and then transpile code to be able to run on the browser (they also use a transpiler) and produce a single JS file (or multiple, based on your configurations).
You can refer the getting started guides of Webpack: https://webpack.js.org/guides/getting-started/
or RollupJS: https://rollupjs.org/guide/en#quick-start
Related
In our AngularJS application we currently have a lot (400+) of files, which get included via <script>-tags. The order these files is something like this:
AngularJS script files
3rd party plugins / modules
business Logic files
modules
services
controllers / components / directives
We would like to move to a better approach utilizing a module bundler and TypeScript. New files are already written in TypeScript but don't make use of import/export. In order to make things easier, we could convert every JavaScript file into a TypeScript file and fix the resulting errors in a feasible time.
However, before we do this, we would like to have a viable strategy on how we could gradually make use of import/export. I'm thinking of something like rewriting one module from time to time, starting with modules deep down in the dependency tree.
However I was not able to achieve this, but I'm quiet sure that others already had to solve this before.
Once you decide on a module bundler, you'll need to learn about its facilities for (1) allowing other JavaScript on the page to access things defined in the bundle and (2) allowing the bundle to access things defined by other JavaScript on the page. (If you're able to migrate in strict dependency order, you might never need #2.) For example, for Webpack, #1 would be the library* output options and #2 would be externals. Then just move the code into modules little by little, adjusting the configuration as necessary so that each part of the code has access to the things it needs from the other part of the code. Since Webpack only supports a single library export module, during the transition, you may have to maintain a dummy module that just re-exports all modules that you need to access from code outside the bundle. This is a little tedious and represents extra work that you wouldn't have to do if you migrated all at once, but you may decide it's worth paying that cost in order to be able to migrate gradually.
If you have issues getting type information from TypeScript module files in non-module TypeScript files, see this answer for a workaround.
I've been using a grunt file to concatenate all my JS into a single file which is then sent to the client. What advantage do I have in using require calls then? The dependencies are inherent from the concatenation order and I don't have to muddy all my JS with extra code and another third-party library.
Further, backbone models (for example) clearly state their inheritance in their definitions. Not to mention that they simply wouldn't work if their dependencies weren't included anyway.
Also, wouldn't maintenance be easier if all comments related to dependencies were in one place (the grunt file) to prevent human error and having to open every JS file to understand its dependencies?
EDIT
My (ordered) file list looks something like:
....
files: [
"js/somelib.js",
"js/somelib2.js",
"js/somelib3.js",
"js/models.js",
"js/views.js",
"js/controllers.js",
"js/main.js"
], ...
So perhaps requireJS isn't worth it for small projects anyway.
Using require.js allow you to break down each part of your application into reusable modules (AMD) and to manage those dependencies easily. It is not easy to manage dependencies in a javascript application with 100 classes, for example.
Also, if you don't want all the overhead of require, check this out (developed by the same guy who created require.js): https://github.com/jrburke/almond
The answer depends on the size of your app and the end use case..
A single site.min.js payload for the front end (client) generally aims for small file sizes and simple architectures (1 single file generated from maybe 10).
back end based (server) apps are usually much bigger and complicated and therefore may warrant the use of another tool to help with managing large code libraries and dependencies (50 files for example).
In general, RequireJS is worthwhile but only if you have many files and dependencies. An alternative for use in the client would be almond. Again, using a tool like this must warrant the need (many files and dependencies).
The answer from orourkedd is also worth reading.
I would like to create a (require.js style) AMD module that can be used in the browser and in node. What is the best way to do this? I keep seeing references to r.js, but still not 100% sure on how to use it, or if it is necessary for my situation.
Also, when including this module in node, do I still run require('module-name'), or will this change as well?
First things first: AMD basics, What all you can do with them, How to optimize them
In extremely simple terms
AMD modules are reusable JS code. Think of them as functions kept in separate files.
AMD loaders are central functions which call all other functions (modules). Think of them as "main" method in C or Java.
RequireJS is a framework which pulls all this scattered code and stitches it in a usable form.
RequireJS works in a browser. As a result, all your code is "stitched" together in a web browser.
r.js works offline (in a web server or on your development machine) to "stitch" all the code offline so that when it reaches a web browser, it's already "stitched".
Use of RequireJS lib is a must no matter you want to "stitch" your code in browser or you want to serve your code "pre-stitched".
Use of r.js is optional. It's needed only if you want to increase performance and decrease HTTP calls.
I'm trying to figure out how to best manage Javascript file dependencies and have that drive the packaging of a 100% front-end app. In short, I'm building an application using backbone.js along with some other libraries. I want an organized codebase and would like to be able to declare dependencies within each file. Ideally those declarations would a) drive the order in which files are loaded by the browser (while in development I want the files to load separately) and drive the order in which the packaging scripts load the scripts for concatenation (I'm aiming to serve a single file for the entire app).
I've been reading on requirejs and commonjs but I'm not convinced.
I have a simple shell script right now that uses cat <file> <file> <file> <file> > concatenated.file to do what I want but it's a pain to maintain that list of files up to date and in the right order. It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
Any suggestions?
Thanks you,
Luis
I am partial to stealjs myself. It's part of JavascriptMVC but no reason why you can't use it with Backbone.js
The nice part about this one is that it builds your app for production including minifying your css and js and neatly packing all of it into 2 files: production.css and production.js.
It can handle loading non JS files too so you can do things like steal('somefile.css').then(function() {...});
For files, its very much like you would do in other languages:
steal(dep1, dep2, dep3).then(function () {
// code
});
For complex frontend apps Asynchronous Module Definition (AMD) format is best choice. And it's alot of great loaders that supports AMD (curl.js, RequireJS).
I recomend this articles to learn about modern approaches in javascript dependecy management:
Writing Modular JavaScript With AMD, CommonJS & ES Harmony
Why AMD?
For packaging take into account CommonJS specifications, there are few implementations and it's a matter of taste, but in any case I recommend to choose tools, that is compliant with some of that specifications.
It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
I have had the same idea several months ago and are working on a dependency resolver for my Resource Builder which already makes it easier for me (including the need to distinuish between development and deployed version, with the optional debug parameter).
JsDoc Toolkit (and related efforts), which syntax is supported e. g. by Eclipse JSDT, provides a #requires tag, so you could use that. But resolving dependencies is still not a trivial task (as you can see in ResourceBuilder::resolveDeps()). (The ultimate goal is to use static code analysis to resolve dependencies automatically, without any tags.)
This would reduce the current
<script type="text/javascript" src="builder?src=object,types,dom,dom/css"></script>
to a mere
<script type="text/javascript" src="builder?src=dom/css"></script>
As for asynchronous loaders: The good thing about asynchronous loaders is that they are fast. The bad thing about asynchronous loaders is that – if they work; they are all based on a non-standard approach – they are so fast that you cannot be sure that the features the scripts provide are available in following scripts. So you have to have your code executed by their listeners. I recommend avoiding them unless you really have features in your application that are only needed on demand.
So we're writing a full-text search framework MongoDb. MongoDB is pretty much javascript-native, so we wrote the javascript library first, and it works.
Now I'm trying to write a python framework for it, which will be partially in python, but partially use those same stored javascript functions - the javascript functions are an intrinsic part of the library. On the other hand, the javascript framework does not depend on python. since they are pretty intertwined it seems like it's worthwhile keeping them in the same repository.
I'm trying to work out a way of structuring the whole project to give the javascript and python frameworks equal status (maybe a ruby driver or whatever in the future?), but still allow the python library to install nicely.
Currently it looks like this: (simplified a little)
javascript/jstest/test1.js
javascript/mongo-fulltext/search.js
javascript/mongo-fulltext/util.js
python/docs/indext.rst
python/tests/search_test.py
python/tests/__init__.py
python/mongofulltextsearch/__init__.py
python/mongofulltextsearch/mongo_search.py
python/mongofulltextsearch/util.py
python/setup.py
I've skipped out a few files for simplicity, but you get the general idea; it' a pretty much standard python project... except that it depends critcally ona whole bunch of javascript which is stored in a sibling directory tree.
What's the preferred setup for dealing with this kind of thing when it comes to setuptools? I can work out how to use package_data etc to install data files that live inside my python project as per the setuptools docs.
The problem is if i want to use setuptools to install stuff, including the javascript files from outside the python code tree, and then also access them in a consistent way when I'm developing the python code and when it is easy_installed to someone's site.
Is that supported behaviour for setuptools? Should i be using paver or distutils2 or Distribute or something? (basic distutils is not an option; the whole reason I'm doing this is to enable requirements tracking) How should i be reading the contents of those files into python scripts?
The short answer is that none of the Python distribution tools is going to do what you want, the exact way you want it. Even if you use distutils' data_files feature, you're still going to have to have your javascript files copied into your Python project directory (i.e., somewhere under the same directory as your setup.py.)
Given that, you might as well just copy the .js files to your package (i.e. alongside mongofulltextsearch/init.py) as part of your build process, and use package_data or include_package_data=True.
(Or alternatively, you could possibly use symlinks, externals, or some such, if your revision control system supports those. I believe that when building source distributions, the Python distribution tools convert symlinks to real files. At least, you could give that a try.)