Develop Reference

Import, Require? How to Mash Javascript Files Together?

This is vague - I apologize in advance, I am trying to be as succinct as I can with my limited understanding, while exposing my potentially incorrect assumptions.
I have a website that's literally one huge HTML file. It runs scripts defined in-line in a <scripts> tag.
My goal is to move all the scripts into individual .js files and pull them into index.html (or into one another where required). I am familiar with the usage of Node's require and I am familiar with import, as used in Angular. I stress usage because I don't really know how they work.
Assumption 1: I cannot use require - it is purely for Node.js. The reason I bring it up is that I am pretty sure I have seen require in AngularJS 1.5 code, so assuming makes me uncomfortable. I am guessing this was stitched together by WebPack, Gulp, or similar.
Assumption 2: I should use import, but import only works with a URL address, if I have my .js hosted on a server or CDN, it will be be pulled. BUT, I cannot give local pathing (on the server) here - index.html will NOT automatically pull in the dependencies while being served. I need npm/Webpack/other to pre-compile my index.html if I want the deps pulled in on the server.
Assumption 3: Pre-compiling into a single, ready-to-go, html file is the desired way to build things, because the file can be served quickly (assuming it's ready to go). I make the assumption with the recent trend of serving Markdown from CDNs, the appearance of the JAMstack, and the number of sites using Jekyll and such (though admittedly for traditional Jekyll sites).
Assumption 4: I also want to go to Typescript eventually, but I assume that changes nothing, since I will just pull in TS to compile it down to .js and then use whatever solution I used above
Question: If it's clear what I am trying to do and what confuses me, is a decent solution to look into npm/Webpack to stich together my .js files? What would prepare them for being stiched together, using import/export? If so, is there an example of how this is usually done?

As you mentioned, require cannot be used for your purposes, since it is a part of CommonJS and NodeJS's module system. More info on require can be found here: What is this Javascript "require"?
Import is a ES2015 standard JS syntax. But ES2015 standard and everything above it has very limited browser support. You can read more about it here: Import Reference
However, you can still code in the latest standard (thereby enabling the use of import/export etc.,) and then transpile the code to be able to run on the browser. Inorder to do this, you require a transpiler. You can refer Babel which is one of the most popular transpilers : https://babeljs.io/
For your exact purpose, you need to use a module bundler. Webpack/Rollup etc., are some popular module bundlers. They can automatically identify all the JS files referenced through import, combine them and then transpile code to be able to run on the browser (they also use a transpiler) and produce a single JS file (or multiple, based on your configurations).
You can refer the getting started guides of Webpack: https://webpack.js.org/guides/getting-started/
or RollupJS: https://rollupjs.org/guide/en#quick-start

Efficiency of Browserify with multiple bundles

I'm new to Browserify and I'm trying to figure out how to make it more efficient with regards to how much the client needs to download.
I have a web app, that uses many different 3rd party libraries and custom code. With Browserify, it seems like the general approach people suggest is to wrap everything up into one big bundle.js. This seems horribly inefficient to me for several reasons:
For example, lets say your bundle.js contains lib1, lib2, lib3, customLib.
If a part of your web app only needs lib1 the client still has to download a huge bundle.js and it ends up not using 75% of it. Wasted bytes downloaded. Unnecessarily increased page load time.
If your customLib is a piece of code that you iterate upon often, then every time it changes, your clients have to redownload bundle.js, again downloading a ton of 3rd party libraries that haven't changed...
Other parts of your web app may use lib2 and lib3 but the client may or may not ever go to there, in which he definitely wasted bandwidth downloading the entire bundle.js.
I've seen suggestions to split up your bundle into multiple bundles. But to what end? If one page uses lib1, another page uses lib1 and lib2 and another page uses lib2 and lib3, then how do you split it up? The more you split it up into multiple bundles aren't you getting away from the advantages of bundle.js?
Browserify seems to be highly regarded so I hope that I am just missing something here. What is the proper approach to bundling together many libraries and custom scripts together? People call Browserify a "script loader" but every script loader I've seen in the past (like yepnope etc), use logic to determine which scripts to download, which seems like a much more efficient solution, while Browserify appears to want the client to download everything...

Not sure if the answer fits SO format well. But nevertheless...
Partitioning Section of handbook describes 2 following techniques
factor-bundle factors 2 or more entry points placing common dependencies into a single bundle.
partition-bundle same as factor-bundle but with runtime loading using async loadjs function.
Factor bundle
<script src="/bundle/common.js"></script>
<script src="/bundle/x.js"></script>
Partition bundle with async loading fallback
loadjs(['./x'], function(x){...});

Substack just published this gist explaining how to split bundles. He suggests to use factor-bundle like this:
browserify x.js y.js -p [ factor-bundle -o bundle/x.js -o y.js ] \
-o bundle/common.js

Why use requireJS instead of an ordered include list?

I've been using a grunt file to concatenate all my JS into a single file which is then sent to the client. What advantage do I have in using require calls then? The dependencies are inherent from the concatenation order and I don't have to muddy all my JS with extra code and another third-party library.
Further, backbone models (for example) clearly state their inheritance in their definitions. Not to mention that they simply wouldn't work if their dependencies weren't included anyway.
Also, wouldn't maintenance be easier if all comments related to dependencies were in one place (the grunt file) to prevent human error and having to open every JS file to understand its dependencies?
EDIT
My (ordered) file list looks something like:
....
files: [
"js/somelib.js",
"js/somelib2.js",
"js/somelib3.js",
"js/models.js",
"js/views.js",
"js/controllers.js",
"js/main.js"
], ...
So perhaps requireJS isn't worth it for small projects anyway.

Using require.js allow you to break down each part of your application into reusable modules (AMD) and to manage those dependencies easily. It is not easy to manage dependencies in a javascript application with 100 classes, for example.
Also, if you don't want all the overhead of require, check this out (developed by the same guy who created require.js): https://github.com/jrburke/almond

The answer depends on the size of your app and the end use case..
A single site.min.js payload for the front end (client) generally aims for small file sizes and simple architectures (1 single file generated from maybe 10).
back end based (server) apps are usually much bigger and complicated and therefore may warrant the use of another tool to help with managing large code libraries and dependencies (50 files for example).
In general, RequireJS is worthwhile but only if you have many files and dependencies. An alternative for use in the client would be almond. Again, using a tool like this must warrant the need (many files and dependencies).
The answer from orourkedd is also worth reading.

When to use Requirejs and when to use bundled javascript?

This may be a dumb question for web guys. But I am a little confused over this. Now, I have an application where I am using a couple of Javascript files to perform different tasks. Now, I am using Javascript bundler to combine and minify all the files. So, at runtime there will be only one app.min.js file. Now, Requirejs is used to load modules or files at runtime. So, the question is if I already have all things in one file, then do I need requirejs? Or what is a use case scenario where I can use requirejs and/or bundler?
Please let me know if any further details are needed.

Generally you only use RequireJS in its loading form during development. Once the site is done and ready for deployment, you minify the code. The advantage here is RequireJS knows exactly what your dependencies are, and thus can easily minify the code in the correct order. Here is what it says on the RequireJS website:
Once you are finished doing development and want to deploy your code for your end users, you can use the optimizer to combine the JavaScript files together and minify it. In the example above, it can combine main.js and helper/util.js into one file and minify the result.

This is a hotly contested issue among many proficient javascript developers. Many other languages have a "compilation" phase where the entire program is bundled up for deployment (JBoss's .WAR files come to mind). Programmers that come from more traditional backgrounds often favor this approach.
Javascript has seen such growth in recent years that it is difficult to chart exact best practices, but those that appreciate the more functional nature of Javascript often prefer the module loading approach (like require.js uses).
I wrote Frame.js which works much like require.js, so my bias is towards the module loader approach.
To answer your question directly, yes, it is one or the other.
Most that argue for packing your scripts into a single file believe it enables more compression and is thus more efficient. I believe the efficiency advantages of packaging are negligible in most cases because: (1) module load times are distributed over the entire session, (2) individual modules can be compressed to nearly the same percentage, (3) individual modules can be cached by the server and routers separately, and (4) loading scripts only when they are needed ultimately allows you load less code for some users and more code overall.
In the long run, if you can see an advantage to dynamic script loading use it. If not, bundle your scripts into a single file.

It depends on your application. If you're making a server-side app with only modest javascript (less than 100kb minified) then go for total bundling, you're probably going to be fine.
But if you're making a javascript app and have a ton of code in it, then your needs are going to be different.
For example, in my app I bundle all the core files. There's jQuery, underscore, backbone, my main app files, my user login system, my layout system, my notifications and chat system, all are part of my big initial file.
But I have many other modules as well that isn't part of the initial bundle, that are loaded after those.
The forums, the wiki, the wysiwyg, color picker, drag/drop, calendar, and some animation files are part of the second category. You need to make reasonable decisions about what's commonly used and needed immediately vs what can be delayed.
If I include everything immediately I can get above a meg of javascript, which would be insane and make the initial boot unacceptably slow.
The second category starts downloading after initSuccess event fires from the initial file.
But the second category is more intelligent than the first in that it loads what's more important first. For example if you're looking at the wiki it'll load the wiki before it loads the color picker.

Javascript dependecy management and packaging

I'm trying to figure out how to best manage Javascript file dependencies and have that drive the packaging of a 100% front-end app. In short, I'm building an application using backbone.js along with some other libraries. I want an organized codebase and would like to be able to declare dependencies within each file. Ideally those declarations would a) drive the order in which files are loaded by the browser (while in development I want the files to load separately) and drive the order in which the packaging scripts load the scripts for concatenation (I'm aiming to serve a single file for the entire app).
I've been reading on requirejs and commonjs but I'm not convinced.
I have a simple shell script right now that uses cat <file> <file> <file> <file> > concatenated.file to do what I want but it's a pain to maintain that list of files up to date and in the right order. It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
Any suggestions?
Thanks you,
Luis

I am partial to stealjs myself. It's part of JavascriptMVC but no reason why you can't use it with Backbone.js
The nice part about this one is that it builds your app for production including minifying your css and js and neatly packing all of it into 2 files: production.css and production.js.
It can handle loading non JS files too so you can do things like steal('somefile.css').then(function() {...});
For files, its very much like you would do in other languages:
steal(dep1, dep2, dep3).then(function () {
// code
});

For complex frontend apps Asynchronous Module Definition (AMD) format is best choice. And it's alot of great loaders that supports AMD (curl.js, RequireJS).
I recomend this articles to learn about modern approaches in javascript dependecy management:
Writing Modular JavaScript With AMD, CommonJS & ES Harmony
Why AMD?
For packaging take into account CommonJS specifications, there are few implementations and it's a matter of taste, but in any case I recommend to choose tools, that is compliant with some of that specifications.

It'd be much easier to be able to declare the dependency at the begining of each javascript file and have the packager and loaders be smart about using that information to concatenate/load scripts.
I have had the same idea several months ago and are working on a dependency resolver for my Resource Builder which already makes it easier for me (including the need to distinuish between development and deployed version, with the optional debug parameter).
JsDoc Toolkit (and related efforts), which syntax is supported e. g. by Eclipse JSDT, provides a #requires tag, so you could use that. But resolving dependencies is still not a trivial task (as you can see in ResourceBuilder::resolveDeps()). (The ultimate goal is to use static code analysis to resolve dependencies automatically, without any tags.)
This would reduce the current
<script type="text/javascript" src="builder?src=object,types,dom,dom/css"></script>
to a mere
<script type="text/javascript" src="builder?src=dom/css"></script>
As for asynchronous loaders: The good thing about asynchronous loaders is that they are fast. The bad thing about asynchronous loaders is that – if they work; they are all based on a non-standard approach – they are so fast that you cannot be sure that the features the scripts provide are available in following scripts. So you have to have your code executed by their listeners. I recommend avoiding them unless you really have features in your application that are only needed on demand.