Develop Reference

What is the best practice to consolidate multiple Javascript files?

For CSS, I can use SASS to import one CSS file to another and produce only single CSS file. What is the similar method for Javascript files?

You might want to check out Closure Compiler (which is a Google product).
You would probably want the Closure Compiler Application form of the product.
A sample workflow would probably look like:
Create a list of your JS files and paths
Run the command to compile and concatenate files (java --jar compiler.js --js path_to_file1.js --js path_to_file2.js (etc.) compiled.js)
Closure Compiler also has a related project, Closure Stylesheets, that does the same thing for stylesheets.
This approach, of course means that there's a pre-compilation step. Depending on your backend, there also exist libraries that do the compilation when the page is built. For example, for JSP, there's Granule a tag library that creates the compiled JS and CSS files at page build.
There's a third possibility: modularization. Since you gave the example of being able to import CSS files in SASS, an analogue for JavaScript is using a module library, using either the CommonJS standard, or (the one I prefer), the AMD (asynchronous module definition) pattern, which I have personally used with RequireJS. RequireJS also comes with a nice optimizing tool that will bundle up (minify, compress, concat etc) all the required files for your application
UPDATE
Since you mentioned that you are using Django in the comments (might be useful to update the question with this info too), see if this answer helps too

You could use minify which allows you to minify and combine javascript files. It also works with CSS.

How to automatically compile client side CoffeeScript files with Node.js / Express

As the title says I'm trying to automatically compile the public static coffeescript files on page load, rather than having to compile them myself and use the .js files, how might I achieve this, I'm trying to maintain a full CoffeeScript stack, this is the only thing I'm having trouble figuring out.
tl;dr: Read the title of the post.

There are a number of ways to accomplish what you're trying to do. Two of the easiest that I know of:
Use the connect-assets module. The idea behind this is that you have an /assets folder in the root, and you instantiate it with express.static, as you normally would with your /public folder. In there, you have two more folders: /js and /css. Your CoffeeScript goes in your /js folder. Then, from within your view template, just call js('yourfile'). It's a wonderfully simple module, but isn't the most advocated asset pipeline.
Use asset-rack. While not as simple to grasp as connect-assets, it's very flexible and is easy to extend. It would be the closest comparator to Rails' asset pipeline and is used by most of the popular JS frameworks (like Sails.js).
However, I would really advise that you refrain from on-the-fly compilation of assets, as it can really drain server performance.
It would be way better to compile on file save using a build system - CoffeeScript ships with Cake, so you can define a watch/compile/build/concatenate step right in your Cakefile and all it would take for you to compile on file save is to type $ cake watch in to the terminal before you change any code.
Alternatively you could write code, then $ cake build. Whichever you prefer. I might also add that cake-flour takes all of the pain out of writing Cake tasks.

coffee --watch --compile .
watches for files changes in . and compiles them as they change. Since you are using expressjs I think you would also like to restart the server as soon as a recompile happens:
coffee --compile --watch . &; nodemon server.js
which uses https://github.com/remy/nodemon

If you only want to have access to compiled CoffeeScript files over HTTP, you could also give connect-coffee-script middleware a try. It has worked quite well for me.
It has the advantage over using coffee --watch --compile that you don't need to have a separate program running.

JavaScript styleguide on organizing files

I have worked in a web project with a heavy part on JavaScript, and I have noticed that there was no style how to use JavaScript. What unsettled me most is that everyone added files here and there, which resulted in a mess to organize and deliver them. Because this will be happen in every new project, I would like to have something like a styleguide for JavaScript. This styleguide should address the following questions:
How should JavaScript files be organized in the file system during development?
How should the JavaScript parts be separated from the HTML and other parts of the application?
How should JavaScript files be delivered in the real application, so that less code has to be loaded on each request and not too much requests have to be sent?
Is there something public available as a starting point for developing our own styleguide? What are your experiences in using your styleguide? Are developers following it easily, what are the simple and what are the difficult parts in it?
(I know, more question than one, but I'm interested in the whole story here. As a background, we have used JQuery and JSF, but I don't think that will have an impact on the answer.)

If you're doing heavy client side, you are probably going the MVC way.
So I'll answer your questions with the approach taken by the brunch. Brunch projects use MVC library Backbone.js, and have strict directory structure.
How should JavaScript files be organized in the file system during development?
src/
app/
collections/
controllers/
models/
styles/
templates/
views/
vendor/
build/
web/
config.yaml
Use Stitch to organize your files as CommonJS modules. Then you will be able to use require() to define dependency between them, as well as to combine them into one file later.
How should the JavaScript parts be separated from the HTML and other parts of the application?
build directory is used to store html; build/web is used to store javascript, images, and css.
How should JavaScript files be delivered in the real application, so that less code has to be loaded on each request and not too much requests have to be sent?
At the build stage, all JavaScript is minified and combined into one file (build/web/js/app.js), so that client will have to make only one HTTP request when he / she visits your site for the first time.
It's probably a good idea to make building process as easy as possible. Brunch does that by offering brunch watch command, which monitors filesystem for changes and builds code instantly with the help of Stitch and some other tools.
(It should be noted that during development brunch projects also use CoffeeScript as the primary language; it is transparently compiled by brunch before stitching the resulting JavaScript. However, this doesn't matter as long as file organization is concerned, and is out of scope of your question.)

For all JavaScript files definitely use a separate directory. Have as many files as possible semantically. One large constructor should correspond to a separate file. Never use filename prefixes where you can create a directory.
Unix-style directory structure is often found on GitHub:
src -- for the source JavaScript.
lib -- for libraries.
tests -- for unit tests.
bin -- for executables.
dist -- for compiled files.
For compiling we use a Makefile with targets for production and development. The production version is all of files JSHint`ed, minified and concatenated into one. The development target is generating a server-side script that includes all JavaScript files dynamically (for easy inclusion into HTML).
But generally it depends. We used a widget directory for one project. This widget directory had a set of separate widget subdirectories (e.g. slider, tabs, modal-window), each of which had the following layout (inspired by DOMLoader):
html -- for HTML templates.
css -- for CSS files necessary for the widget.
js -- for the widget JavaScript constructor.

Crockford has a few stylistic guidelines and the Yahoo exceptional performance site has details which might be useful to you.

I can recommend a book: JavaScript Patterns by Stoyan Stefanov.
I think one of the best book about javascript

How can I convert a multi-file node.js app to a single file?

If I have a node.js application that is filled with many require statements, how can I compile this into a single .js file? I'd have to manually resolve the require statements and ensure that the classes are loaded in the correct order. Is there some tool that does this?
Let me clarify.
The code that is being run on node.js is not node specific. The only thing I'm doing that doesn't have a direct browser equivalent is using require, which is why I'm asking. It is not using any of the node libraries.

You can use webpack with target: 'node', it will inline all required modules and export everything as a single, standalone, one file, nodejs module
https://webpack.js.org/configuration/target/#root
2021 edit: There are now other solutions you could investigate, examples.
Namely:
https://esbuild.github.io
https://github.com/huozhi/bunchee

Try below:
npm i -g #vercel/ncc
ncc build app.ts -o dist
see detail here https://stackoverflow.com/a/65317389/1979406

If you want to send common code to the browser I would personally recommend something like brequire or requireJS which can "compile" your nodeJS source into asynchronously loading code whilst maintaining the order.
For an actual compiler into a single file you might get away with one for requireJS but I would not trust it with large projects with high complexity and edge-cases.
It shouldn't be too hard to write a file like package.json that npm uses to state in which order the files should occur in your packaging. This way it's your responsibility to make sure everything is compacted in the correct order, you can then write a simplistic node application to reads your package.json file and uses file IO to create your compiled script.
Automatically generating the order in which files should be packaged requires building up a dependency tree and doing lots of file parsing. It should be possible but it will probably crash on circular dependencies. I don't know of any libraries out there to do this for you.

Do NOT use requireJS if you value your sanity. I've seen it used in a largish project and it was an absolute disaster ... maybe the worst technical choice made at that company. RequireJS is designed to run in-browser and to asynchronously and recursively load JS dependencies. That is a TERRIBLE idea. Browsers suck at loading lots and lots of little files over the network; every single doc on web performance will tell you this. So you'll very very quickly end up needing a solution to smash your JS files together ... at which point, what's the point of having an in-browser dependency resolution mechanism? And even though your production site will be smashed into a single JS file, with requireJS, your code must constantly assume that any dependency might or might not be loaded yet; in a complex project, this leads to thousands of async load barriers wrapping every interaction point between modules. At my last company, we had some places where the closure stack was 12+ levels deep. All that "if loaded yet" logic makes your code more complex and harder to work with. It also bloats the code increasing the number of bytes sent to the client. Plus, the client has to load the requireJS library itself, which burns another 14.4k. The size alone should tell you something about the level of feature creep in the requireJS project. For comparison, the entire underscore.js toolkit is only 4k.
What you want is a compile-time step for smashing JS together, not a heavyweight framework that will run in the browser....
You should check out https://github.com/substack/node-browserify
Browserify does exactly what you are asking for .... combines multiple NPM modules into a single JS file for distribution to the browser. The consolidated code is functionally identical to the original code, and the overhead is low (approx 4k + 140 bytes per additional file, including the "require('file')" line). If you are picky, you can cut out most of that 4k, which provides wrappers to emulate common node.js globals in the browser (eg "process.nextTick()").

How can I compile CoffeeScript from .NET?

I want to write an HttpHandler that compiles CoffeeScript code on-the-fly and sends the resulting JavaScript code. I have tried MS [JScript][1] and IronJS without success. I don't want to use [Rhino][2] because the Java dependency would make it too difficult to distribute.
How can CoffeeScript be compiled from .NET?

CoffeeScript-dotnet
Command line tool for compiling CoffeeScript. Includes a file system watcher to automatically recompile CoffeeScripts when they change. Roughly equivalent to the coffee-script node package for linux / mac.
CoffeeSharp
Includes a command line tool similar to CoffeeScript-dotnet as well as a http handler that compiles CoffeeScripts when requested from an asp.net site.
SassAndCoffeeScript
Library for Asp.net mvc that compiles sass and coffeescript files on request. Also supports minification and combination.
Manually Compile With IronJS
IronJS is a .NET javascript interpreter that can successfully load the CoffeeScript compiler and compile CoffeeScript.
Manually Compile With Node.js
Get the node binaries and add the bin directory to your path. Write a node.js script to load the CoffeeScript compiler and your CoffeeScript files and save the compiled javascript.

CoffeeScript is now fully supported by Chirpy:
http://chirpy.codeplex.com/

You specifically said that you wanted to write a runtime compiler, so this may not be exactly what you are looking for, but if the main point is to have a way to generate the javascript result, the Mindscape Web Workbench is interesting. It is a free extension for Visual Studio.NET 2010 and available in the Extension Manager. It gives Intellisense, syntax highlighting and compiles to JS as you write. I am just getting started using it but looks promising. Scott Hanselman talks about it here. It also supports LESS and Sass.

I've managed to compile CoffeeScript from .NET using IKVM, jcoffeescript and Rhino. It was straightforward, except that the JCoffeeScriptCompiler constructor overload without parameters didn't work. It ran OK with a java.util.Collections.EMPTY_LIST as parameter.
This is how I did it:
Download IKVM, jcoffeescript and Rhino.
Run ikvmc against js.jar, creating js.dll.
Run ikvmc against the jcoffeescript jar.
Add a reference to the jcoffeescript dll in Visual Studio. More references may be needed, but you will be warned about those.
Run new org.jcoffeescript.JCoffeeScriptCompiler(java.util.Collections.EMPTY_LIST).compile() in your code.
The next step would be to create a build task and/or an HTTP handler.

Check out the new coffeescript-dotnet project, which uses the Jurassic JavaScript implementation.

Since the CoffeeScript compiler now runs on Internet Explorer, after a couple of recent tweaks, it should be good to go within other MS-flavors of JavaScript as well. Try including extras/coffee-script.js from the latest version, and you should be good to go with CoffeeScript.compile(code).

I tried running the bundled extras/coffee-script.js through Windows Based Script Host (or just wscript) and it didn't report any issues. I then added this line:
WScript.Echo(CoffeeScript.compile('a: 1'));
at the end of the file and run it through wscript again and it printed the resulting JavaScript correctly.
Are you using COM objects? Can you share some more of the code responsible for initialising the MScript object reference?

CoffeeScript in Visual Studio 2010
It's Chirpy's fork (chirpy is a tool for mashing, minifing, and validating javascript, stylesheet, and dotless files)
"OK, I think I got it working on my fork, based mostly on other peoples' work. Check it out:
http://chirpy.codeplex.com/SourceControl/network/Forks/Domenic/CoffeeScriptFixes"
from http://chirpy.codeplex.com/workitem/48

I don't have a direct answer, (I hope you find one), but maybe take a look at the following to see how it might be done.
Jint - JavaScript interpreter for .NET
Using IKVM to compile Rhino would get rid of the Java runtime requirement.
jcoffeescript. I haven't looked at jcoffeescript, but I think it depends on JRuby and Rhino. You could possibly IKVM.NET this as well.

IronJS now supports CoffeeScript and is generally faster than the other .NET JS engines:
I have a blog post about wiring the two together here:
http://otac0n.com/blog/2011/06/29/CoffeeDemo-A-Simple-Demo-Of-IronJS-Using-CoffeeScript.aspx

My main editor is VS 2010 and I love the WorkBench extension. it's nice it auto compiles to js everytime you hit save on your .coffee file, also introduces you to SASS which I had read about but never got around.
They offer a pay version to that will autmaically shrink/minify your js and css files as well, since your.coffee and .scss are your source files anyway.
I'd encourage all VS users to go ahead and install this especially if you run VS 2010.
The only knock, and someone please correct me or enlighten me, is that with .coffee syntax it's not highlighted the way say html, js, c# code is. it might be because I am using a color scheme from http://studiostyl.es/ and for the record http://studiostyl.es/schemes/coffee- just shares the name coffee no special syntax highlight support for coffeescript that I am aware of. but no reason not to start using the workbench addin today!
Okay workbench website claims: syntax highlighting so again maybe it's the studiostyle.es i chose.

I know this is old but I came here to answer a very similar question: How do I get my CoffeeScript to compile using Visual Studio 2012 Express? Note that the free Express version does not allow any extensions so I could not continue to use the Mindscape Workbench extension that had served me well for quite some time.
It turns out to be very easy. Just use NuGet to install the Jurassic-Coffee package and off you go.
One advantage of using this package over mindscape workbench is that you can reference your coffee directly from the script tags in the html. It minifies and caches the compiled JS so you only do work if the requested coffee file has changed.
<head>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script src="home.coffee"></script>
</head>
The mindscape workbench allows you to bundle together different coffescript files which is very handy for modularising your coffeescript. You can also do this using Jurassic Coffee by utilising the #= require statement to include other coffee module files, for example:
#= require Classes\GridWrapper.coffee
class UsersGrid
constructor:->
#grid = new GridWrapper()
I think having the #= require staement in the coffee file is actually cleaner and clearer than the mindscape workbench approach, which kind of hides all this behind their interface so you forget easily what dependencies you have.
Note
There is one potential gotcha. The Nuget installer will put in an httphandler entry into your web.config that may not be compatible with IIS Express integrated managed pipeline mode.
You might therefore see the following error:
An ASP.NET setting has been detected that does not apply in Integrated
managed pipeline mode.
To fix this just remove the handler shown below.
<system.web>
//other stuff
<httpHandlers>
<add type="JurassicCoffee.Web.JurassicCoffeeHttpHandler,JurassicCoffee" validate="false" path="*.coffee" verb="*" />
</httpHandlers>
</system.web>

You could simply write a port of it to C#. I have ported Jison to C# (which is the underlying project that makes CoffeeScript run). I would think it may be a bit different, but both Jison parsers work the same.
I have not pull requested it back yet to Jison's main architecture, but will be doing so soon.
https://github.com/robertleeplummerjr

Instead of shelling out to CScript you could shell out to Node.js (here are self-contained Windows binaries)

I've tried to compile the extras/coffee-script.js file, unmodified, to jsc, the JScript.NET compiler for .NET, and I got many errors. Here are the noteworthy ones:
'require' is a new reserved word and should not be used as an identifier
'ensure' is a new reserved word and should not be used as an identifier
Objects of type 'Global Object' do not have such a member
Other errors were caused by the above said errors.

You might also want to check out jurassic-coffee, it is also a coffee-script compiler running the original compiler in jurassic.
It features sprocket style "#= require file.coffee" or "#= require file.js" wich can be used to keep .coffee files modular and combined right before compilation as well as embedding .js files.
It sports a HttpHandler with file watchers for .js and .coffee files that keeps track of what .coffee files needs to be re-compiled and pass through to the compiled *.js files for the rest.
jurassic-coffee is also available as a Nuget package
https://github.com/creamdog/JurassicCoffee

I've done an HttpHandler that uses Windows Script Host behind the scenes: https://github.com/duncansmart/LessCoffee and works great (it also compiles *.less files).
It's on NuGet: http://nuget.org/List/Packages/LessCoffee
It's based on this simple wrapper: https://github.com/duncansmart/coffeescript-windows

I wrote an inteructive shell using v8.
https://github.com/mattn/coffee-script-v8
This work as single executable file. (Don't use external files)
It can't use require(). But enough to learn coffeescript.