i work at the digital department of a public broadcaster, together with two other frontend developers. We're currently looking into improving our Javascript workflow and build processes. Stuff like packaging, minifying, versioning, etcetera.
In an ideal scenario, we would have a global repository for all Javascript-related libraries (like jQuery) and stuff we build ourselves, that can be easily included in lots of projects and versioned.
All of our backend developers use Maven for this process with their Java code. I'm wondering if people have experience with Maven and Javascript, or maybe with other tools that might be good (or better) for the job. And in general if people have good resources about setting up workflow / build processes for frontend and javascript development.
We use Hudson (http://hudson-ci.org/) to continuously build/integrate our Python (Django via zc.buildout), R and several other types of projects. Whenever someone checks in code to the central VCS, a build is triggered and the test suite(s) will run. The build and test status is shown on a central screen in the office.
Hudson not only builds and tests, but also checks for coding style and syntax errors using pep8 and pyflakes, amongst others. Javascript is only checked with jslint at this point, but we may unit-test that in the future using something like xpcshell (more info).
** edit **
Go straight for Jawr, keeping rest in answer for alternatives
** edit **
First of all there's Ruby's rake which in theory can be used through jruby and in theory can be configured and run in the POM.
There's also Jake, but not sure if the latest Rhino supports CommonJS.
Whilst writing this awnser, I just came across Jawr which looks really interresting and there seems to be maven support/plugins.
The last option you already know about is dynamic loading using for example RequireJS or LABJS.
Related
I am not a professional developer - I'm just a self-taught amateur in the field, and I followed the trends with quite some delay. My history with javascript goes back to when I was 14 years old and randomly opened files on my C:/Windows/ directory (it was Windows 98), until I found some intriguing .js files. I learned PHP, C and python, but came back to Javascript and follow his evolution from a scripting language to a full-stack tool - first, I played with Jquery a Jquery / XSLT stack, then made a little videogame using Websocket and Gulp and browserify , then recently learnt some Angular, Ionic and Vue, all based on Webpack.
I'm having though about what tool to use as a task manager. Especially, I tends to think most of the tools used these days are inefficient - hard to configure, needing a thousand plugins to work, which often create incompatibilities between versions,... And more importantly (I may be wrong here) but how come none of them are checking which dependencies need to be rebuild?
Last time, I was launching my test in a typescript project using npm test and got really frustrated. I realized the whole project was compiled every-time I changed a tiny detail in my test files. Having developed in C, I found that really inefficient and wrote a makefile to address that.
Now the bad side of make is that it's kinda hard to make it work with modern javascript tools (tsc explicitely adviced me to use a .tsconfig file instead of the CLI). Another example is there is no official CLI tool to compile a .vue file into a render() function.
Now my question is is there a modern tool that doesn't require me to rebuild, compile, minify, uglify my whole project and lose 30 seconds every time I add a commas in a file? and what's wrong with make?
I would take any idea or correction about that matter with gratitude :)
I don't know the exact nature of your project but I don't think there is a single tool for all the things you want to achieve. That's why we have the term "javascript fatigue".
Currently Webpack is the closest thing we have "one ring to rule them all" in terms of module management in development environments for javascript projects.
Now my question is is there a modern tool that doesn't require me to rebuild, compile, minify, uglify my whole project and lose 30 seconds every time I add a commas in a file? and what's wrong with make?
Since you mention you are familiar with webpack, maybe you can checkout its hot module replacement feature. Here is its description straight from the developers' website:
Hot Module Replacement (HMR) exchanges, adds, or removes modules while an application is running, without a full reload. This can significantly speed up development in a few ways:
Retain application state which is lost during a full reload.
Save valuable development time by only updating what's changed.
Tweak styling faster -- almost comparable to changing styles in the browser's debugger.
It is a sad fact that with each tool comes extra cognitive load. To reduce it, maybe you can:
follow certain patterns and habits in your workflow
have basic setups or boilerplate projects for frequent ones
check out the best practices
take advantage of cheat sheets and API docs
checkout source code before searching on the internet, It maybe less time consuming and less confusing than searching on the internet
try to master few vital tools for your projects
Following the works of developers who move the community forward maybe helpful for staying up to date with technologies, usually they gave speech and write articles.
It may look a lot of work when you see them listed one after the other but you will see it is more rewarding and less time consuming in the long run.
Please remember that, it is a common problem among developers, even the best ones suffer from it. That is why we have these tools in the first place. Good news is that, with every passing day dust is settling and tools are getting better, more mature and more stable.
For more on hot module replacement:
https://webpack.js.org/concepts/hot-module-replacement/
One big benefit I've always perceived with using NodeJS on the server is the potential for sharing bits of code between the server and client side (ex. input validation). Now that I'm actually developing using NodeJS one difficulty that I've found is determining the responsibility and context in which each body of code is executed. Below I'll list a few of the difficulties I've had in hopes gain some enlightenment on conventions or guidance that I may be overlooking that could help elevate these issues.
Build-Time Code
Build time code for projects that use Gulp, Grunt, or vanilla NPM in a way that follow the basic documentation are generally pretty easy to follow. Most smaller projects tend to keep all of the code within a single file and the file tends to be named a conventional name like gulpfile.js, however with bigger projects I've seen these scripts begin to be split out. I've seen some cases where the gulp file is split into multiple files and placed under a separate directory. Even worse I've found cases where the gulpfile.js file isn't even named as such causing new developers to hunt around to find where the gulpfile is located and once it is located the gulp command always has to be run with the specific --gulpfile option.
Run-Time Server-Side Code
The entry point for basic node applications appear to simply require pointing out a specific JavaScript file when running the node command (ex. node script.js). For web server applications, such as those using Express, I've noticed that by convention the entry point file is often called server.js and can usually be found in the root directory of the application. In some other cases however such as when running the web server in a developer environment I've seen gulp tasks take on the responsibility of launching Node. In these cases there seems to be multiple ways to include the entry point but one example I've found is just starting up the webpack complier followed by a require statement for the entry point script. Figuring out how to incorporate normal guidance on how to accomplish a typical node debug command is non-trivial in this type of setup. Besides the entry point of the application, there doesn't seem to be any general guidance on directory structure for NodeJS/Express applications that keeps server-side specific code in it's place to help locate it and to keep it separate from build-time and client-side code.
The server-side story becomes even more complex in cases where the server side code is used both for the purpose for serving up static content, server-side generated views (such as with MVC), as well as for providing an API to the client side. My preference is to separate API from the application project but I get the feeling from others that there is a sense of overcomplexity involved in doing so where I see it as a reasonable separation of concerns.
Run-Time Client-Side Code
Since client-side code can often have various entry points based on the first page that is requested this can be tricky. However because of the general transparency of URLs with how they map to resources for typical cases it as well as how powerful the debugging tools have become in modern browsers, it isn't too much trouble following the follow of the scripts. The difficult instead for the client side code comes more for typical build processes which generally end up copying the files around and placing them into a production like structure under a different name. An example would be a project that has a folder called src or js that holds client-side and server-side code intermingled except for that only a portion of the files happen to be included in a build task which transforms and often concatenates the files and places them in a distribution folder. Common names of these distribution folders that I've seen are dist, public, www, and wwwroot. Often if not always these directories are at the root of the project which at least makes it a bit easier to locate without having to interrogate the build scripts.
My hope is that there is some general guidance on how to put all of this together in a sane way perhaps by an authoritative source mainly to give guidance to those like myself who may want start off on the right foot. As a side effect perhaps being able to reference some sort of standard even if it is a loose one may also reduce the amount of boilerplate a team has to invent and discuss as they get started. Within each of the contexts listed above there will obviously be some technology specific conventions such as those followed for AngularJS, Meteor, or ReactJS on the client-side. The conventions I'm looking for are more specific to separating the main highlevel contexts in end-to-end JavaScript applications where the language and platform no longer become obvious way to differentiate between each.
Build-Time Code
IMHO if you have so much build-time code that it's more than say 1000 lines and requires more than a handful of files, something has gone off the rails. Either you don't know how to make good use of existing packages from npm or you don't understand how to refactor generic code and publish as independent npm packages. If you feel like you need guidance about a project's build-time code because it is so large and complex, my suggestion is to modularize and split out into separate projects - each published independently to npm. Also just check your overall approach. What are you doing that is so custom and requires so much machinery?
Run-Time Server Side Code
Please see my other answer to ExpressJS How to structure an application?
Generally I'd rather see client side code and server side code either completely separate npm packages (separate git repos, separate package.json files, published independently) (if they are large enough) or otherwise co-mingled in the same module and grouped by coupling (all the code relating to a feature kept together including front and back end code), especially if your code base has a substantial amount of code that works in both environments.
I have an open-source full-stack node/JS application called mjournal that keeps browser code and node code alongside each other. You can have a look and see if it feels logical to you and easy to understand where code lives. It is by no means a popular approach, so many folks will dislike it, but it feels good to me personally since I've embraced "group by coupling" as a general principle.
Figuring out how to incorporate normal guidance on how to accomplish a typical node debug command is non-trivial in this type of setup
Yeah, that's nonsense. Your app should start with npm start or something like node server.js. Elaborate gulp/grunt setups that confuse new developers are unnecessary complexity you should just eliminate.
The server-side story becomes even more complex in cases where the server side code is used both for the purpose for serving up static content
In my experience, the code to serve up static content boils down to 5 lines or less, so no big deal. If you have a non-microscopic amount of code dealing with serving static content, again something has gone way off the rails.
Run-Time Client Side Code
My hope is that there is that there is some general guidance on how to put all of this together in a sane way perhaps by an authoritative source mainly to give guidance to those like myself who may want start off on the right foot.
There are some people in the node community that have adopted the "convention over configuration" approach in use by Ruby on Rails, EmberJS, and some other large projects. If you like that approach, check out tools that use it like SailsJS, EmberJS, Yeoman generators, etc.
But in general, looking for a "standard" is not how the node.js/javascript/web community rolls. Small npm packages. File layouts that are forced into obviousness due to smallness. I feel your frustration here as front-end toolchains are so complex, but ultimately it's because JavaScript in the browser took too many decades to create a reasonable module system. Things may start to standardize in the next few years now that ES6 modules are an official spec, but with so much code already written in CommonJS and it's terrible precursors like RequireJS/AMD, we'll be dealing with them probably for the foreseeable future.
I have a web project coded 80% in JavaScript and 20% in Django without a single unit testing as I rushed for Minimum Marketing Features. Now that the project is getting funded, I decided to invest some time to introduce TDD. I had a great deal of inspiration from this KickStarter-funded tutorial.
http://www.letscodejavascript.com/
The author uses Node.js, Jake, Lint, Nodeunit, and Karma to simplify the whole integration process. The server/client tests in all major browsers is done in a single command and I was really hooked to this idea, but it requires switching to Node.js.
I've searched for TDD in Django and ran into this tutorial that makes use of Selenium.
http://www.tdd-django-tutorial.com/
However this TDD was primarily based on unit testing in server. Here are the questions.
Can multiple client JavaScript testing be done in Django/Python?
I assume the answer is no since js files are nothing more than static library in Django. Correct me if I'm wrong.
Is it worth using Node.js just for the sake of TDD Javascript?
My logic was either you use Python or Node.js for the server, but since tools like Karma and Buster.js requires Node.js, I was wondering whether setting up the Node.js alongside Django just for multiple client testing is plausible choice when considering lower cost of maintenance.
Thank you :D
You can take a look at using selenium in your django test suite. Django's official docs cover this in moderate detail
To answer your question about Node.js - I would say that it's probably not worth the complexity to add node.js SOLELY for the purpose of running unit tests. Also, since your javascript is likely built to run in a browser, it's less likely that things will break down if you use a tool like selenium (which runs the code in an actual browser, providing a python scripting interface).
I'm creating an intranet application, the UI of which premoninently uses Ext JS 4.1.1
I have created several custom javascript classes extending Ext JS controls and other code for UI, validation, communication, etc.
Given that my code is not very huge and that my application will be deployed in a controlled & well known environment I'd ideally like to load all .js files upfront at the time of application invokation (boot). I'd like to know an approach to achieve this, mostly from within the relm of Ext JS.
What you want to do is something really usual. Take a look at the deployment section to know how to use the Sencha SDK to create one file containing all your custom javascript classes which extend extjs component here
I hope it is useful ;)
As of this month Sencha (beta) released a revamped build tool. Actually, it is a complete rewrite in Java (I believe). Sencha Cmd is the name, and it is all that the predecessor should have been!
Compile times have been crunched to maybe a 10th of its predecessor and packages/builds are now configured using a powerful command processor.
Integration into other build frameworks and CI servers is now a top priority and achieved by integrating the tool with ANT.
I just spent the past hours to migrate a project from the old Sencha SDK Tools to the new Sencha Cmd. I particularly love the new compiler.
Documentation is already pretty good.
Blog post that introduced Sencha Cmd
Official SDK documentation - there is a total of 10 articles on the subject, don't miss them, particularly the
Compiler documentation
For smaller projects that follow the standard (single-page) pattern suggested by Sencha, there is actually a range of higher-level commands that manage the entire application creation and build process.
Furthermore, Sencha has now a pattern in place for multi-page applications.
And you can always use the compiler directly and implement your own build process on top of it. That's what we did in our project.
There is one class called Ext.Loader
This is more of a server-side question than an Ext JS question. You want to include all your pages up-front? Get a JavaScript compressor to concatenate your files together, minify the whole pack, and add a <script> tag to every page in your web app.
I also work on an intranet web app that heavily uses Ext JS (both 3 and 4), but we have quite a large codebase. Since we use Tomcat, we have a Java class that includes standard files on each page, plus any file groups or individual files we need for that page. Something like that might work for you as well.
You can use Ext.Loader for dynamic loading when you need it, but that doesn't sounds like what you're looking for. Still an option though, should your codebase expand and you need a better solution.
EDIT: As mentioned in the comments, my original answer failed to address the Sencha SDK Tools. I felt it would be more beneficial to make my response part of my revised answer.
While this task may be exactly what the Sencha SDK was designed for, I feel that the product is not yet mature enough for production environments. At the time of writing, the current version is 2.0.0 Beta 3. Beta, meaning it's still under development, and it shows. Documentation is practically non-existent and the official forums are full of topics claiming the product simply does not work. At this point, it's nothing more than a toy to be played with. The most helpful document I've seen is Sencha SDK Tools 2.0 and ExtJS4: The Missing Docs, and the author even ends with "The SDK Tools need a lot of work."
When I'm choosing a product for a production environment, I want something stable, something reliable, something with good documentation, something with quality support. The Sencha SDK Tools are none of those things. Yet.
Are there any must have tools for Java Script (RIA) development like maven, jUnit, Emma, link4j etc. for Javascript? What is the best way to set up a continous integration system for a bigger application or framework? How do projects like jQuery test their code? How to manage dependencies and different project configurations?
tools i know so far:
javascript-maven-tools (is maven the right choice?)
jslint
yuicompressor
sprockets (found it 5 mins ago)
jsunit
selenium
Hudson is a good tool for managing your build and test jobs. And it has a Selenium plugin.
Edited to provide a better answer:
Your question is quite broad, with few details about your situation or the actual problems that you've encountered, so I'll give you a quick overview based on my experience.
Build automation is a good thing. It's good to detect build failures and test failures as soon as possible. I recommend automating your tests in a standard environment so that they run as soon as possible after a commit. And I recommend making your release build as simple and reproducible as pushing a button.
Martin Fowler has a good list of recommended practices.
And a previous question contains a description of how to run javascript unit tests with CI.
Specifically regarding Hudson, we use Hudson because it's flexible, it allows a central server to manage builds on Windows and Linux slaves, and it's got plugins to support lots of different workflows and tools.
As for dependencies, Hudson allows you to trigger builds on the completion of other builds, which is good for firing off various tests after a successful product build. If you're asking how you should deal with external dependencies, you'll have to figure out what policy makes sense based on your application's design and requirements. I've seen a couple other javascript questions go by recently about how to handle such dependencies.
jQuery uses QUnit.
Another compressor is Crockfords JSMIN.