Given a need to write command line utilities to do common tasks like uploading files to a remote FTP site, downloading data from a remote MySQL database etc.
Is it practical to use JavaScript for this sort of thing? I know there are JavaScript interpreters that can be run from the command line, but are there libraries for things like FTP and database access the way there are for e.g. Java? If so, what's the best place to look for them? (Google searches with JavaScript in the keywords always seem to return many pages of browser specific things.)
And is there a way to package a JavaScript program up as a standalone executable on Windows?
Update: I've decided Python is a better tool for this kind of job, but the answers to the original question are still good ones.
Standalone executable?
By the way you ask the question, I'm not sure if you are aware, but the Windows Script Host - included in Windows - allows you to run .js files from the command-line. Your javascript will not be an executable, it will remain a script, a text file. The script runs within cscript.exe, which is provided by WSH. There's no compilation required. Maybe you knew all that.
I use Javascript this way for various utilities on Windows.
I think your instinct is right on the availability of libraries. You are sort of on your own to find all those things. Although, once you find them, it's not hard to package Javascript libraries as COM components and allow re-use from anywhere. See here for an example of packaging the Google Diff/Patch/Match Javascript library in COM.
Addendum: Once a bit of code is available within COM, it can be consumed by any Javascript running on the machine. Some examples of COM objects available to Javascript scripts running in WSH:
MSXML2.XMLHTTP object - used in AJAX, but can be used for any HTTP communication. There also an object for the XSLT engine so you can do transforms from script.
Excel.Application - allows you to open up Excel spreadsheets and automate them from Javascript.
Communicator.UIAutomation - automate MS Communicator (send IM's via script)
COM objects for Google Earth.
SlowAES - an all-Javascript implementation of AES encryption.
You can use Rhino to compile Javascript into Java byte code, and get access to all Java libraries.
Or you could use JScript.net, and get access to the .net libraries. .net includes a jsc.exe that produces exe-files.
Both of these requires the respective framework to be installed to be able to run.
Node.js is by far the best environment for running non-browser JS. I've used Rhino and SpiderMonkey, and there's a pretty huge difference in everything from the basics like how errors are handled to the size of the community using the tool. Node is pitched for "server-side" JS - building server apps in JS. It's great for this. But it works equally well for building command line tools.
The NPM package manager (bundled with Node) provides a nice global directory for finding and installing packages. It works much better than other language equivalents like PECL / Pear / CPAN / etc. Several high quality tools like JSHint, The Jade templating language, and the CoffeeScript compiler are all already available through NPM/Node:
npm install -g jshint, coffee-script, jade
jshint my_code.js
jade < my.jade > my.html
For args parsing, there are packages like commander.js. I currently use a heavily extended version of Commander in my underscore-cli command-line tool.
For messing with JSON or for doing command-line JS work (similar to "perl -pe"), check out underscore-cli - It's a really powerful tool for processing JSON data, processing underscore templates, and running JS expressions from the command-line. I use it for 1001 different things that would otherwise be really annoying to accomplish.
Rhino is bundled with JDK 1.6, jrunscript.exe in the bin directory will allow you to run any Javascript you want. Since it runs under Java you get access to any Java libraries that you may have.
We use it from the command line extensively. It's very good at that.
One way is to write these utilities as AIR applications - They can be written in JavaScript and need not have a UI. They have access to the command line, and there are existing ActionScript 3 libraries that can handle FTP etc. These ActionScript APIs can be called from JS, in AIR applications. AIR applications also have access to a sqlite database.
jslibs is a good standalone JavaScript runtime that support many 3rd party open source libraries like zlib, SQLite, NSPR, libiconv, libTomCrypt, OpenGL, ...
Related
I'm trying to organize my Javascript code and Require.js seems to be the ticket, but it looks like the optimization tool can only be used with Node (NOTE: I know it says the browser can be used too, but it isn't recommended).
If I'm using PHP, is there any way for me to optimize my Javascript?
My current stack: PHP, Slim 3.0, Twig and Webix
You do not have to use r.js to optimize your AMD modules. Any tool that knows how to read AMD modules and how to produce a bundle from them could be used instead of r.js. I don't know if such a tool exists that is implemented in PHP.
If you are going to use r.js then you must have a JavaScript virtual machine that will run its code. r.js only supports Node, Rhino, Nashorn, or the browser. This is a limited list because r.js needs to be able to read files, and how to do this varies from platform to platform. (This also explains why browser usage is not generally recommended: the limitations in the browser are such that it can only be viable for restricted cases and not for general optimization.)
I'm not seeing the need for Node as being particularly onerous. The first projects I used RequireJS with were for applications backed by Django, which is a Python-based web framework. That's similar to your own situation.
I have a project to write to a postgress database from an html, in my project I included packages jquery, html, and css. I read that to access a postgress database it’s needed to have a server side package like nodejs, and I want to include this package in my project. I would appreciate if I can get sample projects or resources to do it.
NodeJS isn't a Javascript package that you can run on a browser.
Let me explain.
NodeJS started out as a project to run Javascript code outside of the browser and on top of your Operating System. This gives it access to OS specific functionality like writing to files, writing to Databases, spawning processes, starting an HTTP server etc. Much like any other general purposes programming language like Python, Ruby, Java, PHP etc.
So NodeJS is two things - the Javascript Interpreter(same as the browsers, V8 to be specific) and Libraries (like fs, vm, zip, etc).
While most of these libraries are inter-operable with the Browser's version of the Javascript interpreter via something like Browserify (Used to run node.js modules on the browser), some of them like the libraries used to write to a database are not and cannot be used on the browser.
To use the postgress database JS library, you'd have to learn node.js and expressjs. But what would be practical is to find a library specific to the framework you are using, whether it is PHP, Ruby on Rails, Django/Python etc.
as a total JavaScript beginner, how do I actually develop for it?
I mean, I've seen https://azure.microsoft.com/en-us/documentation/articles/documentdb-programming/ but there was no mentioning of a development environment.
Is there something like a visual studio project template for server-side javascript?
I use node.js so your mileage may vary if you are developing from .NET, but here's what I do:
First of all, I created an npm package documentdb-utils. It is a wrapper for the DocumentDB node.js package that makes it easier to do a bunch of things.
Then, I created npm package documentdb-mock to write tests for my stored procedures. The source code for documentdb-mock includes 4 example stored procedures along with a test suite for each using nodeunit. You can start with these as they exercise most of the server-side API.
After I have them passing my local unit tests, I write integration tests that exercise my system end-to-end including creating any necessary data for each test run. The only problems that I've found here with sprocs that I didn't see in my mocked testing had to do with reaching certain limits... although, documentdb-mock has been upgraded to simulate many of these now also.
I haven't open sourced this yet, but I have also written a parser/rewriter that will embed any require(d) packages into my sprocs before sending them to DocumentDB. This allows me to write and test in a nicely factored way on node.js even using downloaded packages from within my sprocs, but when they get pushed to DocumentDB any dependencies are automatically embedded inside of the function. I'll open source that at some point (probably adding it to documentdb-utils) but I can share it with you now if you desire.
Here are a few tools that I found helpful for development (especially server-side [by which we mean database-side] scripting):
DocumentDB Studio - https://github.com/mingaliu/DocumentDBStudio/releases
Sample Code - https://github.com/Azure/azure-documentdb-js-server/tree/master/samples/stored-procedures
Here is another nice open source tool for exploring data in documentdb:
documentdb.a7pl.us
How can I (and what tools do I need to) create a makefile that:
Combine all JavaScripts '/js/*.js' (in a manual order - possibly with cat)
Verify that combined script works against unit tests (which using qUnit)
Minify JavaScripts
Verify that minified script works against same unit tests
I would like makefile to work on Mac OS command line.
I will be uploading the Makefile to a GitHub repository, so I would like something that other developers will be able to use easily.
Here is one complex script you could look for inspiration:
https://github.com/mobilizejs/mobilize.js/blob/master/release.py
(check also README)
Generally there are always project specific use cases and one solution does not fit every project, thus everyone is doing ad hoc scripts. Also, makefile is pretty limited in the end - I recommed to pick a real scripting tool.
Thanks for all the suggestions, here is the solution that I have decided to go with:
Makefiles - just as a shortcut really
Google V8 - to execute JavaScript from command-line
I can easily perform minification and unit testing using JavaScript.
I have made a few minor changes (added ability to write files) to the sample shell application that comes with V8 and compiled on OS X. I plan to somehow cross-compile for OS X and Windows in both 32bit and 64-bit. I will include binaries for V8 and custom application source in the GitHub repository. I will include instructions to obtain and compile V8 for those who want to customise or recompile.
V8 will be doing all of the work (essentially). Whilst I have yet to bring all of these things together, I believe that this should work.
The only downside is that DOM based unit testing cannot be automated. It might be possible to get the DOM stuff from the Chromium project, but this is probably going to be too time consuming. I'll leave that to an enthusiastic contributor (should anybody want to).
So we're writing a full-text search framework MongoDb. MongoDB is pretty much javascript-native, so we wrote the javascript library first, and it works.
Now I'm trying to write a python framework for it, which will be partially in python, but partially use those same stored javascript functions - the javascript functions are an intrinsic part of the library. On the other hand, the javascript framework does not depend on python. since they are pretty intertwined it seems like it's worthwhile keeping them in the same repository.
I'm trying to work out a way of structuring the whole project to give the javascript and python frameworks equal status (maybe a ruby driver or whatever in the future?), but still allow the python library to install nicely.
Currently it looks like this: (simplified a little)
javascript/jstest/test1.js
javascript/mongo-fulltext/search.js
javascript/mongo-fulltext/util.js
python/docs/indext.rst
python/tests/search_test.py
python/tests/__init__.py
python/mongofulltextsearch/__init__.py
python/mongofulltextsearch/mongo_search.py
python/mongofulltextsearch/util.py
python/setup.py
I've skipped out a few files for simplicity, but you get the general idea; it' a pretty much standard python project... except that it depends critcally ona whole bunch of javascript which is stored in a sibling directory tree.
What's the preferred setup for dealing with this kind of thing when it comes to setuptools? I can work out how to use package_data etc to install data files that live inside my python project as per the setuptools docs.
The problem is if i want to use setuptools to install stuff, including the javascript files from outside the python code tree, and then also access them in a consistent way when I'm developing the python code and when it is easy_installed to someone's site.
Is that supported behaviour for setuptools? Should i be using paver or distutils2 or Distribute or something? (basic distutils is not an option; the whole reason I'm doing this is to enable requirements tracking) How should i be reading the contents of those files into python scripts?
The short answer is that none of the Python distribution tools is going to do what you want, the exact way you want it. Even if you use distutils' data_files feature, you're still going to have to have your javascript files copied into your Python project directory (i.e., somewhere under the same directory as your setup.py.)
Given that, you might as well just copy the .js files to your package (i.e. alongside mongofulltextsearch/init.py) as part of your build process, and use package_data or include_package_data=True.
(Or alternatively, you could possibly use symlinks, externals, or some such, if your revision control system supports those. I believe that when building source distributions, the Python distribution tools convert symlinks to real files. At least, you could give that a try.)