I'm creating a commandline interface with node.js that runs an external script
> myapp build "path/to/script.js"
myapp is a node.js application that executes the script passed as a commandline argument.
To keep things brief, it basically does this:
var vm = require("vm");
var fs = require("fs");
var code = fs.readFileSync(scriptPath, { encoding: "utf8" }); // read "path/to/script.js"
var script = vm.createScript(code, scriptPath);
var context = vm.createContext({ require: require });
script.runInNewContext(context);
The contents of "path/to/script.js" look something like this:
var fs = require("fs"); // this works
var date = require("./date.js"); // supposed to load "path/to/date.js" but fails
My problem is that require() doesn't work properly in the external script. It works correctly for "native" modules like fs but fails on local files, probably because it doesn't know where to look up the modules.
I've considered to simply use following code:
require(scriptPath);
but then I can't inject my own script context.
I could write my own require function that emulates the built-in require but starts looking for modules in scriptPath but that seems a bit tedious...
I've used sandboxed-module to solve a similar problem.
var Sandbox = require('./sandboxed-module')
Sandbox.require(scriptPath);
Sandbox.require loads code from disk and runs it in a new context. It provides a sandbox-specfiic require function that allows you to tweak how require works.
Related
I'm trying to convert a web application into an electron app. I have multiple functions, in different files that I've imported into my main.js using a transpiler.
However, whenever I try do that in my electron app, I run into an issue with a module I'm using to move away from using php to access my database. Instead I'm using the mysql module on npm.
I want to save this function in its own file, and then require it in main.js. When I try to transpile it with babel, I get an error about Net.Connection not working (or something along those lines). As I understand it, this is because of how Node works. I'm happy to work around this, but I'm hoping there's a way to save this function in another file, and import it without having to use babel.
function loadColourFilter(){
var mysql = require('mysql');
let query_result;
var connection = mysql.createConnection({
host : 'xxxxxxxxxxxx',
user : 'xxxxxxxxxxxx',
password : 'xxxxxxxxxxxx',
database : 'xxxxxxxxxxxx'
});
connection.connect();
let query = "xxxxxxxxxxxxxxxx";
connection.query(query, function (error, results, fields) {
});
connection.end();
return (query_result);
}
EDIT: I've removed some parts of the function to keep credentials safe and whatnot. I'm fairly certain their absence won't change anything when trying to solve this.
EDIT:
My project directory is essentially
src
--- js
--- --- main.js
--- functionFile.js // This would be where my loadColourFilter function above would be saved
--- node_modules
--- --- ...
--- index.html // js/main.js is referenced in a script tag here.
--- main.js // Where the electron window is created.
--- package.json
There should be 2 js contexts, one running in the electron app and one running in node. You won't be able to require you scripts directly from your directory if you are in the electron context (which is like a browser js context).
I'm just assuming this is the case since we don't get a lot of information for your problem, and the other answer should have resolved your problem.
Try to include your js file in your index.html and see what's up.
Edit: Since it's a Transpiling error with babel, babel is probably transpiling for node when it should transpile for the browser.
You can easily make a simple local module using NodeJS by creating a source file and then adding a module.exports assignment to export some functionality/variables/etc from the file. In your case something like a file named colourFilter.js with the contents:
function load(){
var mysql = require('mysql');
let query_result;
var connection = mysql.createConnection({
host : 'xxxxxxxxxxxx',
user : 'xxxxxxxxxxxx',
password : 'xxxxxxxxxxxx',
database : 'xxxxxxxxxxxx'
});
connection.connect();
let query = "xxxxxxxxxxxxxxxx";
connection.query(query, function (error, results, fields) {
});
connection.end();
return (query_result);
}
module.exports = load
And then in your code where you'd like to use it include it by doing something like:
loadColourFilter = require('colourFilter.js')
And use the function like
let result = loadColourFilter()
This is a simple way to split up your code into multiple files/classes/modules but still keep one main file/class/module as the important one which is the public-facing portion or entry point. And of course you don't have to use the names I've used above :P
If you would like to make an object-style module you can instead export an object like
module.exports = {
load
}
Or
module.exports = {
load: loadFunctionNameInThisFile
}
And then use it like
const colourFilter = require('colourFilter.js')
let result = colourFilter.load()
I have the following Node.js project (which is a Minimal Working Example of my problem):
module1.js:
module.exports = function() {
return "this is module1!";
};
module2.js:
var module1 = require('./module1');
module.exports = function() {
return module1()+" and this is module2!";
};
server.js:
var module2 = require('./module2');
console.log(module2()); // prints: "this is module1! and this is module2!"
Now I want to create a client.html file that will also use module2.js. Here is what I tried (and failed):
naive version:
<script src='module2.js'></script>
<script>alert(module2());</script> // should alert: "this is module1! and this is module2!"
This obviously doesn't work - it produces two errors:
ReferenceError: require is not defined.
ReferenceError: module2 is not defined.
Using Node-Browserify: After running:
browserify module2.js > module2.browserified.js
I changed client.html to:
<script src='require.js'></script>
<script>
var module2 = require('module2');
alert(module2());
</script>
This doesn't work - it produces one error:
ReferenceError: module2 is not defined.
Using Smoothie.js by #Torben :
<script src='require.js'></script>
<script>
var module2 = require('module2');
alert(module2());
</script>
This doesn't work - it produces three errors:
syntax error on module2.js line 1.
SmoothieError: unable to load module2 (0 )
TypeError: module2 is not a function
I looked at require.js but it looks too complicated to combine with Node.js - I didn't find a simple example that just takes an existing Node.js module and loads it into a web page (like in the example).
I looked at head.js and lab.js but found no mention of Node.js's require.
So, what should I do in order to use my existing Node.js module, module2.js, from an HTML page?
The problem is that you're using CJS modules, but still try to play old way with inline scripts. That won't work, it's either this or that.
To take full advantage of CJS style, organize your client-side code exactly same way as you would for server-side, so:
Create client.js:
var module2 = require('./module2');
console.log(module2()); // prints: "this is module1! and this is module2!"
Create bundle with Browserify (or other CJS bundler of your choice):
browserify client.js > client.bundle.js
Include generated bundle in HTML:
<script src="client.bundle.js"></script>
After page is loaded you should see "this is module1! and this is module2!" in browser console
You can also try simq with which I can help you.
Your problems with Smoothie Require, were caused by a bug (https://github.com/letorbi/smoothie/issues/3). My latest commit fixed this bug, so your example should work without any changes now.
I am trying to search the best approach to manage different values for same variables in Devlopment, Test and Production environment.
For example, I have variable jsonFile which can be:
var jsonFile = http://localhost:63342/json/appsconfig.json
for development env
var jsonFile = http://192.168.35.59/applications/json/appsconfig.json
for test env
var jsonFile = http://example.com/applications/json/appsconfig.json
for production env
I am trying to read a lot about Frontend Development Stack, but I am confused about what tool to use. I will use Google Closure Tools for minification, can it be also useful to switch variable values? Or can it be considered a Grunt task (even if I am not able to understand how to properly configure Grunt tasks...)?
What might be better is to write the JSON into a JS file that is part of your build artifacts. Something like file-creator that can write a file like so (using a simplistic setup that can obviously be made more dynamic).
In the top of your module.exports for grunt tasks, load in the config file into a var like:
var configData = grunt.file.readJSON('../config/appsconfig.json'),
Then write to a new JS file using the grunt file-creator module
"file-creator": {
'dev': {
'build/config.js': function (fs, fd, done) {
fs.writeSync(fd,
'var yourSiteHere = yourSiteHere || {}; yourSiteHere.config = '
+ JSON.stringify(configData) + ";"
);
done();
}
}
}
Then load this JS file into the page (perhaps even minify it using a separate grunt task). You will be then able to refer to the config data like so:
var apiEndPoint = yourSiteHere.config.api.apiEndPoint,
apiKey = yourSiteHere.config.api.apiKey;
My project has got many folders and I often load my own modules in node.js in the following way:
var request = require("request"),
config = require("../../../modules/config"),
urls = require("../../../modules/urls");
I sometimes move the folders around and the path changes, so I need to adjust the ../ part manually.
I don't want to move my modules into the node_modules folder, but I'd like to load the modules in the following way:
var request = require("request"),
config = require("config"),
urls = require("urls");
or
var request = require("request"),
config = require("modules/config"),
urls = require("modules/urls");
What are my options?
New Answer:
Relative paths seem to be the simplest choice after all, it allows you to run your script from any location.
var config = require("../../config");
Old answer:
I found out that, while not ideal, there's also the possibility to use process.cwd().
var request = require("request"),
config = require(process.cwd() + "/modules/config");
or, if the process.cwd() is set to a global variable in the main js file
global.cwd = process.cwd();
then
var request = require("request"),
config = require(global.cwd + "/modules/config"),
urls = require(global.cwd + "/modules/urls");
You can try to do the following, based on some conditions
if the scripts are exclusively written for your application, meaning it won't work with any other application, and the scripts don't have any dependencies place them under modules directory and try to create expose a variable such as global.basepath and using path.join to construct the filepath and require it.You could also inject module variable after you require them at the main script of your app.
main.js
var config = require('modules/config.js')(_,mongoose,app);
modules/config.js
module.exports=function(_,mongoose,app){
return function(){
// _,mongoose,app
this.loadConfigVariable..
}
}
if the scripts are not one-files that have separate tests, require other vendor modules in order to be executed then create individual node modules and publish them to a registry for convenience when installing the main application.
Just as express does, there is the main application express and there are modules that express uses such as express-validation which in turn has its own dependencies.
You could add a symlink in node_modules, pointing to wherever your modules are.
For example, add a symlink named "my_modules", referencing '../foo/bar/my_modules'. Then, your requires will look like
var config = require('my_modules/config');
var urls = require('my_modules/urls');
Is it possible to use require() (or something similar) on client side?
Example
var myClass = require('./js/myclass.js');
You should look into require.js or head.js for this.
I've been using browserify for that. It also lets me integrate Node.js modules into my client-side code.
I blogged about it here: Add node.js/CommonJS style require() to client-side JavaScript with browserify
If you want to have Node.js style require you can use something like this:
var require = (function () {
var cache = {};
function loadScript(url) {
var xhr = new XMLHttpRequest(),
fnBody;
xhr.open('get', url, false);
xhr.send();
if (xhr.status === 200 && xhr.getResponseHeader('Content-Type') === 'application/x-javascript') {
fnBody = 'var exports = {};\n' + xhr.responseText + '\nreturn exports;';
cache[url] = (new Function(fnBody)).call({});
}
}
function resolve(module) {
//TODO resolve urls
return module;
}
function require(module) {
var url = resolve(module);
if (!Object.prototype.hasOwnProperty.call(cache, url)) {
loadScript(url);
}
return cache[url];
}
require.cache = cache;
require.resolve = resolve;
return require;
}());
Beware: this code works but is incomplete (especially url resolving) and does not implement all Node.js features (I just put this together last night).
YOU SHOULD NOT USE THIS CODE in real apps but it gives you a starting point. I tested it with this simple module and it works:
function hello() {
console.log('Hello world!');
}
exports.hello = hello;
I asked myself the very same questions. When I looked into it I found the choices overwhelming.
Fortunately I found this excellent spreadsheet that helps you choice the best loader based on your requirements:
https://spreadsheets.google.com/lv?key=tDdcrv9wNQRCNCRCflWxhYQ
Take a look at requirejs project.
I have found that in general it is recommended to preprocess scripts at compile time and bundle them in one (or very few) packages with the require being rewritten to some "lightweight shim" also at compile time.
I've Googled out following "new" tools that should be able to do it
http://mixu.net/gluejs/
https://github.com/jrburke/almond
https://github.com/component/builder2.js
And the already mentioned browserify should also fit quite well - http://esa-matti.suuronen.org/blog/2013/04/15/asynchronous-module-loading-with-browserify/
What are the module systems all about?
Older Stack Overflow explanation - Relation between CommonJS, AMD and RequireJS?
Detailed discussion of various module frameworks and the require() they need is in Addy Osmani - Writing Modular JavaScript With AMD, CommonJS & ES Harmony
You can create elements to the DOM, which loads items.
Like such:
var myScript = document.createElement('script'); // Create new script element
myScript.type = 'text/javascript'; // Set appropriate type
myScript.src = './js/myclass.js'; // Load javascript file
Simply use Browserify, what is something like a compiler that process your files before it go into production and packs the file in bundles.
Think you have a main.js file that require the files of your project, when you run browserify in it, it simply process all and creates a bundle with all your files, allowing the use of the require calls synchronously in the browser without HTTP requests and with very little overhead for the performance and for the size of the bundle, for example.
See the link for more info: http://browserify.org/
Some answers already - but I would like to point you to YUI3 and its on-demand module loading. It works on both server (node.js) and client, too - I have a demo website using the exact same JS code running on either client or server to build the pages, but that's another topic.
YUI3: http://developer.yahoo.com/yui/3/
Videos: http://developer.yahoo.com/yui/theater/
Example:
(precondition: the basic YUI3 functions in 7k yui.js have been loaded)
YUI({
//configuration for the loader
}).use('node','io','own-app-module1', function (Y) {
//sandboxed application code
//...
//If you already have a "Y" instance you can use that instead
//of creating a new (sandbox) Y:
// Y.use('moduleX','moduleY', function (Y) {
// });
//difference to YUI().use(): uses the existing "Y"-sandbox
}
This code loads the YUI3 modules "node" and "io", and the module "own-app-module1", and then the callback function is run. A new sandbox "Y" with all the YUI3 and own-app-module1 functions is created. Nothing appears in the global namespace. The loading of the modules (.js files) is handled by the YUI3 loader. It also uses (optional, not show here) configuration to select a -debug or -min(ified) version of the modules to load.
Here's a solution that takes a very different approach: package up all the modules into a JSON object and require modules by reading and executing the file content without additional requests.
https://github.com/STRd6/require/blob/master/main.coffee.md
STRd6/require depends on having a JSON package available at runtime. The require function is generated for that package. The package contains all the files your app could require. No further http requests are made because the package bundles all dependencies. This is as close as one can get to the Node.js style require on the client.
The structure of the package is as follows:
entryPoint: "main"
distribution:
main:
content: "alert(\"It worked!\")"
...
dependencies:
<name>: <a package>
Unlike Node a package doesn't know it's external name. It is up to the pacakge including the dependency to name it. This provides complete encapsulation.
Given all that setup here's a function that loads a file from within a package:
loadModule = (pkg, path) ->
unless (file = pkg.distribution[path])
throw "Could not find file at #{path} in #{pkg.name}"
program = file.content
dirname = path.split(fileSeparator)[0...-1].join(fileSeparator)
module =
path: dirname
exports: {}
context =
require: generateRequireFn(pkg, module)
global: global
module: module
exports: module.exports
PACKAGE: pkg
__filename: path
__dirname: dirname
args = Object.keys(context)
values = args.map (name) -> context[name]
Function(args..., program).apply(module, values)
return module
This external context provides some variable that modules have access to.
A require function is exposed to modules so they may require other modules.
Additional properties such as a reference to the global object and some metadata
are also exposed.
Finally we execute the program within the module and given context.
This answer will be most helpful to those who wish to have a synchronous node.js style require statement in the browser and are not interested in remote script loading solutions.
I find the component project giving a much more streamlined workflow than other solutions (including require.js), so I'd advise checking out https://github.com/component/component . I know this is a bit late answer but may be useful to someone.
Here's a light weight way to use require and exports in your web client. It's a simple wrapper that creates a "namespace" global variable, and you wrap your CommonJS compatible code in a "define" function like this:
namespace.lookup('org.mydomain.mymodule').define(function (exports, require) {
var extern = require('org.other.module');
exports.foo = function foo() { ... };
});
More docs here:
https://github.com/mckoss/namespace
The clientside-require library provides an asynchronous load() function that can be used to load any JS file or NPM module (which uses module.exports), any .css file, any .json, any .html, any any other file as text.
e.g.,
npm install clientside-require --save
<script src = '/node_modules/clientside-require/dist/bundle.js'></script>
<script>
load('color-name') // an npm module
.then(color_name=>{
console.log(color_name.blue); // outputs [0, 0, 255]
})
</script>
A really cool part of this project is that inside of any load()ed script, you can use the synchronous require() function the same way you would expect in node.js!
e.g.,
load('/path/to/functionality.js')
and inside /path/to/functionality.js:
var query_string = require("qs") // an npm module
module.exports = function(name){
return qs.stringify({
name:name,
time:new Date()
}
}
That last part, implementing the synchronous require() method, is what enables it to utilize NPM packages built to run on the server.
This module was designed to implement the require functionality as closely as possible in the browser. Disclaimer: I have written this module.
Yes it is very easy to use, but you need to load javascript file in browser by script tag
<script src="module.js"></script>
and then user in js file like
var moduel = require('./module');
I am making a app using electron and it works as expected.