I've got a question regarding loading external files in node.
Basically I'm loading an external JSON file that contains some configuration, and this file is modified by an external process every 10 minutes. How can i reload this file every 10 minutes, without restarting node?
I've tried this solution:
delete require.cache['/home/conf/myfile.json']
but some people have advised against it. Can anybody help me?
At the end I changed my code and I now use fs.readFile to load my json file, instead of using a require.
I then used node-watch to check for changes in file and reload it.
Couple options. You could just use setInterval for 10 minutes and read the file in the callback. Otherwise you could use fs.watch and trigger the reload when the file actually changes.
update based on comment
If you're using require, it will only load the file once, regardless of how many times you're requiring it and there isn't really a mechanism to invalidate a required file. You could create a wrapper around the functionality and require that instead. Your wrapper is the file you require, and it exposes a function which returns your current config. Inside that module you could create the setTimeout refresh mentioned above.
You can have a look at my module-invalidate module that allows you to invalidate a required module. The module will then be automatically reloaded on further access.
Example:
module ./myModule.js
module.invalidable = true;
var count = 0;
exports.count = function() {
return count++;
}
main module ./index.js
require('module-invalidate');
var myModule = require('./myModule.js');
console.log( myModule.count() ); // 0
console.log( myModule.count() ); // 1
setInterval(function() {
module.constructor.invalidateByExports(myModule);
console.log( myModule.count() ); // 0
console.log( myModule.count() ); // 1
}, 10*60*1000);
Related
EDIT: Additional addition of code for task execution
I have a question I can't answer. I need gulp to execute a stain even if the file is unchanged. Currently only when I change my source file it recreates the new destination file. But I would like it to create it even if it hasn't changed. Thank's you in advance :)
Library external : "gulp-inject-string": "^1.1.2",
// Add timestamp in cache PWA
function pwa_cache() {
return src(`${paths.pwa}/serviceworker.tmp.js`)
.pipe(inject.prepend('const CACHE_VERSION = "pwa-v-'+ new Date().getTime() +'"; \n'))
.pipe(rename('serviceworker.js'))
.pipe(dest(`${paths.js}/pwa/`,{overwrite:true}))
}
// Generate all assets
const generateAssets = parallel(
styles,
scripts,
pwa_cache,
imgCompression
)
exports.default = series(generateAssets)
exports["generate-assets"] = generateAssets
Are you maybe running this task within a gulp.watch() ? If so, you'd need to stop the watch task and re-run it.
well I proceeded differently, if it can help someone, it's the only way I found and I don't have time to find another solution.
Instead of adding I rather assign the variable in my file and then I change its value, and I no longer work on a template file but directly on the original file.
Post and link that helped me find the answer: https://eureka.ykyuen.info/2015/07/13/gulp-rewrite-source-files-instead-of-creating-new-files/, How to inject variable value into JS file from GULP, Inject variable into html via Gulp Task
gulp file :
// Add timestamp in cache PWA
function pwa_cache() {
return src(`${paths.js}/pwa/serviceworker.js`, {base: './'})
.pipe(sourcemaps.init())
.pipe(replace(new RegExp('django-pwa-v-\\d*', 's'), 'django-pwa-v-'+ new Date().getTime()))
.pipe(sourcemaps.write('./'))
.pipe(dest('./'))
}
js file :
// Cache version don't touch string assigned to the variable /
CACHE_VERSION = "django-pwa-v-1580491632502";
/////////////////////////////////////////////////////////////
I've been having problems with conflicting updates in Marklogic. I'm aware of the cause, but I don't know how to fix it.
I have 1 main (.sjs) file which calls two different (.sjs) files which both update a set of documents. In the main file I use: declareUpdate({explicitCommit: true}); and then in the separate files I use the command xdmp.commit(); after updating the documents. However, I'm still getting: XDMP-CONFLICTINGUPDATES.
Part of the code: Main.sjs:
function main(){
declareUpdate({explicitCommit: true});
function.to.file.1();
function.to.file.2();
}
file1.sjs:
//make some docs and insert them into ML
function file1Function(){
for (let d of someCollectionOfData) {
xdmp.documentInsert('/a/uri/filexx.json', d, {collections: aCollectionName1});
};
xdmp.commit();
}
file2.sjs:
//adjust docs made in file1 and put them back into ML
funtion file2Function(){
for (let d of xdmp.directory('/location/of/aCollectionName1/','infinity')) {
let dObj = d.toObject();
dObj.addSomething = 'something';
xdmp.documentInsert(fn.documentUri(d), dObj, {collections: aCollectionName1});
}
xdmp.commit();
}
It must mean your file1 is located inside '/location/of/aCollectionName1/'. Keep in mind that MarkLogic doesn't commit immediately when you invoke xdmp.commit(). Actual persisting is always postponed until after all code has executed. It therefor doesn't make much sense to invoke xdmp.commit() more than once in one request. You won't be able to read your updates after xdmp.commit().
HTH!
I see this question was asked here
load javascript file before onload with requirejs
But the solution there doesn't fit my situation and so I'm wondering if there is a different solution. I can't build with deps and make my code come last because I'm making a library/utility, not an app.
I'm working on the WebGL-Inspector. It works by wrapping HTMLCanvasElement.prototype.getContext and if it sees a "webgl" it then does its thing wrapping the context and allowing you to inspect it.
The WebGL-Inspector works in 3 modes
As a browser extension
As a loader + large compiled script
As a loader + original source (many many scripts)
To use it in modes #2 or #3 above you just insert
<script src="core/embed.js"></script>
Somewhere in the top of your HTML. Because it loads synchronously~ish it will have wrapped HTMLCanvasElement.prototype.getContext before whatever scripts come after it.
These last 2 modes are mostly for debugging/development of the WebGL-Inspector itself. Especially mode #3 because we can edit the inspector's code and refresh the page immediately to see the result, no build step.
I'm in the process of switching to using AMD for all of the WebGL-Inspector. We're using this because we can use webpack to make #2 but still follow the same dev workflow allowing us it use mode #3 above just change the script tag to
<script src="core/require.js" data-main="core/embed.js"></script>
The problem is this no longer works because whatever other code unrelated to the WebGL-Inspector itself runs before core/embed.js has loaded and so calls someCanvas.getContext before we've had a chance to wrap it.
My current solution is sadly to hack in a delay of 1.5 seconds on whatever demo we're using
<script src="core/require.js" data-main="core/embed.js"></script>
<script>
// wait 1.5 seconds for embed.js to load and pray :(
window.onload = setTimeout(reallyRunWebGLCode, 1500);
...
</script>
The previous non-AMD loader doesn't have this async issue. Somehow it manages to load 60 or so .js files before window.onload fires. Is there a way to get require.js to do the same? Or maybe I need to write my own loader?
To put it another way, the issue is as it is now the user adds a single <script> line and makes no other changes to their code and it works. When they're done they remove the single script line.
Switching to AMD/require.js the user is now required to add a single script and re-write code. In my tests went from
window.onload = myapp;
to
require.config({ baseUrl: "/core" });
require("./embed", myapp);
It's minor but now the app is broken when you remove the <script> tag and has to be put back as it was. Requiring those changes is what I'm trying to avoid if possible. In fact with the original style you don't even have to remove the script tag, just run your app in an environment where the script doesn't exist, it will fail to load and your app will run as normal where as with the require method it will fail if the script doesn't exist.
I can require even more code
if (typeof require === 'function' && typeof define === 'function' and define.amd) {
require.config({ baseUrl: "/core" });
require("./embed", myapp);
} else {
window.onload = myapp;
}
But that's even uglier. We went from adding a single script to requiring modifying your app.
Actually it gets even worse in the app I'm currently testing with. It uses a different loader for itself. When I run require version of the code above it fails because in this case require runs myapp before window.onload (strange). I end up having to do this
var tryRunCount = 0;
function tryRunApp() {
++tryRunCount;
// check that tryRunApp has been called twice so we know
// both onload and require have returned
if (tryRunCount === 2) {
myApp();
}
}
window.onload = tryRunApp;
require.config({ baseUrl: "/core" });
require("./embed", tryRunApp);
That's way way more changes than I want to require users to make.
I have a dojo-based module that automates loading a dojo/store/Memory for me, using a bang. It's called like so:
require( [ 'squad_builder/storeBroker!collectionStore' ],
function( collectionStore )
{
// use store...
} );
The first require() call loads the store and subsequent ones load the store from a cache.
This works well but I cannot figure out how to invalidate require's internal cache so I can make the next request load the store with fresh data.
Is there anything built into require() to do this or do I need to mess with some internal workings (and if so, where?)
Edit
In the docs it mentions:
Note: the dojo loader includes the nonstandard function require.undef,
which undefines a module value.
But that method is undefined when I call it. Looking at require() in Firebug with:
for( i in require )
{
console.log( 'require', i, require[ i ] );
}
require.module has a def() method but not an undef() one.
Edit 2
Require.cache only seems to contain dojo and dijit modules. Where are custom namespaced modules cached?
According to the dojo site, the function is not automatically included in "built" dojo files. If you look at an uncompressed dojo.js you will see it there.
From:http://dojotoolkit.org/reference-guide/1.9/loader/amd.html
The following table provides a list of options that are leveraged
within the loader. The first column is the Option/Feature as defined
within the loader, the second options is whether this is a detected
feature (via has.add()) or if it is just an option and its default
value. With “unbuilt” source, all the features and options are
available. If the loader has been built, then some of these features
may have been set as staticHasFeatures and not be configurable
anymore.
Also here it says that you might be able to turn the function back on by adding a line go djconfig
http://dojo-toolkit.33424.n3.nabble.com/require-undef-td3990559.html
dojoConfig = { has : {
"dojo-undef-api": true } }
I spent the better part of last month beating my head against the wall before I came up with an easy way to dynamically load, and chain together HTML canvas classes which are stored on the server, but, obviously, initialized on the client (harder than it sounds when the ordering is important in an asynchronous environment).
I was wondering if someone could help me find a way to load simple javascript scripts. Lets define a load('foo.js') function which instructs the client to load script foo.js from the server and execute it as javascript code.
Given the three files, stored on the server:
A.js
a = 10;
B.js
load('A.js');
b = a + 10;
C.js
load('B.js');
c = b + 10;
If the client issues the command load('C.js'); what's the easiest/most reliable way to implement this. One idea I had was to scan the code serverside and return all the scripts at once. This requires the minimal amount of php requests. However, if the client has already requested C.js before, the script should exist client side, and this would be inneficient, especially if C.js and all its dependent files are large. Another option I considered was to wrap all of these serverside scripts in an object like so, for C.js above:
{
depenencies: ['B.js'] ,
code : 'c.age = b.age + 10;'
}
I just don't know how to 'pause' execution of script C.js after the load('B.js') statement, and then resuming it after B.js has been loaded.
EDIT Thanks to redsqaure for suggesting yepnope and requirejs. Unfortunately, I do not like them for several reasons. For one, requirejs is difficult (I am sure I will come under criticism for this one). My main gripe with this is that, if it is so difficult to learn, I might as well recreate it myself, learning it in the process, AND having greater control over it. Second, it requires you to change your style of writing. Switching to Dojo and having to use dojo.declare("ClassName", [ParentA,ParentB], {...}); to declare classes is one thing, but wrapping every snippet of code in require(['A','B',...], function(){}); is another. Finally, I don't know how simple it will be to instruct where to look for files. I want the user to be able to define a 'PATH' variable server side, and have the search occur in each of the folders/subfolders of the 'PATH'
Depends on how optimized you want it to be. Either you can go the route of synchronous XHR or use a callback (async and recommended). If you were to go the second route your code would look something like:
// Say C.js is dependent on A.js and B.js..
load(["A.js","B.js"], function() {
// code goes here
});
EDIT
Taking a second look after you feedback what you want is somewhat possible, but would be brittle and hard to write in javascript. Below i have a sample/untested implementation of a dependency loader where a file can only have one call to load("file.js") possible. This would be more complex for multiple possible dependencies. Also I'm assuming these files are coming from the same domain.
// Usage: load("A.js")
// A.js -> B.js -> C.js
window.load = (function() {
var loaded = {};
return function(str, /* internally used */ callback) {
if(!loaded[str]) {
loaded[str] = true;
$.get(str, function(data) {
var matches = data.match(/load\(['|"](.*)["|']\)/);
if(matches.length > 1) { // has deps
window.load(matches[1], function() {
window.eval(data);
if(!!callback) callback();
});
} else { // no deps
window.eval(data);
}
});
}
}
})();
Why not look into a script loader like yepnope.js or require.js