Develop Reference

If I read a JSON array from a file, then I get out the old unchanged array

I'm reading about requireing a JSON object from a file. I can read the JSON correctly, but after I add data to it and save it into the file, re-requiring doesn't get the updated JSON, but retrieves the old file. Can someone explain why this is so? How can I fix this?
var file = process.env[(process.platform == 'win32') ? 'USERPROFILE' : 'HOME'] + "\\AppData\\Roaming\\BetterDiscord\\plugins\\Test.json";
var json = require(file);
console.log(JSON.stringify(json.img));
json.img.push({name:"Test"});
console.log(JSON.stringify(json.img));
save(JSON.stringify(json), file);
json = require(file);
console.log(JSON.stringify(json.img));
This is the save method:
var save = function(value, file){
var fs = require('fs');
fs.writeFile(file, value, function(err) {
if (err) {
console.log(err);
}
});
};
This is the Output:
Output 1: [{"name":"A"},{"name":"B"}]
Output 2: [{"name":"A"},{"name":"B"}, {"name":"Test"}]
Output 3: [{"name":"A"},{"name":"B"}]

There's two major problems with your code. The first is that save is an asynchronous function. That means that any code you write directly after it will run before it completes. It works just like setTimeout in that regard.
console.log(1);
setTimeout(function() {
console.log(2);
}, 100);
console.log(3);
Notice how it outputs 3 before 2. That's because setTimeout, like fs.writeFile, is asynchronous. To make sure you run code after it's done, you can pass a callback or use Promises (outside of the scope of this question but very useful). So your code could look something like this.
const save = (path, cb) => {
setTimeout(() => {
cb();
}, 100);
};
let path = 'path/to/file';
save(path, () => {
console.log(`Saved ${path}`);
});
console.log(`Saving ${path}...`);
This whole thing could also be avoided by using the sync version of writeFile.
fs.writeFileSync(file, value);
The next problem stems from how require caches the results. It does this because it's intended for loading in modules and they will not be changing dynamically, most of the time. Instead, load the file from the system directly. This can also be used asynchronously or synchronously. Here's how I would rewrite your code using synchronous code (although asynchronous is generally preferable).
var fs = require('fs');
var save = function(value, file){
return fs.writeFileSync(file, value);
};
var file = process.env[(process.platform == 'win32') ? 'USERPROFILE' : 'HOME'] + "\\AppData\\Roaming\\BetterDiscord\\plugins\\Test.json";
var obj = JSON.parse(fs.readFileSync(file, 'utf8'));
obj.img.push({name:"Test"});
console.log(JSON.stringify(obj.img));
save(JSON.stringify(obj), file);
obj = JSON.parse(fs.readFileSync(file, 'utf8'));
console.log(JSON.stringify(obj.img));

require will cache the file. Instead of requiring, use the fs.readFileSync method to read the file again.
json = fs.readFileSync(file);
Also, as #MikeC points out, you're writing to the file asynchronously, so either switch that out for the synchronous version writeFileSync, or rewrite your code using with readFile and writeFile and use callbacks.

Are Angular's promises asynchronous?

I may have miss something about Angular's promises but I was wondering something : are promises asynchronous ? I'm not sure if 'asynchronous' is the right word but let me explain myself.
In my code I use promises to do a really big process (read and write hundreds of big files) while I display a loading bar to watch the progress of the process. I've noticed that even if my code is in a promise, it seems to not really be asynchronous and freeze the display (that I assume is manage by the main thread).
For example in the code bellow that you can find in this Plnkr, I'm wondering how to let the progress bar move while the big process is done. I understand why it's freezing when I call it in the main thread but not when I'm using Angular's promises.
var app = angular.module('plunker', []);
app.controller('MainCtrl', function($scope, $q) {
function hugeProcess () {
var i = 0;
var start = new Date().getTime();
while (i++ < 100000) {
console.log(i);
}
var end = new Date().getTime();
var time = end - start;
$scope.processTime = 'Done in ' + time + 'ms';
}
$scope.onClickStartHugeProcess = function () {
console.log('onClickStartHugeProcess');
hugeProcess();
};
$scope.onClickStartHugeProcessWithPromise = function () {
console.log('onClickStartHugeProcessWithPromise');
$q.when()
.then(function () {
return hugeProcess();
});
};
});

The issue in your code is that your hugeProcess function never yields. So yes, it's called asynchronously (then callbacks are always called asynchronously in a Promises/A+ promise implementation), but that doesn't change what hugeProcess is doing when it gets called, which is hogging the main UI thread such that nothing else can happen while it's running. There's only one main UI thread, and all of your JavaScript runs on that one main UI thread except web workers.
To make hugeProcess not do that, you have to break it up and have it call itself after a brief delay, via setTimeout (or perhaps something built into Angular).
As Joe Clay points out, this code doesn't make a lot of sense:
$q.when()
.then(function () {
return hugeProcess();
});
That's effectively:
setTimeout(hugeProcess, 0);
...since $q.when() with no arguments returns a resolved promise, and adding a then callback to a resolved promise just results in your callback being called as soon as possible (but asynchronously; e.g., then returns before the callback is called).

So, I've discover Web Workers and here is a first version of my code using them.
app.controller('MainCtrl', function($scope, $q) {
function hugeProcess () {
var i = 0;
var start = new Date().getTime();
while (i++ < 100000) {
console.log(i);
}
var end = new Date().getTime();
var time = end - start;
postMessage(time);
}
var blob = new Blob(["onmessage = " + hugeProcess.toString()]);
// Obtain a blob URL reference to our worker 'file'.
var blobURL = window.URL.createObjectURL(blob);
var worker = new Worker(blobURL);
worker.onmessage = function (message) {
$scope.processTime = 'Done in ' + message.data + 'ms';
$scope.$apply()
};
$scope.onClickStartHugeProcessWithPromise = function () {
console.debug('onClickStartHugeProcessWithPromise');
$q(function () {
worker.postMessage(''); // Start the worker.
});
};
});
I don't think I'm using right but it does what I want ... I've found the package ng-webworker for Angular that seems to mix promises and web workers so that's exactly what I'm looking for.
Thank you all for your help.

Web Worker is right solution. I had similar problem and developed angular plugin ng-vkThread to simplify such kind of tasks.
Basic usage is:
/* function to execute in a thread */
function foo(n, m){
return n + m;
}
/* create an object, which you pass to vkThread as an argument*/
var param = {
fn: foo // <-- function to execute
args: [1, 2] // <-- arguments for this function
};
/* run thread */
vkThread.exec(param).then(
function (data) {
console.log(data); // <-- thread returns 3
},
function(err) {
alert(err); // <-- thread returns error message
}
);
Live demo
--Vadim

How to load images with chained Promises

I'm having trouble loading images with a script of chained promises - they turn up undefined by the end. I also have a 404 error I don't quite understand as I've confirmed that all the sources are valid, but admit I'm still a noob to debugging in an async context. Here is the fiddle in question. I've been chewing on this one for a couple days now and could use a nudge or two to get me headed in the right direction. I think this may be the problem area:
function loadImagePromises() {
var imageLoadPromises = [];
for ( var i = 0; i < sources.length; i++ ) {
imageLoadPromises.push(
imgLoad( sources[i] ).then(function(response) {
var myImage = new Image();
myImage.src = response; // response is a blob
}, function(Error) {
console.log("There was an error when the image was loaded after download: " + Error);
}) );
}
console.log( imageLoadPromises );
return imageLoadPromises;
}
As context, I'm writing an image loader script using promises for a Three.js program I have. No need to load the images to the DOM - I'll use them later as textures in a WebGL visualization.
NOTE:
Here's an earlier and simpler fiddle working end-to-end and outputting to the DOM.

Probably some other considerations here but your success handler isn't returning anything explicitly, so it's implicitly returning undefined:
function(response) {
var myImage = new Image();
myImage.src = response;
// if you want the promise to resolve to a useful value put
// it below, with whatever you want e.g:
return myImage;
}
RE: Errors:
You should probably not shadow Error in your error handler, just use lowercase error. Also, when using console.log/console.error you can use , to chain parts of the message which usually presents a richer message than the string used when concatenating.
e.g.
console.error("There was an error when the image was loaded after download: ", error);
FWIW you could also reduce some of the tedious iteration/collection here by using map to map each source to a promise:
function loadImagePromises() {
return sources.map(function(source) {
return imgLoad(source).then(function(response) {
// ...
});
}
EDIT Re: waiting for image objects to have actually loaded
return imgLoad(source).then(function(response) {
var imageURL = window.URL.createObjectURL(response);
var myImage = new Image();
return new Promise(function(resolve, reject) {
myImage.onload = function () {
// image has loaded
resolve(myImage);
}
// TODO error handling
myImage.src = imageURL;
});
}, console.error);
});

Node module: Don't return until all async requests have finished

I'm new to node and am having trouble understanding node's async behavior. I know this is a very frequently addressed question on SO, but I simply can't understand how to get any of the solutions I've read to work in my context.
I'm writing this module which I want to return an object containing various data.
var myModule = (function () {
var file,
fileArray,
items = [],
getBlock = function (fileArray) {
//get the data from the file that I want, return object
return block;
},
parseBlock = function (block) {
//[find various items in the block, put them into an "myItems" object, then
//take the items and do a look up against a web api as below]...
for (var i = 0, l = myItems.length; i < l; i ++) {
(function (i) {
needle.post(MY_URL, qstring, function(err, resp, body){
if (!err && resp.statusCode === 200){
myItems[i].info = body;
if (i === (myItems.length -1)) {
return myItems;
}
}
});
})(i);
}
},
getSomeOtherData = function (fileArray) {
//parse some other data from the file
}
return {
setFile: function (file) {
fileArray = fs.readFileSync(file).toString().split('\n');
},
render: function () {
var results = [];
results.someOtherData = getsomeOtherData();
var d = getBlock();
results.items = parseBlock(d);
return results;
}
}
})();
When I call this module using:
myModule.setFile('myFile.txt');
var res = myModule.render();
the variable res has the values from the someOtherData property, but not the items property. I understand that my long-running http request has not completed and that node just zooms ahead and finishes executing, but that's not what I want. I looked at a bunch of SO questions on this, and looked at using Q or queue-async, but with no success.
How do I get this module to return no data until all requests have completed? Or is that even possible in node? Is there a better way to design this to achieve my goal?

The problem in your example is your calling getBlock() but you have declared your function as getBlockData(). So you will not get a result. Try changing it to both the same.
Presuming that you have them both the same, your next problem is that your processing data from a file, so I presume that your reading the contents of the file and then parsing it.
If this is the case then there are sync reads that you can use to force sync, however I wouldn't recommend this.
You really want to structure your program based on events. Your thinking in the paradigm of 'call a function, when it returns continue'. You need to be thinking more along the lines of 'call a process and add a listener, the listener then does reply handling'.
This works very well for comms. You receive a request. You need to reply based on contents of file. So you start the read process with two possible results. It calls the completed function or the error function. Both would then call the reply function to process how to handle a reply for the request.
It's important not to block as you will be blocking the thread via which all processes are handled.
Hope that helps, if not add some comments and I will try and elaborate.
Have a look at this answer to another question to see a good example of processing a file using the standard listeners. All async calls have a listener concept for what can happen. All you need to do is pass a function name (or anon if you prefer) to them when you call them.
A quick example (based on node.js stream.Readable API:
fs.createReadStream(filename, {
'flags': 'r'
}).addListener( "data", function(chunk) {
// do your processing logic
}).addListener( "end", function(chunk) {
// do your end logic
response(...);
}).addListener( "error", function(chunk) {
// do your error logic
response(...);
}).addListener( "close",function() {
// do your close logic
});
function response(info) {
}

Using $.Deferred() as a callback

in my project I'm not using callbacks instead I'm trying to use $.Deferred to have uniform logic across all application, I have lots places in my code where I do something like the following:
function someMagicHandler(request) {
var me = this;
var sandbox = me.getSandbox();
var options = request.options;
var deferred = request.deferred;
var result = [];
var databaseOperation = sandbox.database.all('records').done(function (records) {
result.concat(records);
deferred.notify(records);
});
var serverResponse;
var serverOperation = sandbox.server.requestRecords(options).then(function (response) {
// First we are trying to save received records to database
serverResponse = response;
result.concat(response.Records);
deferred.notify(response.Records);
return sandbox.database.put('records', response.Records);
}).done(function() {
sandbox.storage.setTimestamp('records', new Date(serverResponse.Timestamp));
});
$.when(databaseOperation, serverOperation).then(deferred.resolve, deferred.reject);
}
In this code I'm personally don't like one of the last lines:
$.when(databaseOperation, serverOperation).then(deferred.resolve, deferred.reject);
Is there a standard way to express:
$.when(databaseOperation, serverOperation).then(deferred);
which would essentially mean:
$.when(databaseOperation, serverOperation).then(deferred.resolve, deferred.reject, deferred.notify);
Edit
I've investigated this problem deeper, and it seems that below solution is ok only when you are not relying on deferred.progress() which has no memory and therefore will not return any data in case when subsequent async operation is complete synchronously.
Summary
If you are using $.Deferred() as a callback (i.e. when you rely on notify or progress functions and in that case you need pass it as an argument) than you will be obligated to use the ugly
blahblahblah.then(deferred.resolve, deferred.reject, deferred.notify)

You can just replace this
$.when(databaseOperation, serverOperation).then(deferred.resolve, deferred.reject);
with this:
request.deferred = $.when(databaseOperation, serverOperation);
And delete all references to the variable deferred because $.when already creates a promise for you with (as far as I read the manual).