I want to render the output of an async process into the three.js renderer/scene. I came up with 2 ways, both have 2 'loops' though the second one seems more parallel than the other:
Works, you can try it here: The render is ran, which triggers the async process but immediately renders what it has (empty scene), and schedules itself to be run again requestAnimationFrame(render), which doesn't happen immediately. When the async process completes, the results are added to the three.js scene, but is not rendered immediately. The render loop will render it when its ready, which repeats.
Motivation for not wanting to do this: I initially had a possible memory leak issue probably because I was not disposing the resources (geometry, material), and to dispose resources in this style seemed complicated, I came up with the next option:
A loop which does async processing on the video feed and saves it to a variable (in my case, i use a face mesh library (written at the bottom of the question). A threeJS render loop then takes the saved variable (output of async processing), and puts it in a Buffer and renders it. The first method added it to the three.js immediately, whereas this waits.
The issue with my code here is that the 2 loops clump together: loads of render loop run in a row, then loads of async processes run in a row. I discovered this in the debugger/ breakpoints.
Example of Async process:
NPM: #mediapipe/face-mesh API
const faceMeshModel = new FaceMesh({locateFile: (file) => {
return `https://cdn.jsdelivr.net/npm/#mediapipe/face_mesh/${file}`;
}});
// Write the results handler
this.model.onResults((results) => {
/* Preprocess the results into a shape that works with three.js */
// I've got 2 options here: either call the render loop code directly (works) or
// Save this result into a class field/ variable, and access this in the render loop.
// Run the async process here once previous results are generated, so its a loop
faceMeshModel.send({image: htmlVideoElement})
})
// Actually run the process:
faceMeshModel.send({image: htmlVideoElement})
Related
I have a react-native application, it uses react-navigation. There is a functional component with a handler for button click.
I recently had a problem with async/await. I called async method in a non-async method and it did not work as I expected. I debugged it a little and I found out that the async method is called and does everything it should but after that the changes are lost.
The non-async method looked like this:
const handleDone = () => {
api.events.removeEventFromCache(eventId);
navigation.navigate(routes.Events);
};
When the method is called, an object is removed from cache and user is navigated to another screen. api.events.removeEventFromCache(eventId) got called and finished successfully and I even check the cache to see that the object was removed. The thing is that after the navigation.navigate(routes.Events) it is suddenly still in the cache.
Adding async/await keyword solved the problem but I do not really understand why:
const handleDone = async () => {
await api.events.removeEventFromCache(eventId);
navigation.navigate(routes.Events);
};
I though it would do everything without waiting for the result but why did the result disappear? It is not a question about the order of executing and waiting for the result. I do not really care about the result, I just want it to be done.
This is the log made without the keywords:
--> in cache now 3
remove the event from cache
navigate to events
cache length before remove 3
--> in cache now 3
cache length set 2
cache length checked 2
--> in cache now 3
A log with the keywords:
--> in cache now 3
remove the event from cache
cache length before remove 3
cache length set 2
cache length checked 2
navigate to events
--> in cache now 2
Yes, there is a difference in execution but my question is about the result in cache.
When you log the output before and after navigation, you are logging in 2 different contexts.
To explain this, lets say you have a cache object cache from which you wish to remove the event.
The way your code without the keywords executes is as follows:
cache is loaded by the api method to be edited
navigation method executes and it is going to send a copy of the current cache to the next screen and discard the previous.
cache-copy is created and dispatched by the navigation method.
You api method is currently still working with the cache object and not cache-copy.
cache is edited by the api method but is then discarded as the new screen is now using the cache-copy object.
In the second scenario:
The api method receives cache
The event is removed from cache
The navigation method receives the updated cache and creates cache-copy
cache-copy now has the updates list of events
The important thing to note is where and when exactly the cache-copy object is being created. If it is created before the event is removed, the code will work just fine.
Lets say, your navigation method executes the exact instant when the api method has removes the event, your code will run as expected even if async/await isn't used.
async/await is just working as expected. When managing promises, you could have two options:
//Using promises
const handleDone = () => {
api.events.removeEventFromCache(eventId).then(() => {
navigation.navigate(routes.Events);
});//You can manage failure with .catch()
};
and using async/await just as you posted, it waits until the promise is executed, it doesn't stop everything itself. Also, it is a good practice to wrap it inside a try/catch block in case the Promise fails.
The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value.
That means, when you call api.events.removeEventFromCache(eventId) it won't be completed immediately, so you either have to use one of both options.
Context
I'm building a general purpose game playing A.I. framework/library that uses the Monte Carlo Tree Search algorithm. The idea is quite simple, the framework provides the skeleton of the algorithm, the four main steps: Selection, Expansion, Simulation and Backpropagation. All the user needs to do is plug in four simple(ish) game related functions of his making:
a function that takes in a game state and returns all possible legal moves to be played
a function that takes in a game state and an action and returns a new game state after applying the action
a function that takes in a game state and determines if the game is over and returns a boolean and
a function that takes in a state and a player ID and returns a value based on wether the player has won, lost or the game is a draw. With that, the algorithm has all it needs to run and select a move to make.
What I'd like to do
I would love to make use of parallel programming to increase the strength of the algorithm and reduce the time it needs to run each game turn. The problem I'm running into is that, when using Child Processes in NodeJS, you can't pass functions to the child process and my framework is entirely built on using functions passed by the user.
Possible solution
I have looked at this answer but I am not sure this would be the correct implementation for my needs. I don't need to be continually passing functions through messages to the child process, I just need to initialize it with functions that are passed in by my framework's user, when it initializes the framework.
I thought about one way to do it, but it seems so inelegant, on top of probably not being the most secure, that I find myself searching for other solutions. I could, when the user initializes the framework and passes his four functions to it, get a script to write those functions to a new js file (let's call it my-funcs.js) that would look something like:
const func1 = {... function implementation...}
const func2 = {... function implementation...}
const func3 = {... function implementation...}
const func4 = {... function implementation...}
module.exports = {func1, func2, func3, func4}
Then, in the child process worker file, I guess I would have to find a way to lazy load require my-funcs.js. Or maybe I wouldn't, I guess it depends how and when Node.js loads the worker file into memory. This all seems very convoluted.
Can you describe other ways to get the result I want?
child_process is less about running a user's function and more about starting a new thread to exec a file or process.
Node is inherently a single-threaded system, so for I/O-bound things, the Node Event Loop is really good at switching between requests, getting each one a little farther. See https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/
What it looks like you're doing is trying to get JavaScript to run multiple threads simultaniously. Short answer: can't ... or rather it's really hard. See is it possible to achieve multithreading in nodejs?
So how would we do it anyway? You're on the right track: child_process.fork(). But it needs a hard-coded function to run. So how do we get user-generated code into place?
I envision a datastore where you can take userFn.ToString() and save it to a queue. Then fork the process, and let it pick up the next unhandled thing in the queue, marking that it did so. Then write to another queue the results, and this "GUI" thread then polls against that queue, returning the calculated results back to the user. At this point, you've got multi-threading ... and race conditions.
Another idea: create a REST service that accepts the userFn.ToString() content and execs it. Then in this module, you call out to the other "thread" (service), await the results, and return them.
Security: Yeah, we just flung this out the window. Whether you're executing the user's function directly, calling child_process#fork to do it, or shimming it through a service, you're trusting untrusted code. Sadly, there's really no way around this.
Assuming that security isn't an issue you could do something like this.
// Client side
<input class="func1"> // For example user inputs '(gamestate)=>{return 1}'
<input class="func2">
<input class="func3">
<input class="func4">
<script>
socket.on('syntax_error',function(err){alert(err)});
submit_funcs_strs(){
// Get function strings from user input and then put into array
socket.emit('functions',[document.getElementById('func1').value,document.getElementById('func2').value,...
}
</script>
// Server side
// Socket listener is async
socket.on('functions',(funcs_strs)=>{
let funcs = []
for (let i = 0; i < funcs_str.length;i++){
try {
funcs.push(eval(funcs_strs));
} catch (e) {
if (e instanceof SyntaxError) {
socket.emit('syntax_error',e.message);
return;
}
}
}
// Run algorithm here
}
I am currently building a custom task presenter for my PYBOSSA project. I have almost implemented it, but am stuck at the following javascript function -
pybossa.taskLoaded(function(task, deferred) {
if ( !$.isEmptyObject(task) ) {
console.log("Hello from taskLoaded");
// load image from flickr
var img = $('<img />');
img.load(function() {
// continue as soon as the image is loaded
deferred.resolve(task);
pybossaNotify("", false, "loading");
});
img.attr('src', task.info.url).css('height', 460);
img.addClass('img-thumbnail');
task.info.image = img;
console.log("Task ##"+task.id);
}
else {
deferred.resolve(task);
}
});
According to the docs -
The pybossa.taskLoaded method will be in charge of adding new items to the JSON task object and resolve the deferred object once the data has been loaded (i.e. when an image has been downloaded), so another task for the current user can be pre-loaded.
But notice my function. I have logged the task ids, the function loads. It loads 2 tasks. After logging, the console shows -
Task ##256
Task ##257
Also I have tried various other statements. They also execute twice. What I think is that if now I try to insert question of the current task, the function of the next task will also be put along with its respective image. How do I resolve this?
You are seeing double for a good reason :-) PYBOSSA preloads the next task, so the final user does thinks that the next task loads really fast (actually instantly).
While for some projects this might not be a problem, in some cases the user needs to download big images, check other APIs, etc. so it takes 2 or 3 seconds (or even more) to get everything before presenting the task to the user.
PYBOSSA.JS handles this scenario, as soon as the data has been downloaded, it requests a new task, but instead of presenting it, you have it in your window. As you are building your own template, you will have to add that data into the dom (via hidden elements) and then in the pybossa.presentTask method, you will check which task is being loaded, and show/hide the previous one.
In pybossa.saveTask, you can delete the previous DOM elements.
I hope this is now more clear. If you don't want this, you can use jQuery or Axios to request a task, save it and load the next one when you want ;-)
I have the following code that reads a json file:
Meteor.methods({
getPlaces: function(){
return HTTP.get(Meteor.absoluteUrl("/places.json"), function(e, r) {
console.log(r.data);
return r.data;
});
}
})
The console shows that its retrieving the data just fine.
Here is the part of my helper function for the template I want to display 'Places' in:
testing: function(){
return Meteor.call("getPlaces");
}
& here is my loop in the template where it is supposed to show:
{{#each testing}}
<li>{{testing.name}}</li>
{{/each}}
But it seems like I'm not calling the function right as the loop doesn't show anything. I've tested the loop by giving it random data which it works fine with but whenever I call Meteor.call or even HTTP.get directly on 'testing' it doesn't give me anything.
There are two essential problems here:
You're using HTTP.get asynchronously when you don't need to.
You're trying to use Meteor.call synchronously when you can't as it won't work like that.
Explanation of 1:
In supplying a callback to HTTP.get, you're telling Meteor to allow code execution to continue beyond that line, and pass the result of the call to the callback function. As a result, the actual method function finishes execution and will return undefined (which will be passed back to the calling function as null via EJSON) well before your actual HTTP call has returned. When that happens, the result will be logged, but even though you're returning the results in the callback, the enclosing method function won't care as it will have completed execution long before.
There are several ways to deal with this, the simplest being: don't pass a callback. On the server, you can use HTTP.get synchronously by not passing a callback, in which case code will cease executing until the results come back, and they will actually be returned to the client. Note that you cannot do this if you use HTTP.get on the client. Other ways of dealing with this involve Futures or Promises (better), but are unnecessary here.
Explanation of 2:
This is more complicated to resolve, but is fundamental to Meteor and Javascript. If you're calling an asynchronous function and you want to use the result, you need to supply a callback (or use promises). You can't just expect it to work inline for the same reasons given above. So some changes need to be made:
Don't call a method from within a template helper. You've no idea how often the template helper will run (it depends on all sorts of reactive things), so this is an essentially unbounded amount of traffic on the websocket that you're committing to. Call them when something happens (template is rendered, event handler, a specific piece of data changes (i.e. within an autorun block)), but not in a helper function.
Store the result in a reactive data source, otherwise even if you successfully receive it and put it somewhere, your UI won't update with the results.
So:
Template.yourTemplate.onCreated(function () {
this.places = new ReactiveVar()
Meteor.call("getPlaces", (err, res) => {
// do some error handling here
this.places.set(res)
})
})
{{#each Template.instance.places.get}}
<li>{{name}}</li>
{{/each}}
A few notes:
You need to install the reactive-var package, which inexplicably isn't provided out of the box, for this to work: meteor add reactive-var.
You could return the data via a template helper which uses Template.instance().places.get(), but you can just do it in-line in the template, which seems easier to me.
If the result of the first method call aren't sufficient and you need to update the results, do this in an event handler, or an autorun block as required. If the server needs to be able to push data directly to the client rather than waiting for requests for an update, then methods are the wrong tool - you need to be using Meteor's pub/sub model.
I have to do a certain calculation many many times. I want to split it over multiple cores, so I want to use child_process.fork(), and then pass each child a chunk of the work.
This works fine, except that my main process (the one that does the fork()) just keeps going after fork()ing and has already terminated by the time the children complete their work.
What I really want to do is wait for the children to finish before continuing execution.
I can't find any good way to do this in Node. Any advice?
If you spawn a new V8 process via .fork(), it returns a new child object which implements a communication layer. For instance
var cp = require( 'child_process' ),
proc = cp.fork( '/myfile.js' );
proc.on('message', function( msg ) {
// continue whatever you want here
});
and within /myfile.js you just dispatch an event when you're done with the work
process.send({ custom: 'message' });
Be aware of the fact that this method indeed spawns a new V8 instance, which eats a good chunk of memory. So you should use that very thoughtfully. Maybe you don't even need to do it that way, maybe there is a more "node like" solution (using process.nextTick to calculate heavy data).