Develop Reference

Getting deformation object in spark ar environment

I am trying to get deformation object to change it properties by JS, but I cannot even get it by any Spark module.
Spark AR have sample project with face distortion deformation.
https://developers.facebook.com/docs/ar-studio/tutorials-and-samples/samples/face-distortion/
You can even see in tutorial, that there is some morph object attached, which called faceDistortionPack. This object is located in assets, and I tried to get it by different ways in script, but couldn't make it. I want to write some js logic to manipulate deformations.
const Scene = require('Scene');
const Diagnostics = require('Diagnostics');
const faceMesh = Scene.root.find('facemesh_distortion');
Diagnostics.log(faceMesh); // FaceMesh: https://sparkar.facebook.com/ar-studio/learn/documentation/reference/classes/scenemodule.facemesh
Diagnostics.log(faceMesh.deformation); // null
Diagnostics.log(faceMesh.find('faceDistortionPack')); // Exception...
// ....
I want to get 'faceDistortionPack' object to access it properties, like 'nose_z', so I can change them by JS.

Although this is a pretty old question I thought I'd answer anyway if anyone struggles with this and comes across this thread.
First of all: There's an amazing collection of helpful tips, tutorials, snippets etc. called Spark AR Community. There you can find a GitBook with an alternative, better indexed and better working version of the official script object reference. I recommend using it if you're lost in the official reference or it's not working which happens quite often. There you can see, that the BlendShapesMesh, mentioned in the previous answer is deprecated as of Spark AR v85+, so that won't help anyone. ()
So, what you are trying to achieve, if I understand it correctly, is to access the faceMesh's Blendshapes and change their value through script. What you need to do is this:
Follow the instructions in this Tutorial: https://sparkar.facebook.com/ar-studio/learn/tutorials/face-distortion-retouching/
After you applied the Blendshapes, mess around a bit with adjusting the Blendshapes, so you understand what's happening.
Add a script to your assets. You're gonna want to access the faceMesh-Object's Blendshapes' weight Property through script. That can be done by using the getBlendshapes()-Method of the Mesh-Class.
Here's a code example:
//Require Modules
const Scene = require('Scene');
const Diagnostics = require('Diagnostics');
const Reactive = require('Reactive');
//Enable async/await [part 1]
(async function () {
//Load Assets as Promise
const loadAssets = await Promise.all(
[
Scene.root.findFirst("faceMesh0"), //Put name of your facemesh here, default is faceMesh0
]
).then(function(assets){
const faceMesh = assets[0]; //Assign the facemesh from the assets-Array of the promise to a variable you can work with better
const faceMeshShapes = faceMesh.getBlendShapes(); //Access all Blendshapes of the faceMesh-Object and store them in an Array
What you have now is a variable called faceMeshShapes, that's an object with an array of Blenshapes in it. You can console log it with Diagnostics.log(faceMeshShapes) and see that there's an array called "_value" in it, that's filled with all Blendshapes as objects, that have a weight-Property, which specifies the weight of the Blendshape with a scalarValue. You can consolelog this Value by converting it with the pinLastValue()-Method of the scalarValue-Class and you can assign different values by binding it to a custom value, that you convert to a scalarValue by using the val()-Method from the Reactive-Module.
Here's an example:
Diagnostics.log(faceMeshShapes._value[0].weight.pinLastValue()); //console log the Value of the weight property's scalarValue of the Blendshape that's stored at the index 0 of the _value-Array
faceMeshShapes._value[0].weight = Reactive.val(0.5) //set the weight to 0.5
Diagnostics.log(faceMeshShapes._value[0].weight.pinLastValue()); //console log the changed value of the weight property
So basically, that's how you can access every Blendshape's weight. The index should be according to the order in which they are listed in spar AR studio, beginning with 0 for the first BlendShape. Now you can do lots of things with this value, like binding it to an animation that animates it from 0-1 or to the mouth-openess using the face-tracking module and so on.
Last but not least, don't forget any semicolons and close all brackets.
});
})(); // Enable async/await [part 2]
P.S.: It can really help sometimes to understand what's happening and how to access stuff, by console logging a list of all properties of an object. Especially since the spark AR documentation is pretty weak on that part (and pretty weak in general). You can use the following function from MDN to do that. It's nothing fancy, but it does the job and has saved me a couple of times already :)
function listAllProperties(o) {
var objectToInspect;
var result = [];
for(objectToInspect = o; objectToInspect !== null;
objectToInspect = Object.getPrototypeOf(objectToInspect)) {
result = result.concat(
Object.getOwnPropertyNames(objectToInspect)
);
}
return result;
}

The link you posted isn't live any more.
I'm guessing the below links are more relevant now for others looking into this. I animated a series of blendShapes, and these links were useful. Basically cycling through the weights of each blendShape.
https://sparkar.facebook.com/ar-studio/learn/documentation/reference/classes/scenemodule.blendshapesmesh
https://sparkar.facebook.com/ar-studio/learn/documentation/reference/classes/scenemodule.blendshape

How to get the text from an Insert event in CKEditor 5?

I am trying to process an insert event from the CKEditor 5.
editor.document.on("change", (eventInfo, type, data) => {
switch (type) {
case "insert":
console.log(type, data);
break;
}
});
When typing in the editor the call back is called. The data argument in the event callback looks like approximately like this:
{
range: {
start: {
root: { ... },
path: [0, 14]
},
end: {
root: { ... },
path: [0, 15]
}
}
}
I don't see a convenient way to figure out what text was actually inserted. I can call data.range.root.getNodeByPath(data.range.start.path); which seems to get me the text node that the text was inserted in. Should we then look at the text node's data field? Should we assume that the last item in the path is always an offset for the start and end of the range and use that to substring? I think the insert event is also fired for inserting non-text type things (e.g. element). How would we know that this is indeed a text type of an event?
Is there something I am missing, or is there just a different way to do this all together?

First, let me describe how you would do it currently (Jan 2018). Please, keep in mind that CKEditor 5 is now undergoing a big refactoring and things will change. At the end, I will describe how it will look like after we finish this refactoring. You may skip to the later part if you don't mind waiting some more time for the refactoring to come to an end.
EDIT: The 1.0.0-beta.1 was released on 15th of March, so you can jump to the "Since March 2018" section.
Until March 2018 (up to 1.0.0-alpha.2)
(If you need to learn more about some class API or an event, please check out the docs.)
Your best bet would be simply to iterate through the inserted range.
let data = '';
for ( const child of data.range.getItems() ) {
if ( child.is( 'textProxy' ) ) {
data += child.data;
}
}
Note, that a TextProxy instance is always returned when you iterate through the range, even if the whole Text node is included in the range.
(You can read more about stringifying a range in CKEditor5 & Angular2 - Getting exact position of caret on click inside editor to grab data.)
Keep in mind, that InsertOperation may insert multiple nodes of a different kind. Mostly, these are just singular characters or elements, but more nodes can be provided. That's why there is no additional data.item or similar property in data. There could be data.items but those would just be same as Array.from( data.range.getItems() ).
Doing changes on Document#change
You haven't mentioned what you want to do with this information afterwards. Getting the range's content is easy, but if you'd like to somehow react to these changes and change the model, then you need to be careful. When the change event is fired, there might be already more changes enqueued. For example:
more changes can come at once from collaboration service,
a different feature might have already reacted to the same change and enqueued its changes which might make the model different.
If you know exactly what set of features you will use, you may just stick with what I proposed. Just remember that any change you do on the model should be done in a Document#enqueueChanges() block (otherwise, it won't be rendered).
If you would like to have this solution bulletproof, you probably would have to do this:
While iterating over data.range children, if you found a TextProxy, create a LiveRange spanning over that node.
Then, in a enqueueChanges() block, iterate through stored LiveRanges and through their children.
Do your logic for each found TextProxy instance.
Remember to destroy() all the LiveRanges afterwards.
As you can see this seems unnecessarily complicated. There are some drawbacks of providing an open and flexible framework, like CKE5, and having in mind all the edge cases is one of them. However it is true, that it could be simpler, that's why we started refactoring in the first place.
Since March 2018 (starting from 1.0.0-beta.1)
The big change coming in 1.0.0-beta.1 will be the introduction of the model.Differ class, revamped events structure and a new API for big part of the model.
First of all, Document#event:change will be fired after all enqueueChange blocks have finished. This means that you won't have to be worried whether another change won't mess up with the change that you are reacting to in your callback.
Also, engine.Document#registerPostFixer() method will be added and you will be able to use it to register callbacks. change event still will be available, but there will be slight differences between change event and registerPostFixer (we will cover them in a guide and docs).
Second, you will have access to a model.Differ instance, which will store a diff between the model state before the first change and the model state at the moment when you want to react to the changes. You will iterate through all diff items and check what exactly and where has changed.
Other than that, a lot of other changes will be conducted in the refactoring and below code snippet will also reflect them. So, in the new world, it will look like this:
editor.document.registerPostFixer( writer => {
const changes = editor.document.differ.getChanges();
for ( const entry of changes ) {
if ( entry.type == 'insert' && entry.name == '$text' ) {
// Use `writer` to do your logic here.
// `entry` also contains `length` and `position` properties.
}
}
} );
In terms of code, it might be a bit more of it than in the first snippet, but:
The first snippet was incomplete.
There are a lot fewer edge cases to think about in the new approach.
The new approach is easier to grasp - you have all the changes available after they are all done, instead of reacting to a change when other changes are queued and may mess up with the model.
The writer is an object that will be used to do changes on the model (instead of Document#batch API). It will have methods like insertText(), insertElement(), remove(), etc.
You can check model.Differ API and tests already as they are already available on master branch. (The internal code will change, but API will stay as it is.)

#Szymon Cofalik's answer went into a direction "How to apply some changes based on a change listener". This made it far more complex than what's needed to get the text from the Document#change event, which boils down to the following snippet:
let data = '';
for ( const child of data.range.getChildren() ) {
if ( child.is( 'textProxy' ) ) {
data += child.data;
}
}
However, reacting to a change is a tricky task and, therefore, make sure to read Szymon's insightful answer if you plan to do so.

Problems with performance on Game Of life (React.js + Redux)

I am experiencing some problems when my Game Of Life run by itself. To do so I am trying to set a setInterval and trigger the function that now is triggered each time someone clicks on Next. But it is causing me a lot of problems.
The main problem is that when I set a setInterval( () => this.handleChange(), 100) the movement of the game is really slow and it finally crash in codepen.
class Board extends React.Component{
handleChange(){ [.........] //just to indicate that here is more code that is not showing and dont think it is important to the question.
nextMovement() [...........]
render(){
var createBoard = this.props.board.map((idx) => {
return <Cell
onClick={() => this.props.toggleAlive(idx.index)}
key = {idx.index}
index = {idx.index}
col = {idx.col}
row = {idx.row}
val = {idx.val}
/>
});
return(
<div className="board">
{createBoard}
<button className="btn btn-danger" onClick={()=>this.handleChange()}>Next</button>
{setInterval(() => this.handleChange(), 100)}
</div>
);
}
}
/* - - - Reducers - - - */
Here you can find my codePen as well to see the full code.
http://codepen.io/DiazPedroAbel/pen/bwNQAJ
I was also looking at this question on stackOverFlow, who seems to have the same problem as me, but finally to solve it he started using canvas.
I am also wondering if the low performance of my game is due to the way I create the next board. I just have two boards, the actual one and the next one which contains all the new movement, and when I finally fill up this newBoard I change the board on the state triggering an action. Or Maybe the problem is that I am doing something wrong with the setInterval function.
Thanks in advance, any help would be appreciate.

To fix your issue you should add
shouldComponentUpdate(next) {
let props=this.props;
return props.col!==next.col || props.row !== next.row || props.val!==next. val;
}
to your Cell class. This will make sure only changed cells get rerendered.
Redux is normally pretty good at optimizing this stuff but there are two problems with your setup for this to work:
Your Cell class isn't connected. This means Redux can't help control its rendering. If you wanted that then pass the row/col in as the props but have it get its val from Redux in mapStateToProps.
You update the board by replacing. This can work as long as the cells in the new board that have not changed reference the same objects as the old board. This is how Redux checks if cells have changed. So when copying the old board make sure it's a shallow copy, and then when updating cells, replace the old cell object with an entirely new object. Remember: you're not allowed to modify objects that are in the Redux state. But you can modify the array itself because it's a copy and this means you can replace cells without modifying the original cell. If your cells in Redux only contain numbers, then you don't have to worry about it since with Numbers the value is the identity. This means if you write in the same value Redux will see it's the same. Redux cannot check that two different objects have the exact same properties because it takes too long.
Using the shouldComponentUpdate above makes it not necessary to do these things but you should probably will do them because it's the a Redux Way. Without these changes you're not using Redux well. In fact you lose almost all of Redux-React's benefits.
Either way will make your board fast however, provided that only a limited potion of the board actually changes. If the whole board is flickering then there's nothing you can do except use a different technology, such as Canvas or React-Canvas. DOM is slow.

There are 2 points you could try to address:
1) setInterval is a bit dangerous thing to use, if your step may run longer then your interval. You will hog the browser. So, use setTimeout to trigger next step, after step is done.
2) As a side note, DOM operations are slow, compared to running the calculations. Try to minimize DOM operations and update existing components then re-creating the elements (though, I am not sure how your application works in this respect, if it discards the previous board complitely)

Rx.js fromEvent + flatMapLatest broken?

Well, the problem itself is kind of hard to describe briefly, so here's a live example to demonstrate. It seems like I'm misunderstanding something about how Rx.js works, otherwise the functionality here comes from a bug.
What I tried to do was a simple reactive rendering setup, where what you see on the screen, and what events happen are both described in terms of Observables. The problem is that, for some indiscernible reason, the events are dropped entirely when the code is written one way, yet work fine with code that should theoretically be equivalent.
So, let's start with the first case in the example code above:
var dom = makeBox('one');
var clicks = Rx.Observable.fromEvent(dom, 'click');
If you create a DOM fragment, then you can simply use fromEvent to get an Observable for whatever event it emits. So far, so good. You can click this box and see a bunch of lines written to the log.
Now, the next step would be to make the DOM reactive, to express how it changes over time.
var domStream = Rx.Observable.return(makeBox('two'));
var clicks = domStream.flatMapLatest(function(dom) {
return Rx.Observable.fromEvent(dom, 'click');
});
That would make it an Observable, using return here to produce the simplest, constant case. The events you're interested in would be the ones emitted by the latest version of the dom, and that's exactly what the flatMapLatest operator does. This variant still works.
Ultimately, the goal would be to generate the current DOM state based on some application state. That is, map it from one Observable to another. Let's go with the simplest version for now, have a single constant value as the state, and then map it to the same fixed output we used previously:
var updates = Rx.Observable.return(1);
var domStream = updates.map(function (update) {
return makeBox('three');
});
var clicks = domStream.flatMapLatest(function(dom) {
return Rx.Observable.fromEvent(dom, 'click');
});
This should not be any different from the previous version. However, this outputs no events, no matter what you do.
What exactly is going on here? Did I misunderstand some fundamental concept of Rx, or what? I've run into some issues with hot vs cold Observables, but that seems unrelated in this minimal case. So, I'm kind of out of ideas. Can anyone enlighten me?

Sorry to tell you but it is a Hot vs Cold issue.
It is a subtle issue, but the difference between
Rx.Observable.return(makeBox('two'))
and
Rx.Observable.return(1).map(function() {return makeBox('three'); })
Is that the first returns a constant every time you subscribe to it, that is,
a box that you created initially. The second returns a new box every time the Observable is subscribed to, this causes a problem since you actually subscribe to the domStream variable twice, you are creating two instances of Box three, one which has event handlers but isn't shown and one that does not and is shown.
The fix is that you either need to use approach 2 or you need to convert the third into a hot stream either by using:
domStream.replay(1).refCount()
Or by using
domStream.publish()
then after all subscriptions are completed:
domStream.connect()

Never store intermediate program state in DOM?

I managed to run into this funny bug the other day where too quick modifications of the DOM caused the entire internet explorer to crash. So i was thinking, why am i even setting these values if i'm going to change them later anyway? (the modifications are very unpredictable so completely avoiding the scenario is impossible)
Some background, my website is more like a game/application and has alot of custom elements etc. Performance is key and moving around and modifying objects should be fast, smooth and without flicker. In one iteration tons of objects can have their state modified.
The objects in my application right now follows something similar to this pattern. domElement is the actual element used in the DOM, created with document.createElement or getElementById.
function SetWidth(w) {
this.width = w;
this.domElement.style.width = w + "px";
}
Obiously an object has more styles than just width but just to simplify things. This works pretty well right now but what if i for some reason set the width of one object twice inside one "program loop". This will mean the DOM will be modified twice but it's only the second state that should be displayed. In most new browsers this doesn't make a difference because the page is not rendered until all user javascript has completed. But in some browsers you can get updates unpredictably anytime. And even if there is no visible change, does it impact the performance?
Another problem is that some elements depend on being attached to the DOM before you can set/get some properties on them which you can easier avoid with the pattern below.
What i was thinking of doing instead was going back to the old-school render-loop pattern.
So the above object would look like this:
function SetWidth(w) {
this.width = w;
this.stateChanged = true;
}
function Render() {
if(this.stateChanged)
this.domElement.style.width = this.width + "px";
}
Once the "program-loop" is done you loop through every object only to "render" them (or use some more sophisticated structure keeping track of all modified objects).
To me this seems like defeating the whole purpose of having the DOM as you are basically reinventing what is already in place but sometimes it feels like it doesn't work properly so you have to roll your own version.
Has anyone used something similar and is it worth it? What are the pros and cons? What else do i have to think of? Adding and removing objects and managing z-index also should be taken into consideration.

One of the things to watch out for is that if you request the value of property that is dependant on the layout, the browser will recalculate the layout to get the value, which may take a significant amount of time. So "this.style.width = '80px'" will be quick but "this.style.width = '80px'; var wid = this.clientWidth;" will take much, much longer. Of course, if you use the setWidth/render pattern above, you won't be able to get the value of
clientWidth until after the render phase.
The setWidth/render pattern is a reasonable one for your use case, but I'd create a sub object to hold the pending layout values rather than storing them directly on the 'this' object. If you empty/recreate the sub object at the end of the render step, you won't need to store a separate stateChanged variable, the presence of the property in the sub object can serve that purpose.